Class FanOutOneBlockAsyncDFSOutput

java.lang.Object
org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput
All Implemented Interfaces:
Closeable, AutoCloseable, AsyncFSOutput

@Private public class FanOutOneBlockAsyncDFSOutput extends Object implements AsyncFSOutput
An asynchronous HDFS output stream implementation which fans out data to datanode and only supports writing file with only one block.

Use the createOutput method in FanOutOneBlockAsyncDFSOutputHelper to create. The main usage of this class is implementing WAL, so we only expose a little HDFS configurations in the method. And we place it here under io package because we want to make it independent of WAL implementation thus easier to move it to HDFS project finally.

Note that, although we support pipelined flush, i.e, write new data and then flush before the previous flush succeeds, the implementation is not thread safe, so you should not call its methods concurrently.

Advantages compare to DFSOutputStream:

  1. The fan out mechanism. This will reduce the latency.
  2. Fail-fast when connection to datanode error. The WAL implementation could open new writer ASAP.
  3. We could benefit from netty's ByteBuf management mechanism.