Package org.apache.hadoop.hbase.regionserver.wal

Class CombinedWriterBase<T extends WALProvider.WriterBase>

java.lang.Object
org.apache.hadoop.hbase.regionserver.wal.CombinedWriterBase<T>
All Implemented Interfaces:
Closeable, AutoCloseable, WALProvider.WriterBase
Direct Known Subclasses:
CombinedAsyncWriter, CombinedWriter
@Private public class CombinedWriterBase<T extends WALProvider.WriterBase> extends Object implements WALProvider.WriterBase
Base class for combined wal writer implementations.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final org.slf4j.Logger
    LOG
     
    protected final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T>
    writers
     

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    CombinedWriterBase(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T> writers)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    long
    getLength()
     
    long
    getSyncedLength()
    NOTE: We add this method for WALFileLengthProvider used for replication, considering the case if we use AsyncFSWAL,we write to 3 DNs concurrently, according to the visibility guarantee of HDFS, the data will be available immediately when arriving at DN since all the DNs will be considered as the last one in pipeline.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • LOG

      private static final org.slf4j.Logger LOG

    • writers

      protected final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T extends WALProvider.WriterBase> writers

  • Constructor Details

    • CombinedWriterBase

      protected CombinedWriterBase(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T> writers)

  • Method Details

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • getLength

      public long getLength()
      Specified by:
      getLength in interface WALProvider.WriterBase

    • getSyncedLength

      public long getSyncedLength()
      Description copied from interface: WALProvider.WriterBase
      NOTE: We add this method for WALFileLengthProvider used for replication, considering the case if we use AsyncFSWAL,we write to 3 DNs concurrently, according to the visibility guarantee of HDFS, the data will be available immediately when arriving at DN since all the DNs will be considered as the last one in pipeline. This means replication may read uncommitted data and replicate it to the remote cluster and cause data inconsistency. The method WALProvider.WriterBase.getLength() may return length which just in hdfs client buffer and not successfully synced to HDFS, so we use this method to return the length successfully synced to HDFS and replication thread could only read writing WAL file limited by this length. see also HBASE-14004 and this document for more details: https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit#
      Specified by:
      getSyncedLength in interface WALProvider.WriterBase
      Returns:
      byteSize successfully synced to underlying filesystem.