@InterfaceAudience.Private public class CombinedWriterBase<T extends WALProvider.WriterBase> extends Object implements WALProvider.WriterBase
Modifier and Type | Field and Description |
---|---|
private static org.slf4j.Logger |
LOG |
protected org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T> |
writers |
Modifier | Constructor and Description |
---|---|
protected |
CombinedWriterBase(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T> writers) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
long |
getLength() |
long |
getSyncedLength()
NOTE: We add this method for
WALFileLengthProvider used for replication, considering
the case if we use AsyncFSWAL ,we write to 3 DNs concurrently, according to the
visibility guarantee of HDFS, the data will be available immediately when arriving at DN
since all the DNs will be considered as the last one in pipeline. |
private static final org.slf4j.Logger LOG
protected final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T extends WALProvider.WriterBase> writers
protected CombinedWriterBase(org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<T> writers)
public void close() throws IOException
close
in interface Closeable
close
in interface AutoCloseable
IOException
public long getLength()
getLength
in interface WALProvider.WriterBase
public long getSyncedLength()
WALProvider.WriterBase
WALFileLengthProvider
used for replication, considering
the case if we use AsyncFSWAL
,we write to 3 DNs concurrently, according to the
visibility guarantee of HDFS, the data will be available immediately when arriving at DN
since all the DNs will be considered as the last one in pipeline. This means replication may
read uncommitted data and replicate it to the remote cluster and cause data inconsistency.
The method WALProvider.WriterBase.getLength()
may return length which just in hdfs client buffer
and not successfully synced to HDFS, so we use this method to return the length successfully
synced to HDFS and replication thread could only read writing WAL file limited by this
length. see also HBASE-14004 and this document for more details:
https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit#getSyncedLength
in interface WALProvider.WriterBase
Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.