public static interface WALProvider.WriterBase extends Closeable
Modifier and Type | Method and Description |
---|---|
long |
getLength() |
long |
getSyncedLength()
NOTE: We add this method for
WALFileLengthProvider used for replication, considering
the case if we use AsyncFSWAL ,we write to 3 DNs concurrently, according to the
visibility guarantee of HDFS, the data will be available immediately when arriving at DN
since all the DNs will be considered as the last one in pipeline. |
long getLength()
long getSyncedLength()
WALFileLengthProvider
used for replication, considering
the case if we use AsyncFSWAL
,we write to 3 DNs concurrently, according to the
visibility guarantee of HDFS, the data will be available immediately when arriving at DN
since all the DNs will be considered as the last one in pipeline. This means replication may
read uncommitted data and replicate it to the remote cluster and cause data inconsistency.
The method getLength()
may return length which just in hdfs client buffer
and not successfully synced to HDFS, so we use this method to return the length successfully
synced to HDFS and replication thread could only read writing WAL file limited by this
length. see also HBASE-14004 and this document for more details:
https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit#Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.