public static interface WALProvider.WriterBase extends Closeable
Modifier and Type | Method and Description |
---|---|
long |
getLength() |
long |
getSyncedLength()
NOTE: We add this method for
WALFileLengthProvider used for replication,
considering the case if we use AsyncFSWAL ,we write to 3 DNs concurrently,
according to the visibility guarantee of HDFS, the data will be available immediately
when arriving at DN since all the DNs will be considered as the last one in pipeline. |
long getLength()
long getSyncedLength()
WALFileLengthProvider
used for replication,
considering the case if we use AsyncFSWAL
,we write to 3 DNs concurrently,
according to the visibility guarantee of HDFS, the data will be available immediately
when arriving at DN since all the DNs will be considered as the last one in pipeline.
This means replication may read uncommitted data and replicate it to the remote cluster
and cause data inconsistency.
The method getLength()
may return length which just in hdfs client
buffer and not successfully synced to HDFS, so we use this method to return the length
successfully synced to HDFS and replication thread could only read writing WAL file
limited by this length.
see also HBASE-14004 and this document for more details:
https://docs.google.com/document/d/11AyWtGhItQs6vsLRIx32PwTxmBY3libXwGXI25obVEY/edit#Copyright © 2007–2021 The Apache Software Foundation. All rights reserved.