Package org.apache.hadoop.hbase.regionserver.storefiletracker

Class FileBasedStoreFileTracker

java.lang.Object

org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

org.apache.hadoop.hbase.regionserver.storefiletracker.FileBasedStoreFileTracker

All Implemented Interfaces:

StoreFileTracker

@Private
class FileBasedStoreFileTracker
extends StoreFileTrackerBase

A file based store file tracker.

For this tracking way, the store file list will be persistent into a file, so we can write the new store files directly to the final data directory, as we will not load the broken files. This will greatly reduce the time for flush and compaction on some object storages as a rename is actual a copy on them. And it also avoid listing when loading store file list, which could also speed up the loading of store files as listing is also not a fast operation on most object storages.

Field Summary

Fields

Modifier and Type	Field	Description
private final StoreFileListFile	backedFile
private static final org.slf4j.Logger	LOG
private final Map<String,StoreFileInfo>	storefiles

Fields inherited from class org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

conf, ctx, isPrimaryReplica

Constructor Summary

Constructors

Constructor	Description
FileBasedStoreFileTracker(org.apache.hadoop.conf.Configuration conf, boolean isPrimaryReplica, StoreContext ctx)

Method Summary

Modifier and Type	Method	Description
protected void	doAddCompactionResults(Collection<StoreFileInfo> compactedFiles, Collection<StoreFileInfo> newFiles)
protected void	doAddNewStoreFiles(Collection<StoreFileInfo> newFiles)
protected List<StoreFileInfo>	doLoadStoreFiles(boolean readOnly)	For primary replica, we will call load once when opening a region, and the implementation could choose to do some cleanup work.
protected void	doSetStoreFiles(Collection<StoreFileInfo> files)
boolean	requireWritingToTmpDirFirst()	Whether the implementation of this tracker requires you to write to temp directory first, i.e, does not allow broken store files under the actual data directory.
private org.apache.hadoop.hbase.shaded.protobuf.generated.StoreFileTrackerProtos.StoreFileEntry	toStoreFileEntry(StoreFileInfo info)

Methods inherited from class org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

add, createReference, createWriter, getStoreFileInfo, getStoreFileInfo, getTrackerName, hasReferences, load, readReference, replace, set, updateWithTrackerConfigs

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

backedFile

private final StoreFileListFile backedFile

storefiles

private final Map<String,StoreFileInfo> storefiles

LOG

private static final org.slf4j.Logger LOG

Constructor Details

FileBasedStoreFileTracker

public FileBasedStoreFileTracker(org.apache.hadoop.conf.Configuration conf,
 boolean isPrimaryReplica,
 StoreContext ctx)

Method Details

doLoadStoreFiles

protected List<StoreFileInfo> doLoadStoreFiles(boolean readOnly)
                                        throws IOException

Description copied from class: StoreFileTrackerBase

For primary replica, we will call load once when opening a region, and the implementation could choose to do some cleanup work. So here we use readOnly to indicate that whether you are allowed to do the cleanup work. For secondary replicas, we will set readOnly to true.

Specified by:

doLoadStoreFiles in class StoreFileTrackerBase

Throws:

IOException

requireWritingToTmpDirFirst

public boolean requireWritingToTmpDirFirst()

Description copied from interface: StoreFileTracker

Whether the implementation of this tracker requires you to write to temp directory first, i.e, does not allow broken store files under the actual data directory.

toStoreFileEntry

private org.apache.hadoop.hbase.shaded.protobuf.generated.StoreFileTrackerProtos.StoreFileEntry toStoreFileEntry(StoreFileInfo info)

doAddNewStoreFiles

protected void doAddNewStoreFiles(Collection<StoreFileInfo> newFiles)
                           throws IOException

Specified by:

doAddNewStoreFiles in class StoreFileTrackerBase

Throws:

IOException

doAddCompactionResults

protected void doAddCompactionResults(Collection<StoreFileInfo> compactedFiles,
 Collection<StoreFileInfo> newFiles)
                               throws IOException

Specified by:

doAddCompactionResults in class StoreFileTrackerBase

Throws:

IOException

doSetStoreFiles

protected void doSetStoreFiles(Collection<StoreFileInfo> files)
                        throws IOException

Specified by:

doSetStoreFiles in class StoreFileTrackerBase

Throws:

IOException