Package org.apache.hadoop.hbase.regionserver.storefiletracker

Class FileBasedStoreFileTracker

java.lang.Object

org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

org.apache.hadoop.hbase.regionserver.storefiletracker.FileBasedStoreFileTracker

All Implemented Interfaces:

StoreFileTracker

@Private
class FileBasedStoreFileTracker
extends StoreFileTrackerBase

A file based store file tracker.

For this tracking way, the store file list will be persistent into a file, so we can write the new store files directly to the final data directory, as we will not load the broken files. This will greatly reduce the time for flush and compaction on some object storages as a rename is actual a copy on them. And it also avoid listing when loading store file list, which could also speed up the loading of store files as listing is also not a fast operation on most object storages.

Field Summary

Fields

Modifier and Type	Field	Description
private final StoreFileListFile	backedFile
private static final org.slf4j.Logger	LOG
private final Map<String,StoreFileInfo>	storefiles

Fields inherited from class org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

conf, ctx, isPrimaryReplica

Constructor Summary

Constructors

Constructor	Description
FileBasedStoreFileTracker(org.apache.hadoop.conf.Configuration conf, boolean isPrimaryReplica, StoreContext ctx)

Method Summary

Modifier and Type	Method	Description
Reference	createAndCommitReference(Reference reference, org.apache.hadoop.fs.Path path)
HFileLink	createHFileLink(TableName linkedTable, String linkedRegion, String hfileName, boolean createBackRef)	Create a new HFileLink
Reference	createReference(Reference reference, org.apache.hadoop.fs.Path path)
protected void	doAddCompactionResults(Collection<StoreFileInfo> compactedFiles, Collection<StoreFileInfo> newFiles)
protected void	doAddNewStoreFiles(Collection<StoreFileInfo> newFiles)
protected List<StoreFileInfo>	doLoadStoreFiles(boolean readOnly)	For primary replica, we will call load once when opening a region, and the implementation could choose to do some cleanup work.
protected void	doSetStoreFiles(Collection<StoreFileInfo> files)
boolean	hasReferences()	Returns true if the specified family has reference files
Reference	readReference(org.apache.hadoop.fs.Path p)	Reads the reference file from the given path.
boolean	requireWritingToTmpDirFirst()	Whether the implementation of this tracker requires you to write to temp directory first, i.e, does not allow broken store files under the actual data directory.
private org.apache.hadoop.hbase.shaded.protobuf.generated.StoreFileTrackerProtos.StoreFileEntry	toStoreFileEntry(StoreFileInfo info)

Methods inherited from class org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

add, archiveStoreFiles, createAndCommitHFileLink, createFromHFileLink, createWriter, getStoreContext, getStoreFileInfo, getStoreFileInfo, getTrackerName, load, removeStoreFiles, replace, set, updateWithTrackerConfigs

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

backedFile

private final StoreFileListFile backedFile

storefiles

private final Map<String,StoreFileInfo> storefiles

LOG

private static final org.slf4j.Logger LOG

Constructor Details

FileBasedStoreFileTracker

public FileBasedStoreFileTracker(org.apache.hadoop.conf.Configuration conf,
 boolean isPrimaryReplica,
 StoreContext ctx)

Method Details

doLoadStoreFiles

protected List<StoreFileInfo> doLoadStoreFiles(boolean readOnly)
                                        throws IOException

Description copied from class: StoreFileTrackerBase

For primary replica, we will call load once when opening a region, and the implementation could choose to do some cleanup work. So here we use readOnly to indicate that whether you are allowed to do the cleanup work. For secondary replicas, we will set readOnly to true.

Specified by:

doLoadStoreFiles in class StoreFileTrackerBase

Throws:

IOException

requireWritingToTmpDirFirst

public boolean requireWritingToTmpDirFirst()

Description copied from interface: StoreFileTracker

Whether the implementation of this tracker requires you to write to temp directory first, i.e, does not allow broken store files under the actual data directory.

toStoreFileEntry

private org.apache.hadoop.hbase.shaded.protobuf.generated.StoreFileTrackerProtos.StoreFileEntry toStoreFileEntry(StoreFileInfo info)

doAddNewStoreFiles

protected void doAddNewStoreFiles(Collection<StoreFileInfo> newFiles)
                           throws IOException

Specified by:

doAddNewStoreFiles in class StoreFileTrackerBase

Throws:

IOException

doAddCompactionResults

protected void doAddCompactionResults(Collection<StoreFileInfo> compactedFiles,
 Collection<StoreFileInfo> newFiles)
                               throws IOException

Specified by:

doAddCompactionResults in class StoreFileTrackerBase

Throws:

IOException

doSetStoreFiles

protected void doSetStoreFiles(Collection<StoreFileInfo> files)
                        throws IOException

Specified by:

doSetStoreFiles in class StoreFileTrackerBase

Throws:

IOException

readReference

public Reference readReference(org.apache.hadoop.fs.Path p)
                        throws IOException

Description copied from interface: StoreFileTracker

Reads the reference file from the given path.

Specified by:

readReference in interface StoreFileTracker

Overrides:

readReference in class StoreFileTrackerBase

Parameters:

p - the Path to the reference file in the file system.

Returns:

a Reference that points at top/bottom half of a an hfile

Throws:

IOException

hasReferences

public boolean hasReferences()
                      throws IOException

Description copied from class: StoreFileTrackerBase

Returns true if the specified family has reference files

Specified by:

hasReferences in interface StoreFileTracker

Overrides:

hasReferences in class StoreFileTrackerBase

Returns:

true if family contains reference files

Throws:

IOException

createHFileLink

public HFileLink createHFileLink(TableName linkedTable,
 String linkedRegion,
 String hfileName,
 boolean createBackRef)
                          throws IOException

Description copied from interface: StoreFileTracker

Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Specified by:

createHFileLink in interface StoreFileTracker

Overrides:

createHFileLink in class StoreFileTrackerBase

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

createReference

public Reference createReference(Reference reference,
 org.apache.hadoop.fs.Path path)
                          throws IOException

Specified by:

createReference in interface StoreFileTracker

Overrides:

createReference in class StoreFileTrackerBase

Throws:

IOException

createAndCommitReference

public Reference createAndCommitReference(Reference reference,
 org.apache.hadoop.fs.Path path)
                                   throws IOException

Specified by:

createAndCommitReference in interface StoreFileTracker

Overrides:

createAndCommitReference in class StoreFileTrackerBase

Throws:

IOException