Package org.apache.hadoop.hbase.regionserver.storefiletracker

Class StoreFileTrackerBase

java.lang.Object

org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTrackerBase

All Implemented Interfaces:

StoreFileTracker

Direct Known Subclasses:

DefaultStoreFileTracker, FileBasedStoreFileTracker, MigrationStoreFileTracker

@Private
abstract class StoreFileTrackerBase
extends Object
implements StoreFileTracker

Base class for all store file tracker.

Mainly used to place the common logic to skip persistent for secondary replicas.

Field Summary

Fields

Modifier and Type	Field	Description
private boolean	cacheOnWriteLogged
protected final org.apache.hadoop.conf.Configuration	conf
protected final StoreContext	ctx
protected final boolean	isPrimaryReplica
private static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	StoreFileTrackerBase(org.apache.hadoop.conf.Configuration conf, boolean isPrimaryReplica, StoreContext ctx)

Method Summary

Modifier and Type	Method	Description
final void	add(Collection<StoreFileInfo> newFiles)	Add new store files.
protected void	archiveStoreFiles(List<HStoreFile> storeFiles)
private HFileContext	createFileContext(Compression.Algorithm compression, boolean includeMVCCReadpoint, boolean includesTag, Encryption.Context encryptionContext)
String	createFromHFileLink(String hfileLinkName, boolean createBackRef)	Create a new HFileLink starting from a hfileLink name
String	createHFileLink(TableName linkedTable, String linkedRegion, String hfileName, boolean createBackRef)	Create a new HFileLink
Reference	createReference(Reference reference, org.apache.hadoop.fs.Path path)
final StoreFileWriter	createWriter(CreateStoreFileWriterParams params)	Create a writer for writing new store files.
protected abstract void	doAddCompactionResults(Collection<StoreFileInfo> compactedFiles, Collection<StoreFileInfo> newFiles)
protected abstract void	doAddNewStoreFiles(Collection<StoreFileInfo> newFiles)
protected abstract List<StoreFileInfo>	doLoadStoreFiles(boolean readOnly)	For primary replica, we will call load once when opening a region, and the implementation could choose to do some cleanup work.
protected abstract void	doSetStoreFiles(Collection<StoreFileInfo> files)
StoreFileInfo	getStoreFileInfo(org.apache.hadoop.fs.FileStatus fileStatus, org.apache.hadoop.fs.Path initialPath, boolean primaryReplica)
StoreFileInfo	getStoreFileInfo(org.apache.hadoop.fs.Path initialPath, boolean primaryReplica)
protected final String	getTrackerName()
boolean	hasReferences()	Returns true if the specified family has reference files
final List<StoreFileInfo>	load()	Load the store files list when opening a region.
Reference	readReference(org.apache.hadoop.fs.Path p)	Reads the reference file from the given path.
void	removeStoreFiles(List<HStoreFile> storeFiles)	Closes and archives the specified store files from the specified family.
final void	replace(Collection<StoreFileInfo> compactedFiles, Collection<StoreFileInfo> newFiles)	Add new store files and remove compacted store files after compaction.
final void	set(List<StoreFileInfo> files)	Set the store files.
TableDescriptorBuilder	updateWithTrackerConfigs(TableDescriptorBuilder builder)	Adds StoreFileTracker implementations specific configurations into the table descriptor.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.storefiletracker.StoreFileTracker

requireWritingToTmpDirFirst

Field Details

LOG

private static final org.slf4j.Logger LOG

conf

protected final org.apache.hadoop.conf.Configuration conf

isPrimaryReplica

protected final boolean isPrimaryReplica

ctx

protected final StoreContext ctx

cacheOnWriteLogged

private volatile boolean cacheOnWriteLogged

Constructor Details

StoreFileTrackerBase

protected StoreFileTrackerBase(org.apache.hadoop.conf.Configuration conf,
 boolean isPrimaryReplica,
 StoreContext ctx)

Method Details

load

public final List<StoreFileInfo> load()
                               throws IOException

Description copied from interface: StoreFileTracker

Load the store files list when opening a region.

Specified by:

load in interface StoreFileTracker

Throws:

IOException

add

public final void add(Collection<StoreFileInfo> newFiles)
               throws IOException

Description copied from interface: StoreFileTracker

Add new store files.

Used for flush and bulk load.

Specified by:

add in interface StoreFileTracker

Throws:

IOException

replace

public final void replace(Collection<StoreFileInfo> compactedFiles,
 Collection<StoreFileInfo> newFiles)
                   throws IOException

Description copied from interface: StoreFileTracker

Add new store files and remove compacted store files after compaction.

Specified by:

replace in interface StoreFileTracker

Throws:

IOException

set

public final void set(List<StoreFileInfo> files)
               throws IOException

Description copied from interface: StoreFileTracker

Set the store files.

Specified by:

set in interface StoreFileTracker

Throws:

IOException

updateWithTrackerConfigs

public TableDescriptorBuilder updateWithTrackerConfigs(TableDescriptorBuilder builder)

Description copied from interface: StoreFileTracker

Adds StoreFileTracker implementations specific configurations into the table descriptor.

This is used to avoid accidentally data loss when changing the cluster level store file tracker implementation, and also possible misconfiguration between master and region servers.

See HBASE-26246 for more details.

Specified by:

updateWithTrackerConfigs in interface StoreFileTracker

Parameters:

builder - The table descriptor builder for the given table.

getTrackerName

protected final String getTrackerName()

createFileContext

private HFileContext createFileContext(Compression.Algorithm compression,
 boolean includeMVCCReadpoint,
 boolean includesTag,
 Encryption.Context encryptionContext)

createWriter

public final StoreFileWriter createWriter(CreateStoreFileWriterParams params)
                                   throws IOException

Description copied from interface: StoreFileTracker

Create a writer for writing new store files.

Specified by:

createWriter in interface StoreFileTracker

Returns:

Writer for a new StoreFile

Throws:

IOException

createReference

public Reference createReference(Reference reference,
 org.apache.hadoop.fs.Path path)
                          throws IOException

Specified by:

createReference in interface StoreFileTracker

Throws:

IOException

hasReferences

public boolean hasReferences()
                      throws IOException

Returns true if the specified family has reference files

Specified by:

hasReferences in interface StoreFileTracker

Parameters:

familyName - Column Family Name

Returns:

true if family contains reference files

Throws:

IOException

readReference

public Reference readReference(org.apache.hadoop.fs.Path p)
                        throws IOException

Description copied from interface: StoreFileTracker

Reads the reference file from the given path.

Specified by:

readReference in interface StoreFileTracker

Parameters:

p - the Path to the reference file in the file system.

Returns:

a Reference that points at top/bottom half of a an hfile

Throws:

IOException

getStoreFileInfo

public StoreFileInfo getStoreFileInfo(org.apache.hadoop.fs.Path initialPath,
 boolean primaryReplica)
                               throws IOException

Specified by:

getStoreFileInfo in interface StoreFileTracker

Throws:

IOException

getStoreFileInfo

public StoreFileInfo getStoreFileInfo(org.apache.hadoop.fs.FileStatus fileStatus,
 org.apache.hadoop.fs.Path initialPath,
 boolean primaryReplica)
                               throws IOException

Specified by:

getStoreFileInfo in interface StoreFileTracker

Throws:

IOException

createHFileLink

public String createHFileLink(TableName linkedTable,
 String linkedRegion,
 String hfileName,
 boolean createBackRef)
                       throws IOException

Description copied from interface: StoreFileTracker

Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Specified by:

createHFileLink in interface StoreFileTracker

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

createFromHFileLink

public String createFromHFileLink(String hfileLinkName,
 boolean createBackRef)
                           throws IOException

Description copied from interface: StoreFileTracker

Create a new HFileLink starting from a hfileLink name

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Specified by:

createFromHFileLink in interface StoreFileTracker

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

removeStoreFiles

public void removeStoreFiles(List<HStoreFile> storeFiles)
                      throws IOException

Description copied from interface: StoreFileTracker

Closes and archives the specified store files from the specified family.

Specified by:

removeStoreFiles in interface StoreFileTracker

Parameters:

storeFiles - set of store files to remove

Throws:

IOException - if the archiving fails

archiveStoreFiles

protected void archiveStoreFiles(List<HStoreFile> storeFiles)
                          throws IOException

Throws:

IOException

doLoadStoreFiles

protected abstract List<StoreFileInfo> doLoadStoreFiles(boolean readOnly)
                                                 throws IOException

For primary replica, we will call load once when opening a region, and the implementation could choose to do some cleanup work. So here we use readOnly to indicate that whether you are allowed to do the cleanup work. For secondary replicas, we will set readOnly to true.

Throws:

IOException

doAddNewStoreFiles

protected abstract void doAddNewStoreFiles(Collection<StoreFileInfo> newFiles)
                                    throws IOException

Throws:

IOException

doAddCompactionResults

protected abstract void doAddCompactionResults(Collection<StoreFileInfo> compactedFiles,
 Collection<StoreFileInfo> newFiles)
                                        throws IOException

Throws:

IOException

doSetStoreFiles

protected abstract void doSetStoreFiles(Collection<StoreFileInfo> files)
                                 throws IOException

Throws:

IOException