Package org.apache.hadoop.hbase.regionserver

Class DefaultStoreFileManager

java.lang.Object

org.apache.hadoop.hbase.regionserver.DefaultStoreFileManager

All Implemented Interfaces:

StoreFileManager

@Private
class DefaultStoreFileManager
extends Object
implements StoreFileManager

Default implementation of StoreFileManager. Not thread-safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static class	DefaultStoreFileManager.StoreFileList

Field Summary

Fields

Modifier and Type	Field	Description
private final int	blockingFileCount
private final CellComparator	cellComparator
private final CompactionConfiguration	comConf
private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>	compactedfiles	List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.
private final boolean	enableLiveFileTracking
private static final org.slf4j.Logger	LOG
private final Comparator<HStoreFile>	storeFileComparator
private DefaultStoreFileManager.StoreFileList	storeFiles

Constructor Summary

Constructors

Constructor	Description
DefaultStoreFileManager(CellComparator cellComparator, Comparator<HStoreFile> storeFileComparator, org.apache.hadoop.conf.Configuration conf, CompactionConfiguration comConf)

Method Summary

Modifier and Type	Method	Description
void	addCompactionResults(Collection<HStoreFile> newCompactedfiles, Collection<HStoreFile> results)	Adds only the new compaction results into the structure.
Collection<HStoreFile>	clearCompactedFiles()	Clears all the compacted files and returns them.
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>	clearFiles()	Clears all the files currently in use and returns them.
final Iterator<HStoreFile>	getCandidateFilesForRowKeyBefore(KeyValue targetKey)	Gets initial, full list of candidate store files to check for row-key-before.
Collection<HStoreFile>	getCompactedfiles()	List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.
final int	getCompactedFilesCount()	Returns the number of compacted files.
double	getCompactionPressure()
Collection<HStoreFile>	getFilesForScan(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow, boolean onlyLatestVersion)	Gets the store files to scan for a Scan or Get request.
private List<HStoreFile>	getLiveFiles(Collection<HStoreFile> storeFiles)
final Optional<byte[]>	getSplitPoint()	Gets the split point for the split of this set of store files (approx.
int	getStoreCompactionPriority()	Returns The store compaction priority.
Comparator<HStoreFile>	getStoreFileComparator()
final int	getStorefileCount()	Returns the number of files currently in use.
final Collection<HStoreFile>	getStoreFiles()	Gets the snapshot of the store files currently in use.
Collection<HStoreFile>	getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting)
void	insertNewFiles(Collection<HStoreFile> sfs)	Adds new files, either for from MemStore flush or bulk insert, into the structure.
void	loadFiles(List<HStoreFile> storeFiles)	Loads the initial store files into empty StoreFileManager.
void	removeCompactedFiles(Collection<HStoreFile> removedCompactedfiles)	Remove the compacted files
Iterator<HStoreFile>	updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate)	Updates the candidate list for finding row key before.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

cellComparator

private final CellComparator cellComparator

comConf

private final CompactionConfiguration comConf

blockingFileCount

private final int blockingFileCount

storeFileComparator

private final Comparator<HStoreFile> storeFileComparator

storeFiles

private volatile DefaultStoreFileManager.StoreFileList storeFiles

compactedfiles

private volatile org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> compactedfiles

List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction. These compacted files will be deleted/cleared once all the existing readers on these compacted files are done.

enableLiveFileTracking

private final boolean enableLiveFileTracking

Constructor Details

DefaultStoreFileManager

public DefaultStoreFileManager(CellComparator cellComparator,
 Comparator<HStoreFile> storeFileComparator,
 org.apache.hadoop.conf.Configuration conf,
 CompactionConfiguration comConf)

Method Details

getLiveFiles

private List<HStoreFile> getLiveFiles(Collection<HStoreFile> storeFiles)
                               throws IOException

Throws:

IOException

loadFiles

public void loadFiles(List<HStoreFile> storeFiles)
               throws IOException

Description copied from interface: StoreFileManager

Loads the initial store files into empty StoreFileManager.

Specified by:

loadFiles in interface StoreFileManager

Parameters:

storeFiles - The files to load.

Throws:

IOException

getStoreFiles

public final Collection<HStoreFile> getStoreFiles()

Description copied from interface: StoreFileManager

Gets the snapshot of the store files currently in use. Can be used for things like metrics and checks; should not assume anything about relations between store files in the list.

Specified by:

getStoreFiles in interface StoreFileManager

Returns:

The list of StoreFiles.

getCompactedfiles

public Collection<HStoreFile> getCompactedfiles()

Description copied from interface: StoreFileManager

List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction. These compacted files will be deleted/cleared once all the existing readers on these compacted files are done.

Specified by:

getCompactedfiles in interface StoreFileManager

Returns:

the list of compacted files

insertNewFiles

public void insertNewFiles(Collection<HStoreFile> sfs)
                    throws IOException

Description copied from interface: StoreFileManager

Adds new files, either for from MemStore flush or bulk insert, into the structure.

Specified by:

insertNewFiles in interface StoreFileManager

Parameters:

sfs - New store files.

Throws:

IOException

clearFiles

public org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> clearFiles()

Description copied from interface: StoreFileManager

Clears all the files currently in use and returns them.

Specified by:

clearFiles in interface StoreFileManager

Returns:

The files previously in use.

clearCompactedFiles

public Collection<HStoreFile> clearCompactedFiles()

Description copied from interface: StoreFileManager

Clears all the compacted files and returns them. This method is expected to be accessed single threaded.

Specified by:

clearCompactedFiles in interface StoreFileManager

Returns:

The files compacted previously.

getStorefileCount

public final int getStorefileCount()

Description copied from interface: StoreFileManager

Returns the number of files currently in use.

Specified by:

getStorefileCount in interface StoreFileManager

Returns:

The number of files.

getCompactedFilesCount

public final int getCompactedFilesCount()

Description copied from interface: StoreFileManager

Returns the number of compacted files.

Specified by:

getCompactedFilesCount in interface StoreFileManager

Returns:

The number of files.

addCompactionResults

public void addCompactionResults(Collection<HStoreFile> newCompactedfiles,
 Collection<HStoreFile> results)
                          throws IOException

Description copied from interface: StoreFileManager

Adds only the new compaction results into the structure.

Specified by:

addCompactionResults in interface StoreFileManager

Parameters:

newCompactedfiles - The input files for the compaction.

results - The resulting files for the compaction.

Throws:

IOException

removeCompactedFiles

public void removeCompactedFiles(Collection<HStoreFile> removedCompactedfiles)

Description copied from interface: StoreFileManager

Remove the compacted files

Specified by:

removeCompactedFiles in interface StoreFileManager

Parameters:

removedCompactedfiles - the list of compacted files

getCandidateFilesForRowKeyBefore

public final Iterator<HStoreFile> getCandidateFilesForRowKeyBefore(KeyValue targetKey)

Description copied from interface: StoreFileManager

Gets initial, full list of candidate store files to check for row-key-before.

Specified by:

getCandidateFilesForRowKeyBefore in interface StoreFileManager

Parameters:

targetKey - The key that is the basis of the search.

Returns:

The files that may have the key less than or equal to targetKey, in reverse order of new-ness, and preference for target key.

updateCandidateFilesForRowKeyBefore

public Iterator<HStoreFile> updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
 KeyValue targetKey,
 Cell candidate)

Description copied from interface: StoreFileManager

Updates the candidate list for finding row key before. Based on the list of candidates remaining to check from getCandidateFilesForRowKeyBefore, targetKey and current candidate, may trim and reorder the list to remove the files where a better candidate cannot be found.

Specified by:

updateCandidateFilesForRowKeyBefore in interface StoreFileManager

Parameters:

candidateFiles - The candidate files not yet checked for better candidates - return value from StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue), with some files already removed.

targetKey - The key to search for.

candidate - The current best candidate found.

Returns:

The list to replace candidateFiles.

getSplitPoint

public final Optional<byte[]> getSplitPoint()
                                     throws IOException

Description copied from interface: StoreFileManager

Gets the split point for the split of this set of store files (approx. middle).

Specified by:

getSplitPoint in interface StoreFileManager

Returns:

The mid-point if possible.

Throws:

IOException

getFilesForScan

public Collection<HStoreFile> getFilesForScan(byte[] startRow,
 boolean includeStartRow,
 byte[] stopRow,
 boolean includeStopRow,
 boolean onlyLatestVersion)

Description copied from interface: StoreFileManager

Gets the store files to scan for a Scan or Get request.

Specified by:

getFilesForScan in interface StoreFileManager

Parameters:

startRow - Start row of the request.

stopRow - Stop row of the request.

onlyLatestVersion - Scan only latest live version cells.

Returns:

The list of files that are to be read for this request.

getStoreCompactionPriority

public int getStoreCompactionPriority()

Description copied from interface: StoreFileManager

Returns The store compaction priority.

Specified by:

getStoreCompactionPriority in interface StoreFileManager

getUnneededFiles

public Collection<HStoreFile> getUnneededFiles(long maxTs,
 List<HStoreFile> filesCompacting)

Specified by:

getUnneededFiles in interface StoreFileManager

Parameters:

maxTs - Maximum expired timestamp.

filesCompacting - Files that are currently compacting.

Returns:

The files which don't have any necessary data according to TTL and other criteria.

getCompactionPressure

public double getCompactionPressure()

Specified by:

getCompactionPressure in interface StoreFileManager

Returns:

the compaction pressure used for compaction throughput tuning.

See Also:

• HStore.getCompactionPressure()

getStoreFileComparator

public Comparator<HStoreFile> getStoreFileComparator()

Specified by:

getStoreFileComparator in interface StoreFileManager

Returns:

the comparator used to sort storefiles. Usually, the HStoreFile.getMaxSequenceId() is the first priority.