org.apache.hadoop.hbase.regionserver

Class DefaultStoreFileManager

java.lang.Object

org.apache.hadoop.hbase.regionserver.DefaultStoreFileManager

All Implemented Interfaces:

StoreFileManager

@InterfaceAudience.Private
class DefaultStoreFileManager
extends Object
implements StoreFileManager

Default implementation of StoreFileManager. Not thread-safe.

Field Summary

Fields

Modifier and Type	Field and Description
private int	blockingFileCount
private CellComparator	cellComparator
private CompactionConfiguration	comConf
private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>	compactedfiles List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.
private static org.slf4j.Logger	LOG
private Comparator<HStoreFile>	storeFileComparator
private org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile>	storefiles List of store files inside this store.

Constructor Summary

Constructors

Constructor and Description
DefaultStoreFileManager(CellComparator cellComparator, Comparator<HStoreFile> storeFileComparator, org.apache.hadoop.conf.Configuration conf, CompactionConfiguration comConf)

Method Summary

Modifier and Type	Method and Description
void	addCompactionResults(Collection<HStoreFile> newCompactedfiles, Collection<HStoreFile> results) Adds only the new compaction results into the structure.
Collection<HStoreFile>	clearCompactedFiles() Clears all the compacted files and returns them.
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>	clearFiles() Clears all the files currently in use and returns them.
Iterator<HStoreFile>	getCandidateFilesForRowKeyBefore(KeyValue targetKey) Gets initial, full list of candidate store files to check for row-key-before.
Collection<HStoreFile>	getCompactedfiles() List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.
int	getCompactedFilesCount() Returns the number of compacted files.
double	getCompactionPressure()
Collection<HStoreFile>	getFilesForScan(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow) Gets the store files to scan for a Scan or Get request.
Optional<byte[]>	getSplitPoint() Gets the split point for the split of this set of store files (approx.
int	getStoreCompactionPriority()
Comparator<HStoreFile>	getStoreFileComparator()
int	getStorefileCount() Returns the number of files currently in use.
Collection<HStoreFile>	getStorefiles() Gets the snapshot of the store files currently in use.
Collection<HStoreFile>	getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting)
void	insertNewFiles(Collection<HStoreFile> sfs) Adds new files, either for from MemStore flush or bulk insert, into the structure.
void	loadFiles(List<HStoreFile> storeFiles) Loads the initial store files into empty StoreFileManager.
void	removeCompactedFiles(Collection<HStoreFile> removedCompactedfiles) Remove the compacted files
Iterator<HStoreFile>	updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) Updates the candidate list for finding row key before.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

cellComparator

private final CellComparator cellComparator

comConf

private final CompactionConfiguration comConf

blockingFileCount

private final int blockingFileCount

storeFileComparator

private final Comparator<HStoreFile> storeFileComparator

storefiles

private volatile org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> storefiles

List of store files inside this store. This is an immutable list that is atomically replaced when its contents change.

compactedfiles

private volatile org.apache.hbase.thirdparty.com.google.common.collect.ImmutableList<HStoreFile> compactedfiles

List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction. These compacted files will be deleted/cleared once all the existing readers on these compacted files are done.

Constructor Detail

DefaultStoreFileManager

public DefaultStoreFileManager(CellComparator cellComparator,
                               Comparator<HStoreFile> storeFileComparator,
                               org.apache.hadoop.conf.Configuration conf,
                               CompactionConfiguration comConf)

Method Detail

loadFiles

public void loadFiles(List<HStoreFile> storeFiles)

Description copied from interface: StoreFileManager

Loads the initial store files into empty StoreFileManager.

Specified by:

loadFiles in interface StoreFileManager

Parameters:

storeFiles - The files to load.

getStorefiles

public final Collection<HStoreFile> getStorefiles()

Description copied from interface: StoreFileManager

Gets the snapshot of the store files currently in use. Can be used for things like metrics and checks; should not assume anything about relations between store files in the list.

Specified by:

getStorefiles in interface StoreFileManager

Returns:

The list of StoreFiles.

getCompactedfiles

public Collection<HStoreFile> getCompactedfiles()

Description copied from interface: StoreFileManager

List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction. These compacted files will be deleted/cleared once all the existing readers on these compacted files are done.

Specified by:

getCompactedfiles in interface StoreFileManager

Returns:

the list of compacted files

insertNewFiles

public void insertNewFiles(Collection<HStoreFile> sfs)
                    throws IOException

Description copied from interface: StoreFileManager

Adds new files, either for from MemStore flush or bulk insert, into the structure.

Specified by:

insertNewFiles in interface StoreFileManager

Parameters:

sfs - New store files.

Throws:

IOException

clearFiles

public org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> clearFiles()

Description copied from interface: StoreFileManager

Clears all the files currently in use and returns them.

Specified by:

clearFiles in interface StoreFileManager

Returns:

The files previously in use.

clearCompactedFiles

public Collection<HStoreFile> clearCompactedFiles()

Description copied from interface: StoreFileManager

Clears all the compacted files and returns them. This method is expected to be accessed single threaded.

Specified by:

clearCompactedFiles in interface StoreFileManager

Returns:

The files compacted previously.

getStorefileCount

public final int getStorefileCount()

Description copied from interface: StoreFileManager

Returns the number of files currently in use.

Specified by:

getStorefileCount in interface StoreFileManager

Returns:

The number of files.

getCompactedFilesCount

public final int getCompactedFilesCount()

Description copied from interface: StoreFileManager

Returns the number of compacted files.

Specified by:

getCompactedFilesCount in interface StoreFileManager

Returns:

The number of files.

addCompactionResults

public void addCompactionResults(Collection<HStoreFile> newCompactedfiles,
                                 Collection<HStoreFile> results)

Description copied from interface: StoreFileManager

Adds only the new compaction results into the structure.

Specified by:

addCompactionResults in interface StoreFileManager

Parameters:

newCompactedfiles - The input files for the compaction.

results - The resulting files for the compaction.

removeCompactedFiles

public void removeCompactedFiles(Collection<HStoreFile> removedCompactedfiles)
                          throws IOException

Description copied from interface: StoreFileManager

Remove the compacted files

Specified by:

removeCompactedFiles in interface StoreFileManager

Parameters:

removedCompactedfiles - the list of compacted files

Throws:

IOException

getCandidateFilesForRowKeyBefore

public final Iterator<HStoreFile> getCandidateFilesForRowKeyBefore(KeyValue targetKey)

Description copied from interface: StoreFileManager

Gets initial, full list of candidate store files to check for row-key-before.

Specified by:

getCandidateFilesForRowKeyBefore in interface StoreFileManager

Parameters:

targetKey - The key that is the basis of the search.

Returns:

The files that may have the key less than or equal to targetKey, in reverse order of new-ness, and preference for target key.

updateCandidateFilesForRowKeyBefore

public Iterator<HStoreFile> updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
                                                                KeyValue targetKey,
                                                                Cell candidate)

Description copied from interface: StoreFileManager

Updates the candidate list for finding row key before. Based on the list of candidates remaining to check from getCandidateFilesForRowKeyBefore, targetKey and current candidate, may trim and reorder the list to remove the files where a better candidate cannot be found.

Specified by:

updateCandidateFilesForRowKeyBefore in interface StoreFileManager

Parameters:

candidateFiles - The candidate files not yet checked for better candidates - return value from StoreFileManager.getCandidateFilesForRowKeyBefore(KeyValue), with some files already removed.

targetKey - The key to search for.

candidate - The current best candidate found.

Returns:

The list to replace candidateFiles.

getSplitPoint

public final Optional<byte[]> getSplitPoint()
                                     throws IOException

Description copied from interface: StoreFileManager

Gets the split point for the split of this set of store files (approx. middle).

Specified by:

getSplitPoint in interface StoreFileManager

Returns:

The mid-point if possible.

Throws:

IOException

getFilesForScan

public final Collection<HStoreFile> getFilesForScan(byte[] startRow,
                                                    boolean includeStartRow,
                                                    byte[] stopRow,
                                                    boolean includeStopRow)

Description copied from interface: StoreFileManager

Gets the store files to scan for a Scan or Get request.

Specified by:

getFilesForScan in interface StoreFileManager

Parameters:

startRow - Start row of the request.

stopRow - Stop row of the request.

Returns:

The list of files that are to be read for this request.

getStoreCompactionPriority

public int getStoreCompactionPriority()

Specified by:

getStoreCompactionPriority in interface StoreFileManager

Returns:

The store compaction priority.

getUnneededFiles

public Collection<HStoreFile> getUnneededFiles(long maxTs,
                                               List<HStoreFile> filesCompacting)

Specified by:

getUnneededFiles in interface StoreFileManager

Parameters:

maxTs - Maximum expired timestamp.

filesCompacting - Files that are currently compacting.

Returns:

The files which don't have any necessary data according to TTL and other criteria.

getCompactionPressure

public double getCompactionPressure()

Specified by:

getCompactionPressure in interface StoreFileManager

Returns:

the compaction pressure used for compaction throughput tuning.

See Also:

HStore.getCompactionPressure()

getStoreFileComparator

public Comparator<HStoreFile> getStoreFileComparator()

Specified by:

getStoreFileComparator in interface StoreFileManager

Returns:

the comparator used to sort storefiles. Usually, the HStoreFile.getMaxSequenceId() is the first priority.