org.apache.hadoop.hbase.regionserver

Interface StoreFileManager

All Known Implementing Classes:

DefaultStoreFileManager, StripeStoreFileManager

@InterfaceAudience.Private
public interface StoreFileManager

Manages the store files and basic metadata about that that determines the logical structure (e.g. what files to return for scan, how to determine split point, and such). Does NOT affect the physical structure of files in HDFS. Example alternative structures - the default list of files by seqNum; levelDB one sorted by level and seqNum.

Notice that, all the states are only in memory, we do not persist anything here. The only place where we throw an IOException is the getSplitPoint() method, where we need to read startKey, endKey etc, which may lead to an IOException.

Implementations are assumed to be not thread safe.

Method Summary

Modifier and Type	Method and Description
void	addCompactionResults(Collection<HStoreFile> compactedFiles, Collection<HStoreFile> results) Adds only the new compaction results into the structure.
Collection<HStoreFile>	clearCompactedFiles() Clears all the compacted files and returns them.
org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile>	clearFiles() Clears all the files currently in use and returns them.
Iterator<HStoreFile>	getCandidateFilesForRowKeyBefore(KeyValue targetKey) Gets initial, full list of candidate store files to check for row-key-before.
Collection<HStoreFile>	getCompactedfiles() List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction.
int	getCompactedFilesCount() Returns the number of compacted files.
double	getCompactionPressure()
Collection<HStoreFile>	getFilesForScan(byte[] startRow, boolean includeStartRow, byte[] stopRow, boolean includeStopRow) Gets the store files to scan for a Scan or Get request.
Optional<byte[]>	getSplitPoint() Gets the split point for the split of this set of store files (approx.
int	getStoreCompactionPriority() Returns The store compaction priority.
Comparator<HStoreFile>	getStoreFileComparator()
int	getStorefileCount() Returns the number of files currently in use.
Collection<HStoreFile>	getStorefiles() Gets the snapshot of the store files currently in use.
Collection<HStoreFile>	getUnneededFiles(long maxTs, List<HStoreFile> filesCompacting)
void	insertNewFiles(Collection<HStoreFile> sfs) Adds new files, either for from MemStore flush or bulk insert, into the structure.
void	loadFiles(List<HStoreFile> storeFiles) Loads the initial store files into empty StoreFileManager.
void	removeCompactedFiles(Collection<HStoreFile> compactedFiles) Remove the compacted files
Iterator<HStoreFile>	updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles, KeyValue targetKey, Cell candidate) Updates the candidate list for finding row key before.

Method Detail

loadFiles

void loadFiles(List<HStoreFile> storeFiles)

Loads the initial store files into empty StoreFileManager.

Parameters:

storeFiles - The files to load.

insertNewFiles

void insertNewFiles(Collection<HStoreFile> sfs)

Adds new files, either for from MemStore flush or bulk insert, into the structure.

Parameters:

sfs - New store files.

addCompactionResults

void addCompactionResults(Collection<HStoreFile> compactedFiles,
                          Collection<HStoreFile> results)

Adds only the new compaction results into the structure.

Parameters:

compactedFiles - The input files for the compaction.

results - The resulting files for the compaction.

removeCompactedFiles

void removeCompactedFiles(Collection<HStoreFile> compactedFiles)

Remove the compacted files

Parameters:

compactedFiles - the list of compacted files

clearFiles

org.apache.hbase.thirdparty.com.google.common.collect.ImmutableCollection<HStoreFile> clearFiles()

Clears all the files currently in use and returns them.

Returns:

The files previously in use.

clearCompactedFiles

Collection<HStoreFile> clearCompactedFiles()

Clears all the compacted files and returns them. This method is expected to be accessed single threaded.

Returns:

The files compacted previously.

getStorefiles

Collection<HStoreFile> getStorefiles()

Gets the snapshot of the store files currently in use. Can be used for things like metrics and checks; should not assume anything about relations between store files in the list.

Returns:

The list of StoreFiles.

getCompactedfiles

Collection<HStoreFile> getCompactedfiles()

List of compacted files inside this store that needs to be excluded in reads because further new reads will be using only the newly created files out of compaction. These compacted files will be deleted/cleared once all the existing readers on these compacted files are done.

Returns:

the list of compacted files

getStorefileCount

int getStorefileCount()

Returns the number of files currently in use.

Returns:

The number of files.

getCompactedFilesCount

int getCompactedFilesCount()

Returns the number of compacted files.

Returns:

The number of files.

getFilesForScan

Collection<HStoreFile> getFilesForScan(byte[] startRow,
                                       boolean includeStartRow,
                                       byte[] stopRow,
                                       boolean includeStopRow)

Gets the store files to scan for a Scan or Get request.

Parameters:

startRow - Start row of the request.

stopRow - Stop row of the request.

Returns:

The list of files that are to be read for this request.

getCandidateFilesForRowKeyBefore

Iterator<HStoreFile> getCandidateFilesForRowKeyBefore(KeyValue targetKey)

Gets initial, full list of candidate store files to check for row-key-before.

Parameters:

targetKey - The key that is the basis of the search.

Returns:

The files that may have the key less than or equal to targetKey, in reverse order of new-ness, and preference for target key.

updateCandidateFilesForRowKeyBefore

Iterator<HStoreFile> updateCandidateFilesForRowKeyBefore(Iterator<HStoreFile> candidateFiles,
                                                         KeyValue targetKey,
                                                         Cell candidate)

Updates the candidate list for finding row key before. Based on the list of candidates remaining to check from getCandidateFilesForRowKeyBefore, targetKey and current candidate, may trim and reorder the list to remove the files where a better candidate cannot be found.

Parameters:

candidateFiles - The candidate files not yet checked for better candidates - return value from getCandidateFilesForRowKeyBefore(KeyValue), with some files already removed.

targetKey - The key to search for.

candidate - The current best candidate found.

Returns:

The list to replace candidateFiles.

getSplitPoint

Optional<byte[]> getSplitPoint()
                        throws IOException

Gets the split point for the split of this set of store files (approx. middle).

Returns:

The mid-point if possible.

Throws:

IOException

getStoreCompactionPriority

int getStoreCompactionPriority()

Returns The store compaction priority.

getUnneededFiles

Collection<HStoreFile> getUnneededFiles(long maxTs,
                                        List<HStoreFile> filesCompacting)

Parameters:

maxTs - Maximum expired timestamp.

filesCompacting - Files that are currently compacting.

Returns:

The files which don't have any necessary data according to TTL and other criteria.

getCompactionPressure

double getCompactionPressure()

Returns:

the compaction pressure used for compaction throughput tuning.

See Also:

HStore.getCompactionPressure()

getStoreFileComparator

Comparator<HStoreFile> getStoreFileComparator()

Returns:

the comparator used to sort storefiles. Usually, the HStoreFile.getMaxSequenceId() is the first priority.