Package org.apache.hadoop.hbase.regionserver

Class DataTieringManager

java.lang.Object

org.apache.hadoop.hbase.regionserver.DataTieringManager

@Private
public class DataTieringManager
extends Object

The DataTieringManager class categorizes data into hot data and cold data based on the specified DataTieringType when DataTiering is enabled. DataTiering is disabled by default with DataTieringType set to DataTieringType.NONE. The DataTieringType determines the logic for distinguishing data into hot or cold. By default, all data is considered as hot.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DATATIERING_HOT_DATA_AGE_KEY
static final String	DATATIERING_KEY
static final DataTieringType	DEFAULT_DATATIERING
static final long	DEFAULT_DATATIERING_HOT_DATA_AGE
static final boolean	DEFAULT_GLOBAL_DATA_TIERING_ENABLED
static final String	GLOBAL_DATA_TIERING_ENABLED_KEY
private static DataTieringManager	instance
private static final org.slf4j.Logger	LOG
private final Map<String,HRegion>	onlineRegions

Constructor Summary

Constructors

Modifier	Constructor	Description
private	DataTieringManager(Map<String,HRegion> onlineRegions)

Method Summary

Modifier and Type	Method	Description
Set<String>	getColdDataFiles(Set<BlockCacheKey> allCachedBlocks)	Returns a set of cold data filenames from the given set of cached blocks.
Map<String,String>	getColdFilesList()
private org.apache.hadoop.conf.Configuration	getConfiguration(org.apache.hadoop.fs.Path hFilePath)
private long	getCurrentTimestamp()
private long	getDataTieringHotDataAge(org.apache.hadoop.conf.Configuration conf)
private DataTieringType	getDataTieringType(org.apache.hadoop.conf.Configuration conf)
private HRegion	getHRegion(org.apache.hadoop.fs.Path hFilePath)
private HStore	getHStore(org.apache.hadoop.fs.Path hFilePath)
private HStoreFile	getHStoreFile(org.apache.hadoop.fs.Path hFilePath)
static DataTieringManager	getInstance()	Retrieves the instance of DataTieringManager.
private boolean	hotDataValidator(long maxTimestamp, long hotDataAge)
static boolean	instantiate(org.apache.hadoop.conf.Configuration conf, Map<String,HRegion> onlineRegions)	Initializes the DataTieringManager instance with the provided map of online regions, only if the configuration "hbase.regionserver.datatiering.enable" is enabled.
boolean	isDataTieringEnabled(org.apache.hadoop.fs.Path hFilePath)	Determines whether data tiering is enabled for the given HFile path.
boolean	isDataTieringEnabled(BlockCacheKey key)	Determines whether data tiering is enabled for the given block cache key.
private static boolean	isDataTieringFeatureEnabled(org.apache.hadoop.conf.Configuration conf)
boolean	isHotData(long maxTimestamp, org.apache.hadoop.conf.Configuration conf)	Determines whether the data associated with the given time range tracker is considered hot.
boolean	isHotData(org.apache.hadoop.fs.Path hFilePath)	Determines whether the data in the HFile at the given path is considered hot based on the configured data tiering type and hot data age.
boolean	isHotData(BlockCacheKey key)	Determines whether the data associated with the given block cache key is considered hot.
boolean	isHotData(HFileInfo hFileInfo, org.apache.hadoop.conf.Configuration configuration)	Determines whether the data in the HFile being read is considered hot based on the configured data tiering type and hot data age.
static void	resetForTestingOnly()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

GLOBAL_DATA_TIERING_ENABLED_KEY

public static final String GLOBAL_DATA_TIERING_ENABLED_KEY

See Also:

• Constant Field Values

DEFAULT_GLOBAL_DATA_TIERING_ENABLED

public static final boolean DEFAULT_GLOBAL_DATA_TIERING_ENABLED

See Also:

• Constant Field Values

DATATIERING_KEY

public static final String DATATIERING_KEY

See Also:

• Constant Field Values

DATATIERING_HOT_DATA_AGE_KEY

public static final String DATATIERING_HOT_DATA_AGE_KEY

See Also:

• Constant Field Values

DEFAULT_DATATIERING

public static final DataTieringType DEFAULT_DATATIERING

DEFAULT_DATATIERING_HOT_DATA_AGE

public static final long DEFAULT_DATATIERING_HOT_DATA_AGE

See Also:

• Constant Field Values

instance

private static DataTieringManager instance

onlineRegions

private final Map<String,HRegion> onlineRegions

Constructor Details

DataTieringManager

private DataTieringManager(Map<String,HRegion> onlineRegions)

Method Details

instantiate

public static boolean instantiate(org.apache.hadoop.conf.Configuration conf,
 Map<String,HRegion> onlineRegions)

Initializes the DataTieringManager instance with the provided map of online regions, only if the configuration "hbase.regionserver.datatiering.enable" is enabled.

Parameters:

conf - Configuration object.

onlineRegions - A map containing online regions.

Returns:

True if the instance is instantiated successfully, false otherwise.

getInstance

public static DataTieringManager getInstance()

Retrieves the instance of DataTieringManager.

Returns:

The instance of DataTieringManager, if instantiated, null otherwise.

isDataTieringEnabled

public boolean isDataTieringEnabled(BlockCacheKey key)
                             throws DataTieringException

Determines whether data tiering is enabled for the given block cache key.

Parameters:

key - the block cache key

Returns:

true if data tiering is enabled for the HFile associated with the key, false otherwise

Throws:

DataTieringException - if there is an error retrieving the HFile path or configuration

isDataTieringEnabled

public boolean isDataTieringEnabled(org.apache.hadoop.fs.Path hFilePath)
                             throws DataTieringException

Determines whether data tiering is enabled for the given HFile path.

Parameters:

hFilePath - the path to the HFile

Returns:

true if data tiering is enabled, false otherwise

Throws:

DataTieringException - if there is an error retrieving the configuration

isHotData

public boolean isHotData(BlockCacheKey key)
                  throws DataTieringException

Determines whether the data associated with the given block cache key is considered hot. If the data tiering type is set to DataTieringType.TIME_RANGE and maximum timestamp is not present, it considers Long.MAX_VALUE as the maximum timestamp, making the data hot by default.

Parameters:

key - the block cache key

Returns:

true if the data is hot, false otherwise

Throws:

DataTieringException - if there is an error retrieving data tiering information

isHotData

public boolean isHotData(long maxTimestamp,
 org.apache.hadoop.conf.Configuration conf)

Determines whether the data associated with the given time range tracker is considered hot. If the data tiering type is set to DataTieringType.TIME_RANGE, it uses the maximum timestamp from the time range tracker to determine if the data is hot. Otherwise, it considers the data as hot by default.

Parameters:

maxTimestamp - the maximum timestamp associated with the data.

conf - The configuration object to use for determining hot data criteria.

Returns:

true if the data is hot, false otherwise

isHotData

public boolean isHotData(org.apache.hadoop.fs.Path hFilePath)
                  throws DataTieringException

Determines whether the data in the HFile at the given path is considered hot based on the configured data tiering type and hot data age. If the data tiering type is set to DataTieringType.TIME_RANGE and maximum timestamp is not present, it considers Long.MAX_VALUE as the maximum timestamp, making the data hot by default.

Parameters:

hFilePath - the path to the HFile

Returns:

true if the data is hot, false otherwise

Throws:

DataTieringException - if there is an error retrieving data tiering information

isHotData

public boolean isHotData(HFileInfo hFileInfo,
 org.apache.hadoop.conf.Configuration configuration)

Determines whether the data in the HFile being read is considered hot based on the configured data tiering type and hot data age. If the data tiering type is set to DataTieringType.TIME_RANGE and maximum timestamp is not present, it considers Long.MAX_VALUE as the maximum timestamp, making the data hot by default.

Parameters:

hFileInfo - Information about the HFile to determine if its data is hot.

configuration - The configuration object to use for determining hot data criteria.

Returns:

true if the data is hot, false otherwise

hotDataValidator

private boolean hotDataValidator(long maxTimestamp,
 long hotDataAge)

getCurrentTimestamp

private long getCurrentTimestamp()

getColdDataFiles

public Set<String> getColdDataFiles(Set<BlockCacheKey> allCachedBlocks)
                             throws DataTieringException

Returns a set of cold data filenames from the given set of cached blocks. Cold data is determined by the configured data tiering type and hot data age.

Parameters:

allCachedBlocks - a set of all cached block cache keys

Returns:

a set of cold data filenames

Throws:

DataTieringException - if there is an error determining whether a block is hot

getHRegion

private HRegion getHRegion(org.apache.hadoop.fs.Path hFilePath)
                    throws DataTieringException

Throws:

DataTieringException

getHStore

private HStore getHStore(org.apache.hadoop.fs.Path hFilePath)
                  throws DataTieringException

Throws:

DataTieringException

getHStoreFile

private HStoreFile getHStoreFile(org.apache.hadoop.fs.Path hFilePath)
                          throws DataTieringException

Throws:

DataTieringException

getConfiguration

private org.apache.hadoop.conf.Configuration getConfiguration(org.apache.hadoop.fs.Path hFilePath)
                                                       throws DataTieringException

Throws:

DataTieringException

getDataTieringType

private DataTieringType getDataTieringType(org.apache.hadoop.conf.Configuration conf)

getDataTieringHotDataAge

private long getDataTieringHotDataAge(org.apache.hadoop.conf.Configuration conf)

getColdFilesList

public Map<String,String> getColdFilesList()

isDataTieringFeatureEnabled

private static boolean isDataTieringFeatureEnabled(org.apache.hadoop.conf.Configuration conf)

resetForTestingOnly

public static void resetForTestingOnly()