org.apache.hadoop.hbase.quotas

Class SnapshotQuotaObserverChore

java.lang.Object

org.apache.hadoop.hbase.ScheduledChore

org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore

All Implemented Interfaces:

Runnable

@InterfaceAudience.Private
public class SnapshotQuotaObserverChore
extends ScheduledChore

A Master-invoked Chore that computes the size of each snapshot which was created from a table which has a space quota.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
private Connection	conn
private org.apache.hadoop.fs.FileSystem	fs
private static org.slf4j.Logger	LOG
private MetricsMaster	metrics
(package private) static long	SNAPSHOT_QUOTA_CHORE_DELAY_DEFAULT
(package private) static String	SNAPSHOT_QUOTA_CHORE_DELAY_KEY
(package private) static int	SNAPSHOT_QUOTA_CHORE_PERIOD_DEFAULT
(package private) static String	SNAPSHOT_QUOTA_CHORE_PERIOD_KEY
(package private) static String	SNAPSHOT_QUOTA_CHORE_TIMEUNIT_DEFAULT
(package private) static String	SNAPSHOT_QUOTA_CHORE_TIMEUNIT_KEY

Constructor Summary

Constructors

Constructor and Description
SnapshotQuotaObserverChore(Connection conn, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, Stoppable stopper, MetricsMaster metrics)
SnapshotQuotaObserverChore(HMaster master, MetricsMaster metrics)

Method Summary

Modifier and Type	Method and Description
(package private) void	_chore()
protected void	chore() The task to execute on each scheduled execution of the Chore
(package private) Map<String,Long>	computeSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotsToComputeSize) Computes the size of each snapshot provided given the current files referenced by the table.
(package private) static long	getInitialDelay(org.apache.hadoop.conf.Configuration conf) Extracts the initial delay for the chore from the configuration.
(package private) FileArchiverNotifier	getNotifierForTable(TableName tn) Returns the correct instance of FileArchiverNotifier for the given table name.
(package private) static int	getPeriod(org.apache.hadoop.conf.Configuration conf) Extracts the period for the chore from the configuration.
(package private) org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String>	getSnapshotsFromTables(Admin admin, Set<TableName> tablesToFetchSnapshotsFrom) Computes a mapping of originating TableName to snapshots, when the TableName exists in the provided Set.
(package private) org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String>	getSnapshotsToComputeSize() Fetches each table with a quota (table or namespace quota), and then fetch the name of each snapshot which was created from that table.
(package private) static TimeUnit	getTimeUnit(org.apache.hadoop.conf.Configuration conf) Extracts the time unit for the chore period and initial delay from the configuration.
(package private) void	persistSnapshotSizesForNamespaces(Map<String,Long> snapshotSizesByNamespace) Writes the size used by snapshots for each namespace to the quota table.
(package private) void	pruneNamespaceSnapshots(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotsToComputeSize) Removes the snapshot entries that are present in Quota table but not in snapshotsToComputeSize
(package private) void	pruneTableSnapshots(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotsToComputeSize) Removes the snapshot entries that are present in Quota table but not in snapshotsToComputeSize
(package private) void	removeExistingNamespaceSnapshotSizes(Set<String> snapshotEntriesToRemove)
(package private) void	removeExistingSnapshotSizes(List<Delete> deletes)
(package private) void	removeExistingTableSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotEntriesToRemove)

Methods inherited from class org.apache.hadoop.hbase.ScheduledChore

cancel, cancel, choreForTesting, cleanup, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, initialChore, isInitialChoreComplete, isScheduled, run, toString, triggerNow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

SNAPSHOT_QUOTA_CHORE_PERIOD_KEY

static final String SNAPSHOT_QUOTA_CHORE_PERIOD_KEY

See Also:

Constant Field Values

SNAPSHOT_QUOTA_CHORE_PERIOD_DEFAULT

static final int SNAPSHOT_QUOTA_CHORE_PERIOD_DEFAULT

See Also:

Constant Field Values

SNAPSHOT_QUOTA_CHORE_DELAY_KEY

static final String SNAPSHOT_QUOTA_CHORE_DELAY_KEY

See Also:

Constant Field Values

SNAPSHOT_QUOTA_CHORE_DELAY_DEFAULT

static final long SNAPSHOT_QUOTA_CHORE_DELAY_DEFAULT

See Also:

Constant Field Values

SNAPSHOT_QUOTA_CHORE_TIMEUNIT_KEY

static final String SNAPSHOT_QUOTA_CHORE_TIMEUNIT_KEY

See Also:

Constant Field Values

SNAPSHOT_QUOTA_CHORE_TIMEUNIT_DEFAULT

static final String SNAPSHOT_QUOTA_CHORE_TIMEUNIT_DEFAULT

conn

private final Connection conn

conf

private final org.apache.hadoop.conf.Configuration conf

metrics

private final MetricsMaster metrics

fs

private final org.apache.hadoop.fs.FileSystem fs

Constructor Detail

SnapshotQuotaObserverChore

public SnapshotQuotaObserverChore(HMaster master,
                                  MetricsMaster metrics)

SnapshotQuotaObserverChore

SnapshotQuotaObserverChore(Connection conn,
                           org.apache.hadoop.conf.Configuration conf,
                           org.apache.hadoop.fs.FileSystem fs,
                           Stoppable stopper,
                           MetricsMaster metrics)

Method Detail

chore

protected void chore()

Description copied from class: ScheduledChore

The task to execute on each scheduled execution of the Chore

Specified by:

chore in class ScheduledChore

_chore

void _chore()
     throws IOException

Throws:

IOException

pruneTableSnapshots

void pruneTableSnapshots(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotsToComputeSize)
                  throws IOException

Removes the snapshot entries that are present in Quota table but not in snapshotsToComputeSize

Parameters:

snapshotsToComputeSize - list of snapshots to be persisted

Throws:

IOException

pruneNamespaceSnapshots

void pruneNamespaceSnapshots(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotsToComputeSize)
                      throws IOException

Removes the snapshot entries that are present in Quota table but not in snapshotsToComputeSize

Parameters:

snapshotsToComputeSize - list of snapshots to be persisted

Throws:

IOException

getSnapshotsToComputeSize

org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> getSnapshotsToComputeSize()
                                                                                                    throws IOException

Fetches each table with a quota (table or namespace quota), and then fetch the name of each snapshot which was created from that table.

Returns:

A mapping of table to snapshots created from that table

Throws:

IOException

getSnapshotsFromTables

org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> getSnapshotsFromTables(Admin admin,
                                                                                                        Set<TableName> tablesToFetchSnapshotsFrom)
                                                                                                 throws IOException

Computes a mapping of originating TableName to snapshots, when the TableName exists in the provided Set.

Throws:

IOException

computeSnapshotSizes

Map<String,Long> computeSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotsToComputeSize)
                               throws IOException

Computes the size of each snapshot provided given the current files referenced by the table.

Parameters:

snapshotsToComputeSize - The snapshots to compute the size of

Returns:

A mapping of table to snapshot created from that table and the snapshot's size.

Throws:

IOException

getNotifierForTable

FileArchiverNotifier getNotifierForTable(TableName tn)

Returns the correct instance of FileArchiverNotifier for the given table name.

Parameters:

tn - The table name

Returns:

A FileArchiverNotifier instance

persistSnapshotSizesForNamespaces

void persistSnapshotSizesForNamespaces(Map<String,Long> snapshotSizesByNamespace)
                                throws IOException

Writes the size used by snapshots for each namespace to the quota table.

Throws:

IOException

removeExistingTableSnapshotSizes

void removeExistingTableSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> snapshotEntriesToRemove)
                               throws IOException

Throws:

IOException

removeExistingNamespaceSnapshotSizes

void removeExistingNamespaceSnapshotSizes(Set<String> snapshotEntriesToRemove)
                                   throws IOException

Throws:

IOException

removeExistingSnapshotSizes

void removeExistingSnapshotSizes(List<Delete> deletes)
                          throws IOException

Throws:

IOException

getPeriod

static int getPeriod(org.apache.hadoop.conf.Configuration conf)

Extracts the period for the chore from the configuration.

Parameters:

conf - The configuration object.

Returns:

The configured chore period or the default value.

getInitialDelay

static long getInitialDelay(org.apache.hadoop.conf.Configuration conf)

Extracts the initial delay for the chore from the configuration.

Parameters:

conf - The configuration object.

Returns:

The configured chore initial delay or the default value.

getTimeUnit

static TimeUnit getTimeUnit(org.apache.hadoop.conf.Configuration conf)

Extracts the time unit for the chore period and initial delay from the configuration. The configuration value for SNAPSHOT_QUOTA_CHORE_TIMEUNIT_KEY must correspond to a TimeUnit value.

Parameters:

conf - The configuration object.

Returns:

The configured time unit for the chore period and initial delay or the default value.