Package org.apache.hadoop.hbase.quotas
Class SnapshotQuotaObserverChore
java.lang.Object
org.apache.hadoop.hbase.ScheduledChore
org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore
- All Implemented Interfaces:
Runnable
A Master-invoked
Chore
that computes the size of each snapshot which was created from a
table which has a space quota.-
Field Summary
Modifier and TypeFieldDescriptionprivate final org.apache.hadoop.conf.Configuration
private final Connection
private final org.apache.hadoop.fs.FileSystem
private static final org.slf4j.Logger
private final MetricsMaster
(package private) static final long
(package private) static final String
(package private) static final int
(package private) static final String
(package private) static final String
(package private) static final String
-
Constructor Summary
ConstructorDescriptionSnapshotQuotaObserverChore
(Connection conn, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, Stoppable stopper, MetricsMaster metrics) SnapshotQuotaObserverChore
(HMaster master, MetricsMaster metrics) -
Method Summary
Modifier and TypeMethodDescription(package private) void
_chore()
protected void
chore()
The task to execute on each scheduled execution of the ChorecomputeSnapshotSizes
(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotsToComputeSize) Computes the size of each snapshot provided given the current files referenced by the table.(package private) static long
getInitialDelay
(org.apache.hadoop.conf.Configuration conf) Extracts the initial delay for the chore from the configuration.(package private) FileArchiverNotifier
Returns the correct instance ofFileArchiverNotifier
for the given table name.(package private) static int
getPeriod
(org.apache.hadoop.conf.Configuration conf) Extracts the period for the chore from the configuration.getSnapshotsFromTables
(Admin admin, Set<TableName> tablesToFetchSnapshotsFrom) Computes a mapping of originatingTableName
to snapshots, when theTableName
exists in the providedSet
.Fetches each table with a quota (table or namespace quota), and then fetch the name of each snapshot which was created from that table.(package private) static TimeUnit
getTimeUnit
(org.apache.hadoop.conf.Configuration conf) Extracts the time unit for the chore period and initial delay from the configuration.(package private) void
persistSnapshotSizesForNamespaces
(Map<String, Long> snapshotSizesByNamespace) Writes the size used by snapshots for each namespace to the quota table.(package private) void
pruneNamespaceSnapshots
(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotsToComputeSize) Removes the snapshot entries that are present in Quota table but not in snapshotsToComputeSize(package private) void
pruneTableSnapshots
(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotsToComputeSize) Removes the snapshot entries that are present in Quota table but not in snapshotsToComputeSize(package private) void
removeExistingNamespaceSnapshotSizes
(Set<String> snapshotEntriesToRemove) (package private) void
removeExistingSnapshotSizes
(List<Delete> deletes) (package private) void
removeExistingTableSnapshotSizes
(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotEntriesToRemove) Methods inherited from class org.apache.hadoop.hbase.ScheduledChore
cancel, cancel, choreForTesting, cleanup, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, initialChore, isInitialChoreComplete, isScheduled, run, shutdown, shutdown, toString, triggerNow
-
Field Details
-
LOG
-
SNAPSHOT_QUOTA_CHORE_PERIOD_KEY
- See Also:
-
SNAPSHOT_QUOTA_CHORE_PERIOD_DEFAULT
- See Also:
-
SNAPSHOT_QUOTA_CHORE_DELAY_KEY
- See Also:
-
SNAPSHOT_QUOTA_CHORE_DELAY_DEFAULT
- See Also:
-
SNAPSHOT_QUOTA_CHORE_TIMEUNIT_KEY
- See Also:
-
SNAPSHOT_QUOTA_CHORE_TIMEUNIT_DEFAULT
-
conn
-
conf
-
metrics
-
fs
-
-
Constructor Details
-
SnapshotQuotaObserverChore
-
SnapshotQuotaObserverChore
SnapshotQuotaObserverChore(Connection conn, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, Stoppable stopper, MetricsMaster metrics)
-
-
Method Details
-
chore
Description copied from class:ScheduledChore
The task to execute on each scheduled execution of the Chore- Specified by:
chore
in classScheduledChore
-
_chore
- Throws:
IOException
-
pruneTableSnapshots
void pruneTableSnapshots(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotsToComputeSize) throws IOExceptionRemoves the snapshot entries that are present in Quota table but not in snapshotsToComputeSize- Parameters:
snapshotsToComputeSize
- list of snapshots to be persisted- Throws:
IOException
-
pruneNamespaceSnapshots
void pruneNamespaceSnapshots(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotsToComputeSize) throws IOExceptionRemoves the snapshot entries that are present in Quota table but not in snapshotsToComputeSize- Parameters:
snapshotsToComputeSize
- list of snapshots to be persisted- Throws:
IOException
-
getSnapshotsToComputeSize
org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> getSnapshotsToComputeSize() throws IOExceptionFetches each table with a quota (table or namespace quota), and then fetch the name of each snapshot which was created from that table.- Returns:
- A mapping of table to snapshots created from that table
- Throws:
IOException
-
getSnapshotsFromTables
org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> getSnapshotsFromTables(Admin admin, Set<TableName> tablesToFetchSnapshotsFrom) throws IOException Computes a mapping of originatingTableName
to snapshots, when theTableName
exists in the providedSet
.- Throws:
IOException
-
computeSnapshotSizes
Map<String,Long> computeSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotsToComputeSize) throws IOExceptionComputes the size of each snapshot provided given the current files referenced by the table.- Parameters:
snapshotsToComputeSize
- The snapshots to compute the size of- Returns:
- A mapping of table to snapshot created from that table and the snapshot's size.
- Throws:
IOException
-
getNotifierForTable
Returns the correct instance ofFileArchiverNotifier
for the given table name.- Parameters:
tn
- The table name- Returns:
- A
FileArchiverNotifier
instance
-
persistSnapshotSizesForNamespaces
void persistSnapshotSizesForNamespaces(Map<String, Long> snapshotSizesByNamespace) throws IOExceptionWrites the size used by snapshots for each namespace to the quota table.- Throws:
IOException
-
removeExistingTableSnapshotSizes
void removeExistingTableSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotEntriesToRemove) throws IOException- Throws:
IOException
-
removeExistingNamespaceSnapshotSizes
- Throws:
IOException
-
removeExistingSnapshotSizes
- Throws:
IOException
-
getPeriod
Extracts the period for the chore from the configuration.- Parameters:
conf
- The configuration object.- Returns:
- The configured chore period or the default value.
-
getInitialDelay
Extracts the initial delay for the chore from the configuration.- Parameters:
conf
- The configuration object.- Returns:
- The configured chore initial delay or the default value.
-
getTimeUnit
Extracts the time unit for the chore period and initial delay from the configuration. The configuration value forSNAPSHOT_QUOTA_CHORE_TIMEUNIT_KEY
must correspond to aTimeUnit
value.- Parameters:
conf
- The configuration object.- Returns:
- The configured time unit for the chore period and initial delay or the default value.
-