org.apache.hadoop.hbase.master.janitor

Class CatalogJanitor

java.lang.Object

org.apache.hadoop.hbase.ScheduledChore

org.apache.hadoop.hbase.master.janitor.CatalogJanitor

All Implemented Interfaces:

Runnable

@InterfaceAudience.Private
public class CatalogJanitor
extends ScheduledChore

A janitor for the catalog tables. Scans the hbase:meta catalog table on a period. Makes a lastReport on state of hbase:meta. Looks for unused regions to garbage collect. Scan of hbase:meta runs if we are NOT in maintenance mode, if we are NOT shutting down, AND if the assignmentmanager is loaded. Playing it safe, we will garbage collect no-longer needed region references only if there are no regions-in-transition (RIT).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	CatalogJanitor.SplitParentFirstComparator Compare HRegionInfos in a way that has split parents sort BEFORE their daughters.

Field Summary

Fields

Modifier and Type	Field and Description
private AtomicBoolean	alreadyRunning
static int	DEFAULT_HBASE_CATALOGJANITOR_INTERVAL
private AtomicBoolean	enabled
private CatalogJanitorReport	lastReport Saved report from last hbase:meta scan to completion.
private static org.slf4j.Logger	LOG
private MasterServices	services

Constructor Summary

Constructors

Constructor and Description
CatalogJanitor(MasterServices services)

Method Summary

Modifier and Type	Method and Description
private static Pair<Boolean,Boolean>	checkDaughterInFs(MasterServices services, RegionInfo parent, RegionInfo daughter) Checks if a daughter region -- either splitA or splitB -- still holds references to parent.
private static void	checkLog4jProperties()
protected void	chore() The task to execute on each scheduled execution of the Chore
private boolean	cleanMergeRegion(RegionInfo mergedRegion, List<RegionInfo> parents) If merged region no longer holds reference to the merge regions, archive merge region on hdfs and perform deleting references in hbase:meta
(package private) static boolean	cleanParent(MasterServices services, RegionInfo parent, Result rowContent)
private boolean	cleanParent(RegionInfo parent, Result rowContent) If daughters no longer hold reference to the parents, delete the parent.
private TableDescriptor	getDescriptor(TableName tableName)
boolean	getEnabled()
CatalogJanitorReport	getLastReport() Returns Returns last published Report that comes of last successful scan of hbase:meta.
private static boolean	hasNoReferences(Pair<Boolean,Boolean> p)
protected boolean	initialChore() Override to run a task before we start looping.
private static boolean	isMetaLoaded(AssignmentManager am)
private static boolean	isRIT(AssignmentManager am)
static void	main(String[] args) For testing against a cluster.
int	scan() Run janitorial scan of catalog hbase:meta table looking for garbage to collect.
protected CatalogJanitorReport	scanForReport() Scan hbase:meta.
boolean	setEnabled(boolean enabled)
private void	updateAssignmentManagerMetrics()

Methods inherited from class org.apache.hadoop.hbase.ScheduledChore

cancel, cancel, choreForTesting, cleanup, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, isInitialChoreComplete, isScheduled, run, shutdown, shutdown, toString, triggerNow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_HBASE_CATALOGJANITOR_INTERVAL

public static final int DEFAULT_HBASE_CATALOGJANITOR_INTERVAL

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

alreadyRunning

private final AtomicBoolean alreadyRunning

enabled

private final AtomicBoolean enabled

services

private final MasterServices services

lastReport

private volatile CatalogJanitorReport lastReport

Saved report from last hbase:meta scan to completion. May be stale if having trouble completing scan. Check its date.

Constructor Detail

CatalogJanitor

public CatalogJanitor(MasterServices services)

Method Detail

initialChore

protected boolean initialChore()

Description copied from class: ScheduledChore

Override to run a task before we start looping.

Overrides:

initialChore in class ScheduledChore

Returns:

true if initial chore was successful

setEnabled

public boolean setEnabled(boolean enabled)

getEnabled

public boolean getEnabled()

chore

protected void chore()

Description copied from class: ScheduledChore

The task to execute on each scheduled execution of the Chore

Specified by:

chore in class ScheduledChore

isMetaLoaded

private static boolean isMetaLoaded(AssignmentManager am)

isRIT

private static boolean isRIT(AssignmentManager am)

scan

public int scan()
         throws IOException

Run janitorial scan of catalog hbase:meta table looking for garbage to collect.

Returns:

How many items gc'd whether for merge or split. Returns -1 if previous scan is in progress.

Throws:

IOException

scanForReport

protected CatalogJanitorReport scanForReport()
                                      throws IOException

Scan hbase:meta.

Returns:

Return generated CatalogJanitorReport

Throws:

IOException

getLastReport

public CatalogJanitorReport getLastReport()

Returns Returns last published Report that comes of last successful scan of hbase:meta.

cleanMergeRegion

private boolean cleanMergeRegion(RegionInfo mergedRegion,
                                 List<RegionInfo> parents)
                          throws IOException

If merged region no longer holds reference to the merge regions, archive merge region on hdfs and perform deleting references in hbase:meta

Returns:

true if we delete references in merged region on hbase:meta and archive the files on the file system

Throws:

IOException

cleanParent

static boolean cleanParent(MasterServices services,
                           RegionInfo parent,
                           Result rowContent)
                    throws IOException

Throws:

IOException

cleanParent

private boolean cleanParent(RegionInfo parent,
                            Result rowContent)
                     throws IOException

If daughters no longer hold reference to the parents, delete the parent.

Parameters:

parent - RegionInfo of split offlined parent

rowContent - Content of parent row in metaRegionName

Returns:

True if we removed parent from meta table and from the filesystem.

Throws:

IOException

hasNoReferences

private static boolean hasNoReferences(Pair<Boolean,Boolean> p)

Parameters:

p - A pair where the first boolean says whether or not the daughter region directory exists in the filesystem and then the second boolean says whether the daughter has references to the parent.

Returns:

True the passed p signifies no references.

checkDaughterInFs

private static Pair<Boolean,Boolean> checkDaughterInFs(MasterServices services,
                                                       RegionInfo parent,
                                                       RegionInfo daughter)
                                                throws IOException

Checks if a daughter region -- either splitA or splitB -- still holds references to parent.

Parameters:

parent - Parent region

daughter - Daughter region

Returns:

A pair where the first boolean says whether or not the daughter region directory exists in the filesystem and then the second boolean says whether the daughter has references to the parent.

Throws:

IOException

getDescriptor

private TableDescriptor getDescriptor(TableName tableName)
                               throws IOException

Throws:

IOException

updateAssignmentManagerMetrics

private void updateAssignmentManagerMetrics()

checkLog4jProperties

private static void checkLog4jProperties()

main

public static void main(String[] args)
                 throws IOException

For testing against a cluster. Doesn't have a MasterServices context so does not report on good vs bad servers.

Throws:

IOException