org.apache.hadoop.hbase.master

Class CatalogJanitor

java.lang.Object

org.apache.hadoop.hbase.ScheduledChore

org.apache.hadoop.hbase.master.CatalogJanitor

All Implemented Interfaces:

Runnable

@InterfaceAudience.Private
public class CatalogJanitor
extends ScheduledChore

A janitor for the catalog tables. Scans the hbase:meta catalog table on a period. Makes a lastReport on state of hbase:meta. Looks for unused regions to garbage collect. Scan of hbase:meta runs if we are NOT in maintenance mode, if we are NOT shutting down, AND if the assignmentmanager is loaded. Playing it safe, we will garbage collect no-longer needed region references only if there are no regions-in-transition (RIT).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CatalogJanitor.Report Report made by ReportMakingVisitor
(package private) static class	CatalogJanitor.ReportMakingVisitor Visitor we use in here in CatalogJanitor to go against hbase:meta table.
(package private) static class	CatalogJanitor.SplitParentFirstComparator Compare HRegionInfos in a way that has split parents sort BEFORE their daughters.

Field Summary

Fields

Modifier and Type	Field and Description
private AtomicBoolean	alreadyRunning
private AtomicBoolean	enabled
private CatalogJanitor.Report	lastReport Saved report from last hbase:meta scan to completion.
private static org.slf4j.Logger	LOG
private MasterServices	services

Constructor Summary

Constructors

Constructor and Description
CatalogJanitor(MasterServices services)

Method Summary

Modifier and Type	Method and Description
private Pair<Boolean,Boolean>	checkDaughterInFs(RegionInfo parent, RegionInfo daughter) Checks if a daughter region -- either splitA or splitB -- still holds references to parent.
private static void	checkLog4jProperties()
protected void	chore() The task to execute on each scheduled execution of the Chore
boolean	cleanMergeQualifier(RegionInfo region) Checks if the specified region has merge qualifiers, if so, try to clean them.
private boolean	cleanMergeRegion(RegionInfo mergedRegion, List<RegionInfo> parents) If merged region no longer holds reference to the merge regions, archive merge region on hdfs and perform deleting references in hbase:meta
(package private) boolean	cleanParent(RegionInfo parent, Result rowContent) If daughters no longer hold reference to the parents, delete the parent.
private TableDescriptor	getDescriptor(TableName tableName)
(package private) boolean	getEnabled()
CatalogJanitor.Report	getLastReport()
private boolean	hasNoReferences(Pair<Boolean,Boolean> p)
protected boolean	initialChore() Override to run a task before we start looping.
private static boolean	isMetaLoaded(AssignmentManager am)
private static boolean	isRIT(AssignmentManager am)
static void	main(String[] args) For testing against a cluster.
(package private) int	scan() Run janitorial scan of catalog hbase:meta table looking for garbage to collect.
(package private) CatalogJanitor.Report	scanForReport() Scan hbase:meta.
(package private) boolean	setEnabled(boolean enabled)

Methods inherited from class org.apache.hadoop.hbase.ScheduledChore

cancel, cancel, choreForTesting, cleanup, getInitialDelay, getName, getPeriod, getStopper, getTimeUnit, isInitialChoreComplete, isScheduled, run, toString, triggerNow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

alreadyRunning

private final AtomicBoolean alreadyRunning

enabled

private final AtomicBoolean enabled

services

private final MasterServices services

lastReport

private volatile CatalogJanitor.Report lastReport

Saved report from last hbase:meta scan to completion. May be stale if having trouble completing scan. Check its date.

Constructor Detail

CatalogJanitor

CatalogJanitor(MasterServices services)

Method Detail

initialChore

protected boolean initialChore()

Description copied from class: ScheduledChore

Override to run a task before we start looping.

Overrides:

initialChore in class ScheduledChore

Returns:

true if initial chore was successful

setEnabled

boolean setEnabled(boolean enabled)

getEnabled

boolean getEnabled()

chore

protected void chore()

Description copied from class: ScheduledChore

The task to execute on each scheduled execution of the Chore

Specified by:

chore in class ScheduledChore

isMetaLoaded

private static boolean isMetaLoaded(AssignmentManager am)

isRIT

private static boolean isRIT(AssignmentManager am)

scan

int scan()
  throws IOException

Run janitorial scan of catalog hbase:meta table looking for garbage to collect.

Returns:

How many items gc'd whether for merge or split.

Throws:

IOException

scanForReport

CatalogJanitor.Report scanForReport()
                             throws IOException

Scan hbase:meta.

Returns:

Return generated CatalogJanitor.Report

Throws:

IOException

getLastReport

public CatalogJanitor.Report getLastReport()

Returns:

Returns last published Report that comes of last successful scan of hbase:meta.

cleanMergeRegion

private boolean cleanMergeRegion(RegionInfo mergedRegion,
                                 List<RegionInfo> parents)
                          throws IOException

If merged region no longer holds reference to the merge regions, archive merge region on hdfs and perform deleting references in hbase:meta

Returns:

true if we delete references in merged region on hbase:meta and archive the files on the file system

Throws:

IOException

cleanParent

boolean cleanParent(RegionInfo parent,
                    Result rowContent)
             throws IOException

If daughters no longer hold reference to the parents, delete the parent.

Parameters:

parent - RegionInfo of split offlined parent

rowContent - Content of parent row in metaRegionName

Returns:

True if we removed parent from meta table and from the filesystem.

Throws:

IOException

hasNoReferences

private boolean hasNoReferences(Pair<Boolean,Boolean> p)

Parameters:

p - A pair where the first boolean says whether or not the daughter region directory exists in the filesystem and then the second boolean says whether the daughter has references to the parent.

Returns:

True the passed p signifies no references.

checkDaughterInFs

private Pair<Boolean,Boolean> checkDaughterInFs(RegionInfo parent,
                                                RegionInfo daughter)
                                         throws IOException

Checks if a daughter region -- either splitA or splitB -- still holds references to parent.

Parameters:

parent - Parent region

daughter - Daughter region

Returns:

A pair where the first boolean says whether or not the daughter region directory exists in the filesystem and then the second boolean says whether the daughter has references to the parent.

Throws:

IOException

getDescriptor

private TableDescriptor getDescriptor(TableName tableName)
                               throws IOException

Throws:

IOException

cleanMergeQualifier

public boolean cleanMergeQualifier(RegionInfo region)
                            throws IOException

Checks if the specified region has merge qualifiers, if so, try to clean them.

Returns:

true if no info:merge* columns; i.e. the specified region doesn't have any merge qualifiers.

Throws:

IOException

checkLog4jProperties

private static void checkLog4jProperties()

main

public static void main(String[] args)
                 throws IOException

For testing against a cluster. Doesn't have a MasterServices context so does not report on good vs bad servers.

Throws:

IOException