Package org.apache.hadoop.hbase.tool

Class CanaryTool

java.lang.Object

org.apache.hadoop.hbase.tool.CanaryTool

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, Canary, org.apache.hadoop.util.Tool

@LimitedPrivate("Tools")
public class CanaryTool
extends Object
implements org.apache.hadoop.util.Tool, Canary

HBase Canary Tool for "canary monitoring" of a running HBase cluster. There are three modes:

1. region mode (Default): For each region, try to get one row per column family outputting information on failure (ERROR) or else the latency.
2. regionserver mode: For each regionserver try to get one row from one table selected randomly outputting information on failure (ERROR) or else the latency.
3. zookeeper mode: for each zookeeper instance, selects a znode outputting information on failure (ERROR) or else the latency.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CanaryTool.Monitor	A Monitor super-class can be extended by users
private static class	CanaryTool.RegionMonitor	A monitor for region mode.
private static class	CanaryTool.RegionServerMonitor	A monitor for regionserver mode
static class	CanaryTool.RegionServerStdOutSink	By RegionServer, for 'regionserver' mode.
(package private) static class	CanaryTool.RegionServerTask	Run a single RegionServer Task and then exit.
static class	CanaryTool.RegionStdOutSink	By Region, for 'region' mode.
(package private) static class	CanaryTool.RegionTask	Run a single Region Task and then exit.
static class	CanaryTool.RegionTaskResult	Canary region mode-specific data structure which stores information about each region to be scanned
static interface	CanaryTool.Sink	Sink interface used by the canary to output information
static class	CanaryTool.StdOutSink	Simple implementation of canary sink that allows plotting to a file or standard output.
private static class	CanaryTool.ZookeeperMonitor
static class	CanaryTool.ZookeeperStdOutSink	Output for 'zookeeper' mode.
(package private) static class	CanaryTool.ZookeeperTask	Run a single zookeeper Task and then exit.

Field Summary

Fields

Modifier and Type	Field	Description
private static final String	CANARY_TABLE_FAMILY_NAME
private org.apache.hadoop.conf.Configuration	conf
private HashMap<String,Long>	configuredReadTableTimeouts	This is a Map of table to timeout.
private static final long	DEFAULT_INTERVAL
private static final long	DEFAULT_TIMEOUT
static final TableName	DEFAULT_WRITE_TABLE_NAME
private static final int	ERROR_EXIT_CODE
private ExecutorService	executor
private static final int	FAILURE_EXIT_CODE
static final String	HBASE_CANARY_FAIL_ON_ERROR
static final String	HBASE_CANARY_INFO_BINDADDRESS
static final String	HBASE_CANARY_INFO_PORT
static final String	HBASE_CANARY_REGION_READ_TABLE_TIMEOUT
static final String	HBASE_CANARY_REGION_WRITE_SNIFFING
static final String	HBASE_CANARY_REGION_WRITE_TABLE_NAME
static final String	HBASE_CANARY_REGION_WRITE_TABLE_TIMEOUT
static final String	HBASE_CANARY_REGIONSERVER_ALL_REGIONS
static final String	HBASE_CANARY_TIMEOUT
static final String	HBASE_CANARY_USE_REGEX
static final String	HBASE_CANARY_ZOOKEEPER_PERMITTED_FAILURES
private static final int	INIT_ERROR_EXIT_CODE
private long	interval
private static final org.slf4j.Logger	LOG
private static final int	MAX_THREADS_NUM
private boolean	regionServerMode	True if we are to run in 'regionServer' mode.
private CanaryTool.Sink	sink
private static final int	TIMEOUT_ERROR_EXIT_CODE
private static final int	USAGE_EXIT_CODE
private boolean	zookeeperMode	True if we are to run in zookeeper 'mode'.

Constructor Summary

Constructors

Constructor	Description
CanaryTool()
CanaryTool(ExecutorService executor)
CanaryTool(ExecutorService executor, CanaryTool.Sink sink)
CanaryTool(org.apache.hadoop.conf.Configuration conf, ExecutorService executor)
CanaryTool(org.apache.hadoop.conf.Configuration conf, ExecutorService executor, CanaryTool.Sink sink)

Method Summary

Modifier and Type	Method	Description
int	checkRegions(String[] targets)	Run Canary in Region mode.
int	checkRegionServers(String[] targets)	Runs Canary in Region server mode.
int	checkZooKeeper()	Runs Canary in Zookeeper mode.
CanaryTool.Sink	getActiveSink()	Return a CanaryTool.Sink object containing the detailed results of the canary run.
org.apache.hadoop.conf.Configuration	getConf()
Map<String,String>	getReadFailures()
(package private) CanaryTool.Sink	getSink(org.apache.hadoop.conf.Configuration configuration, Class clazz)
Map<String,String>	getWriteFailures()
static void	main(String[] args)
private CanaryTool.Monitor	newMonitor(Connection connection, String[] monitorTargets)	A Factory method for CanaryTool.Monitor.
private int	parseArgs(String[] args)
private void	populateReadTableTimeoutsMap(String configuredReadTableTimeoutsStr)
private void	printUsageAndExit()
private void	putUpWebUI()
int	run(String[] args)
private int	runMonitor(String[] monitorTargets)
void	setConf(org.apache.hadoop.conf.Configuration conf)
private static List<Future<Void>>	sniff(Admin admin, CanaryTool.Sink sink, String tableName, ExecutorService executor, CanaryTool.RegionTask.TaskType taskType, boolean rawScanEnabled, LongAdder readLatency, boolean readAllCF)	Canary entry point for specified table.
private static List<Future<Void>>	sniff(Admin admin, CanaryTool.Sink sink, TableDescriptor tableDesc, ExecutorService executor, CanaryTool.RegionTask.TaskType taskType, boolean rawScanEnabled, LongAdder rwLatency, boolean readAllCF)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

HBASE_CANARY_INFO_PORT

public static final String HBASE_CANARY_INFO_PORT

See Also:

• Constant Field Values

HBASE_CANARY_INFO_BINDADDRESS

public static final String HBASE_CANARY_INFO_BINDADDRESS

See Also:

• Constant Field Values

USAGE_EXIT_CODE

private static final int USAGE_EXIT_CODE

See Also:

• Constant Field Values

INIT_ERROR_EXIT_CODE

private static final int INIT_ERROR_EXIT_CODE

See Also:

• Constant Field Values

TIMEOUT_ERROR_EXIT_CODE

private static final int TIMEOUT_ERROR_EXIT_CODE

See Also:

• Constant Field Values

ERROR_EXIT_CODE

private static final int ERROR_EXIT_CODE

See Also:

• Constant Field Values

FAILURE_EXIT_CODE

private static final int FAILURE_EXIT_CODE

See Also:

• Constant Field Values

DEFAULT_INTERVAL

private static final long DEFAULT_INTERVAL

See Also:

• Constant Field Values

DEFAULT_TIMEOUT

private static final long DEFAULT_TIMEOUT

See Also:

• Constant Field Values

MAX_THREADS_NUM

private static final int MAX_THREADS_NUM

See Also:

• Constant Field Values

LOG

private static final org.slf4j.Logger LOG

DEFAULT_WRITE_TABLE_NAME

public static final TableName DEFAULT_WRITE_TABLE_NAME

CANARY_TABLE_FAMILY_NAME

private static final String CANARY_TABLE_FAMILY_NAME

See Also:

• Constant Field Values

conf

private org.apache.hadoop.conf.Configuration conf

interval

private long interval

sink

private CanaryTool.Sink sink

regionServerMode

private boolean regionServerMode

True if we are to run in 'regionServer' mode.

zookeeperMode

private boolean zookeeperMode

True if we are to run in zookeeper 'mode'.

configuredReadTableTimeouts

private HashMap<String,Long> configuredReadTableTimeouts

This is a Map of table to timeout. The timeout is for reading all regions in the table; i.e. we aggregate time to fetch each region and it needs to be less than this value else we log an ERROR.

HBASE_CANARY_REGIONSERVER_ALL_REGIONS

public static final String HBASE_CANARY_REGIONSERVER_ALL_REGIONS

See Also:

• Constant Field Values

HBASE_CANARY_REGION_WRITE_SNIFFING

public static final String HBASE_CANARY_REGION_WRITE_SNIFFING

See Also:

• Constant Field Values

HBASE_CANARY_REGION_WRITE_TABLE_TIMEOUT

public static final String HBASE_CANARY_REGION_WRITE_TABLE_TIMEOUT

See Also:

• Constant Field Values

HBASE_CANARY_REGION_WRITE_TABLE_NAME

public static final String HBASE_CANARY_REGION_WRITE_TABLE_NAME

See Also:

• Constant Field Values

HBASE_CANARY_REGION_READ_TABLE_TIMEOUT

public static final String HBASE_CANARY_REGION_READ_TABLE_TIMEOUT

See Also:

• Constant Field Values

HBASE_CANARY_ZOOKEEPER_PERMITTED_FAILURES

public static final String HBASE_CANARY_ZOOKEEPER_PERMITTED_FAILURES

See Also:

• Constant Field Values

HBASE_CANARY_USE_REGEX

public static final String HBASE_CANARY_USE_REGEX

See Also:

• Constant Field Values

HBASE_CANARY_TIMEOUT

public static final String HBASE_CANARY_TIMEOUT

See Also:

• Constant Field Values

HBASE_CANARY_FAIL_ON_ERROR

public static final String HBASE_CANARY_FAIL_ON_ERROR

See Also:

• Constant Field Values

executor

private ExecutorService executor

Constructor Details

CanaryTool

public CanaryTool()

CanaryTool

public CanaryTool(ExecutorService executor)

CanaryTool

@Private
CanaryTool(ExecutorService executor,
 CanaryTool.Sink sink)

CanaryTool

CanaryTool(org.apache.hadoop.conf.Configuration conf,
 ExecutorService executor)

CanaryTool

CanaryTool(org.apache.hadoop.conf.Configuration conf,
 ExecutorService executor,
 CanaryTool.Sink sink)

Method Details

putUpWebUI

private void putUpWebUI()
                 throws IOException

Throws:

IOException

checkRegions

public int checkRegions(String[] targets)
                 throws Exception

Description copied from interface: Canary

Run Canary in Region mode.

Specified by:

checkRegions in interface Canary

Parameters:

targets - -- list of monitor tables.

Returns:

the exit code of the Canary tool.

Throws:

Exception

checkRegionServers

public int checkRegionServers(String[] targets)
                       throws Exception

Description copied from interface: Canary

Runs Canary in Region server mode.

Specified by:

checkRegionServers in interface Canary

Parameters:

targets - -- list of monitor tables.

Returns:

the exit code of the Canary tool.

Throws:

Exception

checkZooKeeper

public int checkZooKeeper()
                   throws Exception

Description copied from interface: Canary

Runs Canary in Zookeeper mode.

Specified by:

checkZooKeeper in interface Canary

Returns:

the exit code of the Canary tool.

Throws:

Exception

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

parseArgs

private int parseArgs(String[] args)

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

runMonitor

private int runMonitor(String[] monitorTargets)
                throws Exception

Throws:

Exception

getReadFailures

public Map<String,String> getReadFailures()

Specified by:

getReadFailures in interface Canary

getWriteFailures

public Map<String,String> getWriteFailures()

Specified by:

getWriteFailures in interface Canary

getActiveSink

public CanaryTool.Sink getActiveSink()

Return a CanaryTool.Sink object containing the detailed results of the canary run. The Sink may not have been created if a Monitor thread is not yet running.

Returns:

the active Sink if one exists, null otherwise.

printUsageAndExit

private void printUsageAndExit()

getSink

CanaryTool.Sink getSink(org.apache.hadoop.conf.Configuration configuration,
 Class clazz)

newMonitor

private CanaryTool.Monitor newMonitor(Connection connection,
 String[] monitorTargets)

A Factory method for CanaryTool.Monitor. Makes a RegionServerMonitor, or a ZooKeeperMonitor, or a RegionMonitor.

Returns:

a Monitor instance

populateReadTableTimeoutsMap

private void populateReadTableTimeoutsMap(String configuredReadTableTimeoutsStr)

sniff

private static List<Future<Void>> sniff(Admin admin,
 CanaryTool.Sink sink,
 String tableName,
 ExecutorService executor,
 CanaryTool.RegionTask.TaskType taskType,
 boolean rawScanEnabled,
 LongAdder readLatency,
 boolean readAllCF)
                                 throws Exception

Canary entry point for specified table.

Throws:

Exception - exception

sniff

private static List<Future<Void>> sniff(Admin admin,
 CanaryTool.Sink sink,
 TableDescriptor tableDesc,
 ExecutorService executor,
 CanaryTool.RegionTask.TaskType taskType,
 boolean rawScanEnabled,
 LongAdder rwLatency,
 boolean readAllCF)
                                 throws Exception

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception