org.apache.hadoop.hbase.tool

Class CanaryTool

java.lang.Object

org.apache.hadoop.hbase.tool.CanaryTool

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, Canary, org.apache.hadoop.util.Tool

@InterfaceAudience.LimitedPrivate(value="Tools")
public class CanaryTool
extends Object
implements org.apache.hadoop.util.Tool, Canary

HBase Canary Tool for "canary monitoring" of a running HBase cluster. There are three modes:

1. region mode (Default): For each region, try to get one row per column family outputting information on failure (ERROR) or else the latency.
2. regionserver mode: For each regionserver try to get one row from one table selected randomly outputting information on failure (ERROR) or else the latency.
3. zookeeper mode: for each zookeeper instance, selects a znode outputting information on failure (ERROR) or else the latency.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CanaryTool.Monitor A Monitor super-class can be extended by users
private static class	CanaryTool.RegionMonitor A monitor for region mode.
private static class	CanaryTool.RegionServerMonitor A monitor for regionserver mode
static class	CanaryTool.RegionServerStdOutSink By RegionServer, for 'regionserver' mode.
(package private) static class	CanaryTool.RegionServerTask Run a single RegionServer Task and then exit.
static class	CanaryTool.RegionStdOutSink By Region, for 'region' mode.
(package private) static class	CanaryTool.RegionTask Run a single Region Task and then exit.
static class	CanaryTool.RegionTaskResult Canary region mode-specific data structure which stores information about each region to be scanned
static interface	CanaryTool.Sink Sink interface used by the canary to output information
static class	CanaryTool.StdOutSink Simple implementation of canary sink that allows plotting to a file or standard output.
private static class	CanaryTool.ZookeeperMonitor
static class	CanaryTool.ZookeeperStdOutSink Output for 'zookeeper' mode.
(package private) static class	CanaryTool.ZookeeperTask Run a single zookeeper Task and then exit.

Field Summary

Fields

Modifier and Type	Field and Description
private static String	CANARY_TABLE_FAMILY_NAME
private org.apache.hadoop.conf.Configuration	conf
private HashMap<String,Long>	configuredReadTableTimeouts This is a Map of table to timeout.
static int	DEFAULT_CANARY_INFOPORT
private static long	DEFAULT_INTERVAL
private static long	DEFAULT_TIMEOUT
static TableName	DEFAULT_WRITE_TABLE_NAME
private static int	ERROR_EXIT_CODE
private ExecutorService	executor
private static int	FAILURE_EXIT_CODE
static String	HBASE_CANARY_FAIL_ON_ERROR
static String	HBASE_CANARY_INFO_BINDADDRESS
static String	HBASE_CANARY_INFO_PORT
static String	HBASE_CANARY_REGION_READ_TABLE_TIMEOUT
static String	HBASE_CANARY_REGION_WRITE_SNIFFING
static String	HBASE_CANARY_REGION_WRITE_TABLE_NAME
static String	HBASE_CANARY_REGION_WRITE_TABLE_TIMEOUT
static String	HBASE_CANARY_REGIONSERVER_ALL_REGIONS
static String	HBASE_CANARY_TIMEOUT
static String	HBASE_CANARY_USE_REGEX
static String	HBASE_CANARY_ZOOKEEPER_PERMITTED_FAILURES
private InfoServer	infoServer
private static int	INIT_ERROR_EXIT_CODE
private long	interval
private static org.slf4j.Logger	LOG
private static int	MAX_THREADS_NUM
private boolean	regionServerMode True if we are to run in 'regionServer' mode.
private CanaryTool.Sink	sink
private static int	TIMEOUT_ERROR_EXIT_CODE
private static int	USAGE_EXIT_CODE
private boolean	zookeeperMode True if we are to run in zookeeper 'mode'.

Constructor Summary

Constructors

Constructor and Description
CanaryTool()
CanaryTool(org.apache.hadoop.conf.Configuration conf, ExecutorService executor)
CanaryTool(org.apache.hadoop.conf.Configuration conf, ExecutorService executor, CanaryTool.Sink sink)
CanaryTool(ExecutorService executor)
CanaryTool(ExecutorService executor, CanaryTool.Sink sink)

Method Summary

Modifier and Type	Method and Description
int	checkRegions(String[] targets) Run Canary in Region mode.
int	checkRegionServers(String[] targets) Runs Canary in Region server mode.
int	checkZooKeeper() Runs Canary in Zookeeper mode.
org.apache.hadoop.conf.Configuration	getConf()
Map<String,String>	getReadFailures()
(package private) CanaryTool.Sink	getSink(org.apache.hadoop.conf.Configuration configuration, Class clazz)
Map<String,String>	getWriteFailures()
static void	main(String[] args)
private CanaryTool.Monitor	newMonitor(Connection connection, String[] monitorTargets) A Factory method for CanaryTool.Monitor.
private int	parseArgs(String[] args)
private void	populateReadTableTimeoutsMap(String configuredReadTableTimeoutsStr)
private void	printUsageAndExit()
private void	putUpWebUI()
int	run(String[] args)
private int	runMonitor(String[] monitorTargets)
void	setConf(org.apache.hadoop.conf.Configuration conf)
private static List<Future<Void>>	sniff(Admin admin, CanaryTool.Sink sink, String tableName, ExecutorService executor, CanaryTool.RegionTask.TaskType taskType, boolean rawScanEnabled, LongAdder readLatency, boolean readAllCF) Canary entry point for specified table.
private static List<Future<Void>>	sniff(Admin admin, CanaryTool.Sink sink, TableDescriptor tableDesc, ExecutorService executor, CanaryTool.RegionTask.TaskType taskType, boolean rawScanEnabled, LongAdder rwLatency, boolean readAllCF)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.tool.Canary

create, create

Field Detail

HBASE_CANARY_INFO_PORT

public static final String HBASE_CANARY_INFO_PORT

See Also:

Constant Field Values

DEFAULT_CANARY_INFOPORT

public static final int DEFAULT_CANARY_INFOPORT

See Also:

Constant Field Values

HBASE_CANARY_INFO_BINDADDRESS

public static final String HBASE_CANARY_INFO_BINDADDRESS

See Also:

Constant Field Values

infoServer

private InfoServer infoServer

USAGE_EXIT_CODE

private static final int USAGE_EXIT_CODE

See Also:

Constant Field Values

INIT_ERROR_EXIT_CODE

private static final int INIT_ERROR_EXIT_CODE

See Also:

Constant Field Values

TIMEOUT_ERROR_EXIT_CODE

private static final int TIMEOUT_ERROR_EXIT_CODE

See Also:

Constant Field Values

ERROR_EXIT_CODE

private static final int ERROR_EXIT_CODE

See Also:

Constant Field Values

FAILURE_EXIT_CODE

private static final int FAILURE_EXIT_CODE

See Also:

Constant Field Values

DEFAULT_INTERVAL

private static final long DEFAULT_INTERVAL

See Also:

Constant Field Values

DEFAULT_TIMEOUT

private static final long DEFAULT_TIMEOUT

See Also:

Constant Field Values

MAX_THREADS_NUM

private static final int MAX_THREADS_NUM

See Also:

Constant Field Values

LOG

private static final org.slf4j.Logger LOG

DEFAULT_WRITE_TABLE_NAME

public static final TableName DEFAULT_WRITE_TABLE_NAME

CANARY_TABLE_FAMILY_NAME

private static final String CANARY_TABLE_FAMILY_NAME

See Also:

Constant Field Values

conf

private org.apache.hadoop.conf.Configuration conf

interval

private long interval

sink

private CanaryTool.Sink sink

regionServerMode

private boolean regionServerMode

True if we are to run in 'regionServer' mode.

zookeeperMode

private boolean zookeeperMode

True if we are to run in zookeeper 'mode'.

configuredReadTableTimeouts

private HashMap<String,Long> configuredReadTableTimeouts

This is a Map of table to timeout. The timeout is for reading all regions in the table; i.e. we aggregate time to fetch each region and it needs to be less than this value else we log an ERROR.

HBASE_CANARY_REGIONSERVER_ALL_REGIONS

public static final String HBASE_CANARY_REGIONSERVER_ALL_REGIONS

See Also:

Constant Field Values

HBASE_CANARY_REGION_WRITE_SNIFFING

public static final String HBASE_CANARY_REGION_WRITE_SNIFFING

See Also:

Constant Field Values

HBASE_CANARY_REGION_WRITE_TABLE_TIMEOUT

public static final String HBASE_CANARY_REGION_WRITE_TABLE_TIMEOUT

See Also:

Constant Field Values

HBASE_CANARY_REGION_WRITE_TABLE_NAME

public static final String HBASE_CANARY_REGION_WRITE_TABLE_NAME

See Also:

Constant Field Values

HBASE_CANARY_REGION_READ_TABLE_TIMEOUT

public static final String HBASE_CANARY_REGION_READ_TABLE_TIMEOUT

See Also:

Constant Field Values

HBASE_CANARY_ZOOKEEPER_PERMITTED_FAILURES

public static final String HBASE_CANARY_ZOOKEEPER_PERMITTED_FAILURES

See Also:

Constant Field Values

HBASE_CANARY_USE_REGEX

public static final String HBASE_CANARY_USE_REGEX

See Also:

Constant Field Values

HBASE_CANARY_TIMEOUT

public static final String HBASE_CANARY_TIMEOUT

See Also:

Constant Field Values

HBASE_CANARY_FAIL_ON_ERROR

public static final String HBASE_CANARY_FAIL_ON_ERROR

See Also:

Constant Field Values

executor

private ExecutorService executor

Constructor Detail

CanaryTool

public CanaryTool()

CanaryTool

public CanaryTool(ExecutorService executor)

CanaryTool

CanaryTool(ExecutorService executor,
           CanaryTool.Sink sink)

CanaryTool

CanaryTool(org.apache.hadoop.conf.Configuration conf,
           ExecutorService executor)

CanaryTool

CanaryTool(org.apache.hadoop.conf.Configuration conf,
           ExecutorService executor,
           CanaryTool.Sink sink)

Method Detail

putUpWebUI

private void putUpWebUI()
                 throws IOException

Throws:

IOException

checkRegions

public int checkRegions(String[] targets)
                 throws Exception

Description copied from interface: Canary

Run Canary in Region mode.

Specified by:

checkRegions in interface Canary

Parameters:

targets - -- list of monitor tables.

Returns:

the exit code of the Canary tool.

Throws:

Exception

checkRegionServers

public int checkRegionServers(String[] targets)
                       throws Exception

Description copied from interface: Canary

Runs Canary in Region server mode.

Specified by:

checkRegionServers in interface Canary

Parameters:

targets - -- list of monitor tables.

Returns:

the exit code of the Canary tool.

Throws:

Exception

checkZooKeeper

public int checkZooKeeper()
                   throws Exception

Description copied from interface: Canary

Runs Canary in Zookeeper mode.

Specified by:

checkZooKeeper in interface Canary

Returns:

the exit code of the Canary tool.

Throws:

Exception

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

parseArgs

private int parseArgs(String[] args)

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

runMonitor

private int runMonitor(String[] monitorTargets)
                throws Exception

Throws:

Exception

getReadFailures

public Map<String,String> getReadFailures()

Specified by:

getReadFailures in interface Canary

getWriteFailures

public Map<String,String> getWriteFailures()

Specified by:

getWriteFailures in interface Canary

printUsageAndExit

private void printUsageAndExit()

getSink

CanaryTool.Sink getSink(org.apache.hadoop.conf.Configuration configuration,
                        Class clazz)

newMonitor

private CanaryTool.Monitor newMonitor(Connection connection,
                                      String[] monitorTargets)

A Factory method for CanaryTool.Monitor. Makes a RegionServerMonitor, or a ZooKeeperMonitor, or a RegionMonitor.

Returns:

a Monitor instance

populateReadTableTimeoutsMap

private void populateReadTableTimeoutsMap(String configuredReadTableTimeoutsStr)

sniff

private static List<Future<Void>> sniff(Admin admin,
                                        CanaryTool.Sink sink,
                                        String tableName,
                                        ExecutorService executor,
                                        CanaryTool.RegionTask.TaskType taskType,
                                        boolean rawScanEnabled,
                                        LongAdder readLatency,
                                        boolean readAllCF)
                                 throws Exception

Canary entry point for specified table.

Throws:

Exception - exception

sniff

private static List<Future<Void>> sniff(Admin admin,
                                        CanaryTool.Sink sink,
                                        TableDescriptor tableDesc,
                                        ExecutorService executor,
                                        CanaryTool.RegionTask.TaskType taskType,
                                        boolean rawScanEnabled,
                                        LongAdder rwLatency,
                                        boolean readAllCF)
                                 throws Exception

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception