org.apache.hadoop.hbase.chaos

Class ChaosAgent

java.lang.Object

org.apache.hadoop.hbase.chaos.ChaosAgent

All Implemented Interfaces:

Closeable, AutoCloseable, Runnable, org.apache.zookeeper.Watcher

@InterfaceAudience.Private
public class ChaosAgent
extends Object
implements org.apache.zookeeper.Watcher, Closeable, Runnable

An agent for executing destructive actions for ChaosMonkey. Uses ZooKeeper Watchers and LocalShell, to do the killing and getting status of service on targeted host without SSH. uses given ZNode Structure: /perfChaosTest (root) | | /chaosAgents (Used for registration has hostname ephemeral nodes as children) | | /chaosAgentTaskStatus (Used for task Execution, has hostname persistent nodes as child with tasks as their children) | | /hostname | | /task0000001 (command as data) (has two types of command : 1: starts with "exec" for executing a destructive action. 2: starts with "bool" for getting only status of service.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	ChaosAgent.LocalShell Executes Command locally.

Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher

org.apache.zookeeper.Watcher.Event, org.apache.zookeeper.Watcher.WatcherType

Field Summary

Fields

Modifier and Type	Field and Description
private String	agentName
private org.apache.hadoop.conf.Configuration	conf
private boolean	connected
(package private) org.apache.zookeeper.AsyncCallback.StringCallback	createEphemeralZNodeCallback Callback used while creating a Ephemeral ZNode tries to create ZNode again if Connection was lost in previous try.
(package private) org.apache.zookeeper.AsyncCallback.StringCallback	createZNodeCallback Callback used while creating a Persistent ZNode tries to create ZNode again if Connection was lost in previous try.
(package private) org.apache.zookeeper.AsyncCallback.DataCallback	getTaskForExecutionCallback Callback used by getTasksForAgentCallback while getting command, after getting command successfully, it executes command and set its status with respect to the command type.
(package private) org.apache.zookeeper.AsyncCallback.ChildrenCallback	getTasksForAgentCallback Callback used while getting Tasks for agent if call executed without Exception, It creates a separate thread for each children to execute given Tasks parallely.
private static org.slf4j.Logger	LOG
(package private) org.apache.zookeeper.Watcher	newTaskCreatedWatcher Watcher for notifying if any task is assigned to agent or not, by seeking if any Node is being added to agent as Child.
private String	quorum
private RetryCounterFactory	retryCounterFactory
(package private) org.apache.zookeeper.AsyncCallback.StatCallback	setStatusOfTaskZNodeCallback Callback used while setting status of a given task, Logs given status.
(package private) static AtomicBoolean	stopChaosAgent
private org.apache.zookeeper.ZooKeeper	zk

Constructor Summary

Constructors

Constructor and Description
ChaosAgent(org.apache.hadoop.conf.Configuration conf, String quorum, String agentName)

Method Summary

Modifier and Type	Method and Description
void	close()
void	createEphemeralZNode(String path, byte[] data) Function to create EPHEMERAL ZNODE with given path and data as params.
private void	createIfZNodeNotExists(String path) Checks if given ZNode exists, if not creates a PERSISTENT ZNODE for same.
private void	createZKConnection(org.apache.zookeeper.Watcher watcher) Creates Connection with ZooKeeper.
void	createZNode(String path, byte[] data) Function to create PERSISTENT ZNODE with given path and data given as params
private Pair<Integer,String>	exec(String user, String cmd)
private Pair<Integer,String>	execWithRetries(String user, String cmd) Below function executes command with retries with given user.
private void	getTasks() Gets tasks for execution, basically sets Watch on it's respective host's Znode and waits for tasks to be assigned, also has a getTasksForAgentCallback which handles execution of task.
private void	initChaosAgent(org.apache.hadoop.conf.Configuration conf, String quorum, String agentName) sets global params and initiates connection with ZooKeeper then does registration.
private boolean	isConnected()
void	process(org.apache.zookeeper.WatchedEvent watchedEvent)
private void	recreateZKConnection()
private void	register() registration of ChaosAgent by checking and creating necessary ZNodes.
private <E extends Exception> void	retryOrThrow(RetryCounter retryCounter, E ex, String user, String cmd)
void	run()
void	setStatusOfTaskZNode(String taskZNode, String status) sets given Status for Task Znode

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

stopChaosAgent

static AtomicBoolean stopChaosAgent

zk

private org.apache.zookeeper.ZooKeeper zk

quorum

private String quorum

agentName

private String agentName

conf

private org.apache.hadoop.conf.Configuration conf

retryCounterFactory

private RetryCounterFactory retryCounterFactory

connected

private volatile boolean connected

newTaskCreatedWatcher

org.apache.zookeeper.Watcher newTaskCreatedWatcher

Watcher for notifying if any task is assigned to agent or not, by seeking if any Node is being added to agent as Child.

setStatusOfTaskZNodeCallback

org.apache.zookeeper.AsyncCallback.StatCallback setStatusOfTaskZNodeCallback

Callback used while setting status of a given task, Logs given status.

createZNodeCallback

org.apache.zookeeper.AsyncCallback.StringCallback createZNodeCallback

Callback used while creating a Persistent ZNode tries to create ZNode again if Connection was lost in previous try.

createEphemeralZNodeCallback

org.apache.zookeeper.AsyncCallback.StringCallback createEphemeralZNodeCallback

Callback used while creating a Ephemeral ZNode tries to create ZNode again if Connection was lost in previous try.

getTaskForExecutionCallback

org.apache.zookeeper.AsyncCallback.DataCallback getTaskForExecutionCallback

Callback used by getTasksForAgentCallback while getting command, after getting command successfully, it executes command and set its status with respect to the command type.

getTasksForAgentCallback

org.apache.zookeeper.AsyncCallback.ChildrenCallback getTasksForAgentCallback

Callback used while getting Tasks for agent if call executed without Exception, It creates a separate thread for each children to execute given Tasks parallely.

Constructor Detail

ChaosAgent

public ChaosAgent(org.apache.hadoop.conf.Configuration conf,
                  String quorum,
                  String agentName)

Method Detail

initChaosAgent

private void initChaosAgent(org.apache.hadoop.conf.Configuration conf,
                            String quorum,
                            String agentName)

sets global params and initiates connection with ZooKeeper then does registration.

Parameters:

conf - initial configuration to use

quorum - ZK Quorum

agentName - AgentName to use

createZKConnection

private void createZKConnection(org.apache.zookeeper.Watcher watcher)
                         throws IOException

Creates Connection with ZooKeeper.

Throws:

IOException - if something goes wrong

createZNode

public void createZNode(String path,
                        byte[] data)

Function to create PERSISTENT ZNODE with given path and data given as params

Parameters:

path - Path at which ZNode to create

data - Data to put under ZNode

createEphemeralZNode

public void createEphemeralZNode(String path,
                                 byte[] data)

Function to create EPHEMERAL ZNODE with given path and data as params.

Parameters:

path - Path at which Ephemeral ZNode to create

data - Data to put under ZNode

createIfZNodeNotExists

private void createIfZNodeNotExists(String path)

Checks if given ZNode exists, if not creates a PERSISTENT ZNODE for same.

Parameters:

path - Path to check for ZNode

setStatusOfTaskZNode

public void setStatusOfTaskZNode(String taskZNode,
                                 String status)

sets given Status for Task Znode

Parameters:

taskZNode - ZNode to set status

status - Status value

register

private void register()

registration of ChaosAgent by checking and creating necessary ZNodes.

getTasks

private void getTasks()

Gets tasks for execution, basically sets Watch on it's respective host's Znode and waits for tasks to be assigned, also has a getTasksForAgentCallback which handles execution of task.

execWithRetries

private Pair<Integer,String> execWithRetries(String user,
                                             String cmd)
                                      throws IOException

Below function executes command with retries with given user. Uses LocalShell to execute a command.

Parameters:

user - user name, default none

cmd - Command to execute

Returns:

A pair of Exit Code and Shell output

Throws:

IOException - Exception while executing shell command

exec

private Pair<Integer,String> exec(String user,
                                  String cmd)
                           throws IOException

Throws:

IOException

retryOrThrow

private <E extends Exception> void retryOrThrow(RetryCounter retryCounter,
                                                E ex,
                                                String user,
                                                String cmd)
                                         throws E extends Exception

Throws:

E extends Exception

isConnected

private boolean isConnected()

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

run

public void run()

Specified by:

run in interface Runnable

process

public void process(org.apache.zookeeper.WatchedEvent watchedEvent)

Specified by:

process in interface org.apache.zookeeper.Watcher

recreateZKConnection

private void recreateZKConnection()
                           throws Exception

Throws:

Exception