org.apache.hadoop.hbase

Class RESTApiClusterManager

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.RESTApiClusterManager

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, ClusterManager

public class RESTApiClusterManager
extends org.apache.hadoop.conf.Configured
implements ClusterManager

A ClusterManager implementation designed to control Cloudera Manager (http://www.cloudera.com) clusters via REST API. This API uses HTTP GET requests against the cluster manager server to retrieve information and POST/PUT requests to perform actions. As a simple example, to retrieve a list of hosts from a CM server with login credentials admin:admin, a simple curl command would be curl -X POST -H "Content-Type:application/json" -u admin:admin \ "http://this.is.my.server.com:7180/api/v8/hosts" This command would return a JSON result, which would need to be parsed to retrieve relevant information. This action and many others are covered by this class. A note on nomenclature: while the ClusterManager interface uses a ServiceType enum when referring to things like RegionServers and DataNodes, cluster managers often use different terminology. As an example, Cloudera Manager (http://www.cloudera.com) would refer to a RegionServer as a "role" of the HBase "service." It would further refer to "hbase" as the "serviceType." Apache Ambari (http://ambari.apache.org) would call the RegionServer a "component" of the HBase "service." This class will defer to the ClusterManager terminology in methods that it implements from that interface, but uses Cloudera Manager's terminology when dealing with its API directly. DEBUG-level logging gives more details of the actions this class takes as they happen. Log at TRACE-level to see the API requests and responses. TRACE-level logging on RetryCounter displays wait times, so that can be helpful too.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	RESTApiClusterManager.HealthSummary Represents of the high-level health status of a subject in the cluster.
private static class	RESTApiClusterManager.RoleCommand
private static class	RESTApiClusterManager.RoleState Represents the configured run state of a role.
(package private) static class	RESTApiClusterManager.Service

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.ClusterManager

ClusterManager.ServiceType

Field Summary

Fields

Modifier and Type	Field and Description
private static String	API_VERSION
private javax.ws.rs.client.Client	client
private String	clusterName
private static String	DEFAULT_CLUSTER_NAME
private static String	DEFAULT_SERVER_HOSTNAME
private static String	DEFAULT_SERVER_PASSWORD
private static String	DEFAULT_SERVER_USERNAME
private ClusterManager	hBaseClusterManager
private static org.slf4j.Logger	LOG
private static org.apache.hbase.thirdparty.com.google.gson.JsonParser	parser
private static String	REST_API_CLUSTER_MANAGER_CLUSTER_NAME
private static String	REST_API_CLUSTER_MANAGER_HOSTNAME
private static String	REST_API_CLUSTER_MANAGER_PASSWORD
private static String	REST_API_CLUSTER_MANAGER_USERNAME
private static String	REST_API_DELEGATE_CLUSTER_MANAGER
private org.apache.hadoop.hbase.util.RetryCounterFactory	retryCounterFactory
private static Map<ClusterManager.ServiceType,RESTApiClusterManager.Service>	roleServiceType
private String	serverHostname

Constructor Summary

Constructors

Constructor and Description
RESTApiClusterManager()

Method Summary

Modifier and Type	Method and Description
private static Map<ClusterManager.ServiceType,RESTApiClusterManager.Service>	buildRoleServiceTypeMap()
private long	doRoleCommand(String serviceName, String roleName, RESTApiClusterManager.RoleCommand roleCommand) Issues a command (e.g.
private <T> T	executeWithRetries(Callable<T> callable) Helper method for executing retryable work.
private Optional<org.apache.hbase.thirdparty.com.google.gson.JsonObject>	getCommand(long commandId)
private String	getFromURIGet(URI uri)
private RESTApiClusterManager.HealthSummary	getHealthSummary(String serviceName, String roleType, String hostId)
private String	getHostId(String hostname)
private String	getRoleName(String serviceName, String roleType, String hostId)
private String	getRolePropertyValue(String serviceName, String roleType, String hostId, String property)
private RESTApiClusterManager.RoleState	getRoleState(ClusterManager.ServiceType service, String hostname)
private RESTApiClusterManager.RoleState	getRoleState(String serviceName, String roleType, String hostId)
private String	getServiceName(RESTApiClusterManager.Service service)
private boolean	hasCommandCompleted(long commandId) Return true if the commandId has finished processing.
private boolean	hasCommandCompletedSuccessfully(long commandId) Return true if the commandId has finished successfully.
boolean	isRunning(ClusterManager.ServiceType service, String hostname, int port) Returns whether the service is running on the remote host.
void	kill(ClusterManager.ServiceType service, String hostname, int port) Kills the service running on the given host
private void	performClusterManagerCommand(ClusterManager.ServiceType role, String hostname, RESTApiClusterManager.RoleCommand command)
void	restart(ClusterManager.ServiceType service, String hostname, int port) Restart the service on the given host
void	resume(ClusterManager.ServiceType service, String hostname, int port) Resumes the services running on the given host
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	start(ClusterManager.ServiceType service, String hostname, int port) Start the service on the given host
void	stop(ClusterManager.ServiceType service, String hostname, int port) Stop the service on the given host
void	suspend(ClusterManager.ServiceType service, String hostname, int port) Suspends the service running on the given host
private void	waitFor(Callable<Boolean> predicate)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf

Field Detail

REST_API_CLUSTER_MANAGER_HOSTNAME

private static final String REST_API_CLUSTER_MANAGER_HOSTNAME

See Also:

Constant Field Values

REST_API_CLUSTER_MANAGER_USERNAME

private static final String REST_API_CLUSTER_MANAGER_USERNAME

See Also:

Constant Field Values

REST_API_CLUSTER_MANAGER_PASSWORD

private static final String REST_API_CLUSTER_MANAGER_PASSWORD

See Also:

Constant Field Values

REST_API_CLUSTER_MANAGER_CLUSTER_NAME

private static final String REST_API_CLUSTER_MANAGER_CLUSTER_NAME

See Also:

Constant Field Values

REST_API_DELEGATE_CLUSTER_MANAGER

private static final String REST_API_DELEGATE_CLUSTER_MANAGER

See Also:

Constant Field Values

parser

private static final org.apache.hbase.thirdparty.com.google.gson.JsonParser parser

DEFAULT_SERVER_HOSTNAME

private static final String DEFAULT_SERVER_HOSTNAME

See Also:

Constant Field Values

DEFAULT_SERVER_USERNAME

private static final String DEFAULT_SERVER_USERNAME

See Also:

Constant Field Values

DEFAULT_SERVER_PASSWORD

private static final String DEFAULT_SERVER_PASSWORD

See Also:

Constant Field Values

DEFAULT_CLUSTER_NAME

private static final String DEFAULT_CLUSTER_NAME

See Also:

Constant Field Values

serverHostname

private String serverHostname

clusterName

private String clusterName

API_VERSION

private static final String API_VERSION

See Also:

Constant Field Values

client

private final javax.ws.rs.client.Client client

hBaseClusterManager

private ClusterManager hBaseClusterManager

retryCounterFactory

private org.apache.hadoop.hbase.util.RetryCounterFactory retryCounterFactory

LOG

private static final org.slf4j.Logger LOG

roleServiceType

private static final Map<ClusterManager.ServiceType,RESTApiClusterManager.Service> roleServiceType

Constructor Detail

RESTApiClusterManager

RESTApiClusterManager()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class org.apache.hadoop.conf.Configured

start

public void start(ClusterManager.ServiceType service,
                  String hostname,
                  int port)

Description copied from interface: ClusterManager

Start the service on the given host

Specified by:

start in interface ClusterManager

stop

public void stop(ClusterManager.ServiceType service,
                 String hostname,
                 int port)

Description copied from interface: ClusterManager

Stop the service on the given host

Specified by:

stop in interface ClusterManager

restart

public void restart(ClusterManager.ServiceType service,
                    String hostname,
                    int port)

Description copied from interface: ClusterManager

Restart the service on the given host

Specified by:

restart in interface ClusterManager

isRunning

public boolean isRunning(ClusterManager.ServiceType service,
                         String hostname,
                         int port)

Description copied from interface: ClusterManager

Returns whether the service is running on the remote host. This only checks whether the service still has a pid.

Specified by:

isRunning in interface ClusterManager

kill

public void kill(ClusterManager.ServiceType service,
                 String hostname,
                 int port)
          throws IOException

Description copied from interface: ClusterManager

Kills the service running on the given host

Specified by:

kill in interface ClusterManager

Throws:

IOException

suspend

public void suspend(ClusterManager.ServiceType service,
                    String hostname,
                    int port)
             throws IOException

Description copied from interface: ClusterManager

Suspends the service running on the given host

Specified by:

suspend in interface ClusterManager

Throws:

IOException

resume

public void resume(ClusterManager.ServiceType service,
                   String hostname,
                   int port)
            throws IOException

Description copied from interface: ClusterManager

Resumes the services running on the given host

Specified by:

resume in interface ClusterManager

Throws:

IOException

performClusterManagerCommand

private void performClusterManagerCommand(ClusterManager.ServiceType role,
                                          String hostname,
                                          RESTApiClusterManager.RoleCommand command)

doRoleCommand

private long doRoleCommand(String serviceName,
                           String roleName,
                           RESTApiClusterManager.RoleCommand roleCommand)

Issues a command (e.g. starting or stopping a role).

Returns:

the commandId of a successfully submitted asynchronous command.

getHealthSummary

private RESTApiClusterManager.HealthSummary getHealthSummary(String serviceName,
                                                             String roleType,
                                                             String hostId)

getHostId

private String getHostId(String hostname)

getFromURIGet

private String getFromURIGet(URI uri)

getRoleName

private String getRoleName(String serviceName,
                           String roleType,
                           String hostId)

getRolePropertyValue

private String getRolePropertyValue(String serviceName,
                                    String roleType,
                                    String hostId,
                                    String property)

getRoleState

private RESTApiClusterManager.RoleState getRoleState(ClusterManager.ServiceType service,
                                                     String hostname)

getRoleState

private RESTApiClusterManager.RoleState getRoleState(String serviceName,
                                                     String roleType,
                                                     String hostId)

getServiceName

private String getServiceName(RESTApiClusterManager.Service service)

getCommand

private Optional<org.apache.hbase.thirdparty.com.google.gson.JsonObject> getCommand(long commandId)

hasCommandCompleted

private boolean hasCommandCompleted(long commandId)

Return true if the commandId has finished processing.

hasCommandCompletedSuccessfully

private boolean hasCommandCompletedSuccessfully(long commandId)

Return true if the commandId has finished successfully.

executeWithRetries

private <T> T executeWithRetries(Callable<T> callable)

Helper method for executing retryable work.

waitFor

private void waitFor(Callable<Boolean> predicate)

buildRoleServiceTypeMap

private static Map<ClusterManager.ServiceType,RESTApiClusterManager.Service> buildRoleServiceTypeMap()