ServerManager (Apache HBase 2.0.6 API)

java.lang.Object
- org.apache.hadoop.hbase.master.ServerManager

```
@InterfaceAudience.Private
public class ServerManager
extends Object
```
The ServerManager class manages info about region servers.
Maintains lists of online and dead servers. Processes the startups, shutdowns, and deaths of region servers.
Servers are distinguished in two different ways. A given server has a location, specified by hostname and port, and of which there can only be one online at any given time. A server instance is specified by the location (hostname and port) as well as the startcode (timestamp from when the server was started). This is used to differentiate a restarted instance of a given server from the original instance.
If a sever is known not to be running any more, it is called dead. The dead server needs to be handled by a ServerShutdownHandler. If the handler is not enabled yet, the server can't be handled right away so it is queued up. After the handler is enabled, the server will be submitted to a handler to handle. However, the handler may be just partially enabled. If so, the server cannot be fully processed, and be queued up for further processing. A server is fully processed only after the handler is fully enabled and has completed the handling.

Field Summary

Fields
Modifier and Type	Field and Description
`private AtomicBoolean`	`clusterShutdown`
`private ClusterConnection`	`connection`
`private DeadServer`	`deadservers`
`private ArrayList<ServerName>`	`drainingServers` List of region servers that should not get any more new regions.
`private ConcurrentNavigableMap<byte[],Long>`	`flushedSequenceIdByRegion` The last flushed sequence id for a region.
`private List<ServerListener>`	`listeners` Listeners that are called on server events.
`private static org.slf4j.Logger`	`LOG`
`private MasterServices`	`master`
`private long`	`maxSkew`
`private ConcurrentNavigableMap<ServerName,ServerMetrics>`	`onlineServers` Map of registered servers to their current load
`private RpcControllerFactory`	`rpcControllerFactory`
`private Map<ServerName,org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface>`	`rsAdmins` Map of admin interfaces per registered regionserver; these interfaces we use to control regionservers out on the cluster
`private ConcurrentNavigableMap<byte[],ConcurrentNavigableMap<byte[],Long>>`	`storeFlushedSequenceIdsByRegion` The last flushed sequence id for a store in a region.
`static String`	`WAIT_ON_REGIONSERVERS_INTERVAL`
`static String`	`WAIT_ON_REGIONSERVERS_MAXTOSTART`
`static String`	`WAIT_ON_REGIONSERVERS_MINTOSTART`
`static String`	`WAIT_ON_REGIONSERVERS_TIMEOUT`
`private long`	`warningSkew`

Constructor Summary

Constructors
Constructor and Description

ServerManager(MasterServices master)
Constructor.

Constructors
Constructor and Description
`ServerManager(MasterServices master)` Constructor.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`boolean`	`addServerToDrainList(ServerName sn)` Add the server to the drain list.
`boolean`	`areDeadServersInProgress()` Checks if any dead servers are currently in progress.
`(package private) boolean`	`checkAndRecordNewServer(ServerName serverName, ServerMetrics sl)` Check is a server of same host and port already exists, if not, or the existed one got a smaller start code, record it.
`private void`	`checkClockSkew(ServerName serverName, long serverCurrentTime)` Checks if the clock skew between the server and the master.
`private void`	`checkIsDead(ServerName serverName, String what)` If this server is on the dead list, reject it with a YouAreDeadException.
`(package private) void`	`clearDeadServersWithSameHostNameAndPortOfOnlineServer()` To clear any dead server with same host name and port of any online server
`static void`	`closeRegionSilentlyAndWait(ClusterConnection connection, ServerName server, RegionInfo region, long timeout)` Contacts a region server and waits up to timeout ms to close the region.
`int`	`countOfRegionServers()`
`List<ServerName>`	`createDestinationServersList()` Calls `createDestinationServersList(java.util.List<org.apache.hadoop.hbase.ServerName>)` without server to exclude.
`List<ServerName>`	`createDestinationServersList(List<ServerName> serversToExclude)` Creates a list of possible destinations for a region.
`(package private) boolean`	`expire(ServerName serverName)`
`boolean`	`expireServer(ServerName serverName)` Expire the passed server.
`(package private) void`	`findOutDeadServersAndProcess(Set<ServerName> deadServersFromPE, Set<ServerName> liveServersFromWALDir)` Find out the region servers crashed between the crash of the previous master instance and the current master instance and schedule SCP for them.
`private ServerName`	`findServerWithSameHostnamePortWithLock(ServerName serverName)` Assumes onlineServers is locked.
`double`	`getAverageLoad()` Compute the average load across all region servers.
`DeadServer`	`getDeadServers()`
`List<ServerName>`	`getDrainingServersList()`
`int`	`getInfoPort(ServerName serverName)`
`org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds`	`getLastFlushedSequenceId(byte[] encodedRegionName)`
`ServerMetrics`	`getLoad(ServerName serverName)`
`private int`	`getMinToStart()` Calculate min necessary to start.
`Map<ServerName,ServerMetrics>`	`getOnlineServers()`
`List<ServerName>`	`getOnlineServersList()`
`List<ServerName>`	`getOnlineServersListWithPredicator(List<ServerName> keys, Predicate<ServerMetrics> idleServerPredicator)`
`private List<String>`	`getRegionServersInZK(ZKWatcher zkw)`
`org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface`	`getRsAdmin(ServerName sn)`
`int`	`getServerVersion(ServerName serverName)` May return 0 when server is not online.
`private String`	`getStrForMax(int max)`
`boolean`	`isClusterShutdown()`
`boolean`	`isRegionInServerManagerStates(RegionInfo hri)`
`boolean`	`isServerDead(ServerName serverName)` Check if a server is known to be dead.
`boolean`	`isServerOnline(ServerName serverName)`
`(package private) void`	`letRegionServersShutdown()`
`boolean`	`moveFromOnlineToDeadServers(ServerName sn)`
`private HBaseRpcController`	`newRpcController()`
`(package private) void`	`recordNewServerWithLock(ServerName serverName, ServerMetrics sl)` Adds the onlineServers list.
`void`	`regionServerReport(ServerName sn, ServerMetrics sl)`
`(package private) ServerName`	`regionServerStartup(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStartupRequest request, int versionNumber, InetAddress ia)` Let the server manager know a new regionserver has come online
`void`	`registerListener(ServerListener listener)` Add the listener to the notification list.
`void`	`removeRegion(RegionInfo regionInfo)` Called by delete table and similar to notify the ServerManager that a region was removed.
`void`	`removeRegions(List<RegionInfo> regions)` Called by delete table and similar to notify the ServerManager that a region was removed.
`boolean`	`removeServerFromDrainList(ServerName sn)`
`void`	`sendRegionWarmup(ServerName server, RegionInfo region)` Sends a WARMUP RPC to the specified server to warmup the specified region.
`void`	`shutdownCluster()`
`void`	`stop()` Stop the ServerManager.
`boolean`	`unregisterListener(ServerListener listener)` Remove the listener from the notification list.
`private void`	`updateLastFlushedSequenceIds(ServerName sn, ServerMetrics hsl)` Updates last flushed sequence Ids for the regions on server sn
`void`	`waitForRegionServers(MonitoredTask status)` Wait for the region servers to report in.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - WAIT_ON_REGIONSERVERS_MAXTOSTART
```
public static final String WAIT_ON_REGIONSERVERS_MAXTOSTART
```
    See Also:
    
    Constant Field Values
  - WAIT_ON_REGIONSERVERS_MINTOSTART
```
public static final String WAIT_ON_REGIONSERVERS_MINTOSTART
```
    See Also:
    
    Constant Field Values
  - WAIT_ON_REGIONSERVERS_TIMEOUT
```
public static final String WAIT_ON_REGIONSERVERS_TIMEOUT
```
    See Also:
    
    Constant Field Values
  - WAIT_ON_REGIONSERVERS_INTERVAL
```
public static final String WAIT_ON_REGIONSERVERS_INTERVAL
```
    See Also:
    
    Constant Field Values
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - clusterShutdown
```
private AtomicBoolean clusterShutdown
```
  - flushedSequenceIdByRegion
```
private final ConcurrentNavigableMap<byte[],Long> flushedSequenceIdByRegion
```
    The last flushed sequence id for a region.
  - storeFlushedSequenceIdsByRegion
```
private final ConcurrentNavigableMap<byte[],ConcurrentNavigableMap<byte[],Long>> storeFlushedSequenceIdsByRegion
```
    The last flushed sequence id for a store in a region.
  - onlineServers
```
private final ConcurrentNavigableMap<ServerName,ServerMetrics> onlineServers
```
    Map of registered servers to their current load
  - rsAdmins
```
private final Map<ServerName,org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface> rsAdmins
```
    Map of admin interfaces per registered regionserver; these interfaces we use to control regionservers out on the cluster
  - drainingServers
```
private final ArrayList<ServerName> drainingServers
```
    List of region servers that should not get any more new regions.
  - master
```
private final MasterServices master
```
  - connection
```
private final ClusterConnection connection
```
  - deadservers
```
private final DeadServer deadservers
```
  - maxSkew
```
private final long maxSkew
```
  - warningSkew
```
private final long warningSkew
```
  - rpcControllerFactory
```
private final RpcControllerFactory rpcControllerFactory
```
  - listeners
```
private List<ServerListener> listeners
```
    Listeners that are called on server events.
- Constructor Detail
  - ServerManager
```
public ServerManager(MasterServices master)
```
    Constructor.
- Method Detail
  - registerListener
```
public void registerListener(ServerListener listener)
```
    Add the listener to the notification list.
    
    Parameters:
    
    listener - The ServerListener to register
  - unregisterListener
```
public boolean unregisterListener(ServerListener listener)
```
    Remove the listener from the notification list.
    
    Parameters:
    
    listener - The ServerListener to unregister
  - regionServerStartup
```
ServerName regionServerStartup(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos.RegionServerStartupRequest request,
                               int versionNumber,
                               InetAddress ia)
                        throws IOException
```
    Let the server manager know a new regionserver has come online
    
    Parameters:
    
    request - the startup request
    
    versionNumber - the version of the new regionserver
    
    ia - the InetAddress from which request is received
    
    Returns:
    
    The ServerName we know this server as.
    
    Throws:
    
    IOException
  - updateLastFlushedSequenceIds
```
private void updateLastFlushedSequenceIds(ServerName sn,
                                          ServerMetrics hsl)
```
    Updates last flushed sequence Ids for the regions on server sn
    
    Parameters:
    
    sn -
    
    hsl -
  - regionServerReport
```
public void regionServerReport(ServerName sn,
                               ServerMetrics sl)
                        throws YouAreDeadException
```
    Throws:
    
    YouAreDeadException
  - checkAndRecordNewServer
```
boolean checkAndRecordNewServer(ServerName serverName,
                                ServerMetrics sl)
```
    Check is a server of same host and port already exists, if not, or the existed one got a smaller start code, record it.
    
    Parameters:
    
    serverName - the server to check and record
    
    sl - the server load on the server
    
    Returns:
    
    true if the server is recorded, otherwise, false
  - findOutDeadServersAndProcess
```
void findOutDeadServersAndProcess(Set<ServerName> deadServersFromPE,
                                  Set<ServerName> liveServersFromWALDir)
```
    Find out the region servers crashed between the crash of the previous master instance and the current master instance and schedule SCP for them.
    Since the RegionServerTracker has already helped us to construct the online servers set by scanning zookeeper, now we can compare the online servers with liveServersFromWALDir to find out whether there are servers which are already dead.
    Must be called inside the initialization method of RegionServerTracker to avoid concurrency issue.
    
    Parameters:
    
    deadServersFromPE - the region servers which already have SCP associated.
    
    liveServersFromWALDir - the live region servers from wal directory.
  - checkClockSkew
```
private void checkClockSkew(ServerName serverName,
                            long serverCurrentTime)
                     throws ClockOutOfSyncException
```
    Checks if the clock skew between the server and the master. If the clock skew exceeds the configured max, it will throw an exception; if it exceeds the configured warning threshold, it will log a warning but start normally.
    
    Parameters:
    
    serverName - Incoming servers's name
    
    serverCurrentTime -
    
    Throws:
    
    ClockOutOfSyncException - if the skew exceeds the configured max value
  - checkIsDead
```
private void checkIsDead(ServerName serverName,
                         String what)
                  throws YouAreDeadException
```
    If this server is on the dead list, reject it with a YouAreDeadException. If it was dead but came back with a new start code, remove the old entry from the dead list.
    
    Parameters:
    
    what - START or REPORT
    
    Throws:
    
    YouAreDeadException
  - findServerWithSameHostnamePortWithLock
```
private ServerName findServerWithSameHostnamePortWithLock(ServerName serverName)
```
    Assumes onlineServers is locked.
    
    Returns:
    
    ServerName with matching hostname and port.
  - recordNewServerWithLock
```
void recordNewServerWithLock(ServerName serverName,
                             ServerMetrics sl)
```
    Adds the onlineServers list. onlineServers should be locked.
    
    Parameters:
    
    serverName - The remote servers name.
  - getLastFlushedSequenceId
```
public org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionStoreSequenceIds getLastFlushedSequenceId(byte[] encodedRegionName)
```
  - getLoad
```
public ServerMetrics getLoad(ServerName serverName)
```
    Parameters:
    
    serverName -
    
    Returns:
    
    ServerMetrics if serverName is known else null
  - getAverageLoad
```
public double getAverageLoad()
```
    Compute the average load across all region servers. Currently, this uses a very naive computation - just uses the number of regions being served, ignoring stats about number of requests.
    
    Returns:
    
    the average load
  - countOfRegionServers
```
public int countOfRegionServers()
```
    Returns:
    
    the count of active regionservers
  - getOnlineServers
```
public Map<ServerName,ServerMetrics> getOnlineServers()
```
    Returns:
    
    Read-only map of servers to serverinfo
  - getDeadServers
```
public DeadServer getDeadServers()
```
  - areDeadServersInProgress
```
public boolean areDeadServersInProgress()
```
    Checks if any dead servers are currently in progress.
    
    Returns:
    
    true if any RS are being processed as dead, false if not
  - letRegionServersShutdown
```
void letRegionServersShutdown()
```
  - getRegionServersInZK
```
private List<String> getRegionServersInZK(ZKWatcher zkw)
                                   throws org.apache.zookeeper.KeeperException
```
    Throws:
    
    org.apache.zookeeper.KeeperException
  - expire
```
boolean expire(ServerName serverName)
```
    Returns:
    
    True if we should expire serverName
  - expireServer
```
public boolean expireServer(ServerName serverName)
```
    Expire the passed server. Add it to list of dead servers and queue a shutdown processing.
    
    Returns:
    
    True if we queued a ServerCrashProcedure else false if we did not (could happen for many reasons including the fact that its this server that is going down or we already have queued an SCP for this server or SCP processing is currently disabled because we are in startup phase).
  - moveFromOnlineToDeadServers
```
public boolean moveFromOnlineToDeadServers(ServerName sn)
```
    Returns:
    
    Returns true if was online.
  - removeServerFromDrainList
```
public boolean removeServerFromDrainList(ServerName sn)
```
  - addServerToDrainList
```
public boolean addServerToDrainList(ServerName sn)
```
    Add the server to the drain list.
    
    Parameters:
    
    sn -
    
    Returns:
    
    True if the server is added or the server is already on the drain list.
  - newRpcController
```
private HBaseRpcController newRpcController()
```
  - sendRegionWarmup
```
public void sendRegionWarmup(ServerName server,
                             RegionInfo region)
```
    Sends a WARMUP RPC to the specified server to warmup the specified region.
    A region server could reject the close request because it either does not have the specified region or the region is being split.
    
    Parameters:
    
    server - server to warmup a region
    
    region - region to warmup
  - closeRegionSilentlyAndWait
```
public static void closeRegionSilentlyAndWait(ClusterConnection connection,
                                              ServerName server,
                                              RegionInfo region,
                                              long timeout)
                                       throws IOException,
                                              InterruptedException
```
    Contacts a region server and waits up to timeout ms to close the region. This bypasses the active hmaster.
    
    Throws:
    
    IOException
    
    InterruptedException
  - getRsAdmin
```
public org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.AdminService.BlockingInterface getRsAdmin(ServerName sn)
                                                                                                        throws IOException
```
    Parameters:
    
    sn -
    
    Returns:
    
    Admin interface for the remote regionserver named sn
    
    Throws:
    
    IOException
    
    RetriesExhaustedException - wrapping a ConnectException if failed
  - getMinToStart
```
private int getMinToStart()
```
    Calculate min necessary to start. This is not an absolute. It is just a friction that will cause us hang around a bit longer waiting on RegionServers to check-in.
  - waitForRegionServers
```
public void waitForRegionServers(MonitoredTask status)
                          throws InterruptedException
```
    Wait for the region servers to report in. We will wait until one of this condition is met: - the master is stopped - the 'hbase.master.wait.on.regionservers.maxtostart' number of region servers is reached - the 'hbase.master.wait.on.regionservers.mintostart' is reached AND there have been no new region server in for 'hbase.master.wait.on.regionservers.interval' time AND the 'hbase.master.wait.on.regionservers.timeout' is reached
    
    Throws:
    
    InterruptedException
  - getStrForMax
```
private String getStrForMax(int max)
```
  - getOnlineServersList
```
public List<ServerName> getOnlineServersList()
```
    Returns:
    
    A copy of the internal list of online servers.
  - getOnlineServersListWithPredicator
```
public List<ServerName> getOnlineServersListWithPredicator(List<ServerName> keys,
                                                           Predicate<ServerMetrics> idleServerPredicator)
```
    Parameters:
    
    keys - The target server name
    
    idleServerPredicator - Evaluates the server on the given load
    
    Returns:
    
    A copy of the internal list of online servers matched by the predicator
  - getDrainingServersList
```
public List<ServerName> getDrainingServersList()
```
    Returns:
    
    A copy of the internal list of draining servers.
  - isServerOnline
```
public boolean isServerOnline(ServerName serverName)
```
  - isServerDead
```
public boolean isServerDead(ServerName serverName)
```
    Check if a server is known to be dead. A server can be online, or known to be dead, or unknown to this manager (i.e, not online, not known to be dead either. it is simply not tracked by the master any more, for example, a very old previous instance).
  - shutdownCluster
```
public void shutdownCluster()
```
  - isClusterShutdown
```
public boolean isClusterShutdown()
```
  - stop
```
public void stop()
```
    Stop the ServerManager.
  - createDestinationServersList
```
public List<ServerName> createDestinationServersList(List<ServerName> serversToExclude)
```
    Creates a list of possible destinations for a region. It contains the online servers, but not the draining or dying servers.
    
    Parameters:
    
    serversToExclude - can be null if there is no server to exclude
  - createDestinationServersList
```
public List<ServerName> createDestinationServersList()
```
    Calls createDestinationServersList(java.util.List<org.apache.hadoop.hbase.ServerName>) without server to exclude.
  - clearDeadServersWithSameHostNameAndPortOfOnlineServer
```
void clearDeadServersWithSameHostNameAndPortOfOnlineServer()
```
    To clear any dead server with same host name and port of any online server
  - removeRegion
```
public void removeRegion(RegionInfo regionInfo)
```
    Called by delete table and similar to notify the ServerManager that a region was removed.
  - isRegionInServerManagerStates
```
public boolean isRegionInServerManagerStates(RegionInfo hri)
```
  - removeRegions
```
public void removeRegions(List<RegionInfo> regions)
```
    Called by delete table and similar to notify the ServerManager that a region was removed.
  - getServerVersion
```
public int getServerVersion(ServerName serverName)
```
    May return 0 when server is not online.
  - getInfoPort
```
public int getInfoPort(ServerName serverName)
```

Class ServerManager

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

WAIT_ON_REGIONSERVERS_MAXTOSTART

WAIT_ON_REGIONSERVERS_MINTOSTART

WAIT_ON_REGIONSERVERS_TIMEOUT

WAIT_ON_REGIONSERVERS_INTERVAL

LOG

clusterShutdown

flushedSequenceIdByRegion

storeFlushedSequenceIdsByRegion

onlineServers

rsAdmins

drainingServers

master

connection

deadservers

maxSkew

warningSkew

rpcControllerFactory

listeners

Constructor Detail

ServerManager

Method Detail

registerListener

unregisterListener

regionServerStartup

updateLastFlushedSequenceIds

regionServerReport

checkAndRecordNewServer

findOutDeadServersAndProcess

checkClockSkew

checkIsDead

findServerWithSameHostnamePortWithLock

recordNewServerWithLock

getLastFlushedSequenceId

getLoad

getAverageLoad

countOfRegionServers

getOnlineServers

getDeadServers

areDeadServersInProgress

letRegionServersShutdown

getRegionServersInZK

expire

expireServer

moveFromOnlineToDeadServers

removeServerFromDrainList

addServerToDrainList

newRpcController

sendRegionWarmup

closeRegionSilentlyAndWait

getRsAdmin

getMinToStart

waitForRegionServers

getStrForMax

getOnlineServersList

getOnlineServersListWithPredicator

getDrainingServersList

isServerOnline

isServerDead

shutdownCluster

isClusterShutdown

stop

createDestinationServersList

createDestinationServersList

clearDeadServersWithSameHostNameAndPortOfOnlineServer

removeRegion

isRegionInServerManagerStates

removeRegions

getServerVersion

getInfoPort