org.apache.hadoop.hbase.rsgroup

Class RSGroupInfoManagerImpl

java.lang.Object

org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl

All Implemented Interfaces:

RSGroupInfoManager

@InterfaceAudience.Private
final class RSGroupInfoManagerImpl
extends Object
implements RSGroupInfoManager

This is an implementation of RSGroupInfoManager which makes use of an HBase table as the persistence store for the group information. It also makes use of zookeeper to store group information needed for bootstrapping during offline mode.

Concurrency

RSGroup state is kept locally in Maps. There is a rsgroup name to cached RSGroupInfo Map at rsGroupMap and a Map of tables to the name of the rsgroup they belong too (in tableMap). These Maps are persisted to the hbase:rsgroup table (and cached in zk) on each modification.

Mutations on state are synchronized but reads can continue without having to wait on an instance monitor, mutations do wholesale replace of the Maps on update -- Copy-On-Write; the local Maps of state are read-only, just-in-case (see flushConfig).

Reads must not block else there is a danger we'll deadlock.

Clients of this class, the RSGroupAdminEndpoint for example, want to query and then act on the results of the query modifying cache in zookeeper without another thread making intermediate modifications. These clients synchronize on the 'this' instance so no other has access concurrently. Reads must be able to continue concurrently.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	RSGroupInfoManagerImpl.RSGroupMappingScript
private class	RSGroupInfoManagerImpl.RSGroupStartupWorker
private class	RSGroupInfoManagerImpl.ServerEventsListenerThread Calls updateDefaultServers(SortedSet) to update list of known servers.

Field Summary

Fields

Modifier and Type	Field and Description
private Connection	conn
private static org.slf4j.Logger	LOG
private MasterServices	masterServices
private Set<String>	prevRSGroups
private static TableDescriptor	RSGROUP_TABLE_DESC Table descriptor for hbase:rsgroup catalog table
private Map<String,RSGroupInfo>	rsGroupMap
private RSGroupInfoManagerImpl.RSGroupStartupWorker	rsGroupStartupWorker
(package private) RSGroupInfoManagerImpl.RSGroupMappingScript	script Get rsgroup table mapping script
private RSGroupInfoManagerImpl.ServerEventsListenerThread	serverEventsListenerThread
private Map<TableName,String>	tableMap
private ZKWatcher	watcher

Fields inherited from interface org.apache.hadoop.hbase.rsgroup.RSGroupInfoManager

DEFAULT_REASSIGN_WAIT_INTERVAL, META_FAMILY_BYTES, META_QUALIFIER_BYTES, REASSIGN_WAIT_INTERVAL_KEY, ROW_KEY, RSGROUP_TABLE_NAME, rsGroupZNode

Constructor Summary

Constructors

Modifier	Constructor and Description
private	RSGroupInfoManagerImpl(MasterServices masterServices)

Method Summary

Modifier and Type	Method and Description
void	addRSGroup(RSGroupInfo rsGroupInfo) Add given RSGroupInfo to existing list of group infos.
private void	checkGroupName(String groupName)
RSGroupInfo	determineRSGroupInfoForTable(TableName tableName) Will try to get the rsgroup from tableMap first then try to get the rsgroup from script try to get the rsgroup from the NamespaceDescriptor lastly.
private void	flushConfig()
private void	flushConfig(Map<String,RSGroupInfo> newGroupMap)
private Map<TableName,String>	flushConfigTable(Map<String,RSGroupInfo> groupMap)
private SortedSet<Address>	getDefaultServers()
private SortedSet<Address>	getDefaultServers(List<RSGroupInfo> rsGroupInfoList)
(package private) static RSGroupInfoManager	getInstance(MasterServices master)
private List<ServerName>	getOnlineRS()
private static Set<Address>	getOnlineServers(MasterServices master)
RSGroupInfo	getRSGroup(String groupName) Gets RSGroupInfo for the given group name.
private RSGroupInfo	getRSGroupInfo(String groupName)
RSGroupInfo	getRSGroupOfServer(Address serverHostPort) Gets the group info of server.
String	getRSGroupOfTable(TableName tableName) Get the group membership of a table
private void	init()
private static boolean	isMasterRunning(MasterServices masterServices)
boolean	isOnline() Whether the manager is able to fully return group metadata
List<RSGroupInfo>	listRSGroups() List the existing RSGroupInfos.
Set<Address>	moveServers(Set<Address> servers, String srcGroup, String dstGroup) Move servers to a new group.
void	moveServersAndTables(Set<Address> servers, Set<TableName> tables, String srcGroup, String dstGroup) Move servers and tables to a new group.
void	moveTables(Set<TableName> tableNames, String groupName) Set the group membership of a set of tables
private void	multiMutate(List<Mutation> mutations)
void	refresh() Refresh/reload the group information from the persistent store
private void	refresh(boolean forceOnline) Read rsgroup info from the source of truth, the hbase:rsgroup table.
void	removeRSGroup(String groupName) Remove a region server group.
void	removeServers(Set<Address> servers) Remove decommissioned servers from rsgroup
void	renameRSGroup(String oldName, String newName) Rename RSGroup
private void	resetRSGroupAndTableMaps(Map<String,RSGroupInfo> newRSGroupMap, Map<TableName,String> newTableMap) Make changes visible.
(package private) List<RSGroupInfo>	retrieveGroupListFromGroupTable()
(package private) List<RSGroupInfo>	retrieveGroupListFromZookeeper()
void	start()
private void	updateCacheOfRSGroups(Set<String> currentGroups) Update cache of rsgroups.
private void	updateDefaultServers(SortedSet<Address> servers)
void	updateRSGroupConfig(String groupName, Map<String,String> configuration) Update RSGroup configuration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

RSGROUP_TABLE_DESC

private static final TableDescriptor RSGROUP_TABLE_DESC

Table descriptor for hbase:rsgroup catalog table

rsGroupMap

private volatile Map<String,RSGroupInfo> rsGroupMap

tableMap

private volatile Map<TableName,String> tableMap

masterServices

private final MasterServices masterServices

conn

private final Connection conn

watcher

private final ZKWatcher watcher

rsGroupStartupWorker

private final RSGroupInfoManagerImpl.RSGroupStartupWorker rsGroupStartupWorker

prevRSGroups

private Set<String> prevRSGroups

serverEventsListenerThread

private final RSGroupInfoManagerImpl.ServerEventsListenerThread serverEventsListenerThread

script

RSGroupInfoManagerImpl.RSGroupMappingScript script

Get rsgroup table mapping script

Constructor Detail

RSGroupInfoManagerImpl

private RSGroupInfoManagerImpl(MasterServices masterServices)
                        throws IOException

Throws:

IOException

Method Detail

init

private void init()
           throws IOException

Throws:

IOException

getInstance

static RSGroupInfoManager getInstance(MasterServices master)
                               throws IOException

Throws:

IOException

start

public void start()

Specified by:

start in interface RSGroupInfoManager

addRSGroup

public void addRSGroup(RSGroupInfo rsGroupInfo)
                throws IOException

Description copied from interface: RSGroupInfoManager

Add given RSGroupInfo to existing list of group infos.

Specified by:

addRSGroup in interface RSGroupInfoManager

Throws:

IOException

getRSGroupInfo

private RSGroupInfo getRSGroupInfo(String groupName)
                            throws DoNotRetryIOException

Throws:

DoNotRetryIOException

getOnlineServers

private static Set<Address> getOnlineServers(MasterServices master)

Parameters:

master - the master to get online servers for

Returns:

Set of online Servers named for their hostname and port (not ServerName).

moveServers

public Set<Address> moveServers(Set<Address> servers,
                                String srcGroup,
                                String dstGroup)
                         throws IOException

Description copied from interface: RSGroupInfoManager

Move servers to a new group.

Specified by:

moveServers in interface RSGroupInfoManager

Parameters:

servers - list of servers, must be part of the same group

srcGroup - groupName being moved from

dstGroup - groupName being moved to

Returns:

Set of servers moved (May be a subset of servers).

Throws:

IOException

getRSGroupOfServer

public RSGroupInfo getRSGroupOfServer(Address serverHostPort)
                               throws IOException

Description copied from interface: RSGroupInfoManager

Gets the group info of server.

Specified by:

getRSGroupOfServer in interface RSGroupInfoManager

Throws:

IOException

getRSGroup

public RSGroupInfo getRSGroup(String groupName)

Description copied from interface: RSGroupInfoManager

Gets RSGroupInfo for the given group name.

Specified by:

getRSGroup in interface RSGroupInfoManager

getRSGroupOfTable

public String getRSGroupOfTable(TableName tableName)

Description copied from interface: RSGroupInfoManager

Get the group membership of a table

Specified by:

getRSGroupOfTable in interface RSGroupInfoManager

moveTables

public void moveTables(Set<TableName> tableNames,
                       String groupName)
                throws IOException

Description copied from interface: RSGroupInfoManager

Set the group membership of a set of tables

Specified by:

moveTables in interface RSGroupInfoManager

Parameters:

tableNames - set of tables to move

groupName - name of group of tables to move to

Throws:

IOException

removeRSGroup

public void removeRSGroup(String groupName)
                   throws IOException

Description copied from interface: RSGroupInfoManager

Remove a region server group.

Specified by:

removeRSGroup in interface RSGroupInfoManager

Throws:

IOException

listRSGroups

public List<RSGroupInfo> listRSGroups()

Description copied from interface: RSGroupInfoManager

List the existing RSGroupInfos.

Specified by:

listRSGroups in interface RSGroupInfoManager

isOnline

public boolean isOnline()

Description copied from interface: RSGroupInfoManager

Whether the manager is able to fully return group metadata

Specified by:

isOnline in interface RSGroupInfoManager

Returns:

whether the manager is in online mode

moveServersAndTables

public void moveServersAndTables(Set<Address> servers,
                                 Set<TableName> tables,
                                 String srcGroup,
                                 String dstGroup)
                          throws IOException

Description copied from interface: RSGroupInfoManager

Move servers and tables to a new group.

Specified by:

moveServersAndTables in interface RSGroupInfoManager

Parameters:

servers - list of servers, must be part of the same group

tables - set of tables to move

srcGroup - groupName being moved from

dstGroup - groupName being moved to

Throws:

IOException

removeServers

public void removeServers(Set<Address> servers)
                   throws IOException

Description copied from interface: RSGroupInfoManager

Remove decommissioned servers from rsgroup

Specified by:

removeServers in interface RSGroupInfoManager

Parameters:

servers - set of servers to remove

Throws:

IOException

renameRSGroup

public void renameRSGroup(String oldName,
                          String newName)
                   throws IOException

Description copied from interface: RSGroupInfoManager

Rename RSGroup

Specified by:

renameRSGroup in interface RSGroupInfoManager

Parameters:

oldName - old rsgroup name

newName - new rsgroup name

Throws:

IOException

determineRSGroupInfoForTable

public RSGroupInfo determineRSGroupInfoForTable(TableName tableName)
                                         throws IOException

Will try to get the rsgroup from tableMap first then try to get the rsgroup from script try to get the rsgroup from the NamespaceDescriptor lastly. If still not present, return default group.

Specified by:

determineRSGroupInfoForTable in interface RSGroupInfoManager

Parameters:

tableName - table name

Returns:

RSGroupInfo which table should belong to

Throws:

IOException

updateRSGroupConfig

public void updateRSGroupConfig(String groupName,
                                Map<String,String> configuration)
                         throws IOException

Description copied from interface: RSGroupInfoManager

Update RSGroup configuration

Specified by:

updateRSGroupConfig in interface RSGroupInfoManager

Parameters:

groupName - the group name

configuration - new configuration of the group name to be set

Throws:

IOException - if a remote or network exception occurs

retrieveGroupListFromGroupTable

List<RSGroupInfo> retrieveGroupListFromGroupTable()
                                           throws IOException

Throws:

IOException

retrieveGroupListFromZookeeper

List<RSGroupInfo> retrieveGroupListFromZookeeper()
                                          throws IOException

Throws:

IOException

refresh

public void refresh()
             throws IOException

Description copied from interface: RSGroupInfoManager

Refresh/reload the group information from the persistent store

Specified by:

refresh in interface RSGroupInfoManager

Throws:

IOException

refresh

private void refresh(boolean forceOnline)
              throws IOException

Read rsgroup info from the source of truth, the hbase:rsgroup table. Update zk cache. Called on startup of the manager.

Throws:

IOException

flushConfigTable

private Map<TableName,String> flushConfigTable(Map<String,RSGroupInfo> groupMap)
                                        throws IOException

Throws:

IOException

flushConfig

private void flushConfig()
                  throws IOException

Throws:

IOException

flushConfig

private void flushConfig(Map<String,RSGroupInfo> newGroupMap)
                  throws IOException

Throws:

IOException

resetRSGroupAndTableMaps

private void resetRSGroupAndTableMaps(Map<String,RSGroupInfo> newRSGroupMap,
                                      Map<TableName,String> newTableMap)

Make changes visible. Caller must be synchronized on 'this'.

updateCacheOfRSGroups

private void updateCacheOfRSGroups(Set<String> currentGroups)

Update cache of rsgroups. Caller must be synchronized on 'this'.

Parameters:

currentGroups - Current list of Groups.

getOnlineRS

private List<ServerName> getOnlineRS()
                              throws IOException

Throws:

IOException

getDefaultServers

private SortedSet<Address> getDefaultServers()
                                      throws IOException

Throws:

IOException

getDefaultServers

private SortedSet<Address> getDefaultServers(List<RSGroupInfo> rsGroupInfoList)
                                      throws IOException

Throws:

IOException

updateDefaultServers

private void updateDefaultServers(SortedSet<Address> servers)
                           throws IOException

Throws:

IOException

isMasterRunning

private static boolean isMasterRunning(MasterServices masterServices)

multiMutate

private void multiMutate(List<Mutation> mutations)
                  throws IOException

Throws:

IOException

checkGroupName

private void checkGroupName(String groupName)
                     throws ConstraintException

Throws:

ConstraintException