Class ReplicationPeerManager
java.lang.Object
org.apache.hadoop.hbase.master.replication.ReplicationPeerManager
- All Implemented Interfaces:
ConfigurationObserver
Manages and performs all replication admin operations.
Used to add/remove a replication peer.
Implement ConfigurationObserver
mainly for recreating ReplicationPeerStorage
, for
supporting migrating across different replication peer storages without restarting master.
-
Nested Class Summary
Modifier and TypeClassDescriptionprivate static interface
(package private) static interface
-
Field Summary
Modifier and TypeFieldDescriptionprivate final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableMap<SyncReplicationState,
EnumSet<SyncReplicationState>> private final String
private org.apache.hadoop.conf.Configuration
private final org.apache.hadoop.fs.FileSystem
private static final org.slf4j.Logger
private final ConcurrentMap<String,
ReplicationPeerDescription> private ReplicationPeerStorage
private final ReplicationQueueStorage
private final ZKWatcher
-
Constructor Summary
ConstructorDescriptionReplicationPeerManager
(org.apache.hadoop.fs.FileSystem fs, ZKWatcher zk, ReplicationPeerStorage peerStorage, ReplicationQueueStorage queueStorage, ConcurrentMap<String, ReplicationPeerDescription> peers, org.apache.hadoop.conf.Configuration conf, String clusterId, ReplicationPeerManager.ReplicationQueueStorageInitializer queueStorageInitializer) -
Method Summary
Modifier and TypeMethodDescriptionvoid
addPeer
(String peerId, ReplicationPeerConfig peerConfig, boolean enabled) private void
checkClusterKey
(String clusterKey, ReplicationEndpoint endpoint) private void
checkConfiguredWALEntryFilters
(ReplicationPeerConfig peerConfig) private void
checkNamespacesAndTableCfsConfigConflict
(Set<String> namespaces, Map<TableName, ? extends Collection<String>> tableCfs) Set a namespace in the peer config means that all tables in this namespace will be replicated to the peer cluster.private void
checkPeerConfig
(ReplicationPeerConfig peerConfig) private void
private ReplicationPeerDescription
checkPeerExists
(String peerId) private void
private void
checkQueuesDeleted
(String peerId) private void
private ReplicationQueueData
static ReplicationPeerManager
create
(MasterServices services, String clusterId) private static Pair<ReplicationQueueStorage,
ReplicationPeerManager.ReplicationQueueStorageInitializer> createReplicationQueueStorage
(MasterServices services) void
disablePeer
(String peerId) void
enablePeer
(String peerId) getPeerConfig
(String peerId) boolean
getPeerState
(String peerId) getSerialPeerIdsBelongsTo
(TableName tableName) private void
private boolean
isStringEquals
(String s1, String s2) For replication peer cluster key or endpoint class, null and empty string is same.private void
migrateHFileRefs
(ZKReplicationQueueStorageForMigration oldQueueStorage) private void
migrateLastPushedSeqIds
(ZKReplicationQueueStorageForMigration oldQueueStorage) private void
migrateQueues
(ZKReplicationQueueStorageForMigration oldQueueStorage) (package private) CompletableFuture<Void>
migrateQueuesFromZk
(ZKWatcher zookeeper, ExecutorService executor) Submit the migration tasks to the givenexecutor
.void
onConfigurationChange
(org.apache.hadoop.conf.Configuration conf) This method would be called by theConfigurationManager
object when theConfiguration
object is reloaded from disk.(package private) void
preAddPeer
(String peerId, ReplicationPeerConfig peerConfig) (package private) void
preDisablePeer
(String peerId) (package private) void
preEnablePeer
(String peerId) (package private) ReplicationPeerConfig
preRemovePeer
(String peerId) (package private) ReplicationPeerDescription
preTransitPeerSyncReplicationState
(String peerId, SyncReplicationState state) Returns the old desciption of the peer(package private) ReplicationPeerDescription
preUpdatePeerConfig
(String peerId, ReplicationPeerConfig peerConfig) Return the old peer description.(package private) void
removeAllLastPushedSeqIds
(String peerId) void
removeAllQueues
(String peerId) void
removeAllQueuesAndHFileRefs
(String peerId) void
removePeer
(String peerId) private CompletableFuture<?>
runAsync
(ReplicationPeerManager.ExceptionalRunnable task, ExecutorService executor) void
setPeerNewSyncReplicationState
(String peerId, SyncReplicationState state) private void
setPeerState
(String peerId, boolean enabled) void
transitPeerSyncReplicationState
(String peerId, SyncReplicationState newState) void
updatePeerConfig
(String peerId, ReplicationPeerConfig peerConfig)
-
Field Details
-
LOG
-
peerStorage
-
queueStorage
-
peers
-
allowedTransition
private final org.apache.hbase.thirdparty.com.google.common.collect.ImmutableMap<SyncReplicationState,EnumSet<SyncReplicationState>> allowedTransition -
clusterId
-
conf
-
fs
-
zk
-
queueStorageInitializer
-
-
Constructor Details
-
ReplicationPeerManager
ReplicationPeerManager(org.apache.hadoop.fs.FileSystem fs, ZKWatcher zk, ReplicationPeerStorage peerStorage, ReplicationQueueStorage queueStorage, ConcurrentMap<String, ReplicationPeerDescription> peers, org.apache.hadoop.conf.Configuration conf, String clusterId, ReplicationPeerManager.ReplicationQueueStorageInitializer queueStorageInitializer)
-
-
Method Details
-
checkQueuesDeleted
-
initializeQueueStorage
- Throws:
IOException
-
preAddPeer
void preAddPeer(String peerId, ReplicationPeerConfig peerConfig) throws ReplicationException, IOException - Throws:
ReplicationException
IOException
-
checkPeerExists
- Throws:
DoNotRetryIOException
-
checkPeerInDAStateIfSyncReplication
- Throws:
DoNotRetryIOException
-
preRemovePeer
- Throws:
DoNotRetryIOException
-
preEnablePeer
- Throws:
DoNotRetryIOException
-
preDisablePeer
- Throws:
DoNotRetryIOException
-
preUpdatePeerConfig
ReplicationPeerDescription preUpdatePeerConfig(String peerId, ReplicationPeerConfig peerConfig) throws DoNotRetryIOException Return the old peer description. Can never be null.- Throws:
DoNotRetryIOException
-
preTransitPeerSyncReplicationState
ReplicationPeerDescription preTransitPeerSyncReplicationState(String peerId, SyncReplicationState state) throws DoNotRetryIOException Returns the old desciption of the peer- Throws:
DoNotRetryIOException
-
addPeer
public void addPeer(String peerId, ReplicationPeerConfig peerConfig, boolean enabled) throws ReplicationException - Throws:
ReplicationException
-
removePeer
- Throws:
ReplicationException
-
setPeerState
- Throws:
ReplicationException
-
getPeerState
- Throws:
ReplicationException
-
enablePeer
- Throws:
ReplicationException
-
disablePeer
- Throws:
ReplicationException
-
updatePeerConfig
public void updatePeerConfig(String peerId, ReplicationPeerConfig peerConfig) throws ReplicationException - Throws:
ReplicationException
-
listPeers
-
getPeerConfig
-
removeAllLastPushedSeqIds
- Throws:
ReplicationException
-
setPeerNewSyncReplicationState
public void setPeerNewSyncReplicationState(String peerId, SyncReplicationState state) throws ReplicationException - Throws:
ReplicationException
-
transitPeerSyncReplicationState
public void transitPeerSyncReplicationState(String peerId, SyncReplicationState newState) throws ReplicationException - Throws:
ReplicationException
-
removeAllQueues
- Throws:
ReplicationException
-
removeAllQueuesAndHFileRefs
- Throws:
ReplicationException
-
checkClusterKey
private void checkClusterKey(String clusterKey, ReplicationEndpoint endpoint) throws DoNotRetryIOException - Throws:
DoNotRetryIOException
-
checkPeerConfig
- Throws:
DoNotRetryIOException
-
checkPeerConfigForSyncReplication
private void checkPeerConfigForSyncReplication(ReplicationPeerConfig peerConfig) throws DoNotRetryIOException - Throws:
DoNotRetryIOException
-
checkSyncReplicationPeerConfigConflict
private void checkSyncReplicationPeerConfigConflict(ReplicationPeerConfig peerConfig) throws DoNotRetryIOException - Throws:
DoNotRetryIOException
-
checkNamespacesAndTableCfsConfigConflict
private void checkNamespacesAndTableCfsConfigConflict(Set<String> namespaces, Map<TableName, ? extends Collection<String>> tableCfs) throws DoNotRetryIOExceptionSet a namespace in the peer config means that all tables in this namespace will be replicated to the peer cluster.- If peer config already has a namespace, then not allow set any table of this namespace to the peer config.
- If peer config already has a table, then not allow set this table's namespace to the peer config.
Set a exclude namespace in the peer config means that all tables in this namespace can't be replicated to the peer cluster.
- If peer config already has a exclude namespace, then not allow set any exclude table of this namespace to the peer config.
- If peer config already has a exclude table, then not allow set this table's namespace as a exclude namespace.
- Throws:
DoNotRetryIOException
-
checkConfiguredWALEntryFilters
private void checkConfiguredWALEntryFilters(ReplicationPeerConfig peerConfig) throws DoNotRetryIOException - Throws:
DoNotRetryIOException
-
getSerialPeerIdsBelongsTo
-
getPeerStorage
-
getQueueStorage
-
createReplicationQueueStorage
private static Pair<ReplicationQueueStorage,ReplicationPeerManager.ReplicationQueueStorageInitializer> createReplicationQueueStorage(MasterServices services) throws IOException - Throws:
IOException
-
create
public static ReplicationPeerManager create(MasterServices services, String clusterId) throws ReplicationException, IOException - Throws:
ReplicationException
IOException
-
isStringEquals
For replication peer cluster key or endpoint class, null and empty string is same. So here don't useStringUtils.equals(CharSequence, CharSequence)
directly. -
onConfigurationChange
Description copied from interface:ConfigurationObserver
This method would be called by theConfigurationManager
object when theConfiguration
object is reloaded from disk.- Specified by:
onConfigurationChange
in interfaceConfigurationObserver
-
convert
-
migrateQueues
- Throws:
Exception
-
migrateLastPushedSeqIds
private void migrateLastPushedSeqIds(ZKReplicationQueueStorageForMigration oldQueueStorage) throws Exception - Throws:
Exception
-
migrateHFileRefs
private void migrateHFileRefs(ZKReplicationQueueStorageForMigration oldQueueStorage) throws Exception - Throws:
Exception
-
runAsync
private CompletableFuture<?> runAsync(ReplicationPeerManager.ExceptionalRunnable task, ExecutorService executor) -
migrateQueuesFromZk
Submit the migration tasks to the givenexecutor
.
-