@InterfaceAudience.Private public class SyncReplicationReplayWALManager extends Object
UsedReplayWorkersForPeer
for more details.
Second, the logic for managing the remote wal directory is kept here. Before replaying the wals,
we will rename the remote wal directory, the new name is called 'replay' directory, see
renameToPeerReplayWALDir(String)
. This is used to prevent further writing of remote
wals, which is very important for keeping consistency. And then we will start replaying all the
wals, once a wal has been replayed, we will truncate the file, so that if there are crashes
happen, we do not need to replay all the wals again, see finishReplayWAL(String)
and
isReplayWALFinished(String)
. After replaying all the wals, we will rename the 'replay'
directory, the new name is called 'snapshot' directory. In the directory, we will keep all the
names for the wals being replayed, since all the files should have been truncated. When we
transitting original the ACTIVE cluster to STANDBY later, and there are region server crashes, we
will see the wals in this directory to determine whether a wal should be split and replayed or
not. You can see the code in SplitLogWorker
for more
details.Modifier and Type | Class and Description |
---|---|
private static class |
SyncReplicationReplayWALManager.UsedReplayWorkersForPeer
This class is used to record the used workers(region servers) for a replication peer.
|
Modifier and Type | Field and Description |
---|---|
private org.apache.hadoop.fs.FileSystem |
fs |
private static org.slf4j.Logger |
LOG |
private org.apache.hadoop.fs.Path |
remoteWALDir |
private ServerManager |
serverManager |
private ConcurrentMap<String,SyncReplicationReplayWALManager.UsedReplayWorkersForPeer> |
usedWorkersByPeer |
private org.apache.hadoop.fs.Path |
walRootDir |
Constructor and Description |
---|
SyncReplicationReplayWALManager(MasterServices services) |
Modifier and Type | Method and Description |
---|---|
ServerName |
acquirePeerWorker(String peerId,
Procedure<?> proc)
Get a worker for replaying remote wal for a give peer.
|
void |
addUsedPeerWorker(String peerId,
ServerName worker)
Will only be called when loading procedures, where we need to construct the used worker set for
each peer.
|
void |
createPeerRemoteWALDir(String peerId) |
private void |
deleteDir(org.apache.hadoop.fs.Path dir,
String peerId) |
void |
finishReplayWAL(String wal) |
org.apache.hadoop.fs.Path |
getRemoteWALDir() |
List<org.apache.hadoop.fs.Path> |
getReplayWALsAndCleanUpUnusedFiles(String peerId) |
boolean |
isReplayWALFinished(String wal) |
void |
registerPeer(String peerId) |
void |
releasePeerWorker(String peerId,
ServerName worker,
MasterProcedureScheduler scheduler) |
void |
removePeerRemoteWALs(String peerId) |
String |
removeWALRootPath(org.apache.hadoop.fs.Path path) |
private void |
rename(org.apache.hadoop.fs.Path src,
org.apache.hadoop.fs.Path dst,
String peerId) |
void |
renameToPeerReplayWALDir(String peerId) |
void |
renameToPeerSnapshotWALDir(String peerId) |
void |
unregisterPeer(String peerId) |
private static final org.slf4j.Logger LOG
private final ServerManager serverManager
private final org.apache.hadoop.fs.FileSystem fs
private final org.apache.hadoop.fs.Path walRootDir
private final org.apache.hadoop.fs.Path remoteWALDir
private final ConcurrentMap<String,SyncReplicationReplayWALManager.UsedReplayWorkersForPeer> usedWorkersByPeer
public SyncReplicationReplayWALManager(MasterServices services) throws IOException, ReplicationException
IOException
ReplicationException
public void registerPeer(String peerId)
public void unregisterPeer(String peerId)
public ServerName acquirePeerWorker(String peerId, Procedure<?> proc) throws ProcedureSuspendedException
ProcedureSuspendedException
will be thrown
to suspend the procedure. And it will be woken up later when there are available workers,
either by others release a worker, or there is a new region server joins the cluster.ProcedureSuspendedException
public void releasePeerWorker(String peerId, ServerName worker, MasterProcedureScheduler scheduler)
public void addUsedPeerWorker(String peerId, ServerName worker)
public void createPeerRemoteWALDir(String peerId) throws IOException
IOException
private void rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, String peerId) throws IOException
IOException
public void renameToPeerReplayWALDir(String peerId) throws IOException
IOException
public void renameToPeerSnapshotWALDir(String peerId) throws IOException
IOException
public List<org.apache.hadoop.fs.Path> getReplayWALsAndCleanUpUnusedFiles(String peerId) throws IOException
IOException
private void deleteDir(org.apache.hadoop.fs.Path dir, String peerId) throws IOException
IOException
public void removePeerRemoteWALs(String peerId) throws IOException
IOException
public String removeWALRootPath(org.apache.hadoop.fs.Path path)
public void finishReplayWAL(String wal) throws IOException
IOException
public boolean isReplayWALFinished(String wal) throws IOException
IOException
public org.apache.hadoop.fs.Path getRemoteWALDir()
Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.