Package org.apache.hadoop.hbase.replication

Class FSReplicationPeerStorage

java.lang.Object

org.apache.hadoop.hbase.replication.FSReplicationPeerStorage

All Implemented Interfaces:

ReplicationPeerStorage

@Private
public class FSReplicationPeerStorage
extends Object
implements ReplicationPeerStorage

A filesystem based replication peer storage. The implementation does not require atomic rename so you can use it on cloud OSS.

FileSystem layout:

 hbase
   |
   --peers
       |
       --<peer_id>
           |
           --peer_config
           |
           --disabled
           |
           --sync-rep-state
 

Notice that, if the peer is enabled, we will not have a disabled file.

And for other files, to avoid depending on atomic rename, we will use two files for storing the content. When loading, we will try to read both the files and load the newer one. And when writing, we will write to the older file.

Field Summary

Fields

Modifier and Type	Field	Description
private final org.apache.hadoop.fs.Path	dir
(package private) static final String	DISABLED_FILE
private final org.apache.hadoop.fs.FileSystem	fs
private static final org.slf4j.Logger	LOG
private static final long	MAX_FILE_SIZE
(package private) static final byte[]	NONE_STATE_BYTES
(package private) static final String	PEER_CONFIG_FILE
static final String	PEERS_DIR
static final String	PEERS_DIR_DEFAULT
(package private) static final String	SYNC_REPLICATION_STATE_FILE

Constructor Summary

Constructors

Constructor	Description
FSReplicationPeerStorage(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method	Description
void	addPeer(String peerId, ReplicationPeerConfig peerConfig, boolean enabled, SyncReplicationState syncReplicationState)	Add a replication peer.
private static void	delete(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, String name)
ReplicationPeerConfig	getPeerConfig(String peerId)	Get the peer config of a replication peer.
(package private) org.apache.hadoop.fs.Path	getPeerDir(String peerId)
SyncReplicationState	getPeerNewSyncReplicationState(String peerId)	Get the new sync replication state.
SyncReplicationState	getPeerSyncReplicationState(String peerId)	Get the sync replication state.
private Pair<SyncReplicationState,SyncReplicationState>	getStateAndNewState(String peerId)
boolean	isPeerEnabled(String peerId)	Test whether a replication peer is enabled.
List<String>	listPeerIds()	Return the peer ids of all replication peers.
private static byte[]	read(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, String name)
void	removePeer(String peerId)	Remove a replication peer.
void	setPeerNewSyncReplicationState(String peerId, SyncReplicationState newState)	Set the new sync replication state that we are going to transit to.
void	setPeerState(String peerId, boolean enabled)	Set the state of peer, true to ENABLED, otherwise to DISABLED.
void	transitPeerSyncReplicationState(String peerId)	Overwrite the sync replication state with the new sync replication state which is set with the ReplicationPeerStorage.setPeerNewSyncReplicationState(String, SyncReplicationState) method above, and clear the new sync replication state.
void	updatePeerConfig(String peerId, ReplicationPeerConfig peerConfig)	Update the config a replication peer.
private static void	write(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, String name, byte[] data)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

PEERS_DIR

public static final String PEERS_DIR

See Also:

• Constant Field Values

PEERS_DIR_DEFAULT

public static final String PEERS_DIR_DEFAULT

See Also:

• Constant Field Values

PEER_CONFIG_FILE

static final String PEER_CONFIG_FILE

See Also:

• Constant Field Values

DISABLED_FILE

static final String DISABLED_FILE

See Also:

• Constant Field Values

SYNC_REPLICATION_STATE_FILE

static final String SYNC_REPLICATION_STATE_FILE

See Also:

• Constant Field Values

NONE_STATE_BYTES

static final byte[] NONE_STATE_BYTES

fs

private final org.apache.hadoop.fs.FileSystem fs

dir

private final org.apache.hadoop.fs.Path dir

MAX_FILE_SIZE

private static final long MAX_FILE_SIZE

See Also:

• Constant Field Values

Constructor Details

FSReplicationPeerStorage

public FSReplicationPeerStorage(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.conf.Configuration conf)
                         throws IOException

Throws:

IOException

Method Details

getPeerDir

org.apache.hadoop.fs.Path getPeerDir(String peerId)

addPeer

public void addPeer(String peerId,
 ReplicationPeerConfig peerConfig,
 boolean enabled,
 SyncReplicationState syncReplicationState)
             throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Add a replication peer.

Specified by:

addPeer in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

removePeer

public void removePeer(String peerId)
                throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Remove a replication peer.

Specified by:

removePeer in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

setPeerState

public void setPeerState(String peerId,
 boolean enabled)
                  throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Set the state of peer, true to ENABLED, otherwise to DISABLED.

Specified by:

setPeerState in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

updatePeerConfig

public void updatePeerConfig(String peerId,
 ReplicationPeerConfig peerConfig)
                      throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Update the config a replication peer.

Specified by:

updatePeerConfig in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

listPeerIds

public List<String> listPeerIds()
                         throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Return the peer ids of all replication peers.

Specified by:

listPeerIds in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

isPeerEnabled

public boolean isPeerEnabled(String peerId)
                      throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Test whether a replication peer is enabled.

Specified by:

isPeerEnabled in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

getPeerConfig

public ReplicationPeerConfig getPeerConfig(String peerId)
                                    throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Get the peer config of a replication peer.

Specified by:

getPeerConfig in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

getStateAndNewState

private Pair<SyncReplicationState,SyncReplicationState> getStateAndNewState(String peerId)
                                                                     throws IOException

Throws:

IOException

setPeerNewSyncReplicationState

public void setPeerNewSyncReplicationState(String peerId,
 SyncReplicationState newState)
                                    throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Set the new sync replication state that we are going to transit to.

Specified by:

setPeerNewSyncReplicationState in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

transitPeerSyncReplicationState

public void transitPeerSyncReplicationState(String peerId)
                                     throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Overwrite the sync replication state with the new sync replication state which is set with the ReplicationPeerStorage.setPeerNewSyncReplicationState(String, SyncReplicationState) method above, and clear the new sync replication state.

Specified by:

transitPeerSyncReplicationState in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

getPeerSyncReplicationState

public SyncReplicationState getPeerSyncReplicationState(String peerId)
                                                 throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Get the sync replication state.

Specified by:

getPeerSyncReplicationState in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

getPeerNewSyncReplicationState

public SyncReplicationState getPeerNewSyncReplicationState(String peerId)
                                                    throws ReplicationException

Description copied from interface: ReplicationPeerStorage

Get the new sync replication state. Will return SyncReplicationState.NONE if we are not in a transition.

Specified by:

getPeerNewSyncReplicationState in interface ReplicationPeerStorage

Throws:

ReplicationException - if there are errors accessing the storage service.

read

private static byte[] read(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path dir,
 String name)
                    throws IOException

Throws:

IOException

write

private static void write(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path dir,
 String name,
 byte[] data)
                   throws IOException

Throws:

IOException

delete

private static void delete(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path dir,
 String name)
                    throws IOException

Throws:

IOException