Package org.apache.hadoop.hbase.replication.regionserver

Class ReplicationSourceShipper

java.lang.Object

java.lang.Thread

org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper

All Implemented Interfaces:

Runnable

Direct Known Subclasses:

RecoveredReplicationSourceShipper

@Private
public class ReplicationSourceShipper
extends Thread

This thread reads entries from a queue and ships them. Entries are placed onto the queue by ReplicationSourceWALReaderThread

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	ReplicationSourceShipper.WorkerState

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields

Modifier and Type	Field	Description
private final org.apache.hadoop.conf.Configuration	conf
private org.apache.hadoop.fs.Path	currentPath
private long	currentPosition
private final int	DEFAULT_TIMEOUT
(package private) final ReplicationSourceWALReader	entryReader
private final int	getEntriesTimeout
private static final org.slf4j.Logger	LOG
private final int	maxRetriesMultiplier
private final int	shipEditsTimeout
private final long	sleepForRetries
private final ReplicationSource	source
private ReplicationSourceShipper.WorkerState	state
(package private) final String	walGroupId

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor	Description
ReplicationSourceShipper(org.apache.hadoop.conf.Configuration conf, String walGroupId, ReplicationSource source, ReplicationSourceWALReader walReader)

Method Summary

Modifier and Type	Method	Description
private void	cleanUpHFileRefs(WALEdit edit)
(package private) void	clearWALEntryBatch()	Attempts to properly update ReplicationSourceManager.totalBufferUser, in case there were unprocessed entries batched by the reader to the shipper, but the shipper didn't manage to ship those because the replication source is being terminated.
(package private) org.apache.hadoop.fs.Path	getCurrentPath()
(package private) long	getCurrentPosition()
protected boolean	isActive()
boolean	isFinished()
private void	noMoreData()
protected void	postFinish()
final void	run()
protected final void	setWorkerState(ReplicationSourceShipper.WorkerState state)
private void	shipEdits(WALEntryBatch entryBatch)	Do the shipping logic
void	startup(Thread.UncaughtExceptionHandler handler)
(package private) void	stopWorker()
private boolean	updateLogPosition(WALEntryBatch batch)

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

LOG

private static final org.slf4j.Logger LOG

conf

private final org.apache.hadoop.conf.Configuration conf

walGroupId

final String walGroupId

source

private final ReplicationSource source

currentPosition

private volatile long currentPosition

currentPath

private org.apache.hadoop.fs.Path currentPath

state

private volatile ReplicationSourceShipper.WorkerState state

entryReader

final ReplicationSourceWALReader entryReader

sleepForRetries

private final long sleepForRetries

maxRetriesMultiplier

private final int maxRetriesMultiplier

DEFAULT_TIMEOUT

private final int DEFAULT_TIMEOUT

See Also:

• Constant Field Values

getEntriesTimeout

private final int getEntriesTimeout

shipEditsTimeout

private final int shipEditsTimeout

Constructor Details

ReplicationSourceShipper

public ReplicationSourceShipper(org.apache.hadoop.conf.Configuration conf,
 String walGroupId,
 ReplicationSource source,
 ReplicationSourceWALReader walReader)

Method Details

run

public final void run()

Specified by:

run in interface Runnable

Overrides:

run in class Thread

noMoreData

private void noMoreData()

postFinish

protected void postFinish()

shipEdits

private void shipEdits(WALEntryBatch entryBatch)

Do the shipping logic

cleanUpHFileRefs

private void cleanUpHFileRefs(WALEdit edit)
                       throws IOException

Throws:

IOException

updateLogPosition

private boolean updateLogPosition(WALEntryBatch batch)

startup

public void startup(Thread.UncaughtExceptionHandler handler)

getCurrentPath

org.apache.hadoop.fs.Path getCurrentPath()

getCurrentPosition

long getCurrentPosition()

isActive

protected boolean isActive()

setWorkerState

protected final void setWorkerState(ReplicationSourceShipper.WorkerState state)

stopWorker

void stopWorker()

isFinished

public boolean isFinished()

clearWALEntryBatch

void clearWALEntryBatch()

Attempts to properly update ReplicationSourceManager.totalBufferUser, in case there were unprocessed entries batched by the reader to the shipper, but the shipper didn't manage to ship those because the replication source is being terminated. In that case, it iterates through the batched entries and decrease the pending entries size from ReplicationSourceManager.totalBufferUser

NOTES 1) This method should only be called upon replication source termination. It blocks waiting for both shipper and reader threads termination, to make sure no race conditions when updating ReplicationSourceManager.totalBufferUser. 2) It does not attempt to terminate reader and shipper threads. Those must have been triggered interruption/termination prior to calling this method.