Package org.apache.hadoop.hbase.replication.master

Class ReplicationLogCleaner

java.lang.Object

org.apache.hadoop.hbase.BaseConfigurable

org.apache.hadoop.hbase.master.cleaner.BaseFileCleanerDelegate

org.apache.hadoop.hbase.master.cleaner.BaseLogCleanerDelegate

org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, FileCleanerDelegate, Stoppable

@LimitedPrivate("Configuration")
public class ReplicationLogCleaner
extends BaseLogCleanerDelegate

Implementation of a log cleaner that checks if a log is still scheduled for replication before deleting it when its TTL is over.

The logic is a bit complicated after we switch to use table based replication queue storage, see the design doc in HBASE-27109 and the comments in HBASE-27214 for more details.

Field Summary

Fields

Modifier and Type	Field	Description
private ReplicationLogCleanerBarrier	barrier
private boolean	canFilter
private Supplier<Set<ServerName>>	getNotFullyDeadServers
private static final org.slf4j.Logger	LOG
private MasterServices	masterService
private Set<ServerName>	notFullyDeadServers
private Set<String>	peerIds
private Map<ServerName,Map<String,Map<String,ReplicationGroupOffset>>>	replicationOffsets
private ReplicationPeerManager	rpm
private boolean	stopped

Constructor Summary

Constructors

Constructor	Description
ReplicationLogCleaner()

Method Summary

Modifier and Type	Method	Description
private boolean	filterForDeadRegionServer(ServerName serverName, org.apache.hadoop.fs.FileStatus file)
private boolean	filterForLiveRegionServer(ServerName serverName, org.apache.hadoop.fs.FileStatus file)
Iterable<org.apache.hadoop.fs.FileStatus>	getDeletableFiles(Iterable<org.apache.hadoop.fs.FileStatus> files)	Determines which of the given files are safe to delete
private Set<ServerName>	getNotFullyDeadServers(MasterServices services)
void	init(Map<String,Object> params)	this method is used to pass some instance into subclass
private boolean	isAsyncClusterConnectionClosedOrNull()	Check if asyncClusterConnection is null or closed.
boolean	isStopped()	Returns True if Stoppable.stop(String) has been closed.
void	postClean()	Will be called after cleaner run.
void	preClean()	Used to do some initialize work before every period clean
private boolean	shouldDelete(ReplicationGroupOffset offset, org.apache.hadoop.fs.FileStatus file)
void	stop(String why)	Stop this service.

Methods inherited from class org.apache.hadoop.hbase.master.cleaner.BaseLogCleanerDelegate

isFileDeletable

Methods inherited from class org.apache.hadoop.hbase.BaseConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Methods inherited from interface org.apache.hadoop.hbase.master.cleaner.FileCleanerDelegate

isEmptyDirDeletable

Field Details

LOG

private static final org.slf4j.Logger LOG

notFullyDeadServers

private Set<ServerName> notFullyDeadServers

peerIds

private Set<String> peerIds

replicationOffsets

private Map<ServerName,Map<String,Map<String,ReplicationGroupOffset>>> replicationOffsets

masterService

private MasterServices masterService

barrier

private ReplicationLogCleanerBarrier barrier

rpm

private ReplicationPeerManager rpm

getNotFullyDeadServers

private Supplier<Set<ServerName>> getNotFullyDeadServers

canFilter

private boolean canFilter

stopped

private boolean stopped

Constructor Details

ReplicationLogCleaner

public ReplicationLogCleaner()

Method Details

preClean

public void preClean()

Description copied from interface: FileCleanerDelegate

Used to do some initialize work before every period clean

postClean

public void postClean()

Description copied from interface: FileCleanerDelegate

Will be called after cleaner run.

shouldDelete

private boolean shouldDelete(ReplicationGroupOffset offset,
 org.apache.hadoop.fs.FileStatus file)

filterForLiveRegionServer

private boolean filterForLiveRegionServer(ServerName serverName,
 org.apache.hadoop.fs.FileStatus file)

filterForDeadRegionServer

private boolean filterForDeadRegionServer(ServerName serverName,
 org.apache.hadoop.fs.FileStatus file)

getDeletableFiles

public Iterable<org.apache.hadoop.fs.FileStatus> getDeletableFiles(Iterable<org.apache.hadoop.fs.FileStatus> files)

Description copied from interface: FileCleanerDelegate

Determines which of the given files are safe to delete

This method can be called concurrently by multiple threads. Implementations must be thread safe.

Specified by:

getDeletableFiles in interface FileCleanerDelegate

Overrides:

getDeletableFiles in class BaseFileCleanerDelegate

Parameters:

files - files to check for deletion

Returns:

files that are ok to delete according to this cleaner

getNotFullyDeadServers

private Set<ServerName> getNotFullyDeadServers(MasterServices services)

init

public void init(Map<String,Object> params)

Description copied from interface: FileCleanerDelegate

this method is used to pass some instance into subclass

Specified by:

init in interface FileCleanerDelegate

Overrides:

init in class BaseFileCleanerDelegate

stop

public void stop(String why)

Description copied from interface: Stoppable

Stop this service. Implementers should favor logging errors over throwing RuntimeExceptions.

Parameters:

why - Why we're stopping.

isStopped

public boolean isStopped()

Description copied from interface: Stoppable

Returns True if Stoppable.stop(String) has been closed.

isAsyncClusterConnectionClosedOrNull

private boolean isAsyncClusterConnectionClosedOrNull()

Check if asyncClusterConnection is null or closed.

Returns:

true if asyncClusterConnection is null or is closed, false otherwise