org.apache.hadoop.hbase.io.asyncfs.monitor

Class StreamSlowMonitor

java.lang.Object

org.apache.hadoop.hbase.io.asyncfs.monitor.StreamSlowMonitor

All Implemented Interfaces:

ConfigurationObserver

@InterfaceAudience.Private
public class StreamSlowMonitor
extends Object
implements ConfigurationObserver

Class for monitor the wal file flush performance. Each active wal file has a StreamSlowMonitor.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	StreamSlowMonitor.PacketAckData

Field Summary

Fields

Modifier and Type	Field and Description
private static String	DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH_KEY Configure for the speed check of packet min length.
private static String	DATANODE_SLOW_PACKET_FLUSH_MIN_SPEED_KEY Configure for the check of large packet(which is configured by DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH_KEY) flush speed.
static String	DATANODE_SLOW_PACKET_PROCESS_TIME_KEY Configure for the slow packet process time, a duration from send to ACK.
private org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<org.apache.hadoop.hdfs.protocol.DatanodeInfo,Deque<StreamSlowMonitor.PacketAckData>>	datanodeSlowDataQueue
private static long	DEFAULT_DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH
private static double	DEFAULT_DATANODE_SLOW_PACKET_FLUSH_MIN_SPEED
private static long	DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME
private static long	DEFAULT_WAL_SLOW_DETECT_DATA_TTL
private static int	DEFAULT_WAL_SLOW_DETECT_MIN_COUNT
private ExcludeDatanodeManager	excludeDatanodeManager
private static org.slf4j.Logger	LOG
private long	minLengthForSpeedCheck
private double	minPacketFlushSpeedKBs
private int	minSlowDetectCount
private String	name
private long	slowDataTtl
private long	slowPacketAckMs
private static String	WAL_SLOW_DETECT_DATA_TTL_KEY Configure for the TTL of the data that a datanode detected slow.
private static String	WAL_SLOW_DETECT_MIN_COUNT_KEY Configure for the min count for a datanode detected slow.

Constructor Summary

Constructors

Constructor and Description
StreamSlowMonitor(org.apache.hadoop.conf.Configuration conf, String name, ExcludeDatanodeManager excludeDatanodeManager)

Method Summary

Modifier and Type	Method and Description
private boolean	addSlowAckData(org.apache.hadoop.hdfs.protocol.DatanodeInfo datanodeInfo, long dataLength, long processTime)
void	checkProcessTimeAndSpeed(org.apache.hadoop.hdfs.protocol.DatanodeInfo datanodeInfo, long packetDataLen, long processTimeMs, long lastAckTimestamp, int unfinished) Check if the packet process time shows that the relevant datanode is a slow node.
static StreamSlowMonitor	create(org.apache.hadoop.conf.Configuration conf, String name)
ExcludeDatanodeManager	getExcludeDatanodeManager()
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf) This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
private void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

WAL_SLOW_DETECT_MIN_COUNT_KEY

private static final String WAL_SLOW_DETECT_MIN_COUNT_KEY

Configure for the min count for a datanode detected slow. If a datanode is detected slow times up to this count, then it will be added to the exclude datanode cache by ExcludeDatanodeManager.tryAddExcludeDN(DatanodeInfo, String) of this regionsever.

See Also:

Constant Field Values

DEFAULT_WAL_SLOW_DETECT_MIN_COUNT

private static final int DEFAULT_WAL_SLOW_DETECT_MIN_COUNT

See Also:

Constant Field Values

WAL_SLOW_DETECT_DATA_TTL_KEY

private static final String WAL_SLOW_DETECT_DATA_TTL_KEY

Configure for the TTL of the data that a datanode detected slow.

See Also:

Constant Field Values

DEFAULT_WAL_SLOW_DETECT_DATA_TTL

private static final long DEFAULT_WAL_SLOW_DETECT_DATA_TTL

See Also:

Constant Field Values

DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH_KEY

private static final String DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH_KEY

Configure for the speed check of packet min length. For packets whose data length smaller than this value, check slow by processing time. While for packets whose data length larger than this value, check slow by flushing speed.

See Also:

Constant Field Values

DEFAULT_DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH

private static final long DEFAULT_DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH

See Also:

Constant Field Values

DATANODE_SLOW_PACKET_PROCESS_TIME_KEY

public static final String DATANODE_SLOW_PACKET_PROCESS_TIME_KEY

Configure for the slow packet process time, a duration from send to ACK. The processing time check is for packets that data length smaller than DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH_KEY

See Also:

Constant Field Values

DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME

private static final long DEFAULT_DATANODE_SLOW_PACKET_PROCESS_TIME

See Also:

Constant Field Values

DATANODE_SLOW_PACKET_FLUSH_MIN_SPEED_KEY

private static final String DATANODE_SLOW_PACKET_FLUSH_MIN_SPEED_KEY

Configure for the check of large packet(which is configured by DATANODE_PACKET_FLUSH_CHECK_SPEED_MIN_DATA_LENGTH_KEY) flush speed. e.g. If the configured slow packet process time is smaller than 10s, then here 20KB/s means 64KB should be processed in less than 3.2s.

See Also:

Constant Field Values

DEFAULT_DATANODE_SLOW_PACKET_FLUSH_MIN_SPEED

private static final double DEFAULT_DATANODE_SLOW_PACKET_FLUSH_MIN_SPEED

See Also:

Constant Field Values

name

private final String name

datanodeSlowDataQueue

private final org.apache.hbase.thirdparty.com.google.common.cache.LoadingCache<org.apache.hadoop.hdfs.protocol.DatanodeInfo,Deque<StreamSlowMonitor.PacketAckData>> datanodeSlowDataQueue

excludeDatanodeManager

private final ExcludeDatanodeManager excludeDatanodeManager

minSlowDetectCount

private int minSlowDetectCount

slowDataTtl

private long slowDataTtl

slowPacketAckMs

private long slowPacketAckMs

minPacketFlushSpeedKBs

private double minPacketFlushSpeedKBs

minLengthForSpeedCheck

private long minLengthForSpeedCheck

Constructor Detail

StreamSlowMonitor

public StreamSlowMonitor(org.apache.hadoop.conf.Configuration conf,
                         String name,
                         ExcludeDatanodeManager excludeDatanodeManager)

Method Detail

create

public static StreamSlowMonitor create(org.apache.hadoop.conf.Configuration conf,
                                       String name)

checkProcessTimeAndSpeed

public void checkProcessTimeAndSpeed(org.apache.hadoop.hdfs.protocol.DatanodeInfo datanodeInfo,
                                     long packetDataLen,
                                     long processTimeMs,
                                     long lastAckTimestamp,
                                     int unfinished)

Check if the packet process time shows that the relevant datanode is a slow node.

Parameters:

datanodeInfo - the datanode that processed the packet

packetDataLen - the data length of the packet (in bytes)

processTimeMs - the process time (in ms) of the packet on the datanode,

lastAckTimestamp - the last acked timestamp of the packet on another datanode

unfinished - if the packet is unfinished flushed to the datanode replicas

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: ConfigurationObserver

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

addSlowAckData

private boolean addSlowAckData(org.apache.hadoop.hdfs.protocol.DatanodeInfo datanodeInfo,
                               long dataLength,
                               long processTime)

setConf

private void setConf(org.apache.hadoop.conf.Configuration conf)

getExcludeDatanodeManager

public ExcludeDatanodeManager getExcludeDatanodeManager()