Package org.apache.hadoop.hbase.regionserver

Class HRegion

java.lang.Object

org.apache.hadoop.hbase.regionserver.HRegion

All Implemented Interfaces:

ConfigurationObserver, PropagatingConfigurationObserver, HeapSize, Region

@Private
public class HRegion
extends Object
implements HeapSize, PropagatingConfigurationObserver, Region

Regions store data for a certain region of a table. It stores all columns for each row. A given table consists of one or more Regions.

An Region is defined by its table and its key extent.

Locking at the Region level serves only one purpose: preventing the region from being closed (and consequently split) while other operations are ongoing. Each row level operation obtains both a row lock and a region read lock for the duration of the operation. While a scanner is being constructed, getScanner holds a read lock. If the scanner is successfully constructed, it holds a read lock until it is closed. A close takes out a write lock and consequently will block for ongoing operations and will block new operations from starting while the close is in progress.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	HRegion.BatchOperation<T>	Class that tracks the progress of a batch operations, accumulating status codes and tracking the index at which processing is proceeding.
static interface	HRegion.BulkLoadListener	Listener class to enable callers of bulkLoadHFile() to perform any necessary pre/post processing of a given bulkload call
static interface	HRegion.FlushResult
static class	HRegion.FlushResultImpl	Objects from this class are created when flushing to describe all the different states that that method ends up in.
private static class	HRegion.MutationBatchOperation	Batch of mutation operations.
(package private) static class	HRegion.ObservedExceptionsInBatch	A class that tracks exceptions that have been observed in one batch.
protected static class	HRegion.PrepareFlushResult	A result object from prepare flush cache stage
private static final class	HRegion.ReplayBatchOperation	Deprecated. Since 3.0.0, will be removed in 4.0.0.
(package private) class	HRegion.RowLockContext
static class	HRegion.RowLockImpl	Class used to represent a lock on a row.
(package private) static class	HRegion.WriteState

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.regionserver.Region

Region.Operation, Region.RowLock

Field Summary

Fields

Modifier and Type	Field	Description
private final org.apache.hadoop.conf.Configuration	baseConf
private BlockCache	blockCache
private final LongAdder	blockedRequestsCount
private long	blockingMemStoreSize
(package private) final long	busyWaitDuration
private final CellComparator	cellComparator
(package private) final LongAdder	checkAndMutateChecksFailed
(package private) final LongAdder	checkAndMutateChecksPassed
static final String	CLOSE_WAIT_ABORT
static final String	CLOSE_WAIT_INTERVAL
static final String	CLOSE_WAIT_TIME
(package private) final AtomicBoolean	closed
private final Object	closeLock
(package private) final AtomicBoolean	closing
static final String	COMPACTION_AFTER_BULKLOAD_ENABLE	Parameter name for compaction after bulkload
(package private) final LongAdder	compactionNumBytesCompacted
(package private) final LongAdder	compactionNumFilesCompacted
(package private) final LongAdder	compactionsFailed
(package private) final LongAdder	compactionsFinished
(package private) final LongAdder	compactionsQueued
protected final org.apache.hadoop.conf.Configuration	conf
private ConfigurationManager	configurationManager
private RegionCoprocessorHost	coprocessorHost
private Map<String,org.apache.hbase.thirdparty.com.google.protobuf.Service>	coprocessorServiceHandlers
(package private) final LongAdder	cpRequestsCount
(package private) final LongAdder	dataInMemoryWithoutWAL
static final long	DEEP_OVERHEAD
(package private) static final long	DEFAULT_BUSY_WAIT_DURATION
static final int	DEFAULT_CACHE_FLUSH_INTERVAL	Default interval for the memstore flush
static final boolean	DEFAULT_CLOSE_WAIT_ABORT
static final long	DEFAULT_CLOSE_WAIT_INTERVAL
static final long	DEFAULT_CLOSE_WAIT_TIME
static final boolean	DEFAULT_FAIR_REENTRANT_CLOSE_LOCK
static final long	DEFAULT_FLUSH_PER_CHANGES
static final int	DEFAULT_HBASE_REGIONSERVER_MINIBATCH_SIZE
static final int	DEFAULT_MAX_CELL_SIZE
static final String	DEFAULT_REGION_STORAGE_POLICY
(package private) static final int	DEFAULT_ROWLOCK_WAIT_DURATION
static final boolean	DEFAULT_USE_META_CELL_COMPARATOR
static final boolean	DEFAULT_WAL_HSYNC
static final String	FAIR_REENTRANT_CLOSE_LOCK	Conf key for fair locking policy
(package private) final LongAdder	filteredReadRequestsCount
static final long	FIXED_OVERHEAD
private long	flushCheckInterval
(package private) final LongAdder	flushesQueued
private long	flushPerChanges
private FlushPolicy	flushPolicy
private final HRegionFileSystem	fs
static final String	HBASE_MAX_CELL_SIZE_KEY
static final String	HBASE_REGIONSERVER_MINIBATCH_SIZE
private TableDescriptor	htableDescriptor
private boolean	isLoadingCfsOnDemandDefault	The default setting for whether to enable on-demand CF loading for scan requests to this region.
private boolean	isRestoredRegion
private long	lastFlushOpSeqId	Record the sequence id of last flush operation.
protected long	lastReplayedCompactionSeqId
protected long	lastReplayedOpenRegionSeqId	The sequence id of the last replayed open region event from the primary region.
private long	lastReplayedSequenceId
private final ConcurrentMap<HStore,Long>	lastStoreFlushTimeMap
static final String	LOAD_CFS_ON_DEMAND_CONFIG_KEY
(package private) final ReentrantReadWriteLock	lock
private final ConcurrentHashMap<HashedBytes,HRegion.RowLockContext>	lockedRows
private static final org.slf4j.Logger	LOG
private final AtomicInteger	majorInProgress
static final long	MAX_FLUSH_PER_CHANGES	The following MAX_FLUSH_PER_CHANGES is large enough because each KeyValue has 20+ bytes overhead.
(package private) final long	maxBusyWaitDuration
(package private) final int	maxBusyWaitMultiplier
(package private) final long	maxCellSize
private long	maxFlushedSeqId	The max sequence id of flushed data on this region.
(package private) Map<byte[],Long>	maxSeqIdInStores
static final String	MEMSTORE_FLUSH_PER_CHANGES	Conf key to force a flush if there are already enough changes for one region in memstore
static final String	MEMSTORE_PERIODIC_FLUSH_INTERVAL	Conf key for the periodic flush interval
(package private) long	memstoreFlushSize
private final MemStoreSizing	memStoreSizing
private final MetricsRegion	metricsRegion
private final MetricsRegionWrapperImpl	metricsRegionWrapper
private MetricsTableRequests	metricsTableRequests
private final int	minBlockSizeBytes
private final int	miniBatchSize
private final AtomicInteger	minorInProgress
private MobFileCache	mobFileCache
private final MultiVersionConcurrencyControl	mvcc
(package private) final LongAdder	numMutationsWithoutWAL
private long	openSeqNum	The sequence ID that was enLongAddered when this region was opened.
private HRegion.PrepareFlushResult	prepareFlushResult	Saved state from replaying prepare flush cache
(package private) final LongAdder	readRequestsCount
static final String	RECOVERED_EDITS_IGNORE_EOF	Mainly used for master local region, where we will replay the WAL file directly without splitting, so it is possible to have WAL files which are not closed cleanly, in this way, hitting EOF is expected so should not consider it as a critical problem.
static final String	REGION_STORAGE_POLICY_KEY
private final Durability	regionDurability
(package private) final ConcurrentHashMap<Thread,Boolean>	regionLockHolders
protected Optional<RegionReplicationSink>	regionReplicationSink
(package private) RegionServicesForStores	regionServicesForStores
private final boolean	regionStatsEnabled
private org.apache.hadoop.fs.Path	regionWalDir
private Lock	replayLock
private final NavigableMap<byte[],Integer>	replicationScope
private final int	rowLockWaitDuration
private RegionServerAccounting	rsAccounting
protected RegionServerServices	rsServices
(package private) final ConcurrentHashMap<RegionScanner,Long>	scannerReadPoints
(package private) final ReadPointCalculationLock	smallestReadPointCalcLock
static final String	SPECIAL_RECOVERED_EDITS_DIR	This is for for using HRegion as a local storage, where we may put the recovered edits in a special place.
static final String	SPLIT_IGNORE_BLOCKING_ENABLED_KEY	Config for allow split when file count greater than the configured blocking file count
private RegionSplitPolicy	splitPolicy
private RegionSplitRestriction	splitRestriction
private final StoreHotnessProtector	storeHotnessProtector
protected final Map<byte[],HStore>	stores
static final int	SYSTEM_CACHE_FLUSH_INTERVAL	Default interval for System tables memstore flush
private Long	timeoutForWriteLock
(package private) final long	timestampSlop
private final ReentrantReadWriteLock	updatesLock
static final String	USE_META_CELL_COMPARATOR	Whether to use MetaCellComparator even if we are not meta region.
private final WAL	wal
static final String	WAL_HSYNC_CONF_KEY
private org.apache.hadoop.fs.FileSystem	walFS
(package private) final LongAdder	writeRequestsCount
(package private) final HRegion.WriteState	writestate

Constructor Summary

Constructors

Constructor	Description
HRegion(org.apache.hadoop.fs.Path tableDir, WAL wal, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration confParam, RegionInfo regionInfo, TableDescriptor htd, RegionServerServices rsServices)	Deprecated. Use other constructors.
HRegion(HRegionFileSystem fs, WAL wal, org.apache.hadoop.conf.Configuration confParam, TableDescriptor htd, RegionServerServices rsServices)	HRegion constructor.

Method Summary

Modifier and Type	Method	Description
void	addReadRequestsCount(long readRequestsCount)
void	addRegionToSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription desc, ForeignExceptionSnare exnSnare)	Complete taking the snapshot on the region.
void	addWriteRequestsCount(long writeRequestsCount)
Result	append(Append append)	Perform one or more append operations on a row.
Result	append(Append append, long nonceGroup, long nonce)
private void	applyToMemStore(HStore store, List<ExtendedCell> cells, boolean delta, MemStoreSizing memstoreAccounting)
boolean	areWritesEnabled()
private void	attachRegionReplicationInWALAppend(HRegion.BatchOperation<?> batchOp, MiniBatchOperationInProgress<Mutation> miniBatchOp, WALKeyImpl walKey, WALEdit walEdit, MultiVersionConcurrencyControl.WriteEntry writeEntry)	Attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the mvcc writeEntry for replicating to region replica.
private void	attachRegionReplicationToFlushOpSeqIdMVCCEntry(MultiVersionConcurrencyControl.WriteEntry flushOpSeqIdMVCCEntry, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor desc, RegionReplicationSink sink)	Create WALEdit for WALProtos.FlushDescriptor and attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the flushOpSeqIdMVCCEntry.
OperationStatus[]	batchMutate(Mutation[] mutations)	Perform a batch of mutations.
(package private) OperationStatus[]	batchMutate(Mutation[] mutations, boolean atomic)
OperationStatus[]	batchMutate(Mutation[] mutations, boolean atomic, long nonceGroup, long nonce)
private OperationStatus[]	batchMutate(HRegion.BatchOperation<?> batchOp)	Perform a batch of mutations.
(package private) OperationStatus[]	batchReplay(WALSplitUtil.MutationReplay[] mutations, long replaySeqId)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
void	blockUpdates()
Map<byte[],List<org.apache.hadoop.fs.Path>>	bulkLoadHFiles(Collection<Pair<byte[],String>> familyPaths, boolean assignSeqId, HRegion.BulkLoadListener bulkLoadListener)	Attempts to atomically load a group of hfiles.
Map<byte[],List<org.apache.hadoop.fs.Path>>	bulkLoadHFiles(Collection<Pair<byte[],String>> familyPaths, boolean assignSeqId, HRegion.BulkLoadListener bulkLoadListener, boolean copyFile, List<String> clusterIds, boolean replicate)	Attempts to atomically load a group of hfiles.
boolean	checkAndMutate(byte[] row, byte[] family, byte[] qualifier, CompareOperator op, ByteArrayComparable comparator, TimeRange timeRange, Mutation mutation)	Deprecated.
boolean	checkAndMutate(byte[] row, Filter filter, TimeRange timeRange, Mutation mutation)	Deprecated.
CheckAndMutateResult	checkAndMutate(CheckAndMutate checkAndMutate)	Atomically checks if a row matches the conditions and if it does, it performs the actions.
CheckAndMutateResult	checkAndMutate(CheckAndMutate checkAndMutate, long nonceGroup, long nonce)
private CheckAndMutateResult	checkAndMutateInternal(CheckAndMutate checkAndMutate, long nonceGroup, long nonce)
boolean	checkAndRowMutate(byte[] row, byte[] family, byte[] qualifier, CompareOperator op, ByteArrayComparable comparator, TimeRange timeRange, RowMutations rm)	Deprecated.
boolean	checkAndRowMutate(byte[] row, Filter filter, TimeRange timeRange, RowMutations rm)	Deprecated.
private void	checkFamilies(Collection<byte[]> families, Durability durability)
private void	checkFamily(byte[] family)
private void	checkFamily(byte[] family, Durability durability)
(package private) void	checkInterrupt()	Check thread interrupt status and throw an exception if interrupted.
private void	checkMutationType(Mutation mutation)
private void	checkNegativeMemStoreDataSize(long memStoreDataSize, long delta)
private void	checkReadOnly()
private void	checkReadsEnabled()
private void	checkResources()	Check if resources to support an update.
(package private) void	checkRow(byte[] row, String op)	Make sure this is a valid row for the HRegion
private void	checkRow(Row action, byte[] row)
Optional<byte[]>	checkSplit()
Optional<byte[]>	checkSplit(boolean force)	Return the split point.
private void	checkTargetRegion(byte[] encodedRegionName, String exceptionMsg, Object payload)	Checks whether the given regionName is either equal to our region, or that the regionName is the primary region to our corresponding range for the secondary replica.
void	checkTimestamps(Map<byte[],List<Cell>> familyMap, long now)	Check the collection of families for valid timestamps
Map<byte[],List<HStoreFile>>	close()	Close down this HRegion.
Map<byte[],List<HStoreFile>>	close(boolean abort)
Map<byte[],List<HStoreFile>>	close(boolean abort, boolean ignoreStatus)	Close this HRegion.
Map<byte[],List<HStoreFile>>	close(boolean abort, boolean ignoreStatus, boolean isGracefulStop)	Close down this HRegion.
private void	closeBulkRegionOperation()	Closes the lock.
void	closeRegionOperation()	Closes the region operation lock.
void	closeRegionOperation(Region.Operation operation)	Closes the region operation lock.
void	compact(boolean majorCompaction)	Synchronously compact all stores in the region.
boolean	compact(CompactionContext compaction, HStore store, ThroughputController throughputController)	Called by compaction thread and after region is opened to compact the HStores if necessary.
boolean	compact(CompactionContext compaction, HStore store, ThroughputController throughputController, User user)	We are trying to remove / relax the region read lock for compaction.
(package private) void	compactStore(byte[] family, ThroughputController throughputController)	This is a helper function that compact the given store.
void	compactStores()	This is a helper function that compact all the stores synchronously.
static HDFSBlocksDistribution	computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf, TableDescriptor tableDescriptor, RegionInfo regionInfo)	This is a helper function to compute HDFS block distribution on demand
static HDFSBlocksDistribution	computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf, TableDescriptor tableDescriptor, RegionInfo regionInfo, org.apache.hadoop.fs.Path tablePath)	This is a helper function to compute HDFS block distribution on demand
static HRegion	createHRegion(org.apache.hadoop.conf.Configuration conf, RegionInfo regionInfo, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, TableDescriptor tableDesc)	Create a region under the given table directory.
static HRegion	createHRegion(RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, TableDescriptor hTableDescriptor, WAL wal)
static HRegion	createHRegion(RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, TableDescriptor hTableDescriptor, WAL wal, boolean initialize)	Convenience method creating new HRegions.
static HRegion	createHRegion(RegionInfo info, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.conf.Configuration conf, TableDescriptor hTableDescriptor, WAL wal, boolean initialize, RegionServerServices rsRpcServices)	Convenience method creating new HRegions.
static HRegionFileSystem	createRegionDir(org.apache.hadoop.conf.Configuration configuration, RegionInfo ri, org.apache.hadoop.fs.Path rootDir)	Create the region directory in the filesystem.
(package private) io.opentelemetry.api.trace.Span	createRegionSpan(String name)
private WALKeyImpl	createWALKeyForWALAppend(boolean isReplay, HRegion.BatchOperation<?> batchOp, long now, long nonceGroup, long nonce)
private static void	decorateRegionConfiguration(org.apache.hadoop.conf.Configuration conf)	This method modifies the region's configuration in order to inject replication-related features
void	decrementCompactionsQueuedCount()
protected void	decrementFlushesQueuedCount()
private void	decrMemStoreSize(long dataSizeDelta, long heapSizeDelta, long offHeapSizeDelta, int cellsCountDelta)
(package private) void	decrMemStoreSize(MemStoreSize mss)
void	delete(Delete delete)	Deletes the specified cells/row.
private void	deleteRecoveredEdits(org.apache.hadoop.fs.FileSystem fs, Iterable<org.apache.hadoop.fs.Path> files)
void	deregisterChildren(ConfigurationManager manager)	Needs to be called to deregister the children from the manager.
(package private) void	disableInterrupts()	If a handler thread is eligible for interrupt, make it ineligible.
private void	doAbortFlushToWAL(WAL wal, long flushOpSeqId, Map<byte[],List<org.apache.hadoop.fs.Path>> committedFiles)
private void	doAttachReplicateRegionReplicaAction(WALKeyImpl walKey, WALEdit walEdit, MultiVersionConcurrencyControl.WriteEntry writeEntry)	Attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the mvcc writeEntry for replicating to region replica.
private Map<byte[],List<HStoreFile>>	doClose(boolean abort, MonitoredTask status)
private MemStoreSize	doDropStoreMemStoreContentsForSeqId(HStore s, long currentSeqId)
private void	doMiniBatchMutate(HRegion.BatchOperation<?> batchOp)	Called to do a piece of the batch that came in to batchMutate(Mutation[]) In here we also handle replay of edits on region recover.
protected void	doRegionCompactionPrep()	Do preparation for pending compaction.
private static void	doSyncOfUnflushedWALChanges(WAL wal, RegionInfo hri)	Sync unflushed WAL changes.
private MultiVersionConcurrencyControl.WriteEntry	doWALAppend(WALEdit walEdit, HRegion.BatchOperation<?> batchOp, MiniBatchOperationInProgress<Mutation> miniBatchOp, long now, NonceKey nonceKey)	Returns writeEntry associated with this append
private MemStoreSize	dropMemStoreContents()	Be careful, this method will drop all data in the memstore of this region.
private MemStoreSize	dropMemStoreContentsForSeqId(long seqId, HStore store)	Drops the memstore contents after replaying a flush descriptor or region open event replay if the memstore edits have seqNums smaller than the given seq id
private void	dropPrepareFlushIfPossible()	If all stores ended up dropping their snapshots, we can safely drop the prepareFlushResult
(package private) void	enableInterrupts()	If a handler thread was made ineligible for interrupt via {disableInterrupts(), make it eligible again.
boolean	equals(Object o)
org.apache.hbase.thirdparty.com.google.protobuf.Message	execService(org.apache.hbase.thirdparty.com.google.protobuf.RpcController controller, org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.CoprocessorServiceCall call)	Executes a single protocol buffer coprocessor endpoint Service method using the registered protocol handlers.
HRegion.FlushResult	flush(boolean flushAllStores)	Flush the cache.
HRegion.FlushResultImpl	flushcache(boolean flushAllStores, boolean writeFlushRequestWalMarker, FlushLifeCycleTracker tracker)
HRegion.FlushResultImpl	flushcache(List<byte[]> families, boolean writeFlushRequestWalMarker, FlushLifeCycleTracker tracker)	Flush the cache.
Result	get(Get get)	Do a get based on the get parameter.
List<Cell>	get(Get get, boolean withCoprocessor)	Do a get based on the get parameter.
private List<Cell>	get(Get get, boolean withCoprocessor, long nonceGroup, long nonce)
(package private) org.apache.hadoop.conf.Configuration	getBaseConf()	A split takes the config from the parent region & passes it to the daughter region's constructor.
BlockCache	getBlockCache()
long	getBlockedRequestsCount()	Returns the number of blocked requests
CellComparator	getCellComparator()	The comparator to be used with the region
long	getCheckAndMutateChecksFailed()	Returns the number of failed checkAndMutate guards
long	getCheckAndMutateChecksPassed()	Returns the number of checkAndMutate guards that passed
CompactionState	getCompactionState()	Returns if a given region is in compaction now.
int	getCompactPriority()	Returns The priority that this region should have in the compaction queue
RegionCoprocessorHost	getCoprocessorHost()	Returns the coprocessor host
long	getCpRequestsCount()	Returns coprocessor requests count for this region
long	getDataInMemoryWithoutWAL()	Returns the size of data processed bypassing the WAL, in bytes
long	getEarliestFlushTimeForAllStores()
private Durability	getEffectiveDurability(Durability d)	Returns effective durability from the passed durability and the table descriptor.
org.apache.hadoop.fs.FileSystem	getFilesystem()	Returns FileSystem being used by this region
long	getFilteredReadRequestsCount()	Returns filtered read requests count for this region
HDFSBlocksDistribution	getHDFSBlocksDistribution()
private List<Cell>	getInternal(Get get, boolean withCoprocessor, long nonceGroup, long nonce)
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.RegionLoadStats	getLoadStatistics()	Returns statistics about the current load of the region
ConcurrentHashMap<HashedBytes,HRegion.RowLockContext>	getLockedRows()
long	getMaxFlushedSeqId()
Map<byte[],Long>	getMaxStoreSeqId()
long	getMemStoreDataSize()
long	getMemStoreFlushSize()
long	getMemStoreHeapSize()
long	getMemStoreOffHeapSize()
MetricsRegion	getMetrics()
MetricsTableRequests	getMetricsTableRequests()
int	getMinBlockSizeBytes()	The minimum block size configuration from all relevant column families.
MobFileCache	getMobFileCache()
MultiVersionConcurrencyControl	getMVCC()
protected long	getNextSequenceId(WAL wal)	Method to safely get the next sequence number.
long	getNumMutationsWithoutWAL()	Returns the number of mutations processed bypassing the WAL
long	getOldestHfileTs(boolean majorCompactionOnly)	This can be used to determine the last time all files of this region were major compacted.
long	getOldestSeqIdOfStore(byte[] familyName)
private static ThreadPoolExecutor	getOpenAndCloseThreadPool(int maxThreads, String threadNamePrefix)
long	getOpenSeqNum()	Returns the latest sequence number that was read from storage when this region was opened
(package private) HRegion.PrepareFlushResult	getPrepareFlushResult()
int	getReadLockCount()
org.apache.hadoop.conf.Configuration	getReadOnlyConfiguration()
long	getReadPoint(IsolationLevel isolationLevel)	Returns readpoint considering given IsolationLevel.
long	getReadRequestsCount()	Returns read requests count for this region
static org.apache.hadoop.fs.Path	getRegionDir(org.apache.hadoop.fs.Path tabledir, String name)	Deprecated. For tests only; to be removed.
HRegionFileSystem	getRegionFileSystem()	Returns the HRegionFileSystem used by this region
RegionInfo	getRegionInfo()	Returns region information for this region
Optional<RegionReplicationSink>	getRegionReplicationSink()
(package private) RegionServerServices	getRegionServerServices()	Returns Instance of RegionServerServices used by this HRegion.
RegionServicesForStores	getRegionServicesForStores()	Returns store services for this region, to access services required by store level needs
(package private) HRegionWALFileSystem	getRegionWALFileSystem()	Returns the WAL HRegionFileSystem used by this region
NavigableMap<byte[],Integer>	getReplicationScope()
Region.RowLock	getRowLock(byte[] row)	Get an exclusive ( write lock ) lock on a given row.
Region.RowLock	getRowLock(byte[] row, boolean readLock)	Get a row lock for the specified row.
private Region.RowLock	getRowLock(byte[] row, boolean readLock, Region.RowLock prevRowLock)
protected Region.RowLock	getRowLockInternal(byte[] row, boolean readLock, Region.RowLock prevRowLock)
RegionScannerImpl	getScanner(Scan scan)	Return an iterator that scans over the HRegion, returning the indicated columns and rows specified by the Scan.
RegionScannerImpl	getScanner(Scan scan, List<KeyValueScanner> additionalScanners)	Return an iterator that scans over the HRegion, returning the indicated columns and rows specified by the Scan.
private RegionScannerImpl	getScanner(Scan scan, List<KeyValueScanner> additionalScanners, long nonceGroup, long nonce)
long	getSmallestReadPoint()
private Collection<HStore>	getSpecificStores(List<byte[]> families)	get stores which matches the specified families
(package private) RegionSplitPolicy	getSplitPolicy()	Returns split policy for this region.
HStore	getStore(byte[] column)	Return the Store for the given family
private HStore	getStore(Cell cell)	Return HStore instance.
List<String>	getStoreFileList(byte[][] columns)	Returns list of store file names for the given families
(package private) ThreadPoolExecutor	getStoreFileOpenAndCloseThreadPool(String threadNamePrefix)
private NavigableMap<byte[],List<org.apache.hadoop.fs.Path>>	getStoreFiles()	Returns Map of StoreFiles by column family
private ThreadPoolExecutor	getStoreOpenAndCloseThreadPool(String threadNamePrefix)
List<HStore>	getStores()	Return the list of Stores managed by this region
private Collection<HStore>	getStoresToFlush(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flushDesc)
TableDescriptor	getTableDescriptor()	Returns table descriptor for this region
WAL	getWAL()	Returns WAL in use for this region
(package private) org.apache.hadoop.fs.FileSystem	getWalFileSystem()	Returns the WAL FileSystem being used by this region
org.apache.hadoop.fs.Path	getWALRegionDir()
long	getWriteRequestsCount()	Returns write request count for this region
private void	handleException(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path edits, IOException e)
int	hashCode()
private static boolean	hasMultipleColumnFamilies(Collection<Pair<byte[],String>> familyPaths)	Determines whether multiple column families are present Precondition: familyPaths is not null
boolean	hasReferences()	Returns True if this region has references.
long	heapSize()	Return the approximate 'exclusive deep size' of implementing object.
(package private) void	incMemStoreSize(long dataSizeDelta, long heapSizeDelta, long offHeapSizeDelta, int cellsCountDelta)
private void	incMemStoreSize(MemStoreSize mss)	Increase the size of mem store in this region and the size of global mem store
Result	increment(Increment increment)	Perform one or more increment operations on a row.
Result	increment(Increment increment, long nonceGroup, long nonce)
void	incrementCompactionsQueuedCount()
void	incrementFlushesQueuedCount()
long	initialize()	Deprecated. use HRegion.createHRegion() or HRegion.openHRegion()
(package private) long	initialize(CancelableProgressable reporter)	Initialize this region.
private long	initializeRegionInternals(CancelableProgressable reporter, MonitoredTask status)
private void	initializeRegionReplicationSink(CancelableProgressable reporter, MonitoredTask status)
private long	initializeStores(CancelableProgressable reporter, MonitoredTask status)	Open all Stores.
private long	initializeStores(CancelableProgressable reporter, MonitoredTask status, boolean warmup)
private void	initializeWarmup(CancelableProgressable reporter)
protected HStore	instantiateHStore(ColumnFamilyDescriptor family, boolean warmup)
protected RegionScannerImpl	instantiateRegionScanner(Scan scan, List<KeyValueScanner> additionalScanners, long nonceGroup, long nonce)
private HRegion.FlushResultImpl	internalFlushcache(Collection<HStore> storesToFlush, MonitoredTask status, boolean writeFlushWalMarker, FlushLifeCycleTracker tracker)	Flushing given stores.
private HRegion.FlushResult	internalFlushcache(MonitoredTask status)	Flushing all stores.
protected HRegion.FlushResultImpl	internalFlushcache(WAL wal, long myseqid, Collection<HStore> storesToFlush, MonitoredTask status, boolean writeFlushWalMarker, FlushLifeCycleTracker tracker)	Flush the memstore.
(package private) HRegion.FlushResultImpl	internalFlushCacheAndCommit(WAL wal, MonitoredTask status, HRegion.PrepareFlushResult prepareResult, Collection<HStore> storesToFlush)
protected HRegion.PrepareFlushResult	internalPrepareFlushCache(WAL wal, long myseqid, Collection<HStore> storesToFlush, MonitoredTask status, boolean writeFlushWalMarker, FlushLifeCycleTracker tracker)
private void	interruptRegionOperations()	Interrupt any region options that have acquired the region lock via startRegionOperation(org.apache.hadoop.hbase.regionserver.Region.Operation), or startBulkRegionOperation(boolean).
private boolean	isAllFamilies(Collection<HStore> families)	Returns True if passed Set is all families in the region.
boolean	isAvailable()	Returns true if region is available (not closed and not closing)
boolean	isClosed()	Returns true if region is closed
boolean	isClosing()	Returns True if closing process has started
private boolean	isFlushSize(MemStoreSize size)
boolean	isLoadingCfsOnDemandDefault()
boolean	isMergeable()	Returns true if region is mergeable
boolean	isReadOnly()	Returns True if region is read only
(package private) boolean	isReadsEnabled()
boolean	isSplittable()	Returns true if region is splittable
private static boolean	isZeroLengthThenDelete(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus stat, org.apache.hadoop.fs.Path p)	make sure have been through lease recovery before get file status, so the file length can be trusted.
private long	loadRecoveredHFilesIfAny(Collection<HStore> stores)
private void	lock(Lock lock)
private void	lock(Lock lock, int multiplier)	Try to acquire a lock.
private void	logFatLineOnFlush(Collection<HStore> storesToFlush, long sequenceId)	Utility method broken out of internalPrepareFlushCache so that method is smaller.
private void	logRegionFiles()
private boolean	matches(CompareOperator op, int compareResult)
(package private) void	metricsUpdateForGet()
private OperationStatus	mutate(Mutation mutation)
private OperationStatus	mutate(Mutation mutation, boolean atomic)
private OperationStatus	mutate(Mutation mutation, boolean atomic, long nonceGroup, long nonce)
Result	mutateRow(RowMutations rm)	Performs multiple mutations atomically on a single row.
Result	mutateRow(RowMutations rm, long nonceGroup, long nonce)
void	mutateRowsWithLocks(Collection<Mutation> mutations, Collection<byte[]> rowsToLock, long nonceGroup, long nonce)	Perform atomic (all or none) mutations within the region.
static HRegion	newHRegion(org.apache.hadoop.fs.Path tableDir, WAL wal, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, RegionInfo regionInfo, TableDescriptor htd, RegionServerServices rsServices)	A utility method to create new instances of HRegion based on the HConstants.REGION_IMPL configuration property.
void	onConfigurationChange(org.apache.hadoop.conf.Configuration conf)	This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.
static HRegion	openHRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, RegionInfo info, TableDescriptor htd, WAL wal)	Open a Region.
static HRegion	openHRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, RegionInfo info, TableDescriptor htd, WAL wal, RegionServerServices rsServices, CancelableProgressable reporter)	Open a Region.
static HRegion	openHRegion(org.apache.hadoop.fs.Path rootDir, RegionInfo info, TableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf)	Open a Region.
static HRegion	openHRegion(org.apache.hadoop.fs.Path rootDir, RegionInfo info, TableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf, RegionServerServices rsServices, CancelableProgressable reporter)	Open a Region.
static HRegion	openHRegion(RegionInfo info, TableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf)	Open a Region.
static HRegion	openHRegion(RegionInfo info, TableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf, RegionServerServices rsServices, CancelableProgressable reporter)	Open a Region.
static HRegion	openHRegion(HRegion other, CancelableProgressable reporter)	Useful when reopening a closed region (normally for unit tests)
static Region	openHRegion(Region other, CancelableProgressable reporter)
private HRegion	openHRegion(CancelableProgressable reporter)	Open HRegion.
static HRegion	openHRegionFromTableDir(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo info, TableDescriptor htd, WAL wal, RegionServerServices rsServices, CancelableProgressable reporter)	Open a Region.
static HRegion	openReadOnlyFileSystemHRegion(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo info, TableDescriptor htd)	Open a Region on a read-only file-system (like hdfs snapshots)
private void	prepareDelete(Delete delete)	Prepare a delete for a row mutation processor
private void	prepareDeleteTimestamps(Mutation mutation, Map<byte[],List<ExtendedCell>> familyMap, byte[] byteNow)	Set up correct timestamps in the KVs in Delete object.
(package private) void	prepareGet(Get get)
void	put(Put put)	Puts some data in the table.
private void	recordMutationWithoutWal(Map<byte[],List<Cell>> familyMap)	Update LongAdders for number of puts without wal and the size of possible data loss.
boolean	refreshStoreFiles()	Check the region's underlying store files, open the files that have not been opened yet, and remove the store file readers for store files no longer available.
protected boolean	refreshStoreFiles(boolean force)
void	registerChildren(ConfigurationManager manager)	Needs to be called to register the children to the manager.
boolean	registerService(org.apache.hbase.thirdparty.com.google.protobuf.Service instance)	Registers a new protocol buffer Service subclass as a coprocessor endpoint to be available for handling execService(RpcController, CoprocessorServiceCall) calls.
private void	releaseRowLocks(List<Region.RowLock> rowLocks)
private void	replayFlushInStores(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush, HRegion.PrepareFlushResult prepareFlushResult, boolean dropMemstoreSnapshot)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
private long	replayRecoveredEdits(org.apache.hadoop.fs.Path edits, Map<byte[],Long> maxSeqIdInStores, CancelableProgressable reporter, org.apache.hadoop.fs.FileSystem fs)
private long	replayRecoveredEditsForPaths(long minSeqIdForTheRegion, org.apache.hadoop.fs.FileSystem fs, NavigableSet<org.apache.hadoop.fs.Path> files, CancelableProgressable reporter, org.apache.hadoop.fs.Path regionDir)
(package private) long	replayRecoveredEditsIfAny(Map<byte[],Long> maxSeqIdInStores, CancelableProgressable reporter, MonitoredTask status)	Read the edits put under this region by wal splitting process.
private void	replayWALBatchMutate(Map<byte[],List<ExtendedCell>> family2Cells)	Replay the batch mutate for secondary replica.
(package private) void	replayWALBulkLoadEventMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadEvent)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
(package private) void	replayWALCompactionMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor compaction, boolean pickCompactionFiles, boolean removeFiles, long replaySeqId)	Call to complete a compaction.
(package private) void	replayWALEntry(org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.WALEntry entry, CellScanner cells)	Replay remote wal entry sent by primary replica.
private void	replayWALFlushAbortMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush)
private void	replayWALFlushCannotFlushMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush, long replaySeqId)
(package private) void	replayWALFlushCommitMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
(package private) void	replayWALFlushMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush, long replaySeqId)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
(package private) HRegion.PrepareFlushResult	replayWALFlushStartMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
private void	replayWALMetaEdit(Cell cell)	Replay the meta edits, i.e, flush marker, compaction marker, bulk load marker, region event marker, etc.
(package private) void	replayWALRegionEventMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEvent)	Deprecated. Since 3.0.0, will be removed in 4.0.0.
void	reportCompactionRequestEnd(boolean isMajor, int numFiles, long filesSizeCompacted)
void	reportCompactionRequestFailure()
void	reportCompactionRequestStart(boolean isMajor)
void	requestCompaction(byte[] family, String why, int priority, boolean major, CompactionLifeCycleTracker tracker)	Request compaction for the given family
void	requestCompaction(String why, int priority, boolean major, CompactionLifeCycleTracker tracker)	Request compaction on this region.
private void	requestFlush()
void	requestFlush(FlushLifeCycleTracker tracker)	Request flush on this region.
private void	requestFlush0(FlushLifeCycleTracker tracker)
private void	requestFlushIfNeeded()
protected void	restoreEdit(HStore s, ExtendedCell cell, MemStoreSizing memstoreAccounting)	Used by tests
private void	rewriteCellTags(Map<byte[],List<ExtendedCell>> familyMap, Mutation m)	Possibly rewrite incoming cell tags.
static boolean	rowIsInRange(RegionInfo info, byte[] row)	Determines if the specified row is within the row range specified by the specified RegionInfo
static boolean	rowIsInRange(RegionInfo info, byte[] row, int offset, short length)
void	setBlockCache(BlockCache blockCache)	Only used for unit test which doesn't start region server.
void	setClosing(boolean closing)	Exposed for some very specific unit tests.
(package private) org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad.Builder	setCompleteSequenceId(org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad.Builder regionLoadBldr)
void	setCoprocessorHost(RegionCoprocessorHost coprocessorHost)
private void	setHTableSpecificConf()
void	setMobFileCache(MobFileCache mobFileCache)	Only used for unit test which doesn't start region server.
void	setReadsEnabled(boolean readsEnabled)
void	setRestoredRegion(boolean restoredRegion)
void	setTableDescriptor(TableDescriptor desc)
void	setTimeoutForWriteLock(long timeoutForWriteLock)	The doClose(boolean, org.apache.hadoop.hbase.monitoring.MonitoredTask) will block forever if someone tries proving the dead lock via the unit test.
(package private) boolean	shouldFlush(StringBuilder whyFlush)	Should the memstore be flushed now
(package private) boolean	shouldFlushStore(HStore store)	Should the store be flushed because it is old enough.
private boolean	shouldForbidMajorCompaction()
private boolean	shouldSyncWAL()	Check whether we should sync the wal from the table's durability settings
private void	startBulkRegionOperation(boolean writeLockNeeded)	This method needs to be called before any public call that reads or modifies stores in bulk.
void	startRegionOperation()	This method needs to be called before any public call that reads or modifies data.
void	startRegionOperation(Region.Operation op)	This method needs to be called before any public call that reads or modifies data.
private void	sync(long txid, Durability durability)	Calls sync with the given transaction ID
(package private) void	throwException(String title, String regionName)
(package private) IOException	throwOnInterrupt(Throwable t)	Throw the correct exception upon interrupt
String	toString()
void	unblockUpdates()
private static void	updateCellTimestamps(Iterable<List<ExtendedCell>> cellItr, byte[] now)	Replace any cell timestamps set to HConstants.LATEST_TIMESTAMP provided current timestamp.
private void	updateDeleteLatestVersionTimestamp(Cell cell, Get get, int count, byte[] byteNow)
private void	updateSequenceId(Iterable<List<ExtendedCell>> cellItr, long sequenceId)
void	waitForFlushes()	Wait for all current flushes of the region to complete
boolean	waitForFlushes(long timeout)	Wait for all current flushes of the region to complete
void	waitForFlushesAndCompactions()	Wait for all current flushes and compactions of the region to complete
static HRegion	warmupHRegion(RegionInfo info, TableDescriptor htd, WAL wal, org.apache.hadoop.conf.Configuration conf, RegionServerServices rsServices, CancelableProgressable reporter)
private boolean	worthPreFlushing()	Returns True if its worth doing a flush before we put up the close flag.
private boolean	writeCanNotFlushMarkerToWAL(MultiVersionConcurrencyControl.WriteEntry flushOpSeqIdMVCCEntry, WAL wal, boolean writeFlushWalMarker)	This method is only used when we flush but the memstore is empty,if writeFlushWalMarker is true,we write the WALProtos.FlushDescriptor.FlushAction.CANNOT_FLUSH flush marker to WAL when the memstore is empty.
private void	writeRegionCloseMarker(WAL wal)
protected void	writeRegionOpenMarker(WAL wal, long openSeqId)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.regionserver.Region

checkAndMutate, checkAndMutate, checkAndRowMutate, checkAndRowMutate

Field Details

LOG

private static final org.slf4j.Logger LOG

LOAD_CFS_ON_DEMAND_CONFIG_KEY

public static final String LOAD_CFS_ON_DEMAND_CONFIG_KEY

See Also:

• Constant Field Values

HBASE_MAX_CELL_SIZE_KEY

public static final String HBASE_MAX_CELL_SIZE_KEY

DEFAULT_MAX_CELL_SIZE

public static final int DEFAULT_MAX_CELL_SIZE

See Also:

• Constant Field Values

HBASE_REGIONSERVER_MINIBATCH_SIZE

public static final String HBASE_REGIONSERVER_MINIBATCH_SIZE

DEFAULT_HBASE_REGIONSERVER_MINIBATCH_SIZE

public static final int DEFAULT_HBASE_REGIONSERVER_MINIBATCH_SIZE

See Also:

• Constant Field Values

WAL_HSYNC_CONF_KEY

public static final String WAL_HSYNC_CONF_KEY

See Also:

• Constant Field Values

DEFAULT_WAL_HSYNC

public static final boolean DEFAULT_WAL_HSYNC

See Also:

• Constant Field Values

COMPACTION_AFTER_BULKLOAD_ENABLE

public static final String COMPACTION_AFTER_BULKLOAD_ENABLE

Parameter name for compaction after bulkload

See Also:

• Constant Field Values

SPLIT_IGNORE_BLOCKING_ENABLED_KEY

public static final String SPLIT_IGNORE_BLOCKING_ENABLED_KEY

Config for allow split when file count greater than the configured blocking file count

See Also:

• Constant Field Values

REGION_STORAGE_POLICY_KEY

public static final String REGION_STORAGE_POLICY_KEY

See Also:

• Constant Field Values

DEFAULT_REGION_STORAGE_POLICY

public static final String DEFAULT_REGION_STORAGE_POLICY

See Also:

• Constant Field Values

SPECIAL_RECOVERED_EDITS_DIR

public static final String SPECIAL_RECOVERED_EDITS_DIR

This is for for using HRegion as a local storage, where we may put the recovered edits in a special place. Once this is set, we will only replay the recovered edits under this directory and ignore the original replay directory configs.

See Also:

• Constant Field Values

RECOVERED_EDITS_IGNORE_EOF

public static final String RECOVERED_EDITS_IGNORE_EOF

Mainly used for master local region, where we will replay the WAL file directly without splitting, so it is possible to have WAL files which are not closed cleanly, in this way, hitting EOF is expected so should not consider it as a critical problem.

See Also:

• Constant Field Values

USE_META_CELL_COMPARATOR

public static final String USE_META_CELL_COMPARATOR

Whether to use MetaCellComparator even if we are not meta region. Used when creating master local region.

See Also:

• Constant Field Values

DEFAULT_USE_META_CELL_COMPARATOR

public static final boolean DEFAULT_USE_META_CELL_COMPARATOR

See Also:

• Constant Field Values

closed

final AtomicBoolean closed

closing

final AtomicBoolean closing

maxFlushedSeqId

private volatile long maxFlushedSeqId

The max sequence id of flushed data on this region. There is no edit in memory that is less that this sequence id.

lastFlushOpSeqId

private volatile long lastFlushOpSeqId

Record the sequence id of last flush operation. Can be in advance of maxFlushedSeqId when flushing a single column family. In this case, maxFlushedSeqId will be older than the oldest edit in memory.

lastReplayedOpenRegionSeqId

protected volatile long lastReplayedOpenRegionSeqId

The sequence id of the last replayed open region event from the primary region. This is used to skip entries before this due to the possibility of replay edits coming out of order from replication.

lastReplayedCompactionSeqId

protected volatile long lastReplayedCompactionSeqId

lockedRows

private final ConcurrentHashMap<HashedBytes,HRegion.RowLockContext> lockedRows

stores

protected final Map<byte[],HStore> stores

coprocessorServiceHandlers

private Map<String,org.apache.hbase.thirdparty.com.google.protobuf.Service> coprocessorServiceHandlers

memStoreSizing

private final MemStoreSizing memStoreSizing

regionServicesForStores

RegionServicesForStores regionServicesForStores

numMutationsWithoutWAL

final LongAdder numMutationsWithoutWAL

dataInMemoryWithoutWAL

final LongAdder dataInMemoryWithoutWAL

checkAndMutateChecksPassed

final LongAdder checkAndMutateChecksPassed

checkAndMutateChecksFailed

final LongAdder checkAndMutateChecksFailed

readRequestsCount

final LongAdder readRequestsCount

cpRequestsCount

final LongAdder cpRequestsCount

filteredReadRequestsCount

final LongAdder filteredReadRequestsCount

writeRequestsCount

final LongAdder writeRequestsCount

blockedRequestsCount

private final LongAdder blockedRequestsCount

compactionsFinished

final LongAdder compactionsFinished

compactionsFailed

final LongAdder compactionsFailed

compactionNumFilesCompacted

final LongAdder compactionNumFilesCompacted

compactionNumBytesCompacted

final LongAdder compactionNumBytesCompacted

compactionsQueued

final LongAdder compactionsQueued

flushesQueued

final LongAdder flushesQueued

blockCache

private BlockCache blockCache

mobFileCache

private MobFileCache mobFileCache

wal

private final WAL wal

fs

private final HRegionFileSystem fs

conf

protected final org.apache.hadoop.conf.Configuration conf

baseConf

private final org.apache.hadoop.conf.Configuration baseConf

rowLockWaitDuration

private final int rowLockWaitDuration

DEFAULT_ROWLOCK_WAIT_DURATION

static final int DEFAULT_ROWLOCK_WAIT_DURATION

See Also:

• Constant Field Values

regionWalDir

private org.apache.hadoop.fs.Path regionWalDir

walFS

private org.apache.hadoop.fs.FileSystem walFS

isRestoredRegion

private boolean isRestoredRegion

metricsTableRequests

private MetricsTableRequests metricsTableRequests

busyWaitDuration

final long busyWaitDuration

DEFAULT_BUSY_WAIT_DURATION

static final long DEFAULT_BUSY_WAIT_DURATION

See Also:

• Constant Field Values

maxBusyWaitMultiplier

final int maxBusyWaitMultiplier

maxBusyWaitDuration

final long maxBusyWaitDuration

maxCellSize

final long maxCellSize

miniBatchSize

private final int miniBatchSize

scannerReadPoints

final ConcurrentHashMap<RegionScanner,Long> scannerReadPoints

smallestReadPointCalcLock

final ReadPointCalculationLock smallestReadPointCalcLock

openSeqNum

private long openSeqNum

The sequence ID that was enLongAddered when this region was opened.

isLoadingCfsOnDemandDefault

private boolean isLoadingCfsOnDemandDefault

The default setting for whether to enable on-demand CF loading for scan requests to this region. Requests can override it.

majorInProgress

private final AtomicInteger majorInProgress

minorInProgress

private final AtomicInteger minorInProgress

maxSeqIdInStores

Map<byte[],Long> maxSeqIdInStores

replayLock

private Lock replayLock

prepareFlushResult

private HRegion.PrepareFlushResult prepareFlushResult

Saved state from replaying prepare flush cache

lastReplayedSequenceId

private long lastReplayedSequenceId

configurationManager

private volatile ConfigurationManager configurationManager

timeoutForWriteLock

private volatile Long timeoutForWriteLock

cellComparator

private final CellComparator cellComparator

minBlockSizeBytes

private final int minBlockSizeBytes

writestate

final HRegion.WriteState writestate

memstoreFlushSize

long memstoreFlushSize

timestampSlop

final long timestampSlop

lastStoreFlushTimeMap

private final ConcurrentMap<HStore,Long> lastStoreFlushTimeMap

rsServices

protected RegionServerServices rsServices

rsAccounting

private RegionServerAccounting rsAccounting

flushCheckInterval

private long flushCheckInterval

flushPerChanges

private long flushPerChanges

blockingMemStoreSize

private long blockingMemStoreSize

lock

final ReentrantReadWriteLock lock

regionLockHolders

final ConcurrentHashMap<Thread,Boolean> regionLockHolders

updatesLock

private final ReentrantReadWriteLock updatesLock

mvcc

private final MultiVersionConcurrencyControl mvcc

coprocessorHost

private volatile RegionCoprocessorHost coprocessorHost

htableDescriptor

private TableDescriptor htableDescriptor

splitPolicy

private RegionSplitPolicy splitPolicy

splitRestriction

private RegionSplitRestriction splitRestriction

flushPolicy

private FlushPolicy flushPolicy

metricsRegion

private final MetricsRegion metricsRegion

metricsRegionWrapper

private final MetricsRegionWrapperImpl metricsRegionWrapper

regionDurability

private final Durability regionDurability

regionStatsEnabled

private final boolean regionStatsEnabled

replicationScope

private final NavigableMap<byte[],Integer> replicationScope

storeHotnessProtector

private final StoreHotnessProtector storeHotnessProtector

regionReplicationSink

protected Optional<RegionReplicationSink> regionReplicationSink

closeLock

private final Object closeLock

FAIR_REENTRANT_CLOSE_LOCK

public static final String FAIR_REENTRANT_CLOSE_LOCK

Conf key for fair locking policy

See Also:

• Constant Field Values

DEFAULT_FAIR_REENTRANT_CLOSE_LOCK

public static final boolean DEFAULT_FAIR_REENTRANT_CLOSE_LOCK

See Also:

• Constant Field Values

MEMSTORE_PERIODIC_FLUSH_INTERVAL

public static final String MEMSTORE_PERIODIC_FLUSH_INTERVAL

Conf key for the periodic flush interval

DEFAULT_CACHE_FLUSH_INTERVAL

public static final int DEFAULT_CACHE_FLUSH_INTERVAL

Default interval for the memstore flush

See Also:

• Constant Field Values

SYSTEM_CACHE_FLUSH_INTERVAL

public static final int SYSTEM_CACHE_FLUSH_INTERVAL

Default interval for System tables memstore flush

See Also:

• Constant Field Values

MEMSTORE_FLUSH_PER_CHANGES

public static final String MEMSTORE_FLUSH_PER_CHANGES

Conf key to force a flush if there are already enough changes for one region in memstore

See Also:

• Constant Field Values

DEFAULT_FLUSH_PER_CHANGES

public static final long DEFAULT_FLUSH_PER_CHANGES

See Also:

• Constant Field Values

MAX_FLUSH_PER_CHANGES

public static final long MAX_FLUSH_PER_CHANGES

The following MAX_FLUSH_PER_CHANGES is large enough because each KeyValue has 20+ bytes overhead. Therefore, even 1G empty KVs occupy at least 20GB memstore size for a single region

See Also:

• Constant Field Values

CLOSE_WAIT_ABORT

public static final String CLOSE_WAIT_ABORT

See Also:

• Constant Field Values

DEFAULT_CLOSE_WAIT_ABORT

public static final boolean DEFAULT_CLOSE_WAIT_ABORT

See Also:

• Constant Field Values

CLOSE_WAIT_TIME

public static final String CLOSE_WAIT_TIME

See Also:

• Constant Field Values

DEFAULT_CLOSE_WAIT_TIME

public static final long DEFAULT_CLOSE_WAIT_TIME

See Also:

• Constant Field Values

CLOSE_WAIT_INTERVAL

public static final String CLOSE_WAIT_INTERVAL

See Also:

• Constant Field Values

DEFAULT_CLOSE_WAIT_INTERVAL

public static final long DEFAULT_CLOSE_WAIT_INTERVAL

See Also:

• Constant Field Values

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

DEEP_OVERHEAD

public static final long DEEP_OVERHEAD

Constructor Details

HRegion

@Deprecated
public HRegion(org.apache.hadoop.fs.Path tableDir,
 WAL wal,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.conf.Configuration confParam,
 RegionInfo regionInfo,
 TableDescriptor htd,
 RegionServerServices rsServices)

Deprecated.

Use other constructors.

HRegion constructor. This constructor should only be used for testing and extensions. Instances of HRegion should be instantiated with the createHRegion(org.apache.hadoop.hbase.client.RegionInfo, org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, org.apache.hadoop.hbase.client.TableDescriptor, org.apache.hadoop.hbase.wal.WAL, boolean) or openHRegion(org.apache.hadoop.hbase.client.RegionInfo, org.apache.hadoop.hbase.client.TableDescriptor, org.apache.hadoop.hbase.wal.WAL, org.apache.hadoop.conf.Configuration) method.

Parameters:

tableDir - qualified path of directory where region should be located, usually the table directory.

wal - The WAL is the outbound log for any updates to the HRegion The wal file is a logfile from the previous execution that's custom-computed for this HRegion. The HRegionServer computes and sorts the appropriate wal info for this HRegion. If there is a previous wal file (implying that the HRegion has been written-to before), then read it from the supplied path.

fs - is the filesystem.

confParam - is global configuration settings.

regionInfo - - RegionInfo that describes the region is new), then read them from the supplied path.

htd - the table descriptor

rsServices - reference to RegionServerServices or null

HRegion

public HRegion(HRegionFileSystem fs,
 WAL wal,
 org.apache.hadoop.conf.Configuration confParam,
 TableDescriptor htd,
 RegionServerServices rsServices)

HRegion constructor. This constructor should only be used for testing and extensions. Instances of HRegion should be instantiated with the createHRegion(org.apache.hadoop.hbase.client.RegionInfo, org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, org.apache.hadoop.hbase.client.TableDescriptor, org.apache.hadoop.hbase.wal.WAL, boolean) or openHRegion(org.apache.hadoop.hbase.client.RegionInfo, org.apache.hadoop.hbase.client.TableDescriptor, org.apache.hadoop.hbase.wal.WAL, org.apache.hadoop.conf.Configuration) method.

Parameters:

fs - is the filesystem.

wal - The WAL is the outbound log for any updates to the HRegion The wal file is a logfile from the previous execution that's custom-computed for this HRegion. The HRegionServer computes and sorts the appropriate wal info for this HRegion. If there is a previous wal file (implying that the HRegion has been written-to before), then read it from the supplied path.

confParam - is global configuration settings.

htd - the table descriptor

rsServices - reference to RegionServerServices or null

Method Details

setRestoredRegion

public void setRestoredRegion(boolean restoredRegion)

getMetricsTableRequests

public MetricsTableRequests getMetricsTableRequests()

getSmallestReadPoint

public long getSmallestReadPoint()

Returns:

The smallest mvcc readPoint across all the scanners in this region. Writes older than this readPoint, are included in every read operation.

setHTableSpecificConf

private void setHTableSpecificConf()

initialize

@Deprecated
public long initialize()
                throws IOException

Deprecated.

use HRegion.createHRegion() or HRegion.openHRegion()

Initialize this region. Used only by tests and SplitTransaction to reopen the region. You should use createHRegion() or openHRegion()

Returns:

What the next sequence (edit) id should be.

Throws:

IOException - e

initialize

long initialize(CancelableProgressable reporter)
         throws IOException

Initialize this region.

Parameters:

reporter - Tickle every so often if initialize is taking a while.

Returns:

What the next sequence (edit) id should be.

Throws:

IOException

initializeRegionInternals

private long initializeRegionInternals(CancelableProgressable reporter,
 MonitoredTask status)
                                throws IOException

Throws:

IOException

initializeRegionReplicationSink

private void initializeRegionReplicationSink(CancelableProgressable reporter,
 MonitoredTask status)

initializeStores

private long initializeStores(CancelableProgressable reporter,
 MonitoredTask status)
                       throws IOException

Open all Stores.

Returns:

Highest sequenceId found out in a Store.

Throws:

IOException

initializeStores

private long initializeStores(CancelableProgressable reporter,
 MonitoredTask status,
 boolean warmup)
                       throws IOException

Throws:

IOException

initializeWarmup

private void initializeWarmup(CancelableProgressable reporter)
                       throws IOException

Throws:

IOException

getStoreFiles

private NavigableMap<byte[],List<org.apache.hadoop.fs.Path>> getStoreFiles()

Returns Map of StoreFiles by column family

writeRegionOpenMarker

protected void writeRegionOpenMarker(WAL wal,
 long openSeqId)
                              throws IOException

Throws:

IOException

writeRegionCloseMarker

private void writeRegionCloseMarker(WAL wal)
                             throws IOException

Throws:

IOException

hasReferences

public boolean hasReferences()

Returns True if this region has references.

blockUpdates

public void blockUpdates()

unblockUpdates

public void unblockUpdates()

getHDFSBlocksDistribution

public HDFSBlocksDistribution getHDFSBlocksDistribution()

computeHDFSBlocksDistribution

public static HDFSBlocksDistribution computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf,
 TableDescriptor tableDescriptor,
 RegionInfo regionInfo)
                                                            throws IOException

This is a helper function to compute HDFS block distribution on demand

Parameters:

conf - configuration

tableDescriptor - TableDescriptor of the table

regionInfo - encoded name of the region

Returns:

The HDFS blocks distribution for the given region.

Throws:

IOException

computeHDFSBlocksDistribution

public static HDFSBlocksDistribution computeHDFSBlocksDistribution(org.apache.hadoop.conf.Configuration conf,
 TableDescriptor tableDescriptor,
 RegionInfo regionInfo,
 org.apache.hadoop.fs.Path tablePath)
                                                            throws IOException

This is a helper function to compute HDFS block distribution on demand

Parameters:

conf - configuration

tableDescriptor - TableDescriptor of the table

regionInfo - encoded name of the region

tablePath - the table directory

Returns:

The HDFS blocks distribution for the given region.

Throws:

IOException

incMemStoreSize

private void incMemStoreSize(MemStoreSize mss)

Increase the size of mem store in this region and the size of global mem store

incMemStoreSize

void incMemStoreSize(long dataSizeDelta,
 long heapSizeDelta,
 long offHeapSizeDelta,
 int cellsCountDelta)

decrMemStoreSize

void decrMemStoreSize(MemStoreSize mss)

decrMemStoreSize

private void decrMemStoreSize(long dataSizeDelta,
 long heapSizeDelta,
 long offHeapSizeDelta,
 int cellsCountDelta)

checkNegativeMemStoreDataSize

private void checkNegativeMemStoreDataSize(long memStoreDataSize,
 long delta)

getRegionInfo

public RegionInfo getRegionInfo()

Description copied from interface: Region

Returns region information for this region

Specified by:

getRegionInfo in interface Region

getRegionServerServices

RegionServerServices getRegionServerServices()

Returns Instance of RegionServerServices used by this HRegion. Can be null.

getReadRequestsCount

public long getReadRequestsCount()

Description copied from interface: Region

Returns read requests count for this region

Specified by:

getReadRequestsCount in interface Region

getCpRequestsCount

public long getCpRequestsCount()

Description copied from interface: Region

Returns coprocessor requests count for this region

Specified by:

getCpRequestsCount in interface Region

getFilteredReadRequestsCount

public long getFilteredReadRequestsCount()

Description copied from interface: Region

Returns filtered read requests count for this region

Specified by:

getFilteredReadRequestsCount in interface Region

getWriteRequestsCount

public long getWriteRequestsCount()

Description copied from interface: Region

Returns write request count for this region

Specified by:

getWriteRequestsCount in interface Region

getMemStoreDataSize

public long getMemStoreDataSize()

Specified by:

getMemStoreDataSize in interface Region

Returns:

memstore size for this region, in bytes. It just accounts data size of cells added to the memstores of this Region. Means size in bytes for key, value and tags within Cells. It wont consider any java heap overhead for the cell objects or any other.

getMemStoreHeapSize

public long getMemStoreHeapSize()

Specified by:

getMemStoreHeapSize in interface Region

Returns:

memstore heap size for this region, in bytes. It accounts data size of cells added to the memstores of this Region, as well as java heap overhead for the cell objects or any other.

getMemStoreOffHeapSize

public long getMemStoreOffHeapSize()

Specified by:

getMemStoreOffHeapSize in interface Region

Returns:

memstore off-heap size for this region, in bytes. It accounts data size of cells added to the memstores of this Region, as well as overhead for the cell objects or any other that is allocated off-heap.

getRegionServicesForStores

public RegionServicesForStores getRegionServicesForStores()

Returns store services for this region, to access services required by store level needs

getNumMutationsWithoutWAL

public long getNumMutationsWithoutWAL()

Description copied from interface: Region

Returns the number of mutations processed bypassing the WAL

Specified by:

getNumMutationsWithoutWAL in interface Region

getDataInMemoryWithoutWAL

public long getDataInMemoryWithoutWAL()

Description copied from interface: Region

Returns the size of data processed bypassing the WAL, in bytes

Specified by:

getDataInMemoryWithoutWAL in interface Region

getBlockedRequestsCount

public long getBlockedRequestsCount()

Description copied from interface: Region

Returns the number of blocked requests

Specified by:

getBlockedRequestsCount in interface Region

getCheckAndMutateChecksPassed

public long getCheckAndMutateChecksPassed()

Description copied from interface: Region

Returns the number of checkAndMutate guards that passed

Specified by:

getCheckAndMutateChecksPassed in interface Region

getCheckAndMutateChecksFailed

public long getCheckAndMutateChecksFailed()

Description copied from interface: Region

Returns the number of failed checkAndMutate guards

Specified by:

getCheckAndMutateChecksFailed in interface Region

getMetrics

public MetricsRegion getMetrics()

isClosed

public boolean isClosed()

Description copied from interface: Region

Returns true if region is closed

Specified by:

isClosed in interface Region

isClosing

public boolean isClosing()

Description copied from interface: Region

Returns True if closing process has started

Specified by:

isClosing in interface Region

isReadOnly

public boolean isReadOnly()

Description copied from interface: Region

Returns True if region is read only

Specified by:

isReadOnly in interface Region

isAvailable

public boolean isAvailable()

Description copied from interface: Region

Returns true if region is available (not closed and not closing)

Specified by:

isAvailable in interface Region

isSplittable

public boolean isSplittable()

Description copied from interface: Region

Returns true if region is splittable

Specified by:

isSplittable in interface Region

isMergeable

public boolean isMergeable()

Description copied from interface: Region

Returns true if region is mergeable

Specified by:

isMergeable in interface Region

areWritesEnabled

public boolean areWritesEnabled()

getMVCC

public MultiVersionConcurrencyControl getMVCC()

getMaxFlushedSeqId

public long getMaxFlushedSeqId()

Specified by:

getMaxFlushedSeqId in interface Region

Returns:

the max sequence id of flushed data on this region; no edit in memory will have a sequence id that is less that what is returned here.

getReadPoint

public long getReadPoint(IsolationLevel isolationLevel)

Returns readpoint considering given IsolationLevel. Pass null for default

isLoadingCfsOnDemandDefault

public boolean isLoadingCfsOnDemandDefault()

close

public Map<byte[],List<HStoreFile>> close()
                                   throws IOException

Close down this HRegion. Flush the cache, shut down each HStore, don't service any more calls.

This method could take some time to execute, so don't call it from a time-sensitive thread.

Returns:

Vector of all the storage files that the HRegion's component HStores make use of. It's a list of all StoreFile objects. Returns empty vector if already closed and null if judged that it should not close.

Throws:

IOException - e

DroppedSnapshotException - Thrown when replay of wal is required because a Snapshot was not properly persisted. The region is put in closing mode, and the caller MUST abort after this.

close

public Map<byte[],List<HStoreFile>> close(boolean abort)
                                   throws IOException

Throws:

IOException

close

public Map<byte[],List<HStoreFile>> close(boolean abort,
 boolean ignoreStatus)
                                   throws IOException

Close this HRegion.

Parameters:

abort - true if server is aborting (only during testing)

ignoreStatus - true if ignore the status (won't be showed on task list)

Returns:

Vector of all the storage files that the HRegion's component HStores make use of. It's a list of StoreFile objects. Can be null if we are not to close at this time, or we are already closed.

Throws:

IOException - e

DroppedSnapshotException - Thrown when replay of wal is required because a Snapshot was not properly persisted. The region is put in closing mode, and the caller MUST abort after this.

close

public Map<byte[],List<HStoreFile>> close(boolean abort,
 boolean ignoreStatus,
 boolean isGracefulStop)
                                   throws IOException

Close down this HRegion. Flush the cache unless abort parameter is true, Shut down each HStore, don't service any more calls. This method could take some time to execute, so don't call it from a time-sensitive thread.

Parameters:

abort - true if server is aborting (only during testing)

ignoreStatus - true if ignore the status (wont be showed on task list)

isGracefulStop - true if region is being closed during graceful stop and the blocks in the BucketCache should not be evicted.

Returns:

Vector of all the storage files that the HRegion's component HStores make use of. It's a list of StoreFile objects. Can be null if we are not to close at this time or we are already closed.

Throws:

IOException - e

DroppedSnapshotException - Thrown when replay of wal is required because a Snapshot was not properly persisted. The region is put in closing mode, and the caller MUST abort after this.

setClosing

public void setClosing(boolean closing)

Exposed for some very specific unit tests.

setTimeoutForWriteLock

public void setTimeoutForWriteLock(long timeoutForWriteLock)

The doClose(boolean, org.apache.hadoop.hbase.monitoring.MonitoredTask) will block forever if someone tries proving the dead lock via the unit test. Instead of blocking, the doClose(boolean, org.apache.hadoop.hbase.monitoring.MonitoredTask) will throw exception if you set the timeout.

Parameters:

timeoutForWriteLock - the second time to wait for the write lock in doClose(boolean, org.apache.hadoop.hbase.monitoring.MonitoredTask)

doClose

private Map<byte[],List<HStoreFile>> doClose(boolean abort,
 MonitoredTask status)
                                      throws IOException

Throws:

IOException

waitForFlushesAndCompactions

public void waitForFlushesAndCompactions()

Wait for all current flushes and compactions of the region to complete

waitForFlushes

public void waitForFlushes()

Wait for all current flushes of the region to complete

waitForFlushes

public boolean waitForFlushes(long timeout)

Description copied from interface: Region

Wait for all current flushes of the region to complete

Specified by:

waitForFlushes in interface Region

Parameters:

timeout - The maximum time to wait in milliseconds.

Returns:

False when timeout elapsed but flushes are not over. True when flushes are over within max wait time period.

getReadOnlyConfiguration

public org.apache.hadoop.conf.Configuration getReadOnlyConfiguration()

Specified by:

getReadOnlyConfiguration in interface Region

Returns:

a read only configuration of this region; throws UnsupportedOperationException if you try to set a configuration.

getMinBlockSizeBytes

public int getMinBlockSizeBytes()

Description copied from interface: Region

The minimum block size configuration from all relevant column families. This is used when estimating quota consumption.

Specified by:

getMinBlockSizeBytes in interface Region

getStoreOpenAndCloseThreadPool

private ThreadPoolExecutor getStoreOpenAndCloseThreadPool(String threadNamePrefix)

getStoreFileOpenAndCloseThreadPool

ThreadPoolExecutor getStoreFileOpenAndCloseThreadPool(String threadNamePrefix)

getOpenAndCloseThreadPool

private static ThreadPoolExecutor getOpenAndCloseThreadPool(int maxThreads,
 String threadNamePrefix)

worthPreFlushing

private boolean worthPreFlushing()

Returns True if its worth doing a flush before we put up the close flag.

getTableDescriptor

public TableDescriptor getTableDescriptor()

Description copied from interface: Region

Returns table descriptor for this region

Specified by:

getTableDescriptor in interface Region

setTableDescriptor

public void setTableDescriptor(TableDescriptor desc)

getWAL

public WAL getWAL()

Returns WAL in use for this region

getBlockCache

public BlockCache getBlockCache()

setBlockCache

public void setBlockCache(BlockCache blockCache)

Only used for unit test which doesn't start region server.

getMobFileCache

public MobFileCache getMobFileCache()

setMobFileCache

public void setMobFileCache(MobFileCache mobFileCache)

Only used for unit test which doesn't start region server.

getSplitPolicy

RegionSplitPolicy getSplitPolicy()

Returns split policy for this region.

getBaseConf

org.apache.hadoop.conf.Configuration getBaseConf()

A split takes the config from the parent region & passes it to the daughter region's constructor. If 'conf' was passed, you would end up using the HTD of the parent region in addition to the new daughter HTD. Pass 'baseConf' to the daughter regions to avoid this tricky dedupe problem.

Returns:

Configuration object

getFilesystem

public org.apache.hadoop.fs.FileSystem getFilesystem()

Returns FileSystem being used by this region

getRegionFileSystem

public HRegionFileSystem getRegionFileSystem()

Returns the HRegionFileSystem used by this region

getRegionWALFileSystem

HRegionWALFileSystem getRegionWALFileSystem()
                                     throws IOException

Returns the WAL HRegionFileSystem used by this region

Throws:

IOException

getWalFileSystem

org.apache.hadoop.fs.FileSystem getWalFileSystem()
                                          throws IOException

Returns the WAL FileSystem being used by this region

Throws:

IOException

getWALRegionDir

public org.apache.hadoop.fs.Path getWALRegionDir()
                                          throws IOException

Returns:

the Region directory under WALRootDirectory

Throws:

IOException - if there is an error getting WALRootDir

getEarliestFlushTimeForAllStores

public long getEarliestFlushTimeForAllStores()

Specified by:

getEarliestFlushTimeForAllStores in interface Region

Returns:

The earliest time a store in the region was flushed. All other stores in the region would have been flushed either at, or after this time.

getOldestHfileTs

public long getOldestHfileTs(boolean majorCompactionOnly)
                      throws IOException

Description copied from interface: Region

This can be used to determine the last time all files of this region were major compacted.

Specified by:

getOldestHfileTs in interface Region

Parameters:

majorCompactionOnly - Only consider HFile that are the result of major compaction

Returns:

the timestamp of the oldest HFile for all stores of this region

Throws:

IOException

setCompleteSequenceId

org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad.Builder setCompleteSequenceId(org.apache.hadoop.hbase.shaded.protobuf.generated.ClusterStatusProtos.RegionLoad.Builder regionLoadBldr)

doRegionCompactionPrep

protected void doRegionCompactionPrep()
                               throws IOException

Do preparation for pending compaction.

Throws:

IOException

compact

public void compact(boolean majorCompaction)
             throws IOException

Synchronously compact all stores in the region.

This operation could block for a long time, so don't call it from a time-sensitive thread.

Note that no locks are taken to prevent possible conflicts between compaction and splitting activities. The regionserver does not normally compact and split in parallel. However by calling this method you may introduce unexpected and unhandled concurrency. Don't do this unless you know what you are doing.

Parameters:

majorCompaction - True to force a major compaction regardless of thresholds

Throws:

IOException

compactStores

public void compactStores()
                   throws IOException

This is a helper function that compact all the stores synchronously.

It is used by utilities and testing

Throws:

IOException

compactStore

void compactStore(byte[] family,
 ThroughputController throughputController)
           throws IOException

This is a helper function that compact the given store.

It is used by utilities and testing

Throws:

IOException

compact

public boolean compact(CompactionContext compaction,
 HStore store,
 ThroughputController throughputController)
                throws IOException

Called by compaction thread and after region is opened to compact the HStores if necessary.

This operation could block for a long time, so don't call it from a time-sensitive thread. Note that no locking is necessary at this level because compaction only conflicts with a region split, and that cannot happen because the region server does them sequentially and not in parallel.

Parameters:

compaction - Compaction details, obtained by requestCompaction()

Returns:

whether the compaction completed

Throws:

IOException

shouldForbidMajorCompaction

private boolean shouldForbidMajorCompaction()

compact

public boolean compact(CompactionContext compaction,
 HStore store,
 ThroughputController throughputController,
 User user)
                throws IOException

We are trying to remove / relax the region read lock for compaction. Let's see what are the potential race conditions among the operations (user scan, region split, region close and region bulk load).

   user scan ---> region read lock
   region split --> region close first --> region write lock
   region close --> region write lock
   region bulk load --> region write lock
 

read lock is compatible with read lock. ---> no problem with user scan/read region bulk load does not cause problem for compaction (no consistency problem, store lock will help the store file accounting). They can run almost concurrently at the region level.

The only remaining race condition is between the region close and compaction. So we will evaluate, below, how region close intervenes with compaction if compaction does not acquire region read lock.

Here are the steps for compaction:

1. obtain list of StoreFile's
2. create StoreFileScanner's based on list from #1
3. perform compaction and save resulting files under tmp dir
4. swap in compacted files

#1 is guarded by store lock. This patch does not change this --> no worse or better For #2, we obtain smallest read point (for region) across all the Scanners (for both default compactor and stripe compactor). The read points are for user scans. Region keeps the read points for all currently open user scanners. Compaction needs to know the smallest read point so that during re-write of the hfiles, it can remove the mvcc points for the cells if their mvccs are older than the smallest since they are not needed anymore. This will not conflict with compaction.

For #3, it can be performed in parallel to other operations.

For #4 bulk load and compaction don't conflict with each other on the region level (for multi-family atomicy).

Region close and compaction are guarded pretty well by the 'writestate'. In HRegion#doClose(), we have :

 synchronized (writestate) {
   // Disable compacting and flushing by background threads for this
   // region.
   canFlush = !writestate.readOnly;
   writestate.writesEnabled = false;
   LOG.debug("Closing " + this + ": disabling compactions & flushes");
   waitForFlushesAndCompactions();
 }
 

waitForFlushesAndCompactions() would wait for writestate.compacting to come down to 0. and in HRegion.compact()

   try {
     synchronized (writestate) {
       if (writestate.writesEnabled) {
         wasStateSet = true;
         ++writestate.compacting;
       } else {
         String msg = "NOT compacting region " + this + ". Writes disabled.";
         LOG.info(msg);
         status.abort(msg);
         return false;
       }
     }
   }
 

Also in compactor.performCompaction(): check periodically to see if a system stop is requested

 if (closeChecker != null && closeChecker.isTimeLimit(store, now)) {
   progress.cancel();
   return false;
 }
 if (closeChecker != null && closeChecker.isSizeLimit(store, len)) {
   progress.cancel();
   return false;
 }
 

Throws:

IOException

flush

public HRegion.FlushResult flush(boolean flushAllStores)
                          throws IOException

Flush the cache.

When this method is called the cache will be flushed unless:

1. the cache is empty
2. the region is closed.
3. a flush is already in progress
4. writes are disabled

This method may block for some time, so it should not be called from a time-sensitive thread.

Parameters:

flushAllStores - whether we want to force a flush of all stores

Returns:

FlushResult indicating whether the flush was successful or not and if the region needs compacting

Throws:

IOException - general io exceptions because a snapshot was not properly persisted.

flushcache

public HRegion.FlushResultImpl flushcache(boolean flushAllStores,
 boolean writeFlushRequestWalMarker,
 FlushLifeCycleTracker tracker)
                                   throws IOException

Throws:

IOException

flushcache

public HRegion.FlushResultImpl flushcache(List<byte[]> families,
 boolean writeFlushRequestWalMarker,
 FlushLifeCycleTracker tracker)
                                   throws IOException

Flush the cache. When this method is called the cache will be flushed unless:

1. the cache is empty
2. the region is closed.
3. a flush is already in progress
4. writes are disabled

This method may block for some time, so it should not be called from a time-sensitive thread.

Parameters:

families - stores of region to flush.

writeFlushRequestWalMarker - whether to write the flush request marker to WAL

tracker - used to track the life cycle of this flush

Returns:

whether the flush is success and whether the region needs compacting

Throws:

IOException - general io exceptions

DroppedSnapshotException - Thrown when replay of wal is required because a Snapshot was not properly persisted. The region is put in closing mode, and the caller MUST abort after this.

getSpecificStores

private Collection<HStore> getSpecificStores(List<byte[]> families)

get stores which matches the specified families

Returns:

the stores need to be flushed.

shouldFlushStore

boolean shouldFlushStore(HStore store)

Should the store be flushed because it is old enough.

Every FlushPolicy should call this to determine whether a store is old enough to flush (except that you always flush all stores). Otherwise the method will always returns true which will make a lot of flush requests.

shouldFlush

boolean shouldFlush(StringBuilder whyFlush)

Should the memstore be flushed now

internalFlushcache

private HRegion.FlushResult internalFlushcache(MonitoredTask status)
                                        throws IOException

Flushing all stores.

Throws:

IOException

See Also:

• internalFlushcache(Collection, MonitoredTask, boolean, FlushLifeCycleTracker)

internalFlushcache

private HRegion.FlushResultImpl internalFlushcache(Collection<HStore> storesToFlush,
 MonitoredTask status,
 boolean writeFlushWalMarker,
 FlushLifeCycleTracker tracker)
                                            throws IOException

Flushing given stores.

Throws:

IOException

See Also:

• internalFlushcache(WAL, long, Collection, MonitoredTask, boolean, FlushLifeCycleTracker)

internalFlushcache

protected HRegion.FlushResultImpl internalFlushcache(WAL wal,
 long myseqid,
 Collection<HStore> storesToFlush,
 MonitoredTask status,
 boolean writeFlushWalMarker,
 FlushLifeCycleTracker tracker)
                                              throws IOException

Flush the memstore. Flushing the memstore is a little tricky. We have a lot of updates in the memstore, all of which have also been written to the wal. We need to write those updates in the memstore out to disk, while being able to process reads/writes as much as possible during the flush operation.

This method may block for some time. Every time you call it, we up the regions sequence id even if we don't flush; i.e. the returned region id will be at least one larger than the last edit applied to this region. The returned id does not refer to an actual edit. The returned id can be used for say installing a bulk loaded file just ahead of the last hfile that was the result of this flush, etc.

Parameters:

wal - Null if we're NOT to go via wal.

myseqid - The seqid to use if wal is null writing out flush file.

storesToFlush - The list of stores to flush.

Returns:

object describing the flush's state

Throws:

IOException - general io exceptions

DroppedSnapshotException - Thrown when replay of WAL is required.

internalPrepareFlushCache

protected HRegion.PrepareFlushResult internalPrepareFlushCache(WAL wal,
 long myseqid,
 Collection<HStore> storesToFlush,
 MonitoredTask status,
 boolean writeFlushWalMarker,
 FlushLifeCycleTracker tracker)
                                                        throws IOException

Throws:

IOException

logFatLineOnFlush

private void logFatLineOnFlush(Collection<HStore> storesToFlush,
 long sequenceId)

Utility method broken out of internalPrepareFlushCache so that method is smaller.

doAbortFlushToWAL

private void doAbortFlushToWAL(WAL wal,
 long flushOpSeqId,
 Map<byte[],List<org.apache.hadoop.fs.Path>> committedFiles)

doSyncOfUnflushedWALChanges

private static void doSyncOfUnflushedWALChanges(WAL wal,
 RegionInfo hri)
                                         throws IOException

Sync unflushed WAL changes. See HBASE-8208 for details

Throws:

IOException

isAllFamilies

private boolean isAllFamilies(Collection<HStore> families)

Returns True if passed Set is all families in the region.

writeCanNotFlushMarkerToWAL

private boolean writeCanNotFlushMarkerToWAL(MultiVersionConcurrencyControl.WriteEntry flushOpSeqIdMVCCEntry,
 WAL wal,
 boolean writeFlushWalMarker)

This method is only used when we flush but the memstore is empty,if writeFlushWalMarker is true,we write the WALProtos.FlushDescriptor.FlushAction.CANNOT_FLUSH flush marker to WAL when the memstore is empty. Ignores exceptions from WAL. Returns whether the write succeeded.

Returns:

whether WAL write was successful

attachRegionReplicationToFlushOpSeqIdMVCCEntry

private void attachRegionReplicationToFlushOpSeqIdMVCCEntry(MultiVersionConcurrencyControl.WriteEntry flushOpSeqIdMVCCEntry,
 org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor desc,
 RegionReplicationSink sink)

Create WALEdit for WALProtos.FlushDescriptor and attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the flushOpSeqIdMVCCEntry.

internalFlushCacheAndCommit

HRegion.FlushResultImpl internalFlushCacheAndCommit(WAL wal,
 MonitoredTask status,
 HRegion.PrepareFlushResult prepareResult,
 Collection<HStore> storesToFlush)
                                             throws IOException

Throws:

IOException

getNextSequenceId

protected long getNextSequenceId(WAL wal)
                          throws IOException

Method to safely get the next sequence number.

Returns:

Next sequence number unassociated with any actual edit.

Throws:

IOException

getScanner

public RegionScannerImpl getScanner(Scan scan)
                             throws IOException

Description copied from interface: Region

Return an iterator that scans over the HRegion, returning the indicated columns and rows specified by the Scan.

This Iterator must be closed by the caller.

Specified by:

getScanner in interface Region

Parameters:

scan - configured Scan

Throws:

IOException - read exceptions

getScanner

public RegionScannerImpl getScanner(Scan scan,
 List<KeyValueScanner> additionalScanners)
                             throws IOException

Description copied from interface: Region

Return an iterator that scans over the HRegion, returning the indicated columns and rows specified by the Scan. The scanner will also include the additional scanners passed along with the scanners for the specified Scan instance. Should be careful with the usage to pass additional scanners only within this Region

This Iterator must be closed by the caller.

Specified by:

getScanner in interface Region

Parameters:

scan - configured Scan

additionalScanners - Any additional scanners to be used

Throws:

IOException - read exceptions

getScanner

private RegionScannerImpl getScanner(Scan scan,
 List<KeyValueScanner> additionalScanners,
 long nonceGroup,
 long nonce)
                              throws IOException

Throws:

IOException

instantiateRegionScanner

protected RegionScannerImpl instantiateRegionScanner(Scan scan,
 List<KeyValueScanner> additionalScanners,
 long nonceGroup,
 long nonce)
                                              throws IOException

Throws:

IOException

prepareDelete

private void prepareDelete(Delete delete)
                    throws IOException

Prepare a delete for a row mutation processor

Parameters:

delete - The passed delete is modified by this method. WARNING!

Throws:

IOException

delete

public void delete(Delete delete)
            throws IOException

Description copied from interface: Region

Deletes the specified cells/row.

Specified by:

delete in interface Region

Throws:

IOException

prepareDeleteTimestamps

private void prepareDeleteTimestamps(Mutation mutation,
 Map<byte[],List<ExtendedCell>> familyMap,
 byte[] byteNow)
                              throws IOException

Set up correct timestamps in the KVs in Delete object.

Caller should have the row and region locks.

Throws:

IOException

updateDeleteLatestVersionTimestamp

private void updateDeleteLatestVersionTimestamp(Cell cell,
 Get get,
 int count,
 byte[] byteNow)
                                         throws IOException

Throws:

IOException

put

public void put(Put put)
         throws IOException

Description copied from interface: Region

Puts some data in the table.

Specified by:

put in interface Region

Throws:

IOException

batchMutate

public OperationStatus[] batchMutate(Mutation[] mutations,
 boolean atomic,
 long nonceGroup,
 long nonce)
                              throws IOException

Throws:

IOException

batchMutate

public OperationStatus[] batchMutate(Mutation[] mutations)
                              throws IOException

Description copied from interface: Region

Perform a batch of mutations.

Please do not operate on a same column of a single row in a batch, we will not consider the previous operation in the same batch when performing the operations in the batch.

Specified by:

batchMutate in interface Region

Parameters:

mutations - the list of mutations

Returns:

an array of OperationStatus which internally contains the OperationStatusCode and the exceptionMessage if any.

Throws:

IOException

batchMutate

OperationStatus[] batchMutate(Mutation[] mutations,
 boolean atomic)
                       throws IOException

Throws:

IOException

batchReplay

@Deprecated
OperationStatus[] batchReplay(WALSplitUtil.MutationReplay[] mutations,
 long replaySeqId)
                       throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Now we use replayWALEntry(WALEntry, CellScanner) for replaying edits at secondary replica side.

Throws:

IOException

batchMutate

private OperationStatus[] batchMutate(HRegion.BatchOperation<?> batchOp)
                               throws IOException

Perform a batch of mutations.

Operations in a batch are stored with highest durability specified of for all operations in a batch, except for Durability.SKIP_WAL.

This function is called from batchReplay(WALSplitUtil.MutationReplay[], long) with HRegion.ReplayBatchOperation instance and batchMutate(Mutation[]) with HRegion.MutationBatchOperation instance as an argument. As the processing of replay batch and mutation batch is very similar, lot of code is shared by providing generic methods in base class HRegion.BatchOperation. The logic for this method and doMiniBatchMutate(BatchOperation) is implemented using methods in base class which are overridden by derived classes to implement special behavior.

Parameters:

batchOp - contains the list of mutations

Returns:

an array of OperationStatus which internally contains the OperationStatusCode and the exceptionMessage if any.

Throws:

IOException - if an IO problem is encountered

doMiniBatchMutate

private void doMiniBatchMutate(HRegion.BatchOperation<?> batchOp)
                        throws IOException

Called to do a piece of the batch that came in to batchMutate(Mutation[]) In here we also handle replay of edits on region recover. Also gets change in size brought about by applying batchOp.

Throws:

IOException

getEffectiveDurability

private Durability getEffectiveDurability(Durability d)

Returns effective durability from the passed durability and the table descriptor.

checkAndMutate

@Deprecated
public boolean checkAndMutate(byte[] row,
 byte[] family,
 byte[] qualifier,
 CompareOperator op,
 ByteArrayComparable comparator,
 TimeRange timeRange,
 Mutation mutation)
                       throws IOException

Deprecated.

Description copied from interface: Region

Atomically checks if a row/family/qualifier value matches the expected value and if it does, it performs the mutation. If the passed value is null, the lack of column value (ie: non-existence) is used. See checkAndRowMutate to do many checkAndPuts at a time on a single row.

Specified by:

checkAndMutate in interface Region

Parameters:

row - to check

family - column family to check

qualifier - column qualifier to check

op - the comparison operator

comparator - the expected value

timeRange - time range to check

mutation - data to put if check succeeds

Returns:

true if mutation was applied, false otherwise

Throws:

IOException

checkAndMutate

@Deprecated
public boolean checkAndMutate(byte[] row,
 Filter filter,
 TimeRange timeRange,
 Mutation mutation)
                       throws IOException

Deprecated.

Description copied from interface: Region

Atomically checks if a row value matches the filter and if it does, it performs the mutation. See checkAndRowMutate to do many checkAndPuts at a time on a single row.

Specified by:

checkAndMutate in interface Region

Parameters:

row - to check

filter - the filter

timeRange - time range to check

mutation - data to put if check succeeds

Returns:

true if mutation was applied, false otherwise

Throws:

IOException

checkAndRowMutate

@Deprecated
public boolean checkAndRowMutate(byte[] row,
 byte[] family,
 byte[] qualifier,
 CompareOperator op,
 ByteArrayComparable comparator,
 TimeRange timeRange,
 RowMutations rm)
                          throws IOException

Deprecated.

Description copied from interface: Region

Atomically checks if a row/family/qualifier value matches the expected values and if it does, it performs the row mutations. If the passed value is null, the lack of column value (ie: non-existence) is used. Use to do many mutations on a single row. Use checkAndMutate to do one checkAndMutate at a time.

Specified by:

checkAndRowMutate in interface Region

Parameters:

row - to check

family - column family to check

qualifier - column qualifier to check

op - the comparison operator

comparator - the expected value

timeRange - time range to check

rm - data to put if check succeeds

Returns:

true if mutations were applied, false otherwise

Throws:

IOException

checkAndRowMutate

@Deprecated
public boolean checkAndRowMutate(byte[] row,
 Filter filter,
 TimeRange timeRange,
 RowMutations rm)
                          throws IOException

Deprecated.

Description copied from interface: Region

Atomically checks if a row matches the filter and if it does, it performs the row mutations. Use to do many mutations on a single row. Use checkAndMutate to do one checkAndMutate at a time.

Specified by:

checkAndRowMutate in interface Region

Parameters:

row - to check

filter - the filter

timeRange - time range to check

rm - data to put if check succeeds

Returns:

true if mutations were applied, false otherwise

Throws:

IOException

checkAndMutate

public CheckAndMutateResult checkAndMutate(CheckAndMutate checkAndMutate)
                                    throws IOException

Description copied from interface: Region

Atomically checks if a row matches the conditions and if it does, it performs the actions. Use to do many mutations on a single row. Use checkAndMutate to do one checkAndMutate at a time.

Specified by:

checkAndMutate in interface Region

Parameters:

checkAndMutate - the CheckAndMutate object

Returns:

true if mutations were applied, false otherwise

Throws:

IOException - if an error occurred in this method

checkAndMutate

public CheckAndMutateResult checkAndMutate(CheckAndMutate checkAndMutate,
 long nonceGroup,
 long nonce)
                                    throws IOException

Throws:

IOException

checkAndMutateInternal

private CheckAndMutateResult checkAndMutateInternal(CheckAndMutate checkAndMutate,
 long nonceGroup,
 long nonce)
                                             throws IOException

Throws:

IOException

checkMutationType

private void checkMutationType(Mutation mutation)
                        throws DoNotRetryIOException

Throws:

DoNotRetryIOException

checkRow

private void checkRow(Row action,
 byte[] row)
               throws DoNotRetryIOException

Throws:

DoNotRetryIOException

matches

private boolean matches(CompareOperator op,
 int compareResult)

mutate

private OperationStatus mutate(Mutation mutation)
                        throws IOException

Throws:

IOException

mutate

private OperationStatus mutate(Mutation mutation,
 boolean atomic)
                        throws IOException

Throws:

IOException

mutate

private OperationStatus mutate(Mutation mutation,
 boolean atomic,
 long nonceGroup,
 long nonce)
                        throws IOException

Throws:

IOException

addRegionToSnapshot

public void addRegionToSnapshot(org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription desc,
 ForeignExceptionSnare exnSnare)
                         throws IOException

Complete taking the snapshot on the region. Writes the region info and adds references to the working snapshot directory. TODO for api consistency, consider adding another version with no ForeignExceptionSnare arg. (In the future other cancellable HRegion methods could eventually add a ForeignExceptionSnare, or we could do something fancier).

Parameters:

desc - snapshot description object

exnSnare - ForeignExceptionSnare that captures external exceptions in case we need to bail out. This is allowed to be null and will just be ignored in that case.

Throws:

IOException - if there is an external or internal error causing the snapshot to fail

updateSequenceId

private void updateSequenceId(Iterable<List<ExtendedCell>> cellItr,
 long sequenceId)
                       throws IOException

Throws:

IOException

updateCellTimestamps

private static void updateCellTimestamps(Iterable<List<ExtendedCell>> cellItr,
 byte[] now)
                                  throws IOException

Replace any cell timestamps set to HConstants.LATEST_TIMESTAMP provided current timestamp.

Throws:

IOException

rewriteCellTags

private void rewriteCellTags(Map<byte[],List<ExtendedCell>> familyMap,
 Mutation m)

Possibly rewrite incoming cell tags.

checkResources

private void checkResources()
                     throws RegionTooBusyException

Check if resources to support an update.

We throw RegionTooBusyException if above memstore limit and expect client to retry using some kind of backoff

Throws:

RegionTooBusyException

checkReadOnly

private void checkReadOnly()
                    throws IOException

Throws:

IOException - Throws exception if region is in read-only mode.

checkReadsEnabled

private void checkReadsEnabled()
                        throws IOException

Throws:

IOException

setReadsEnabled

public void setReadsEnabled(boolean readsEnabled)

applyToMemStore

private void applyToMemStore(HStore store,
 List<ExtendedCell> cells,
 boolean delta,
 MemStoreSizing memstoreAccounting)

Parameters:

delta - If we are doing delta changes -- e.g. increment/append -- then this flag will be set; when set we will run operations that make sense in the increment/append scenario but that do not make sense otherwise.

checkFamilies

private void checkFamilies(Collection<byte[]> families,
 Durability durability)
                    throws NoSuchColumnFamilyException,
InvalidMutationDurabilityException

Throws:

NoSuchColumnFamilyException

InvalidMutationDurabilityException

checkFamily

private void checkFamily(byte[] family,
 Durability durability)
                  throws NoSuchColumnFamilyException,
InvalidMutationDurabilityException

Throws:

NoSuchColumnFamilyException

InvalidMutationDurabilityException

checkFamily

private void checkFamily(byte[] family)
                  throws NoSuchColumnFamilyException

Throws:

NoSuchColumnFamilyException

checkTimestamps

public void checkTimestamps(Map<byte[],List<Cell>> familyMap,
 long now)
                     throws FailedSanityCheckException

Check the collection of families for valid timestamps

Parameters:

now - current timestamp

Throws:

FailedSanityCheckException

isFlushSize

private boolean isFlushSize(MemStoreSize size)

deleteRecoveredEdits

private void deleteRecoveredEdits(org.apache.hadoop.fs.FileSystem fs,
 Iterable<org.apache.hadoop.fs.Path> files)
                           throws IOException

Throws:

IOException

replayRecoveredEditsIfAny

long replayRecoveredEditsIfAny(Map<byte[],Long> maxSeqIdInStores,
 CancelableProgressable reporter,
 MonitoredTask status)
                        throws IOException

Read the edits put under this region by wal splitting process. Put the recovered edits back up into this region.

We can ignore any wal message that has a sequence ID that's equal to or lower than minSeqId. (Because we know such messages are already reflected in the HFiles.)

While this is running we are putting pressure on memory yet we are outside of our usual accounting because we are not yet an onlined region (this stuff is being run as part of Region initialization). This means that if we're up against global memory limits, we'll not be flagged to flush because we are not online. We can't be flushed by usual mechanisms anyways; we're not yet online so our relative sequenceids are not yet aligned with WAL sequenceids -- not till we come up online, post processing of split edits.

But to help relieve memory pressure, at least manage our own heap size flushing if are in excess of per-region limits. Flushing, though, we have to be careful and avoid using the regionserver/wal sequenceid. Its running on a different line to whats going on in here in this region context so if we crashed replaying these edits, but in the midst had a flush that used the regionserver wal with a sequenceid in excess of whats going on in here in this region and with its split editlogs, then we could miss edits the next time we go to recover. So, we have to flush inline, using seqids that make sense in a this single region context only -- until we online.

Parameters:

maxSeqIdInStores - Any edit found in split editlogs needs to be in excess of the maxSeqId for the store to be applied, else its skipped.

Returns:

the sequence id of the last edit added to this region out of the recovered edits log or minSeqId if nothing added from editlogs.

Throws:

IOException

replayRecoveredEditsForPaths

private long replayRecoveredEditsForPaths(long minSeqIdForTheRegion,
 org.apache.hadoop.fs.FileSystem fs,
 NavigableSet<org.apache.hadoop.fs.Path> files,
 CancelableProgressable reporter,
 org.apache.hadoop.fs.Path regionDir)
                                   throws IOException

Throws:

IOException

handleException

private void handleException(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path edits,
 IOException e)
                      throws IOException

Throws:

IOException

replayRecoveredEdits

private long replayRecoveredEdits(org.apache.hadoop.fs.Path edits,
 Map<byte[],Long> maxSeqIdInStores,
 CancelableProgressable reporter,
 org.apache.hadoop.fs.FileSystem fs)
                           throws IOException

Parameters:

edits - File of recovered edits.

maxSeqIdInStores - Maximum sequenceid found in each store. Edits in wal must be larger than this to be replayed for each store.

Returns:

the sequence id of the last edit added to this region out of the recovered edits log or minSeqId if nothing added from editlogs.

Throws:

IOException

replayWALCompactionMarker

void replayWALCompactionMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor compaction,
 boolean pickCompactionFiles,
 boolean removeFiles,
 long replaySeqId)
                        throws IOException

Call to complete a compaction. Its for the case where we find in the WAL a compaction that was not finished. We could find one recovering a WAL after a regionserver crash. See HBASE-2331.

Throws:

IOException

replayWALFlushMarker

@Deprecated
void replayWALFlushMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush,
 long replaySeqId)
                   throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Only for keep compatibility for old region replica implementation.

Throws:

IOException

getStoresToFlush

private Collection<HStore> getStoresToFlush(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flushDesc)

replayWALFlushStartMarker

@Deprecated
HRegion.PrepareFlushResult replayWALFlushStartMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush)
                                              throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Only for keep compatibility for old region replica implementation.

Replay the flush marker from primary region by creating a corresponding snapshot of the store memstores, only if the memstores do not have a higher seqId from an earlier wal edit (because the events may be coming out of order).

Throws:

IOException

replayWALFlushCommitMarker

@Deprecated
void replayWALFlushCommitMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush)
                         throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Only for keep compatibility for old region replica implementation.

Throws:

IOException

replayFlushInStores

@Deprecated
private void replayFlushInStores(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush,
 HRegion.PrepareFlushResult prepareFlushResult,
 boolean dropMemstoreSnapshot)
                          throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Only for keep compatibility for old region replica implementation.

Replays the given flush descriptor by opening the flush files in stores and dropping the memstore snapshots if requested.

Throws:

IOException

loadRecoveredHFilesIfAny

private long loadRecoveredHFilesIfAny(Collection<HStore> stores)
                               throws IOException

Throws:

IOException

dropMemStoreContents

private MemStoreSize dropMemStoreContents()
                                   throws IOException

Be careful, this method will drop all data in the memstore of this region. Currently, this method is used to drop memstore to prevent memory leak when replaying recovered.edits while opening region.

Throws:

IOException

dropMemStoreContentsForSeqId

private MemStoreSize dropMemStoreContentsForSeqId(long seqId,
 HStore store)
                                           throws IOException

Drops the memstore contents after replaying a flush descriptor or region open event replay if the memstore edits have seqNums smaller than the given seq id

Throws:

IOException

doDropStoreMemStoreContentsForSeqId

private MemStoreSize doDropStoreMemStoreContentsForSeqId(HStore s,
 long currentSeqId)
                                                  throws IOException

Throws:

IOException

replayWALFlushAbortMarker

private void replayWALFlushAbortMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush)

replayWALFlushCannotFlushMarker

private void replayWALFlushCannotFlushMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor flush,
 long replaySeqId)

getPrepareFlushResult

HRegion.PrepareFlushResult getPrepareFlushResult()

replayWALRegionEventMarker

@Deprecated
void replayWALRegionEventMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEvent)
                         throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Only for keep compatibility for old region replica implementation.

Throws:

IOException

replayWALBulkLoadEventMarker

@Deprecated
void replayWALBulkLoadEventMarker(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadEvent)
                           throws IOException

Deprecated.

Since 3.0.0, will be removed in 4.0.0. Only for keep compatibility for old region replica implementation.

Throws:

IOException

replayWALBatchMutate

private void replayWALBatchMutate(Map<byte[],List<ExtendedCell>> family2Cells)
                           throws IOException

Replay the batch mutate for secondary replica.

We will directly apply the cells to the memstore. This is because:

1. All the cells are gotten from WALEdit, so we only have Put and Delete here
2. The replay is single threaded, we do not need to acquire row lock, as the region is read only so no one else can write it.
3. We do not need to write WAL.
4. We will advance MVCC in the caller directly.

Throws:

IOException

replayWALMetaEdit

private void replayWALMetaEdit(Cell cell)
                        throws IOException

Replay the meta edits, i.e, flush marker, compaction marker, bulk load marker, region event marker, etc.

For all events other than start flush, we will just call refreshStoreFiles() as the logic is straight-forward and robust. For start flush, we need to snapshot the memstore, so later refreshStoreFiles() call could drop the snapshot, otherwise we may run out of memory.

Throws:

IOException

replayWALEntry

void replayWALEntry(org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos.WALEntry entry,
 CellScanner cells)
             throws IOException

Replay remote wal entry sent by primary replica.

Should only call this method on secondary replicas.

Throws:

IOException

dropPrepareFlushIfPossible

private void dropPrepareFlushIfPossible()

If all stores ended up dropping their snapshots, we can safely drop the prepareFlushResult

refreshStoreFiles

public boolean refreshStoreFiles()
                          throws IOException

Description copied from interface: Region

Check the region's underlying store files, open the files that have not been opened yet, and remove the store file readers for store files no longer available.

Specified by:

refreshStoreFiles in interface Region

Throws:

IOException

refreshStoreFiles

protected boolean refreshStoreFiles(boolean force)
                             throws IOException

Throws:

IOException

logRegionFiles

private void logRegionFiles()

checkTargetRegion

private void checkTargetRegion(byte[] encodedRegionName,
 String exceptionMsg,
 Object payload)
                        throws WrongRegionException

Checks whether the given regionName is either equal to our region, or that the regionName is the primary region to our corresponding range for the secondary replica.

Throws:

WrongRegionException

restoreEdit

protected void restoreEdit(HStore s,
 ExtendedCell cell,
 MemStoreSizing memstoreAccounting)

Used by tests

Parameters:

s - Store to add edit too.

cell - Cell to add.

isZeroLengthThenDelete

private static boolean isZeroLengthThenDelete(org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.FileStatus stat,
 org.apache.hadoop.fs.Path p)
                                       throws IOException

make sure have been through lease recovery before get file status, so the file length can be trusted.

Parameters:

p - File to check.

Returns:

True if file was zero-length (and if so, we'll delete it in here).

Throws:

IOException

instantiateHStore

protected HStore instantiateHStore(ColumnFamilyDescriptor family,
 boolean warmup)
                            throws IOException

Throws:

IOException

getStore

public HStore getStore(byte[] column)

Description copied from interface: Region

Return the Store for the given family

Use with caution. Exposed for use of fixup utilities.

Specified by:

getStore in interface Region

Returns:

the Store for the given family

getStore

private HStore getStore(Cell cell)

Return HStore instance. Does not do any copy: as the number of store is limited, we iterate on the list.

getStores

public List<HStore> getStores()

Description copied from interface: Region

Return the list of Stores managed by this region

Use with caution. Exposed for use of fixup utilities.

Specified by:

getStores in interface Region

Returns:

a list of the Stores managed by this region

getStoreFileList

public List<String> getStoreFileList(byte[][] columns)
                              throws IllegalArgumentException

Description copied from interface: Region

Returns list of store file names for the given families

Specified by:

getStoreFileList in interface Region

Throws:

IllegalArgumentException

checkRow

void checkRow(byte[] row,
 String op)
       throws IOException

Make sure this is a valid row for the HRegion

Throws:

IOException

getRowLock

public Region.RowLock getRowLock(byte[] row)
                          throws IOException

Get an exclusive ( write lock ) lock on a given row.

Parameters:

row - Which row to lock.

Returns:

A locked RowLock. The lock is exclusive and already aqquired.

Throws:

IOException

getRowLock

public Region.RowLock getRowLock(byte[] row,
 boolean readLock)
                          throws IOException

Description copied from interface: Region

Get a row lock for the specified row. All locks are reentrant. Before calling this function make sure that a region operation has already been started (the calling thread has already acquired the region-close-guard lock).

The obtained locks should be released after use by Region.RowLock.release()

NOTE: the boolean passed here has changed. It used to be a boolean that stated whether or not to wait on the lock. Now it is whether it an exclusive lock is requested.

Specified by:

getRowLock in interface Region

Parameters:

row - The row actions will be performed against

readLock - is the lock reader or writer. True indicates that a non-exclusive lock is requested

Throws:

IOException

See Also:

• Region.startRegionOperation()
• Region.startRegionOperation(Operation)

createRegionSpan

io.opentelemetry.api.trace.Span createRegionSpan(String name)

getRowLockInternal

protected Region.RowLock getRowLockInternal(byte[] row,
 boolean readLock,
 Region.RowLock prevRowLock)
                                     throws IOException

Throws:

IOException

getRowLock

private Region.RowLock getRowLock(byte[] row,
 boolean readLock,
 Region.RowLock prevRowLock)
                           throws IOException

Throws:

IOException

releaseRowLocks

private void releaseRowLocks(List<Region.RowLock> rowLocks)

getReadLockCount

public int getReadLockCount()

getLockedRows

public ConcurrentHashMap<HashedBytes,HRegion.RowLockContext> getLockedRows()

hasMultipleColumnFamilies

private static boolean hasMultipleColumnFamilies(Collection<Pair<byte[],String>> familyPaths)

Determines whether multiple column families are present Precondition: familyPaths is not null

Parameters:

familyPaths - List of (column family, hfilePath)

bulkLoadHFiles

public Map<byte[],List<org.apache.hadoop.fs.Path>> bulkLoadHFiles(Collection<Pair<byte[],String>> familyPaths,
 boolean assignSeqId,
 HRegion.BulkLoadListener bulkLoadListener)
                                                           throws IOException

Attempts to atomically load a group of hfiles. This is critical for loading rows with multiple column families atomically.

Parameters:

familyPaths - List of Pair<byte[] column family, String hfilePath>

bulkLoadListener - Internal hooks enabling massaging/preparation of a file about to be bulk loaded

Returns:

Map from family to List of store file paths if successful, null if failed recoverably

Throws:

IOException - if failed unrecoverably.

bulkLoadHFiles

public Map<byte[],List<org.apache.hadoop.fs.Path>> bulkLoadHFiles(Collection<Pair<byte[],String>> familyPaths,
 boolean assignSeqId,
 HRegion.BulkLoadListener bulkLoadListener,
 boolean copyFile,
 List<String> clusterIds,
 boolean replicate)
                                                           throws IOException

Attempts to atomically load a group of hfiles. This is critical for loading rows with multiple column families atomically.

Parameters:

familyPaths - List of Pair<byte[] column family, String hfilePath>

bulkLoadListener - Internal hooks enabling massaging/preparation of a file about to be bulk loaded

copyFile - always copy hfiles if true

clusterIds - ids from clusters that had already handled the given bulkload event.

Returns:

Map from family to List of store file paths if successful, null if failed recoverably

Throws:

IOException - if failed unrecoverably.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object

newHRegion

public static HRegion newHRegion(org.apache.hadoop.fs.Path tableDir,
 WAL wal,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.conf.Configuration conf,
 RegionInfo regionInfo,
 TableDescriptor htd,
 RegionServerServices rsServices)

A utility method to create new instances of HRegion based on the HConstants.REGION_IMPL configuration property.

Parameters:

tableDir - qualified path of directory where region should be located, usually the table directory.

wal - The WAL is the outbound log for any updates to the HRegion The wal file is a logfile from the previous execution that's custom-computed for this HRegion. The HRegionServer computes and sorts the appropriate wal info for this HRegion. If there is a previous file (implying that the HRegion has been written-to before), then read it from the supplied path.

fs - is the filesystem.

conf - is global configuration settings.

regionInfo - - RegionInfo that describes the region is new), then read them from the supplied path.

htd - the table descriptor

Returns:

the new instance

createHRegion

public static HRegion createHRegion(RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 TableDescriptor hTableDescriptor,
 WAL wal,
 boolean initialize)
                             throws IOException

Convenience method creating new HRegions. Used by createTable.

Parameters:

info - Info for region to create.

rootDir - Root directory for HBase instance

wal - shared WAL

initialize - - true to initialize the region

Returns:

new HRegion

Throws:

IOException

createHRegion

public static HRegion createHRegion(RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 TableDescriptor hTableDescriptor,
 WAL wal,
 boolean initialize,
 RegionServerServices rsRpcServices)
                             throws IOException

Convenience method creating new HRegions. Used by createTable.

Parameters:

info - Info for region to create.

rootDir - Root directory for HBase instance

wal - shared WAL

initialize - - true to initialize the region

rsRpcServices - An interface we can request flushes against.

Returns:

new HRegion

Throws:

IOException

createHRegion

public static HRegion createHRegion(org.apache.hadoop.conf.Configuration conf,
 RegionInfo regionInfo,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path tableDir,
 TableDescriptor tableDesc)
                             throws IOException

Create a region under the given table directory.

Throws:

IOException

createRegionDir

public static HRegionFileSystem createRegionDir(org.apache.hadoop.conf.Configuration configuration,
 RegionInfo ri,
 org.apache.hadoop.fs.Path rootDir)
                                         throws IOException

Create the region directory in the filesystem.

Throws:

IOException

createHRegion

public static HRegion createHRegion(RegionInfo info,
 org.apache.hadoop.fs.Path rootDir,
 org.apache.hadoop.conf.Configuration conf,
 TableDescriptor hTableDescriptor,
 WAL wal)
                             throws IOException

Throws:

IOException

openHRegion

public static HRegion openHRegion(RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 org.apache.hadoop.conf.Configuration conf)
                           throws IOException

Open a Region.

Parameters:

info - Info for region to be opened.

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

Returns:

new HRegion

Throws:

IOException

openHRegion

public static HRegion openHRegion(RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 org.apache.hadoop.conf.Configuration conf,
 RegionServerServices rsServices,
 CancelableProgressable reporter)
                           throws IOException

Open a Region.

Parameters:

info - Info for region to be opened

htd - the table descriptor

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

conf - The Configuration object to use.

rsServices - An interface we can request flushes against.

reporter - An interface we can report progress against.

Returns:

new HRegion

Throws:

IOException

openHRegion

public static HRegion openHRegion(org.apache.hadoop.fs.Path rootDir,
 RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 org.apache.hadoop.conf.Configuration conf)
                           throws IOException

Open a Region.

Parameters:

rootDir - Root directory for HBase instance

info - Info for region to be opened.

htd - the table descriptor

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

conf - The Configuration object to use.

Returns:

new HRegion

Throws:

IOException

openHRegion

public static HRegion openHRegion(org.apache.hadoop.fs.Path rootDir,
 RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 org.apache.hadoop.conf.Configuration conf,
 RegionServerServices rsServices,
 CancelableProgressable reporter)
                           throws IOException

Open a Region.

Parameters:

rootDir - Root directory for HBase instance

info - Info for region to be opened.

htd - the table descriptor

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

conf - The Configuration object to use.

rsServices - An interface we can request flushes against.

reporter - An interface we can report progress against.

Returns:

new HRegion

Throws:

IOException

openHRegion

public static HRegion openHRegion(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path rootDir,
 RegionInfo info,
 TableDescriptor htd,
 WAL wal)
                           throws IOException

Open a Region.

Parameters:

conf - The Configuration object to use.

fs - Filesystem to use

rootDir - Root directory for HBase instance

info - Info for region to be opened.

htd - the table descriptor

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

Returns:

new HRegion

Throws:

IOException

openHRegion

public static HRegion openHRegion(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path rootDir,
 RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 RegionServerServices rsServices,
 CancelableProgressable reporter)
                           throws IOException

Open a Region.

Parameters:

conf - The Configuration object to use.

fs - Filesystem to use

rootDir - Root directory for HBase instance

info - Info for region to be opened.

htd - the table descriptor

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

rsServices - An interface we can request flushes against.

reporter - An interface we can report progress against.

Returns:

new HRegion

Throws:

IOException

openHRegionFromTableDir

public static HRegion openHRegionFromTableDir(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path tableDir,
 RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 RegionServerServices rsServices,
 CancelableProgressable reporter)
                                       throws IOException

Open a Region.

Parameters:

conf - The Configuration object to use.

fs - Filesystem to use

info - Info for region to be opened.

htd - the table descriptor

wal - WAL for region to use. This method will call WAL#setSequenceNumber(long) passing the result of the call to HRegion#getMinSequenceId() to ensure the wal id is properly kept up. HRegionStore does this every time it opens a new region.

rsServices - An interface we can request flushes against.

reporter - An interface we can report progress against.

Returns:

new HRegion

Throws:

NullPointerException - if info is null

IOException

getReplicationScope

public NavigableMap<byte[],Integer> getReplicationScope()

openHRegion

public static HRegion openHRegion(HRegion other,
 CancelableProgressable reporter)
                           throws IOException

Useful when reopening a closed region (normally for unit tests)

Parameters:

other - original object

reporter - An interface we can report progress against.

Returns:

new HRegion

Throws:

IOException

openHRegion

public static Region openHRegion(Region other,
 CancelableProgressable reporter)
                          throws IOException

Throws:

IOException

openHRegion

private HRegion openHRegion(CancelableProgressable reporter)
                     throws IOException

Open HRegion.

Calls initialize and sets sequenceId.

Returns:

Returns this

Throws:

IOException

openReadOnlyFileSystemHRegion

public static HRegion openReadOnlyFileSystemHRegion(org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.fs.FileSystem fs,
 org.apache.hadoop.fs.Path tableDir,
 RegionInfo info,
 TableDescriptor htd)
                                             throws IOException

Open a Region on a read-only file-system (like hdfs snapshots)

Parameters:

conf - The Configuration object to use.

fs - Filesystem to use

info - Info for region to be opened.

htd - the table descriptor

Returns:

new HRegion

Throws:

NullPointerException - if info is null

IOException

warmupHRegion

public static HRegion warmupHRegion(RegionInfo info,
 TableDescriptor htd,
 WAL wal,
 org.apache.hadoop.conf.Configuration conf,
 RegionServerServices rsServices,
 CancelableProgressable reporter)
                             throws IOException

Throws:

IOException

getRegionDir

@Deprecated
public static org.apache.hadoop.fs.Path getRegionDir(org.apache.hadoop.fs.Path tabledir,
 String name)

Deprecated.

For tests only; to be removed.

Computes the Path of the HRegion

Parameters:

tabledir - qualified path for table

name - ENCODED region name

Returns:

Path of HRegion directory

rowIsInRange

public static boolean rowIsInRange(RegionInfo info,
 byte[] row)

Determines if the specified row is within the row range specified by the specified RegionInfo

Parameters:

info - RegionInfo that specifies the row range

row - row to be checked

Returns:

true if the row is within the range specified by the RegionInfo

rowIsInRange

public static boolean rowIsInRange(RegionInfo info,
 byte[] row,
 int offset,
 short length)

get

public Result get(Get get)
           throws IOException

Description copied from interface: Region

Do a get based on the get parameter.

Specified by:

get in interface Region

Parameters:

get - query parameters

Returns:

result of the operation

Throws:

IOException

prepareGet

void prepareGet(Get get)
         throws IOException

Throws:

IOException

get

public List<Cell> get(Get get,
 boolean withCoprocessor)
               throws IOException

Description copied from interface: Region

Do a get based on the get parameter.

Specified by:

get in interface Region

Parameters:

get - query parameters

withCoprocessor - invoke coprocessor or not. We don't want to always invoke cp.

Returns:

list of cells resulting from the operation

Throws:

IOException

get

private List<Cell> get(Get get,
 boolean withCoprocessor,
 long nonceGroup,
 long nonce)
                throws IOException

Throws:

IOException

getInternal

private List<Cell> getInternal(Get get,
 boolean withCoprocessor,
 long nonceGroup,
 long nonce)
                        throws IOException

Throws:

IOException

metricsUpdateForGet

void metricsUpdateForGet()

mutateRow

public Result mutateRow(RowMutations rm)
                 throws IOException

Description copied from interface: Region

Performs multiple mutations atomically on a single row.

Specified by:

mutateRow in interface Region

Parameters:

rm - object that specifies the set of mutations to perform atomically

Returns:

results of Increment/Append operations. If no Increment/Append operations, it returns null

Throws:

IOException

mutateRow

public Result mutateRow(RowMutations rm,
 long nonceGroup,
 long nonce)
                 throws IOException

Throws:

IOException

mutateRowsWithLocks

public void mutateRowsWithLocks(Collection<Mutation> mutations,
 Collection<byte[]> rowsToLock,
 long nonceGroup,
 long nonce)
                         throws IOException

Perform atomic (all or none) mutations within the region.

Specified by:

mutateRowsWithLocks in interface Region

Parameters:

mutations - The list of mutations to perform. mutations can contain operations for multiple rows. Caller has to ensure that all rows are contained in this region.

rowsToLock - Rows to lock

nonceGroup - Optional nonce group of the operation (client Id)

nonce - Optional nonce of the operation (unique random id to ensure "more idempotence") If multiple rows are locked care should be taken that rowsToLock is sorted in order to avoid deadlocks.

Throws:

IOException

getLoadStatistics

public org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.RegionLoadStats getLoadStatistics()

Returns statistics about the current load of the region

append

public Result append(Append append)
              throws IOException

Description copied from interface: Region

Perform one or more append operations on a row.

Specified by:

append in interface Region

Returns:

result of the operation

Throws:

IOException

append

public Result append(Append append,
 long nonceGroup,
 long nonce)
              throws IOException

Throws:

IOException

increment

public Result increment(Increment increment)
                 throws IOException

Description copied from interface: Region

Perform one or more increment operations on a row.

Specified by:

increment in interface Region

Returns:

result of the operation

Throws:

IOException

increment

public Result increment(Increment increment,
 long nonceGroup,
 long nonce)
                 throws IOException

Throws:

IOException

createWALKeyForWALAppend

private WALKeyImpl createWALKeyForWALAppend(boolean isReplay,
 HRegion.BatchOperation<?> batchOp,
 long now,
 long nonceGroup,
 long nonce)

doWALAppend

private MultiVersionConcurrencyControl.WriteEntry doWALAppend(WALEdit walEdit,
 HRegion.BatchOperation<?> batchOp,
 MiniBatchOperationInProgress<Mutation> miniBatchOp,
 long now,
 NonceKey nonceKey)
                                                       throws IOException

Returns writeEntry associated with this append

Throws:

IOException

attachRegionReplicationInWALAppend

private void attachRegionReplicationInWALAppend(HRegion.BatchOperation<?> batchOp,
 MiniBatchOperationInProgress<Mutation> miniBatchOp,
 WALKeyImpl walKey,
 WALEdit walEdit,
 MultiVersionConcurrencyControl.WriteEntry writeEntry)

Attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the mvcc writeEntry for replicating to region replica.

doAttachReplicateRegionReplicaAction

private void doAttachReplicateRegionReplicaAction(WALKeyImpl walKey,
 WALEdit walEdit,
 MultiVersionConcurrencyControl.WriteEntry writeEntry)

Attach RegionReplicationSink.add(org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit, org.apache.hadoop.hbase.ipc.ServerCall<?>) to the mvcc writeEntry for replicating to region replica.

heapSize

public long heapSize()

Description copied from interface: HeapSize

Return the approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.

Specified by:

heapSize in interface HeapSize

registerService

public boolean registerService(org.apache.hbase.thirdparty.com.google.protobuf.Service instance)

Registers a new protocol buffer Service subclass as a coprocessor endpoint to be available for handling execService(RpcController, CoprocessorServiceCall) calls.

Only a single instance may be registered per region for a given Service subclass (the instances are keyed on Descriptors.ServiceDescriptor.getFullName().. After the first registration, subsequent calls with the same service name will fail with a return value of false.

Parameters:

instance - the Service subclass instance to expose as a coprocessor endpoint

Returns:

true if the registration was successful, false otherwise

execService

public org.apache.hbase.thirdparty.com.google.protobuf.Message execService(org.apache.hbase.thirdparty.com.google.protobuf.RpcController controller,
 org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos.CoprocessorServiceCall call)
                                                                    throws IOException

Executes a single protocol buffer coprocessor endpoint Service method using the registered protocol handlers. Service implementations must be registered via the registerService(Service) method before they are available.

Parameters:

controller - an RpcContoller implementation to pass to the invoked service

call - a CoprocessorServiceCall instance identifying the service, method, and parameters for the method invocation

Returns:

a protocol buffer Message instance containing the method's result

Throws:

IOException - if no registered service handler is found or an error occurs during the invocation

See Also:

• registerService(Service)

checkSplit

public Optional<byte[]> checkSplit()

checkSplit

public Optional<byte[]> checkSplit(boolean force)

Return the split point. An empty result indicates the region isn't splittable.

getCompactPriority

public int getCompactPriority()

Returns The priority that this region should have in the compaction queue

getCoprocessorHost

public RegionCoprocessorHost getCoprocessorHost()

Returns the coprocessor host

setCoprocessorHost

public void setCoprocessorHost(RegionCoprocessorHost coprocessorHost)

Parameters:

coprocessorHost - the new coprocessor host

startRegionOperation

public void startRegionOperation()
                          throws IOException

Description copied from interface: Region

This method needs to be called before any public call that reads or modifies data. Acquires a read lock and checks if the region is closing or closed.

Region.closeRegionOperation() MUST then always be called after the operation has completed, whether it succeeded or failed.

Specified by:

startRegionOperation in interface Region

Throws:

IOException

startRegionOperation

public void startRegionOperation(Region.Operation op)
                          throws IOException

Description copied from interface: Region

This method needs to be called before any public call that reads or modifies data. Acquires a read lock and checks if the region is closing or closed.

Region.closeRegionOperation() MUST then always be called after the operation has completed, whether it succeeded or failed.

Specified by:

startRegionOperation in interface Region

Parameters:

op - The operation is about to be taken on the region

Throws:

IOException

closeRegionOperation

public void closeRegionOperation()
                          throws IOException

Description copied from interface: Region

Closes the region operation lock.

Specified by:

closeRegionOperation in interface Region

Throws:

IOException

closeRegionOperation

public void closeRegionOperation(Region.Operation operation)
                          throws IOException

Description copied from interface: Region

Closes the region operation lock. This needs to be called in the finally block corresponding to the try block of Region.startRegionOperation(Operation)

Specified by:

closeRegionOperation in interface Region

Throws:

IOException

startBulkRegionOperation

private void startBulkRegionOperation(boolean writeLockNeeded)
                               throws IOException

This method needs to be called before any public call that reads or modifies stores in bulk. It has to be called just before a try. #closeBulkRegionOperation needs to be called in the try's finally block Acquires a writelock and checks if the region is closing or closed.

Throws:

NotServingRegionException - when the region is closing or closed

RegionTooBusyException - if failed to get the lock in time

InterruptedIOException - if interrupted while waiting for a lock

IOException

closeBulkRegionOperation

private void closeBulkRegionOperation()

Closes the lock. This needs to be called in the finally block corresponding to the try block of #startRegionOperation

recordMutationWithoutWal

private void recordMutationWithoutWal(Map<byte[],List<Cell>> familyMap)

Update LongAdders for number of puts without wal and the size of possible data loss. These information are exposed by the region server metrics.

lock

private void lock(Lock lock)
           throws IOException

Throws:

IOException

lock

private void lock(Lock lock,
 int multiplier)
           throws IOException

Try to acquire a lock. Throw RegionTooBusyException if failed to get the lock in time. Throw InterruptedIOException if interrupted while waiting for the lock.

Throws:

IOException

sync

private void sync(long txid,
 Durability durability)
           throws IOException

Calls sync with the given transaction ID

Parameters:

txid - should sync up to which transaction

Throws:

IOException - If anything goes wrong with DFS

shouldSyncWAL

private boolean shouldSyncWAL()

Check whether we should sync the wal from the table's durability settings

getOpenSeqNum

public long getOpenSeqNum()

Returns the latest sequence number that was read from storage when this region was opened

getMaxStoreSeqId

public Map<byte[],Long> getMaxStoreSeqId()

Specified by:

getMaxStoreSeqId in interface Region

Returns:

map of column family names to max sequence id that was read from storage when this region was opened

getOldestSeqIdOfStore

public long getOldestSeqIdOfStore(byte[] familyName)

getCompactionState

public CompactionState getCompactionState()

Description copied from interface: Region

Returns if a given region is in compaction now.

Specified by:

getCompactionState in interface Region

reportCompactionRequestStart

public void reportCompactionRequestStart(boolean isMajor)

reportCompactionRequestEnd

public void reportCompactionRequestEnd(boolean isMajor,
 int numFiles,
 long filesSizeCompacted)

reportCompactionRequestFailure

public void reportCompactionRequestFailure()

incrementCompactionsQueuedCount

public void incrementCompactionsQueuedCount()

decrementCompactionsQueuedCount

public void decrementCompactionsQueuedCount()

incrementFlushesQueuedCount

public void incrementFlushesQueuedCount()

decrementFlushesQueuedCount

protected void decrementFlushesQueuedCount()

disableInterrupts

void disableInterrupts()

If a handler thread is eligible for interrupt, make it ineligible. Should be paired with {enableInterrupts().

enableInterrupts

void enableInterrupts()

If a handler thread was made ineligible for interrupt via {disableInterrupts(), make it eligible again. No-op if interrupts are already enabled.

interruptRegionOperations

private void interruptRegionOperations()

Interrupt any region options that have acquired the region lock via startRegionOperation(org.apache.hadoop.hbase.regionserver.Region.Operation), or startBulkRegionOperation(boolean).

checkInterrupt

void checkInterrupt()
             throws NotServingRegionException,
InterruptedIOException

Check thread interrupt status and throw an exception if interrupted.

Throws:

NotServingRegionException - if region is closing

InterruptedIOException - if interrupted but region is not closing

throwOnInterrupt

IOException throwOnInterrupt(Throwable t)

Throw the correct exception upon interrupt

Parameters:

t - cause

onConfigurationChange

public void onConfigurationChange(org.apache.hadoop.conf.Configuration conf)

This method would be called by the ConfigurationManager object when the Configuration object is reloaded from disk.

Specified by:

onConfigurationChange in interface ConfigurationObserver

registerChildren

public void registerChildren(ConfigurationManager manager)

Needs to be called to register the children to the manager.

Specified by:

registerChildren in interface PropagatingConfigurationObserver

Parameters:

manager - : to register to

deregisterChildren

public void deregisterChildren(ConfigurationManager manager)

Needs to be called to deregister the children from the manager.

Specified by:

deregisterChildren in interface PropagatingConfigurationObserver

Parameters:

manager - : to deregister from

getCellComparator

public CellComparator getCellComparator()

Description copied from interface: Region

The comparator to be used with the region

Specified by:

getCellComparator in interface Region

getMemStoreFlushSize

public long getMemStoreFlushSize()

throwException

void throwException(String title,
 String regionName)

requestCompaction

public void requestCompaction(String why,
 int priority,
 boolean major,
 CompactionLifeCycleTracker tracker)
                       throws IOException

Description copied from interface: Region

Request compaction on this region.

Specified by:

requestCompaction in interface Region

Throws:

IOException

requestCompaction

public void requestCompaction(byte[] family,
 String why,
 int priority,
 boolean major,
 CompactionLifeCycleTracker tracker)
                       throws IOException

Description copied from interface: Region

Request compaction for the given family

Specified by:

requestCompaction in interface Region

Throws:

IOException

requestFlushIfNeeded

private void requestFlushIfNeeded()
                           throws RegionTooBusyException

Throws:

RegionTooBusyException

requestFlush

private void requestFlush()

requestFlush0

private void requestFlush0(FlushLifeCycleTracker tracker)

requestFlush

public void requestFlush(FlushLifeCycleTracker tracker)
                  throws IOException

Description copied from interface: Region

Request flush on this region.

Specified by:

requestFlush in interface Region

Throws:

IOException

decorateRegionConfiguration

private static void decorateRegionConfiguration(org.apache.hadoop.conf.Configuration conf)

This method modifies the region's configuration in order to inject replication-related features

Parameters:

conf - region configurations

getRegionReplicationSink

public Optional<RegionReplicationSink> getRegionReplicationSink()

addReadRequestsCount

public void addReadRequestsCount(long readRequestsCount)

addWriteRequestsCount

public void addWriteRequestsCount(long writeRequestsCount)

isReadsEnabled

boolean isReadsEnabled()