Package org.apache.hadoop.hbase.coprocessor.example

Class WriteHeavyIncrementObserver

java.lang.Object

org.apache.hadoop.hbase.coprocessor.example.WriteHeavyIncrementObserver

All Implemented Interfaces:

Coprocessor, RegionCoprocessor, RegionObserver

@Private
public class WriteHeavyIncrementObserver
extends Object
implements RegionCoprocessor, RegionObserver

An example for implementing a counter that reads is much less than writes, i.e, write heavy.

We will convert increment to put, and do aggregating when get. And of course the return value of increment is useless then.

Notice that this is only an example so we do not handle most corner cases, for example, you must provide a qualifier when doing a get.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.Coprocessor

Coprocessor.State

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

RegionObserver.MutationType

Field Summary

Fields

Modifier and Type	Field	Description
private final org.apache.commons.lang3.mutable.MutableLong[]	lastTimestamps
private final int	mask

Fields inherited from interface org.apache.hadoop.hbase.Coprocessor

PRIORITY_HIGHEST, PRIORITY_LOWEST, PRIORITY_SYSTEM, PRIORITY_USER, VERSION

Constructor Summary

Constructors

Constructor	Description
WriteHeavyIncrementObserver()

Method Summary

Modifier and Type	Method	Description
private Cell	createCell(byte[] row, byte[] family, byte[] qualifier, long ts, long value)
Optional<RegionObserver>	getRegionObserver()
private long	getUniqueTimestamp(byte[] row)
InternalScanner	preCompact(ObserverContext<? extends RegionCoprocessorEnvironment> c, Store store, InternalScanner scanner, ScanType scanType, CompactionLifeCycleTracker tracker, CompactionRequest request)	Called prior to writing the StoreFiles selected for compaction into a new StoreFile.
void	preCompactScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> c, Store store, ScanType scanType, ScanOptions options, CompactionLifeCycleTracker tracker, CompactionRequest request)	Called before we open store scanner for compaction.
InternalScanner	preFlush(ObserverContext<? extends RegionCoprocessorEnvironment> c, Store store, InternalScanner scanner, FlushLifeCycleTracker tracker)	Called before a Store's memstore is flushed to disk.
void	preFlushScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> c, Store store, ScanOptions options, FlushLifeCycleTracker tracker)	Called before we open store scanner for flush.
void	preGetOp(ObserverContext<? extends RegionCoprocessorEnvironment> c, Get get, List<Cell> result)	Called before the client performs a Get
Result	preIncrement(ObserverContext<? extends RegionCoprocessorEnvironment> c, Increment increment)	Called before Increment.
InternalScanner	preMemStoreCompactionCompact(ObserverContext<? extends RegionCoprocessorEnvironment> c, Store store, InternalScanner scanner)	Called before we do in memory compaction.
void	preMemStoreCompactionCompactScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> c, Store store, ScanOptions options)	Called before we open store scanner for in memory compaction.
void	preStoreScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> ctx, Store store, ScanOptions options)	Called before a store opens a new scanner.
private InternalScanner	wrap(byte[] family, InternalScanner scanner)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.Coprocessor

getServices, start, stop

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.RegionCoprocessor

getBulkLoadObserver, getEndpointObserver

Methods inherited from interface org.apache.hadoop.hbase.coprocessor.RegionObserver

postAppend, postAppend, postAppendBeforeWAL, postBatchMutate, postBatchMutateIndispensably, postBulkLoadHFile, postCheckAndDelete, postCheckAndDelete, postCheckAndMutate, postCheckAndPut, postCheckAndPut, postClose, postCloseRegionOperation, postCommitStoreFile, postCompact, postCompactSelection, postDelete, postDelete, postExists, postFlush, postFlush, postGetOp, postIncrement, postIncrement, postIncrementBeforeWAL, postInstantiateDeleteTracker, postMemStoreCompaction, postMutationBeforeWAL, postOpen, postPut, postPut, postReplayWALs, postScannerClose, postScannerFilterRow, postScannerNext, postScannerOpen, postStartRegionOperation, postStoreFileReaderOpen, preAppend, preAppend, preAppendAfterRowLock, preBatchMutate, preBulkLoadHFile, preCheckAndDelete, preCheckAndDelete, preCheckAndDeleteAfterRowLock, preCheckAndDeleteAfterRowLock, preCheckAndMutate, preCheckAndMutateAfterRowLock, preCheckAndPut, preCheckAndPut, preCheckAndPutAfterRowLock, preCheckAndPutAfterRowLock, preClose, preCommitStoreFile, preCompactSelection, preDelete, preDelete, preExists, preFlush, preIncrement, preIncrementAfterRowLock, preMemStoreCompaction, preOpen, prePrepareTimeStampForDeleteVersion, prePut, prePut, preReplayWALs, preScannerClose, preScannerNext, preScannerOpen, preStoreFileReaderOpen, preWALAppend

Field Details

mask

private final int mask

lastTimestamps

private final org.apache.commons.lang3.mutable.MutableLong[] lastTimestamps

Constructor Details

WriteHeavyIncrementObserver

public WriteHeavyIncrementObserver()

Method Details

getRegionObserver

public Optional<RegionObserver> getRegionObserver()

Specified by:

getRegionObserver in interface RegionCoprocessor

preFlushScannerOpen

public void preFlushScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Store store,
 ScanOptions options,
 FlushLifeCycleTracker tracker)
                         throws IOException

Description copied from interface: RegionObserver

Called before we open store scanner for flush. You can use the options to change max versions and TTL for the scanner being opened.

Specified by:

preFlushScannerOpen in interface RegionObserver

Parameters:

c - the environment provided by the region server

store - the store where flush is being requested

options - used to change max versions and TTL for the scanner being opened

Throws:

IOException

createCell

private Cell createCell(byte[] row,
 byte[] family,
 byte[] qualifier,
 long ts,
 long value)

wrap

private InternalScanner wrap(byte[] family,
 InternalScanner scanner)

preFlush

public InternalScanner preFlush(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Store store,
 InternalScanner scanner,
 FlushLifeCycleTracker tracker)
                         throws IOException

Description copied from interface: RegionObserver

Called before a Store's memstore is flushed to disk.

Specified by:

preFlush in interface RegionObserver

Parameters:

c - the environment provided by the region server

store - the store where flush is being requested

scanner - the scanner over existing data used in the memstore

tracker - tracker used to track the life cycle of a flush

Returns:

the scanner to use during flush. Should not be null unless the implementation is writing new store files on its own.

Throws:

IOException

preCompactScannerOpen

public void preCompactScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Store store,
 ScanType scanType,
 ScanOptions options,
 CompactionLifeCycleTracker tracker,
 CompactionRequest request)
                           throws IOException

Description copied from interface: RegionObserver

Called before we open store scanner for compaction. You can use the options to change max versions and TTL for the scanner being opened.

Specified by:

preCompactScannerOpen in interface RegionObserver

Parameters:

c - the environment provided by the region server

store - the store being compacted

scanType - type of Scan

options - used to change max versions and TTL for the scanner being opened

tracker - tracker used to track the life cycle of a compaction

request - the requested compaction

Throws:

IOException

preCompact

public InternalScanner preCompact(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Store store,
 InternalScanner scanner,
 ScanType scanType,
 CompactionLifeCycleTracker tracker,
 CompactionRequest request)
                           throws IOException

Description copied from interface: RegionObserver

Called prior to writing the StoreFiles selected for compaction into a new StoreFile.

To override or modify the compaction process, implementing classes can wrap the provided InternalScanner with a custom implementation that is returned from this method. The custom scanner can then inspect Cells from the wrapped scanner, applying its own policy to what gets written.

If implementations are wrapping the passed in InternalScanner, they can also have their implementation implement Shipper and delegate to the original scanner. This will cause compactions to free up memory as they progress, which is especially important for people using off-heap memory pools.

Keep in mind that when Shipper.shipped() is called, any cell references you maintain in your implementation may get corrupted. As such you should make sure to deep clone any cells that you need to keep reference to across invocations of shipped.

Specified by:

preCompact in interface RegionObserver

Parameters:

c - the environment provided by the region server

store - the store being compacted

scanner - the scanner over existing data used in the store file rewriting

scanType - type of Scan

tracker - tracker used to track the life cycle of a compaction

request - the requested compaction

Returns:

the scanner to use during compaction. Should not be null unless the implementation is writing new store files on its own.

Throws:

IOException

preMemStoreCompactionCompactScannerOpen

public void preMemStoreCompactionCompactScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Store store,
 ScanOptions options)
                                             throws IOException

Description copied from interface: RegionObserver

Called before we open store scanner for in memory compaction. You can use the options to change max versions and TTL for the scanner being opened. Notice that this method will only be called when you use eager mode. For basic mode we will not drop any cells thus we do not open a store scanner.

Specified by:

preMemStoreCompactionCompactScannerOpen in interface RegionObserver

Parameters:

c - the environment provided by the region server

store - the store where in memory compaction is being requested

options - used to change max versions and TTL for the scanner being opened

Throws:

IOException

preMemStoreCompactionCompact

public InternalScanner preMemStoreCompactionCompact(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Store store,
 InternalScanner scanner)
                                             throws IOException

Description copied from interface: RegionObserver

Called before we do in memory compaction. Notice that this method will only be called when you use eager mode. For basic mode we will not drop any cells thus there is no InternalScanner.

Specified by:

preMemStoreCompactionCompact in interface RegionObserver

Parameters:

c - the environment provided by the region server

store - the store where in memory compaction is being executed

scanner - the scanner over existing data used in the memstore segments being compact

Returns:

the scanner to use during in memory compaction. Must be non-null.

Throws:

IOException

preGetOp

public void preGetOp(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Get get,
 List<Cell> result)
              throws IOException

Description copied from interface: RegionObserver

Called before the client performs a Get

Call CoprocessorEnvironment#bypass to skip default actions. If 'bypass' is set, we skip out on calling any subsequent chained coprocessors.

Specified by:

preGetOp in interface RegionObserver

Parameters:

c - the environment provided by the region server

get - the Get request

result - The result to return to the client if default processing is bypassed. Can be modified. Will not be used if default processing is not bypassed.

Throws:

IOException

getUniqueTimestamp

private long getUniqueTimestamp(byte[] row)

preIncrement

public Result preIncrement(ObserverContext<? extends RegionCoprocessorEnvironment> c,
 Increment increment)
                    throws IOException

Description copied from interface: RegionObserver

Called before Increment.

Call CoprocessorEnvironment#bypass to skip default actions. If 'bypass' is set, we skip out on calling any subsequent chained coprocessors.

Note: Do not retain references to any Cells in 'increment' beyond the life of this invocation. If need a Cell reference for later use, copy the cell and use that.

Specified by:

preIncrement in interface RegionObserver

Parameters:

c - the environment provided by the region server

increment - increment object

Returns:

result to return to the client if bypassing default processing

Throws:

IOException

preStoreScannerOpen

public void preStoreScannerOpen(ObserverContext<? extends RegionCoprocessorEnvironment> ctx,
 Store store,
 ScanOptions options)
                         throws IOException

Description copied from interface: RegionObserver

Called before a store opens a new scanner.

This hook is called when a "user" scanner is opened. Use preFlushScannerOpen and preCompactScannerOpen to inject flush/compaction.

Notice that, this method is used to change the inherent max versions and TTL for a Store. For example, you can change the max versions option for a Scan object to 10 in preScannerOpen, but if the max versions config on the Store is 1, then you still can only read 1 version. You need also to inject here to change the max versions to 10 if you want to get more versions.

Specified by:

preStoreScannerOpen in interface RegionObserver

Parameters:

ctx - the environment provided by the region server

store - the store which we want to get scanner from

options - used to change max versions and TTL for the scanner being opened

Throws:

IOException

See Also:

• RegionObserver.preFlushScannerOpen(ObserverContext, Store, ScanOptions, FlushLifeCycleTracker)
• RegionObserver.preCompactScannerOpen(ObserverContext, Store, ScanType, ScanOptions, CompactionLifeCycleTracker, CompactionRequest)