Package org.apache.hadoop.hbase.regionserver

Class TestCompaction

java.lang.Object

org.apache.hadoop.hbase.regionserver.TestCompaction

Direct Known Subclasses:

TestCompactionWithCoprocessor

public class TestCompaction
extends Object

Test compaction framework and common functions

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	TestCompaction.BlockingStoreMockMaker
static class	TestCompaction.DummyCompactor
(package private) class	TestCompaction.StoreMockMaker
static class	TestCompaction.Tracker	Simple CompactionLifeCycleTracker on which you can wait until the requested compaction finishes.
static class	TestCompaction.WaitThroughPutController	Simple CompactionLifeCycleTracker on which you can wait until the requested compaction finishes.

Field Summary

Fields

Modifier and Type	Field	Description
static final HBaseClassTestRule	CLASS_RULE
private static final byte[]	COLUMN_FAMILY
private static final byte[]	COLUMN_FAMILY_TEXT
private int	compactionThreshold
protected org.apache.hadoop.conf.Configuration	conf
private final byte[]	FAMILY
private static final long	MAX_FILES_TO_COMPACT
org.junit.rules.TestName	name
private org.apache.hadoop.hbase.regionserver.HRegion	r
private byte[]	secondRowBytes
private final byte[]	STARTROW
private org.apache.hadoop.hbase.client.TableDescriptor	tableDescriptor
private byte[]	thirdRowBytes
private static final HBaseTestingUtil	UTIL

Constructor Summary

Constructors

Constructor	Description
TestCompaction()	constructor

Method Summary

Modifier and Type	Method	Description
private int	count()
private static org.apache.hadoop.hbase.regionserver.HStoreFile	createFile()
private void	createStoreFile(org.apache.hadoop.hbase.regionserver.HRegion region)
private void	createStoreFile(org.apache.hadoop.hbase.regionserver.HRegion region, String family)
void	setUp()
void	tearDown()
void	testCompactionFailure()
void	testCompactionQueuePriorities()	Test compaction priority management and multiple compactions per store (HBASE-8665).
void	testCompactionSeqId()	Firstly write 10 cells (with different time stamp) to a qualifier and flush to hfile1, then write 10 cells (with different time stamp) to the same qualifier and flush to hfile2.
void	testCompactionWithCorruptBlock()	This test uses a hand-modified HFile, which is loaded in from the resources' path.
void	testCompactionWithCorruptResult()
void	testInterruptCompactionBySize()	Verify that you can stop a long-running compaction (used during RS shutdown)
void	testInterruptCompactionByTime()
void	testInterruptingRunningCompactions()
void	testMultipleCustomCompactionRequests()	HBASE-7947: Regression test to ensure adding to the correct list in the CompactSplit
void	testStopStartCompaction()	Test no new Compaction requests are generated after calling stop compactions
void	testTrackingCompactionRequest()	Create a custom compaction request and be sure that we can track it through the queue, knowing when the compaction is completed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CLASS_RULE

public static final HBaseClassTestRule CLASS_RULE

name

public org.junit.rules.TestName name

UTIL

private static final HBaseTestingUtil UTIL

conf

protected org.apache.hadoop.conf.Configuration conf

r

private org.apache.hadoop.hbase.regionserver.HRegion r

tableDescriptor

private org.apache.hadoop.hbase.client.TableDescriptor tableDescriptor

COLUMN_FAMILY

private static final byte[] COLUMN_FAMILY

STARTROW

private final byte[] STARTROW

COLUMN_FAMILY_TEXT

private static final byte[] COLUMN_FAMILY_TEXT

compactionThreshold

private int compactionThreshold

secondRowBytes

private byte[] secondRowBytes

thirdRowBytes

private byte[] thirdRowBytes

MAX_FILES_TO_COMPACT

private static final long MAX_FILES_TO_COMPACT

See Also:

• Constant Field Values

FAMILY

private final byte[] FAMILY

Constructor Details

TestCompaction

public TestCompaction()

constructor

Method Details

setUp

public void setUp()
           throws Exception

Throws:

Exception

tearDown

public void tearDown()
              throws Exception

Throws:

Exception

testInterruptCompactionBySize

public void testInterruptCompactionBySize()
                                   throws Exception

Verify that you can stop a long-running compaction (used during RS shutdown)

Throws:

Exception

testInterruptCompactionByTime

public void testInterruptCompactionByTime()
                                   throws Exception

Throws:

Exception

count

private int count()
           throws IOException

Throws:

IOException

createStoreFile

private void createStoreFile(org.apache.hadoop.hbase.regionserver.HRegion region)
                      throws IOException

Throws:

IOException

createStoreFile

private void createStoreFile(org.apache.hadoop.hbase.regionserver.HRegion region,
 String family)
                      throws IOException

Throws:

IOException

testCompactionWithCorruptResult

public void testCompactionWithCorruptResult()
                                     throws Exception

Throws:

Exception

testCompactionWithCorruptBlock

public void testCompactionWithCorruptBlock()
                                    throws Exception

This test uses a hand-modified HFile, which is loaded in from the resources' path. That file was generated from the test support code in this class and then edited to corrupt the GZ-encoded block by zeroing-out the first two bytes of the GZip header, the "standard declaration" of 1f 8b, found at offset 33 in the file. I'm not sure why, but it seems that in this test context we do not enforce CRC checksums. Thus, this corruption manifests in the Decompressor rather than in the reader when it loads the block bytes and compares vs. the header.

Throws:

Exception

testTrackingCompactionRequest

public void testTrackingCompactionRequest()
                                   throws Exception

Create a custom compaction request and be sure that we can track it through the queue, knowing when the compaction is completed.

Throws:

Exception

testCompactionFailure

public void testCompactionFailure()
                           throws Exception

Throws:

Exception

testStopStartCompaction

public void testStopStartCompaction()
                             throws IOException

Test no new Compaction requests are generated after calling stop compactions

Throws:

IOException

testInterruptingRunningCompactions

public void testInterruptingRunningCompactions()
                                        throws Exception

Throws:

Exception

testMultipleCustomCompactionRequests

public void testMultipleCustomCompactionRequests()
                                          throws Exception

HBASE-7947: Regression test to ensure adding to the correct list in the CompactSplit

Throws:

Exception - on failure

testCompactionQueuePriorities

public void testCompactionQueuePriorities()
                                   throws Exception

Test compaction priority management and multiple compactions per store (HBASE-8665).

Throws:

Exception

testCompactionSeqId

public void testCompactionSeqId()
                         throws Exception

Firstly write 10 cells (with different time stamp) to a qualifier and flush to hfile1, then write 10 cells (with different time stamp) to the same qualifier and flush to hfile2. The latest cell (cell-A) in hfile1 and the oldest cell (cell-B) in hfile2 are with the same time stamp but different sequence id, and will get scanned successively during compaction.

We set compaction.kv.max to 10 so compaction will scan 10 versions each round, meanwhile we set keepSeqIdPeriod=0 in TestCompaction.DummyCompactor so all 10 versions of hfile2 will be written out with seqId cleaned (set to 0) including cell-B, then when scanner goes to cell-A it will cause a scan out-of-order assertion error before HBASE-16931 if error occurs during the test

Throws:

Exception

createFile

private static org.apache.hadoop.hbase.regionserver.HStoreFile createFile()
                                                                   throws Exception

Throws:

Exception