public class TestIOFencing extends Object
If we happen to assign a region before it fully done with in its old location -- i.e. it is on two servers at the same time -- all can work fine until the case where the region on the dying server decides to compact or otherwise change the region file set. The region in its new location will then get a surprise when it tries to do something w/ a file removed by the region in its old location on dying server.
Making a test for this case is a little tough in that even if a file is deleted up on the namenode, if the file was opened before the delete, it will continue to let reads happen until something changes the state of cached blocks in the dfsclient that was already open (a block from the deleted file is cleaned from the datanode by NN).
What we will do below is do an explicit check for existence on the files listed in the region that has had some files removed because of a compaction. This sort of hurry's along and makes certain what is a chance occurance.
Modifier and Type | Class and Description |
---|---|
static class |
TestIOFencing.BlockCompactionsInCompletionHStore |
static class |
TestIOFencing.BlockCompactionsInCompletionRegion
An override of HRegion that allows us park compactions in a holding pattern and
then when appropriate for the test, allow them proceed again.
|
static class |
TestIOFencing.BlockCompactionsInPrepRegion
An override of HRegion that allows us park compactions in a holding pattern and
then when appropriate for the test, allow them proceed again.
|
static class |
TestIOFencing.CompactionBlockerRegion |
Modifier and Type | Field and Description |
---|---|
static HBaseClassTestRule |
CLASS_RULE |
private static byte[] |
FAMILY |
private static int |
FIRST_BATCH_COUNT |
private static org.slf4j.Logger |
LOG |
private static int |
SECOND_BATCH_COUNT |
private static org.apache.hadoop.hbase.TableName |
TABLE_NAME |
private static HBaseTestingUtility |
TEST_UTIL |
Constructor and Description |
---|
TestIOFencing() |
Modifier and Type | Method and Description |
---|---|
void |
doTest(Class<?> regionClass,
org.apache.hadoop.hbase.MemoryCompactionPolicy policy) |
void |
testFencingAroundCompaction()
Test that puts up a regionserver, starts a compaction on a loaded region but holds the
compaction until after we have killed the server and the region has come up on
a new regionserver altogether.
|
void |
testFencingAroundCompactionAfterWALSync()
Test that puts up a regionserver, starts a compaction on a loaded region but holds the
compaction completion until after we have killed the server and the region has come up on
a new regionserver altogether.
|
public static final HBaseClassTestRule CLASS_RULE
private static final org.slf4j.Logger LOG
private static final HBaseTestingUtility TEST_UTIL
private static final org.apache.hadoop.hbase.TableName TABLE_NAME
private static final byte[] FAMILY
private static final int FIRST_BATCH_COUNT
private static final int SECOND_BATCH_COUNT
public TestIOFencing()
public void testFencingAroundCompaction() throws Exception
Exception
public void testFencingAroundCompactionAfterWALSync() throws Exception
Exception
Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.