Package org.apache.hadoop.hbase.mapreduce

Class TestWALInputFormat

java.lang.Object

org.apache.hadoop.hbase.mapreduce.TestWALInputFormat

@Tag("org.apache.hadoop.hbase.testclassification.MapReduceTests") @Tag("org.apache.hadoop.hbase.testclassification.MediumTests")
public class TestWALInputFormat
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
private static final HBaseTestingUtil	TEST_UTIL

Constructor Summary

Constructors

Constructor	Description
TestWALInputFormat()

Method Summary

Modifier and Type	Method	Description
private List<org.apache.hadoop.mapreduce.InputSplit>	getSplitsForEmptyFile(boolean ignoreEmptyFiles)
private static boolean	isKept(org.apache.hadoop.fs.FileSystem fs, long created, long mtime, long start, long end)
private static boolean	isKept(org.apache.hadoop.fs.FileSystem fs, long created, long staleMtime, long refreshedMtime, long start, long end)
static void	setupClass()
void	testAddFile()	Test the primitive start/end time filtering.
void	testAddFileUsesModificationTimeOfClosedFilesOnly()	The name of a WAL carries its creation time, which only bounds its entries from below.
void	testEmptyFileIsIgnoredWhenConfigured()
void	testEmptyFileIsIncludedWhenNotIgnored()
void	testHandlesArchivedWALFiles()
void	testHandlesEmptyWALFile()	Test that an empty WAL file (which causes WALHeaderEOFException) is gracefully handled and skipped rather than causing the job to fail.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

TEST_UTIL

private static final HBaseTestingUtil TEST_UTIL

Constructor Details

TestWALInputFormat

public TestWALInputFormat()

Method Details

setupClass

@BeforeAll
public static void setupClass()
                       throws Exception

Throws:

Exception

testAddFile

@Test
public void testAddFile()

Test the primitive start/end time filtering.

isKept

private static boolean isKept(org.apache.hadoop.fs.FileSystem fs,
 long created,
 long mtime,
 long start,
 long end)

isKept

private static boolean isKept(org.apache.hadoop.fs.FileSystem fs,
 long created,
 long staleMtime,
 long refreshedMtime,
 long start,
 long end)

testAddFileUsesModificationTimeOfClosedFilesOnly

@Test
public void testAddFileUsesModificationTimeOfClosedFilesOnly()
                                                      throws Exception

The name of a WAL carries its creation time, which only bounds its entries from below. A WAL stays open until it rolls, so one created before startTime can still hold entries in range. Only a closed file has a final modification time that can rule it out.

Throws:

Exception

testHandlesArchivedWALFiles

@Test
public void testHandlesArchivedWALFiles()
                                 throws Exception

Throws:

Exception

testEmptyFileIsIgnoredWhenConfigured

@Test
public void testEmptyFileIsIgnoredWhenConfigured()
                                          throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

testEmptyFileIsIncludedWhenNotIgnored

@Test
public void testEmptyFileIsIncludedWhenNotIgnored()
                                           throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

getSplitsForEmptyFile

private List<org.apache.hadoop.mapreduce.InputSplit> getSplitsForEmptyFile(boolean ignoreEmptyFiles)
                                                                    throws IOException,
InterruptedException

Throws:

IOException

InterruptedException

testHandlesEmptyWALFile

@Test
public void testHandlesEmptyWALFile()
                             throws Exception

Test that an empty WAL file (which causes WALHeaderEOFException) is gracefully handled and skipped rather than causing the job to fail.

Throws:

Exception