org.apache.hadoop.hbase

Class HFilePerformanceEvaluation

java.lang.Object

org.apache.hadoop.hbase.HFilePerformanceEvaluation

@InterfaceAudience.LimitedPrivate(value="Tools")
public class HFilePerformanceEvaluation
extends Object

This class runs performance benchmarks for HFile.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	HFilePerformanceEvaluation.GaussianRandomReadBenchmark
(package private) static class	HFilePerformanceEvaluation.ReadBenchmark
(package private) static class	HFilePerformanceEvaluation.RowOrientedBenchmark
(package private) static class	HFilePerformanceEvaluation.SequentialReadBenchmark
(package private) static class	HFilePerformanceEvaluation.SequentialWriteBenchmark
(package private) static class	HFilePerformanceEvaluation.UniformRandomReadBenchmark
(package private) static class	HFilePerformanceEvaluation.UniformRandomSmallScan

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOG
private static int	RFILE_BLOCKSIZE
private static int	ROW_COUNT
private static int	ROW_LENGTH
private static StringBuilder	testSummary

Constructor Summary

Constructors

Constructor and Description
HFilePerformanceEvaluation()

Method Summary

Modifier and Type	Method and Description
(package private) static org.apache.hadoop.hbase.Cell	createCell(byte[] keyRow)
(package private) static org.apache.hadoop.hbase.Cell	createCell(byte[] keyRow, byte[] value)
(package private) static org.apache.hadoop.hbase.Cell	createCell(int i)
(package private) static org.apache.hadoop.hbase.Cell	createCell(int i, byte[] value) HFile is Cell-based.
(package private) static byte[]	format(int i)
(package private) static org.apache.hadoop.hbase.io.ImmutableBytesWritable	format(int i, org.apache.hadoop.hbase.io.ImmutableBytesWritable w)
private String	getCipherName(org.apache.hadoop.conf.Configuration conf, String cipherName)
static void	main(String[] args)
protected void	runBenchmark(HFilePerformanceEvaluation.RowOrientedBenchmark benchmark, int rowCount, String codec, String cipher)
private void	runBenchmarks() Add any supported codec or cipher to test the HFile read/write performance.
private void	runReadBenchmark(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path mf, String codec, String cipher) Run all the read benchmarks for the test HFile
private void	runWriteBenchmark(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path mf, String codec, String cipher) Write a test HFile with the given codec & cipher

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ROW_LENGTH

private static final int ROW_LENGTH

See Also:

Constant Field Values

ROW_COUNT

private static final int ROW_COUNT

See Also:

Constant Field Values

RFILE_BLOCKSIZE

private static final int RFILE_BLOCKSIZE

See Also:

Constant Field Values

testSummary

private static StringBuilder testSummary

LOG

private static final org.slf4j.Logger LOG

Constructor Detail

HFilePerformanceEvaluation

public HFilePerformanceEvaluation()

Method Detail

format

static byte[] format(int i)

format

static org.apache.hadoop.hbase.io.ImmutableBytesWritable format(int i,
                                                                org.apache.hadoop.hbase.io.ImmutableBytesWritable w)

createCell

static org.apache.hadoop.hbase.Cell createCell(int i)

createCell

static org.apache.hadoop.hbase.Cell createCell(int i,
                                               byte[] value)

HFile is Cell-based. It used to be byte arrays. Doing this test, pass Cells. All Cells intentionally have same coordinates in all fields but row.

Parameters:

i - Integer to format as a row Key.

value - Value to use

Returns:

Created Cell.

createCell

static org.apache.hadoop.hbase.Cell createCell(byte[] keyRow)

createCell

static org.apache.hadoop.hbase.Cell createCell(byte[] keyRow,
                                               byte[] value)

runBenchmarks

private void runBenchmarks()
                    throws Exception

Add any supported codec or cipher to test the HFile read/write performance. Specify "none" to disable codec or cipher or both.

Throws:

Exception

runWriteBenchmark

private void runWriteBenchmark(org.apache.hadoop.conf.Configuration conf,
                               org.apache.hadoop.fs.FileSystem fs,
                               org.apache.hadoop.fs.Path mf,
                               String codec,
                               String cipher)
                        throws Exception

Write a test HFile with the given codec & cipher

Parameters:

conf -

fs -

mf -

codec - "none", "lzo", "gz", "snappy"

cipher - "none", "aes"

Throws:

Exception

runReadBenchmark

private void runReadBenchmark(org.apache.hadoop.conf.Configuration conf,
                              org.apache.hadoop.fs.FileSystem fs,
                              org.apache.hadoop.fs.Path mf,
                              String codec,
                              String cipher)

Run all the read benchmarks for the test HFile

Parameters:

conf -

fs -

mf -

codec - "none", "lzo", "gz", "snappy"

cipher - "none", "aes"

runBenchmark

protected void runBenchmark(HFilePerformanceEvaluation.RowOrientedBenchmark benchmark,
                            int rowCount,
                            String codec,
                            String cipher)
                     throws Exception

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Parameters:

args -

Throws:

Exception

IOException

getCipherName

private String getCipherName(org.apache.hadoop.conf.Configuration conf,
                             String cipherName)