Package org.apache.hadoop.hbase.io.compress.aircompressor

Class ZstdCodec

java.lang.Object

org.apache.hadoop.hbase.io.compress.aircompressor.ZstdCodec

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.compress.CompressionCodec

@Private
public class ZstdCodec
extends Object
implements org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.compress.CompressionCodec

Hadoop codec implementation for Zstandard, implemented with aircompressor.

Unlike the other codecs this one should be considered as under development and unstable (as in changing), reflecting the status of aircompressor's zstandard implementation.

NOTE: This codec is NOT data format compatible with the Hadoop native zstandard codec. There are issues with both framing and limitations of the aircompressor zstandard compressor. This codec can be used as an alternative to the native codec, if the native codec cannot be made available and/or an eventual migration will never be necessary (i.e. this codec's performance meets anticipated requirements). Once you begin using this alternative you will be locked into it.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	ZstdCodec.HadoopZstdCompressor
class	ZstdCodec.HadoopZstdDecompressor

Nested classes/interfaces inherited from interface org.apache.hadoop.io.compress.CompressionCodec

org.apache.hadoop.io.compress.CompressionCodec.Util

Field Summary

Fields

Modifier and Type	Field	Description
private int	bufferSize
private org.apache.hadoop.conf.Configuration	conf
static final int	ZSTD_BUFFER_SIZE_DEFAULT
static final String	ZSTD_BUFFER_SIZE_KEY

Constructor Summary

Constructors

Constructor	Description
ZstdCodec()

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.io.compress.Compressor	createCompressor()
org.apache.hadoop.io.compress.Decompressor	createDecompressor()
org.apache.hadoop.io.compress.CompressionInputStream	createInputStream(InputStream in)
org.apache.hadoop.io.compress.CompressionInputStream	createInputStream(InputStream in, org.apache.hadoop.io.compress.Decompressor d)
org.apache.hadoop.io.compress.CompressionOutputStream	createOutputStream(OutputStream out)
org.apache.hadoop.io.compress.CompressionOutputStream	createOutputStream(OutputStream out, org.apache.hadoop.io.compress.Compressor c)
(package private) static int	getBufferSize(org.apache.hadoop.conf.Configuration conf)
Class<? extends org.apache.hadoop.io.compress.Compressor>	getCompressorType()
org.apache.hadoop.conf.Configuration	getConf()
Class<? extends org.apache.hadoop.io.compress.Decompressor>	getDecompressorType()
String	getDefaultExtension()
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

ZSTD_BUFFER_SIZE_KEY

public static final String ZSTD_BUFFER_SIZE_KEY

See Also:

• Constant Field Values

ZSTD_BUFFER_SIZE_DEFAULT

public static final int ZSTD_BUFFER_SIZE_DEFAULT

See Also:

• Constant Field Values

conf

private org.apache.hadoop.conf.Configuration conf

bufferSize

private int bufferSize

Constructor Details

ZstdCodec

public ZstdCodec()

Method Details

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

createCompressor

public org.apache.hadoop.io.compress.Compressor createCompressor()

Specified by:

createCompressor in interface org.apache.hadoop.io.compress.CompressionCodec

createDecompressor

public org.apache.hadoop.io.compress.Decompressor createDecompressor()

Specified by:

createDecompressor in interface org.apache.hadoop.io.compress.CompressionCodec

createInputStream

public org.apache.hadoop.io.compress.CompressionInputStream createInputStream(InputStream in)
                                                                       throws IOException

Specified by:

createInputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

createInputStream

public org.apache.hadoop.io.compress.CompressionInputStream createInputStream(InputStream in,
 org.apache.hadoop.io.compress.Decompressor d)
                                                                       throws IOException

Specified by:

createInputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

createOutputStream

public org.apache.hadoop.io.compress.CompressionOutputStream createOutputStream(OutputStream out)
                                                                         throws IOException

Specified by:

createOutputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

createOutputStream

public org.apache.hadoop.io.compress.CompressionOutputStream createOutputStream(OutputStream out,
 org.apache.hadoop.io.compress.Compressor c)
                                                                         throws IOException

Specified by:

createOutputStream in interface org.apache.hadoop.io.compress.CompressionCodec

Throws:

IOException

getCompressorType

public Class<? extends org.apache.hadoop.io.compress.Compressor> getCompressorType()

Specified by:

getCompressorType in interface org.apache.hadoop.io.compress.CompressionCodec

getDecompressorType

public Class<? extends org.apache.hadoop.io.compress.Decompressor> getDecompressorType()

Specified by:

getDecompressorType in interface org.apache.hadoop.io.compress.CompressionCodec

getDefaultExtension

public String getDefaultExtension()

Specified by:

getDefaultExtension in interface org.apache.hadoop.io.compress.CompressionCodec

getBufferSize

static int getBufferSize(org.apache.hadoop.conf.Configuration conf)