FixedFileTrailer (Apache HBase 2.3.0 API)

java.lang.Object
- org.apache.hadoop.hbase.io.hfile.FixedFileTrailer

```
@InterfaceAudience.Private
public class FixedFileTrailer
extends Object
```
The HFile has a fixed trailer which contains offsets to other variable parts of the file. Also includes basic metadata on this file. The trailer size is fixed within a given HFile format version only, but we always store the version number as the last four-byte integer of the file. The version number itself is split into two portions, a major version and a minor version. The last three bytes of a file are the major version and a single preceding byte is the minor number. The major version determines which readers/writers to use to read/write a hfile while a minor version determines smaller changes in hfile format that do not need a new reader/writer type.

Field Summary

Fields
Modifier and Type	Field and Description
`private String`	`comparatorClassName` Raw key comparator class name in version 3
`private Compression.Algorithm`	`compressionCodec` The compression codec used for all blocks.
`private int`	`dataIndexCount` The number of entries in the root data index.
`private byte[]`	`encryptionKey` The encryption key
`private long`	`entryCount` The number of key/value pairs in the file.
`private long`	`fileInfoOffset` Offset to the fileinfo data, a small block of vitals.
`private long`	`firstDataBlockOffset` The offset of the first data block.
`private long`	`lastDataBlockOffset` It is guaranteed that no key/value data blocks start after this offset in the file.
`private long`	`loadOnOpenDataOffset` In version 1, the offset to the data block index.
`private static org.slf4j.Logger`	`LOG`
`private int`	`majorVersion` The `HFile` format major version.
`private static int`	`MAX_COMPARATOR_NAME_LENGTH` We store the comparator class name as a fixed-length field in the trailer.
`private static int`	`MAX_TRAILER_SIZE`
`private int`	`metaIndexCount` The number of entries in the meta index
`private int`	`minorVersion` The `HFile` format minor version.
`private static int`	`NOT_PB_SIZE`
`private int`	`numDataIndexLevels` The number of levels in the potentially multi-level data index.
`private long`	`totalUncompressedBytes` The total uncompressed size of keys/values stored in the file.
`private static int[]`	`TRAILER_SIZE`
`private long`	`uncompressedDataIndexSize` Total uncompressed size of all blocks of the data index

Constructor Summary

Constructors
Constructor and Description

FixedFileTrailer(int majorVersion, int minorVersion)

Constructors
Constructor and Description
`FixedFileTrailer(int majorVersion, int minorVersion)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`private void`	`append(StringBuilder sb, String s)`
`private static int[]`	`computeTrailerSizeByVersion()`
`(package private) CellComparator`	`createComparator()`
`(package private) static CellComparator`	`createComparator(String comparatorClassName)`
`(package private) void`	`deserialize(DataInputStream inputStream)` Deserialize the fixed file trailer from the given stream.
`(package private) void`	`deserializeFromPB(DataInputStream inputStream)` Deserialize the file trailer as protobuf
`(package private) void`	`deserializeFromWritable(DataInput input)` Deserialize the file trailer as writable data
`void`	`expectAtLeastMajorVersion(int lowerBound)`
`void`	`expectMajorVersion(int expected)`
`void`	`expectMinorVersion(int expected)`
`private static int`	`extractMajorVersion(int serializedVersion)` Extracts the major version for a 4-byte serialized version data.
`private static int`	`extractMinorVersion(int serializedVersion)` Extracts the minor version for a 4-byte serialized version data.
`private static Class<? extends CellComparator>`	`getComparatorClass(String comparatorClassName)`
`String`	`getComparatorClassName()`
`Compression.Algorithm`	`getCompressionCodec()`
`int`	`getDataIndexCount()`
`byte[]`	`getEncryptionKey()`
`long`	`getEntryCount()`
`long`	`getFileInfoOffset()`
`long`	`getFirstDataBlockOffset()`
`private String`	`getHBase1CompatibleName(String comparator)` Deprecated. Since hbase-2.0.0. Will be removed in hbase-3.0.0.
`long`	`getLastDataBlockOffset()`
`long`	`getLoadOnOpenDataOffset()`
`int`	`getMajorVersion()` Returns the major version of this HFile format
`private static int`	`getMaxTrailerSize()`
`int`	`getMetaIndexCount()`
`int`	`getMinorVersion()` Returns the minor version of this HFile format
`int`	`getNumDataIndexLevels()`
`long`	`getTotalUncompressedBytes()`
`int`	`getTrailerSize()`
`(package private) static int`	`getTrailerSize(int version)`
`long`	`getUncompressedDataIndexSize()`
`(package private) static int`	`materializeVersion(int majorVersion, int minorVersion)` Create a 4 byte serialized version number by combining the minor and major version numbers.
`static FixedFileTrailer`	`readFromStream(org.apache.hadoop.fs.FSDataInputStream istream, long fileSize)` Reads a file trailer from the given file.
`(package private) void`	`serialize(DataOutputStream outputStream)` Write the trailer to a data stream.
`(package private) void`	`serializeAsPB(DataOutputStream output)` Write trailer data as protobuf.
`void`	`setComparatorClass(Class<? extends CellComparator> klass)`
`void`	`setCompressionCodec(Compression.Algorithm compressionCodec)`
`void`	`setDataIndexCount(int dataIndexCount)`
`void`	`setEncryptionKey(byte[] keyBytes)`
`void`	`setEntryCount(long newEntryCount)`
`void`	`setFileInfoOffset(long fileInfoOffset)`
`void`	`setFirstDataBlockOffset(long firstDataBlockOffset)`
`void`	`setLastDataBlockOffset(long lastDataBlockOffset)`
`void`	`setLoadOnOpenOffset(long loadOnOpenDataOffset)`
`void`	`setMetaIndexCount(int metaIndexCount)`
`void`	`setNumDataIndexLevels(int numDataIndexLevels)`
`void`	`setTotalUncompressedBytes(long totalUncompressedBytes)`
`void`	`setUncompressedDataIndexSize(long uncompressedDataIndexSize)`
`(package private) org.apache.hadoop.hbase.shaded.protobuf.generated.HFileProtos.FileTrailerProto`	`toProtobuf()`
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.slf4j.Logger LOG
```
  - MAX_COMPARATOR_NAME_LENGTH
```
private static final int MAX_COMPARATOR_NAME_LENGTH
```
    We store the comparator class name as a fixed-length field in the trailer.
    
    See Also:
    
    Constant Field Values
  - fileInfoOffset
```
private long fileInfoOffset
```
    Offset to the fileinfo data, a small block of vitals. Necessary in v1 but only potentially useful for pretty-printing in v2.
  - loadOnOpenDataOffset
```
private long loadOnOpenDataOffset
```
    In version 1, the offset to the data block index. Starting from version 2, the meaning of this field is the offset to the section of the file that should be loaded at the time the file is being opened: i.e. on open we load the root index, file info, etc. See http://hbase.apache.org/book.html#_hfile_format_2 in the reference guide.
  - dataIndexCount
```
private int dataIndexCount
```
    The number of entries in the root data index.
  - uncompressedDataIndexSize
```
private long uncompressedDataIndexSize
```
    Total uncompressed size of all blocks of the data index
  - metaIndexCount
```
private int metaIndexCount
```
    The number of entries in the meta index
  - totalUncompressedBytes
```
private long totalUncompressedBytes
```
    The total uncompressed size of keys/values stored in the file.
  - entryCount
```
private long entryCount
```
    The number of key/value pairs in the file. This field was int in version 1, but is now long.
  - compressionCodec
```
private Compression.Algorithm compressionCodec
```
    The compression codec used for all blocks.
  - numDataIndexLevels
```
private int numDataIndexLevels
```
    The number of levels in the potentially multi-level data index. Used from version 2 onwards.
  - firstDataBlockOffset
```
private long firstDataBlockOffset
```
    The offset of the first data block.
  - lastDataBlockOffset
```
private long lastDataBlockOffset
```
    It is guaranteed that no key/value data blocks start after this offset in the file.
  - comparatorClassName
```
private String comparatorClassName
```
    Raw key comparator class name in version 3
  - encryptionKey
```
private byte[] encryptionKey
```
    The encryption key
  - majorVersion
```
private final int majorVersion
```
    The HFile format major version.
  - minorVersion
```
private final int minorVersion
```
    The HFile format minor version.
  - TRAILER_SIZE
```
private static final int[] TRAILER_SIZE
```
  - MAX_TRAILER_SIZE
```
private static final int MAX_TRAILER_SIZE
```
  - NOT_PB_SIZE
```
private static final int NOT_PB_SIZE
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - FixedFileTrailer
```
FixedFileTrailer(int majorVersion,
                 int minorVersion)
```
- Method Detail
  - computeTrailerSizeByVersion
```
private static int[] computeTrailerSizeByVersion()
```
  - getMaxTrailerSize
```
private static int getMaxTrailerSize()
```
  - getTrailerSize
```
static int getTrailerSize(int version)
```
  - getTrailerSize
```
public int getTrailerSize()
```
  - serialize
```
void serialize(DataOutputStream outputStream)
        throws IOException
```
    Write the trailer to a data stream. We support writing version 1 for testing and for determining version 1 trailer size. It is also easy to see what fields changed in version 2.
    
    Throws:
    
    IOException
  - toProtobuf
```
org.apache.hadoop.hbase.shaded.protobuf.generated.HFileProtos.FileTrailerProto toProtobuf()
```
  - serializeAsPB
```
void serializeAsPB(DataOutputStream output)
            throws IOException
```
    Write trailer data as protobuf. NOTE: we run a translation on the comparator name and will serialize the old hbase-1.x where it makes sense. See getHBase1CompatibleName(String).
    
    Throws:
    
    IOException
  - deserialize
```
void deserialize(DataInputStream inputStream)
          throws IOException
```
    Deserialize the fixed file trailer from the given stream. The version needs to already be specified. Make sure this is consistent with serialize(DataOutputStream).
    
    Throws:
    
    IOException
  - deserializeFromPB
```
void deserializeFromPB(DataInputStream inputStream)
                throws IOException
```
    Deserialize the file trailer as protobuf
    
    Throws:
    
    IOException
  - deserializeFromWritable
```
void deserializeFromWritable(DataInput input)
                      throws IOException
```
    Deserialize the file trailer as writable data
    
    Throws:
    
    IOException
  - append
```
private void append(StringBuilder sb,
                    String s)
```
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - readFromStream
```
public static FixedFileTrailer readFromStream(org.apache.hadoop.fs.FSDataInputStream istream,
                                              long fileSize)
                                       throws IOException
```
    Reads a file trailer from the given file.
    
    Parameters:
    
    istream - the input stream with the ability to seek. Does not have to be buffered, as only one read operation is made.
    
    fileSize - the file size. Can be obtained using FileSystem.getFileStatus( org.apache.hadoop.fs.Path).
    
    Returns:
    
    the fixed file trailer read
    
    Throws:
    
    IOException - if failed to read from the underlying stream, or the trailer is corrupted, or the version of the trailer is unsupported
  - expectMajorVersion
```
public void expectMajorVersion(int expected)
```
  - expectMinorVersion
```
public void expectMinorVersion(int expected)
```
  - expectAtLeastMajorVersion
```
public void expectAtLeastMajorVersion(int lowerBound)
```
  - getFileInfoOffset
```
public long getFileInfoOffset()
```
  - setFileInfoOffset
```
public void setFileInfoOffset(long fileInfoOffset)
```
  - getLoadOnOpenDataOffset
```
public long getLoadOnOpenDataOffset()
```
  - setLoadOnOpenOffset
```
public void setLoadOnOpenOffset(long loadOnOpenDataOffset)
```
  - getDataIndexCount
```
public int getDataIndexCount()
```
  - setDataIndexCount
```
public void setDataIndexCount(int dataIndexCount)
```
  - getMetaIndexCount
```
public int getMetaIndexCount()
```
  - setMetaIndexCount
```
public void setMetaIndexCount(int metaIndexCount)
```
  - getTotalUncompressedBytes
```
public long getTotalUncompressedBytes()
```
  - setTotalUncompressedBytes
```
public void setTotalUncompressedBytes(long totalUncompressedBytes)
```
  - getEntryCount
```
public long getEntryCount()
```
  - setEntryCount
```
public void setEntryCount(long newEntryCount)
```
  - getCompressionCodec
```
public Compression.Algorithm getCompressionCodec()
```
  - setCompressionCodec
```
public void setCompressionCodec(Compression.Algorithm compressionCodec)
```
  - getNumDataIndexLevels
```
public int getNumDataIndexLevels()
```
  - setNumDataIndexLevels
```
public void setNumDataIndexLevels(int numDataIndexLevels)
```
  - getLastDataBlockOffset
```
public long getLastDataBlockOffset()
```
  - setLastDataBlockOffset
```
public void setLastDataBlockOffset(long lastDataBlockOffset)
```
  - getFirstDataBlockOffset
```
public long getFirstDataBlockOffset()
```
  - setFirstDataBlockOffset
```
public void setFirstDataBlockOffset(long firstDataBlockOffset)
```
  - getComparatorClassName
```
public String getComparatorClassName()
```
  - getMajorVersion
```
public int getMajorVersion()
```
    Returns the major version of this HFile format
  - getMinorVersion
```
public int getMinorVersion()
```
    Returns the minor version of this HFile format
  - setComparatorClass
```
public void setComparatorClass(Class<? extends CellComparator> klass)
```
  - getHBase1CompatibleName
```
@Deprecated
private String getHBase1CompatibleName(String comparator)
```
    Deprecated. Since hbase-2.0.0. Will be removed in hbase-3.0.0.
    
    If a 'standard' Comparator, write the old name for the Comparator when we serialize rather than the new name; writing the new name will make it so newly-written hfiles are not parseable by hbase-1.x, a facility we'd like to preserve across rolling upgrade and hbase-1.x clusters reading hbase-2.x produce.
    The Comparators in hbase-2.x work the same as they did in hbase-1.x; they compare KeyValues. In hbase-2.x they were renamed making use of the more generic 'Cell' nomenclature to indicate that we intend to move away from KeyValues post hbase-2. A naming change is not reason enough to make it so hbase-1.x cannot read hbase-2.x files given the structure goes unchanged (hfile v3). So, lets write the old names for Comparators into the hfile tails in hbase-2. Here is where we do the translation. getComparatorClass(String) does translation going the other way.
    The translation is done on the serialized Protobuf only.
    
    Parameters:
    
    comparator - String class name of the Comparator used in this hfile.
    
    Returns:
    
    What to store in the trailer as our comparator name.
    
    Since:
    
    hbase-2.0.0.
    
    See Also:
    
    getComparatorClass(String)
  - getComparatorClass
```
private static Class<? extends CellComparator> getComparatorClass(String comparatorClassName)
                                                           throws IOException
```
    Throws:
    
    IOException
  - createComparator
```
static CellComparator createComparator(String comparatorClassName)
                                throws IOException
```
    Throws:
    
    IOException
  - createComparator
```
CellComparator createComparator()
                         throws IOException
```
    Throws:
    
    IOException
  - getUncompressedDataIndexSize
```
public long getUncompressedDataIndexSize()
```
  - setUncompressedDataIndexSize
```
public void setUncompressedDataIndexSize(long uncompressedDataIndexSize)
```
  - getEncryptionKey
```
public byte[] getEncryptionKey()
```
  - setEncryptionKey
```
public void setEncryptionKey(byte[] keyBytes)
```
  - extractMajorVersion
```
private static int extractMajorVersion(int serializedVersion)
```
    Extracts the major version for a 4-byte serialized version data. The major version is the 3 least significant bytes
  - extractMinorVersion
```
private static int extractMinorVersion(int serializedVersion)
```
    Extracts the minor version for a 4-byte serialized version data. The major version are the 3 the most significant bytes
  - materializeVersion
```
static int materializeVersion(int majorVersion,
                              int minorVersion)
```
    Create a 4 byte serialized version number by combining the minor and major version numbers.

Class FixedFileTrailer

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

MAX_COMPARATOR_NAME_LENGTH

fileInfoOffset

loadOnOpenDataOffset

dataIndexCount

uncompressedDataIndexSize

metaIndexCount

totalUncompressedBytes

entryCount

compressionCodec

numDataIndexLevels

firstDataBlockOffset

lastDataBlockOffset

comparatorClassName

encryptionKey

majorVersion

minorVersion

TRAILER_SIZE

MAX_TRAILER_SIZE

NOT_PB_SIZE

Constructor Detail

FixedFileTrailer

Method Detail

computeTrailerSizeByVersion

getMaxTrailerSize

getTrailerSize

getTrailerSize

serialize

toProtobuf

serializeAsPB

deserialize

deserializeFromPB

deserializeFromWritable

append

toString

readFromStream

expectMajorVersion

expectMinorVersion

expectAtLeastMajorVersion

getFileInfoOffset

setFileInfoOffset

getLoadOnOpenDataOffset

setLoadOnOpenOffset

getDataIndexCount

setDataIndexCount

getMetaIndexCount

setMetaIndexCount

getTotalUncompressedBytes

setTotalUncompressedBytes

getEntryCount

setEntryCount

getCompressionCodec

setCompressionCodec

getNumDataIndexLevels

setNumDataIndexLevels

getLastDataBlockOffset

setLastDataBlockOffset

getFirstDataBlockOffset

setFirstDataBlockOffset

getComparatorClassName

getMajorVersion

getMinorVersion

setComparatorClass

getHBase1CompatibleName

getComparatorClass

createComparator

createComparator

getUncompressedDataIndexSize

setUncompressedDataIndexSize

getEncryptionKey

setEncryptionKey

extractMajorVersion

extractMinorVersion

materializeVersion