ProtobufLogReader (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.regionserver.wal.ReaderBase
- - org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader

All Implemented Interfaces:

Closeable, AutoCloseable, DefaultWALProvider.Reader, WAL.Reader

Direct Known Subclasses:

SecureProtobufLogReader
```
@InterfaceAudience.LimitedPrivate(value={"Coprocesssor","Phoenix","Configuration"})
public class ProtobufLogReader
extends ReaderBase
```
A Protobuf based WAL has the following structure:
<PB_WAL_MAGIC><WALHeader><WALEdits>...<WALEdits><Trailer> <TrailerSize> <PB_WAL_COMPLETE_MAGIC>
The Reader reads meta information (WAL Compression state, WALTrailer, etc) in ProtobufLogReader#initReader(FSDataInputStream). A WALTrailer is an extensible structure which is appended at the end of the WAL. This is empty for now; it can contain some meta information such as Region level stats, etc in future.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

(package private) static class ProtobufLogReader.WALHdrContext

(package private) static class ProtobufLogReader.WALHdrResult

Nested Classes
Modifier and Type	Class and Description
`(package private) static class`	`ProtobufLogReader.WALHdrContext`
`(package private) static class`	`ProtobufLogReader.WALHdrResult`

Field Summary

Fields
Modifier and Type	Field and Description
`protected WALCellCodec.ByteStringUncompressor`	`byteStringUncompressor`
`protected Codec.Decoder`	`cellDecoder`
`private String`	`codecClsName`
`(package private) static int`	`DEFAULT_WAL_TRAILER_WARN_SIZE`
`protected boolean`	`hasCompression`
`protected boolean`	`hasTagCompression`
`protected org.apache.hadoop.fs.FSDataInputStream`	`inputStream`
`private static org.apache.commons.logging.Log`	`LOG`
`static byte[]`	`PB_WAL_COMPLETE_MAGIC`
`static byte[]`	`PB_WAL_MAGIC`
`protected org.apache.hadoop.hbase.protobuf.generated.WALProtos.WALTrailer`	`trailer`
`private boolean`	`trailerPresent`
`protected int`	`trailerWarnSize`
`(package private) static String`	`WAL_TRAILER_WARN_SIZE` Configuration name of WAL Trailer's warning size.
`private long`	`walEditsStopOffset`
`private static List<String>`	`writerClsNames`

Fields inherited from class org.apache.hadoop.hbase.regionserver.wal.ReaderBase
compressionContext, conf, edit, emptyCompressionContext, fileLength, fs, path

Constructor Summary

Constructors
Constructor and Description

ProtobufLogReader()

Constructors
Constructor and Description
`ProtobufLogReader()`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`close()`
`private IOException`	`extractHiddenEof(Exception ex)`
`protected WALCellCodec`	`getCodec(org.apache.hadoop.conf.Configuration conf, String cellCodecClsName, CompressionContext compressionContext)`
`String`	`getCodecClsName()`
`long`	`getPosition()`
`List<String>`	`getWriterClsNames()`
`protected boolean`	`hasCompression()`
`protected boolean`	`hasTagCompression()`
`void`	`init(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FSDataInputStream stream)`
`protected void`	`initAfterCompression()` Initializes the compression after the shared stuff has been initialized.
`protected void`	`initAfterCompression(String cellCodecClsName)` Initializes the compression after the shared stuff has been initialized.
`private String`	`initInternal(org.apache.hadoop.fs.FSDataInputStream stream, boolean isFirst)`
`protected String`	`initReader(org.apache.hadoop.fs.FSDataInputStream stream)` Initializes the log reader with a particular stream (may be null).
`protected ProtobufLogReader.WALHdrContext`	`readHeader(org.apache.hadoop.hbase.protobuf.generated.WALProtos.WALHeader.Builder builder, org.apache.hadoop.fs.FSDataInputStream stream)`
`protected boolean`	`readNext(WAL.Entry entry)` Read next entry.
`void`	`reset()`
`protected void`	`seekOnFs(long pos)` Performs a filesystem-level seek to a certain position in an underlying file.
`private boolean`	`setTrailerIfPresent()` To check whether a trailer is present in a WAL, it seeks to position (fileLength - PB_WAL_COMPLETE_MAGIC.size() - Bytes.SIZEOF_INT).
`long`	`trailerSize()`

Methods inherited from class org.apache.hadoop.hbase.regionserver.wal.ReaderBase
next, next, seek

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - PB_WAL_MAGIC
```
@InterfaceAudience.Private
public static final byte[] PB_WAL_MAGIC
```
  - PB_WAL_COMPLETE_MAGIC
```
@InterfaceAudience.Private
public static final byte[] PB_WAL_COMPLETE_MAGIC
```
  - WAL_TRAILER_WARN_SIZE
```
static final String WAL_TRAILER_WARN_SIZE
```
    Configuration name of WAL Trailer's warning size. If a waltrailer's size is greater than the configured size, providers should log a warning. e.g. this is used with Protobuf reader/writer.
    
    See Also:
    Constant Field Values
  - DEFAULT_WAL_TRAILER_WARN_SIZE
```
static final int DEFAULT_WAL_TRAILER_WARN_SIZE
```
    See Also:
    Constant Field Values
  - inputStream
```
protected org.apache.hadoop.fs.FSDataInputStream inputStream
```
  - cellDecoder
```
protected Codec.Decoder cellDecoder
```
  - byteStringUncompressor
```
protected WALCellCodec.ByteStringUncompressor byteStringUncompressor
```
  - hasCompression
```
protected boolean hasCompression
```
  - hasTagCompression
```
protected boolean hasTagCompression
```
  - walEditsStopOffset
```
private long walEditsStopOffset
```
  - trailerPresent
```
private boolean trailerPresent
```
  - trailer
```
protected org.apache.hadoop.hbase.protobuf.generated.WALProtos.WALTrailer trailer
```
  - trailerWarnSize
```
protected int trailerWarnSize
```
  - writerClsNames
```
private static List<String> writerClsNames
```
  - codecClsName
```
private String codecClsName
```
- Constructor Detail
  - ProtobufLogReader
```
public ProtobufLogReader()
```
- Method Detail
  - trailerSize
```
@InterfaceAudience.Private
public long trailerSize()
```
  - close
```
public void close()
           throws IOException
```
    Throws:
    
    IOException
  - getPosition
```
public long getPosition()
                 throws IOException
```
    Throws:
    
    IOException
  - reset
```
public void reset()
           throws IOException
```
    Throws:
    
    IOException
  - init
```
public void init(org.apache.hadoop.fs.FileSystem fs,
        org.apache.hadoop.fs.Path path,
        org.apache.hadoop.conf.Configuration conf,
        org.apache.hadoop.fs.FSDataInputStream stream)
          throws IOException
```
    Specified by:
    
    init in interface DefaultWALProvider.Reader
    
    Overrides:
    
    init in class ReaderBase
    
    Parameters:
    fs - File system.
    path - Path.
    conf - Configuration.
    stream - Input stream that may have been pre-opened by the caller; may be null.
    
    Throws:
    
    IOException
  - initReader
```
protected String initReader(org.apache.hadoop.fs.FSDataInputStream stream)
                     throws IOException
```
    Description copied from class: ReaderBase
    
    Initializes the log reader with a particular stream (may be null). Reader assumes ownership of the stream if not null and may use it. Called once.
    
    Specified by:
    
    initReader in class ReaderBase
    
    Returns:
    the class name of cell Codec, null if such information is not available
    
    Throws:
    
    IOException
  - getWriterClsNames
```
public List<String> getWriterClsNames()
```
  - getCodecClsName
```
public String getCodecClsName()
```
  - readHeader
```
protected ProtobufLogReader.WALHdrContext readHeader(org.apache.hadoop.hbase.protobuf.generated.WALProtos.WALHeader.Builder builder,
                                         org.apache.hadoop.fs.FSDataInputStream stream)
                                              throws IOException
```
    Throws:
    
    IOException
  - initInternal
```
private String initInternal(org.apache.hadoop.fs.FSDataInputStream stream,
                  boolean isFirst)
                     throws IOException
```
    Throws:
    
    IOException
  - setTrailerIfPresent
```
private boolean setTrailerIfPresent()
```
    To check whether a trailer is present in a WAL, it seeks to position (fileLength - PB_WAL_COMPLETE_MAGIC.size() - Bytes.SIZEOF_INT). It reads the int value to know the size of the trailer, and checks whether the trailer is present at the end or not by comparing the last PB_WAL_COMPLETE_MAGIC.size() bytes. In case trailer is not present, it returns false; otherwise, sets the trailer and sets this.walEditsStopOffset variable up to the point just before the trailer.
    - fileLength is 0 or not correct (when file is under recovery, etc).
    - the trailer size is negative.
    In case the trailer size > this.trailerMaxSize, it is read after a WARN message.
    Returns:
    true if a valid trailer is present
    
    Throws:
    
    IOException
  - getCodec
```
protected WALCellCodec getCodec(org.apache.hadoop.conf.Configuration conf,
                    String cellCodecClsName,
                    CompressionContext compressionContext)
                         throws IOException
```
    Throws:
    
    IOException
  - initAfterCompression
```
protected void initAfterCompression()
                             throws IOException
```
    Description copied from class: ReaderBase
    
    Initializes the compression after the shared stuff has been initialized. Called once.
    
    Specified by:
    
    initAfterCompression in class ReaderBase
    
    Throws:
    
    IOException
  - initAfterCompression
```
protected void initAfterCompression(String cellCodecClsName)
                             throws IOException
```
    Description copied from class: ReaderBase
    
    Initializes the compression after the shared stuff has been initialized. Called once.
    
    Specified by:
    
    initAfterCompression in class ReaderBase
    
    Parameters:
    cellCodecClsName - class name of cell Codec
    
    Throws:
    
    IOException
  - hasCompression
```
protected boolean hasCompression()
```
    Specified by:
    
    hasCompression in class ReaderBase
    
    Returns:
    Whether compression is enabled for this log.
  - hasTagCompression
```
protected boolean hasTagCompression()
```
    Specified by:
    
    hasTagCompression in class ReaderBase
    
    Returns:
    Whether tag compression is enabled for this log.
  - readNext
```
protected boolean readNext(WAL.Entry entry)
                    throws IOException
```
    Description copied from class: ReaderBase
    
    Read next entry.
    
    Specified by:
    
    readNext in class ReaderBase
    
    Parameters:
    entry - The entry to read into.
    
    Returns:
    Whether there was anything to read.
    
    Throws:
    
    IOException
  - extractHiddenEof
```
private IOException extractHiddenEof(Exception ex)
```
  - seekOnFs
```
protected void seekOnFs(long pos)
                 throws IOException
```
    Description copied from class: ReaderBase
    
    Performs a filesystem-level seek to a certain position in an underlying file.
    
    Specified by:
    
    seekOnFs in class ReaderBase
    
    Throws:
    
    IOException

Class ProtobufLogReader

Nested Class Summary

Field Summary

Fields inherited from class org.apache.hadoop.hbase.regionserver.wal.ReaderBase

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.hbase.regionserver.wal.ReaderBase

Methods inherited from class java.lang.Object

Field Detail

LOG

PB_WAL_MAGIC

PB_WAL_COMPLETE_MAGIC

WAL_TRAILER_WARN_SIZE

DEFAULT_WAL_TRAILER_WARN_SIZE

inputStream

cellDecoder

byteStringUncompressor

hasCompression

hasTagCompression

walEditsStopOffset

trailerPresent

trailer

trailerWarnSize

writerClsNames

codecClsName

Constructor Detail

ProtobufLogReader

Method Detail

trailerSize

close

getPosition

reset

init

initReader

getWriterClsNames

getCodecClsName

readHeader

initInternal

setTrailerIfPresent

getCodec

initAfterCompression

initAfterCompression

hasCompression

hasTagCompression

readNext

extractHiddenEof

seekOnFs