org.apache.hadoop.hbase.wal

Class WALEdit

java.lang.Object

org.apache.hadoop.hbase.wal.WALEdit

All Implemented Interfaces:

HeapSize

@InterfaceAudience.LimitedPrivate(value={"Replication","Coprocesssor"})
public class WALEdit
extends Object
implements HeapSize

Used in HBase's transaction log (WAL) to represent a collection of edits (Cell/KeyValue objects) that came in as a single transaction. All the edits for a given transaction are written out as a single record, in PB format, followed (optionally) by Cells written via the WALCellEncoder.

This class is LimitedPrivate for CPs to read-only. The add(org.apache.hadoop.hbase.Cell, byte[]) methods are classified as private methods, not for use by CPs.

WALEdit will accumulate a Set of all column family names referenced by the Cells add(Cell)'d. This is an optimization. Usually when loading a WALEdit, we have the column family name to-hand.. just shove it into the WALEdit if available. Doing this, we can save on a parse of each Cell to figure column family down the line when we go to add the WALEdit to the WAL file. See the hand-off in FSWALEntry Constructor.

Field Summary

Fields

Modifier and Type	Field and Description
static byte[]	BULK_LOAD
private ArrayList<Cell>	cells
static byte[]	COMPACTION
private Set<byte[]>	families All the Cell families in cells.
static byte[]	FLUSH
private static org.slf4j.Logger	LOG
static byte[]	METAFAMILY
static byte[]	METAROW
static byte[]	REGION_EVENT
private boolean	replay

Constructor Summary

Constructors

Constructor and Description
WALEdit()
WALEdit(boolean isReplay) Deprecated. since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.
WALEdit(int cellCount) Deprecated. since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.
WALEdit(int cellCount, boolean isReplay)

Method Summary

Modifier and Type	Method and Description
WALEdit	add(Cell cell)
WALEdit	add(Cell cell, byte[] family)
void	add(Map<byte[],List<Cell>> familyMap) Append the given map of family->edits to a WALEdit data structure.
private WALEdit	addCell(Cell cell)
private void	addFamily(byte[] family)
static WALEdit	createBulkLoadEvent(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor) Create a bulk loader WALEdit
static WALEdit	createCompaction(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c) Create a compaction WALEdit
static WALEdit	createFlushWALEdit(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f)
static WALEdit	createRegionEventWALEdit(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)
long	estimatedSerializedSizeOf()
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor	getBulkLoadDescriptor(Cell cell) Deserialized and returns a BulkLoadDescriptor from the passed in Cell
ArrayList<Cell>	getCells()
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor	getCompaction(Cell kv) Deserialized and returns a CompactionDescriptor is the KeyValue contains one.
Set<byte[]>	getFamilies() For use by FSWALEntry ONLY.
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor	getFlushDescriptor(Cell cell)
private Set<byte[]>	getOrCreateFamilies()
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor	getRegionEventDescriptor(Cell cell)
static byte[]	getRowForRegion(RegionInfo hri)
long	heapSize()
static boolean	isCompactionMarker(Cell cell) Returns true if the given cell is a serialized WALProtos.CompactionDescriptor
boolean	isEmpty()
boolean	isMetaEdit()
static boolean	isMetaEditFamily(byte[] f)
static boolean	isMetaEditFamily(Cell cell)
boolean	isReplay()
int	readFromCells(Codec.Decoder cellDecoder, int expectedCount) Reads WALEdit from cells.
void	setCells(ArrayList<Cell> cells) This is not thread safe.
int	size()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

METAFAMILY

public static final byte[] METAFAMILY

METAROW

public static final byte[] METAROW

COMPACTION

public static final byte[] COMPACTION

FLUSH

public static final byte[] FLUSH

REGION_EVENT

public static final byte[] REGION_EVENT

BULK_LOAD

public static final byte[] BULK_LOAD

replay

private final boolean replay

cells

private ArrayList<Cell> cells

families

private Set<byte[]> families

All the Cell families in cells. Updated by add(Cell) and add(Map). This Set is passed to the FSWALEntry so it does not have to recalculate the Set of families in a transaction; makes for a bunch of CPU savings. An optimization that saves on CPU-expensive Cell-parsing.

Constructor Detail

WALEdit

public WALEdit()

WALEdit

@Deprecated
public WALEdit(boolean isReplay)

Deprecated. since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.

See Also:

WALEdit(int, boolean), HBASE-20781

WALEdit

@Deprecated
public WALEdit(int cellCount)

Deprecated. since 2.0.1 and will be removed in 4.0.0. Use WALEdit(int, boolean) instead.

See Also:

WALEdit(int, boolean), HBASE-20781

WALEdit

public WALEdit(int cellCount,
               boolean isReplay)

Parameters:

cellCount - Pass so can pre-size the WALEdit. Optimization.

Method Detail

getOrCreateFamilies

private Set<byte[]> getOrCreateFamilies()

getFamilies

public Set<byte[]> getFamilies()

For use by FSWALEntry ONLY. An optimization.

Returns:

All families in getCells(); may be null.

isMetaEditFamily

public static boolean isMetaEditFamily(byte[] f)

Returns:

True is f is METAFAMILY

isMetaEditFamily

public static boolean isMetaEditFamily(Cell cell)

isMetaEdit

public boolean isMetaEdit()

isReplay

public boolean isReplay()

Returns:

True when current WALEdit is created by log replay. Replication skips WALEdits from replay.

add

@InterfaceAudience.Private
public WALEdit add(Cell cell,
                                              byte[] family)

add

@InterfaceAudience.Private
public WALEdit add(Cell cell)

isEmpty

public boolean isEmpty()

size

public int size()

getCells

public ArrayList<Cell> getCells()

setCells

@InterfaceAudience.Private
public void setCells(ArrayList<Cell> cells)

This is not thread safe. This will change the WALEdit and shouldn't be used unless you are sure that nothing else depends on the contents being immutable.

Parameters:

cells - the list of cells that this WALEdit now contains.

readFromCells

public int readFromCells(Codec.Decoder cellDecoder,
                         int expectedCount)
                  throws IOException

Reads WALEdit from cells.

Parameters:

cellDecoder - Cell decoder.

expectedCount - Expected cell count.

Returns:

Number of KVs read.

Throws:

IOException

heapSize

public long heapSize()

Specified by:

heapSize in interface HeapSize

Returns:

Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.

estimatedSerializedSizeOf

public long estimatedSerializedSizeOf()

toString

public String toString()

Overrides:

toString in class Object

createFlushWALEdit

public static WALEdit createFlushWALEdit(RegionInfo hri,
                                         org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f)

getFlushDescriptor

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor getFlushDescriptor(Cell cell)
                                                                                                      throws IOException

Throws:

IOException

createRegionEventWALEdit

public static WALEdit createRegionEventWALEdit(RegionInfo hri,
                                               org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)

getRegionEventDescriptor

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor getRegionEventDescriptor(Cell cell)
                                                                                                                  throws IOException

Throws:

IOException

createCompaction

public static WALEdit createCompaction(RegionInfo hri,
                                       org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c)

Create a compaction WALEdit

Parameters:

c -

Returns:

A WALEdit that has c serialized as its value

getRowForRegion

public static byte[] getRowForRegion(RegionInfo hri)

getCompaction

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor getCompaction(Cell kv)
                                                                                                      throws IOException

Deserialized and returns a CompactionDescriptor is the KeyValue contains one.

Parameters:

kv - the key value

Returns:

deserialized CompactionDescriptor or null.

Throws:

IOException

isCompactionMarker

public static boolean isCompactionMarker(Cell cell)

Returns true if the given cell is a serialized WALProtos.CompactionDescriptor

See Also:

getCompaction(Cell)

createBulkLoadEvent

public static WALEdit createBulkLoadEvent(RegionInfo hri,
                                          org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor)

Create a bulk loader WALEdit

Parameters:

hri - The RegionInfo for the region in which we are bulk loading

bulkLoadDescriptor - The descriptor for the Bulk Loader

Returns:

The WALEdit for the BulkLoad

getBulkLoadDescriptor

public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor getBulkLoadDescriptor(Cell cell)
                                                                                                            throws IOException

Deserialized and returns a BulkLoadDescriptor from the passed in Cell

Parameters:

cell - the key value

Returns:

deserialized BulkLoadDescriptor or null.

Throws:

IOException

add

public void add(Map<byte[],List<Cell>> familyMap)

Append the given map of family->edits to a WALEdit data structure. This does not write to the WAL itself. Note that as an optimization, we will stamp the Set of column families into the WALEdit to save on our having to calculate it subsequently way down in the actual WAL writing.

Parameters:

familyMap - map of family->edits

addFamily

private void addFamily(byte[] family)

addCell

private WALEdit addCell(Cell cell)