@InterfaceAudience.LimitedPrivate(value={"Replication","Coprocesssor"}) public class WALEdit extends Object implements HeapSize
This class is LimitedPrivate for CPs to read-only. The add(org.apache.hadoop.hbase.Cell, byte[])
methods are classified as
private methods, not for use by CPs.
A particular WALEdit 'type' is the 'meta' type used to mark key operational events in the WAL
such as compaction, flush, or region open. These meta types do not traverse hbase memstores. They
are edits made by the hbase system rather than edit data submitted by clients. They only show in
the WAL. These 'Meta' types have not been formally specified (or made into an explicit class
type). They evolved organically. HBASE-8457 suggests codifying a WALEdit 'type' by adding a type
field to WALEdit that gets serialized into the WAL. TODO. Would have to work on the
consumption-side. Reading WALs on replay we seem to consume a Cell-at-a-time rather than by
WALEdit. We are already in the below going out of our way to figure particular types -- e.g. if a
compaction, replay, or close meta Marker -- during normal processing so would make sense to do
this. Current system is an awkward marking of Cell columnfamily as METAFAMILY
and then
setting qualifier based off meta edit type. For replay-time where we read Cell-at-a-time, there
are utility methods below for figuring meta type. See also
createBulkLoadEvent(RegionInfo, WALProtos.BulkLoadDescriptor)
, etc., for where we create
meta WALEdit instances.
WALEdit will accumulate a Set of all column family names referenced by the Cells
add(Cell)
'd. This is an optimization. Usually when loading a WALEdit, we have the column
family name to-hand.. just shove it into the WALEdit if available. Doing this, we can save on a
parse of each Cell to figure column family down the line when we go to add the WALEdit to the WAL
file. See the hand-off in FSWALEntry Constructor.
WALKey
Modifier and Type | Field and Description |
---|---|
static byte[] |
BULK_LOAD |
private ArrayList<Cell> |
cells |
static byte[] |
COMPACTION
Deprecated.
Since 2.3.0. Make it protected, internal-use only. Use
isCompactionMarker(Cell) |
private Set<byte[]> |
families
All the Cell families in
cells . |
static byte[] |
FLUSH
Deprecated.
Since 2.3.0. Make it protected, internal-use only.
|
static byte[] |
METAFAMILY |
static byte[] |
METAROW
Deprecated.
Since 2.3.0. Not used.
|
static byte[] |
REGION_EVENT
Deprecated.
Since 2.3.0. Remove. Not for external use. Not used.
|
private static byte[] |
REGION_EVENT_CLOSE
We use this define figuring if we are carrying a close event.
|
private static byte[] |
REGION_EVENT_PREFIX |
private static String |
REGION_EVENT_PREFIX_STR |
private static String |
REGION_EVENT_STR
Qualifier for region event meta 'Marker' WALEdits start with the
REGION_EVENT_PREFIX
prefix ('HBASE::REGION_EVENT::'). |
private boolean |
replay |
Constructor and Description |
---|
WALEdit() |
WALEdit(boolean replay)
Deprecated.
since 2.0.1 and will be removed in 4.0.0. Use
WALEdit(int, boolean)
instead. |
WALEdit(int cellCount)
Deprecated.
since 2.0.1 and will be removed in 4.0.0. Use
WALEdit(int, boolean)
instead. |
WALEdit(int cellCount,
boolean isReplay) |
Modifier and Type | Method and Description |
---|---|
WALEdit |
add(Cell cell) |
WALEdit |
add(Cell cell,
byte[] family) |
void |
add(Map<byte[],List<Cell>> familyMap)
Append the given map of family->edits to a WALEdit data structure.
|
private WALEdit |
addCell(Cell cell) |
private void |
addFamily(byte[] family) |
static WALEdit |
createBulkLoadEvent(RegionInfo hri,
org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor)
Create a bulk loader WALEdit
|
static WALEdit |
createCompaction(RegionInfo hri,
org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c)
Returns A Marker WALEdit that has
c serialized as its value |
static WALEdit |
createFlushWALEdit(RegionInfo hri,
org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f) |
static byte[] |
createRegionEventDescriptorQualifier(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor.EventType t) |
static WALEdit |
createRegionEventWALEdit(byte[] rowForRegion,
org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc) |
static WALEdit |
createRegionEventWALEdit(RegionInfo hri,
org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc) |
long |
estimatedSerializedSizeOf() |
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor |
getBulkLoadDescriptor(Cell cell)
Deserialized and returns a BulkLoadDescriptor from the passed in Cell
|
ArrayList<Cell> |
getCells() |
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor |
getCompaction(Cell kv)
Deserialized and returns a CompactionDescriptor is the KeyValue contains one.
|
Set<byte[]> |
getFamilies()
For use by FSWALEntry ONLY.
|
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor |
getFlushDescriptor(Cell cell) |
private Set<byte[]> |
getOrCreateFamilies() |
static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor |
getRegionEventDescriptor(Cell cell) |
static byte[] |
getRowForRegion(RegionInfo hri) |
long |
heapSize()
Return the approximate 'exclusive deep size' of implementing object.
|
static boolean |
isCompactionMarker(Cell cell)
Returns true if the given cell is a serialized
WALProtos.CompactionDescriptor |
boolean |
isEmpty() |
boolean |
isMetaEdit() |
static boolean |
isMetaEditFamily(byte[] f)
Deprecated.
Since 2.3.0. Do not expose. Make protected.
|
static boolean |
isMetaEditFamily(Cell cell)
Replaying WALs can read Cell-at-a-time so need this method in those cases.
|
boolean |
isRegionCloseMarker()
Public so can be accessed from regionserver.wal package.
|
boolean |
isReplay() |
int |
readFromCells(Codec.Decoder cellDecoder,
int expectedCount)
Reads WALEdit from cells.
|
void |
setCells(ArrayList<Cell> cells)
This is not thread safe.
|
int |
size() |
String |
toString() |
public static final byte[] METAFAMILY
@Deprecated public static final byte[] METAROW
@Deprecated @InterfaceAudience.Private public static final byte[] COMPACTION
isCompactionMarker(Cell)
@Deprecated @InterfaceAudience.Private public static final byte[] FLUSH
private static final String REGION_EVENT_STR
REGION_EVENT_PREFIX
prefix ('HBASE::REGION_EVENT::'). After the prefix, we note the type of the event which we get
from the RegionEventDescriptor protobuf instance type (A RegionEventDescriptor protobuf
instance is written as the meta Marker Cell value). Adding a type suffix means we do not have
to deserialize the protobuf to figure out what type of event this is.. .just read the qualifier
suffix. For example, a close region event descriptor will have a qualifier of
HBASE::REGION_EVENT::REGION_CLOSE. See WAL.proto and the EventType in RegionEventDescriptor
protos for all possible event types.private static final String REGION_EVENT_PREFIX_STR
private static final byte[] REGION_EVENT_PREFIX
@Deprecated public static final byte[] REGION_EVENT
private static final byte[] REGION_EVENT_CLOSE
@InterfaceAudience.Private public static final byte[] BULK_LOAD
private final transient boolean replay
public WALEdit()
@Deprecated public WALEdit(boolean replay)
WALEdit(int, boolean)
instead.WALEdit(int, boolean)
,
HBASE-20781@Deprecated public WALEdit(int cellCount)
WALEdit(int, boolean)
instead.WALEdit(int, boolean)
,
HBASE-20781public WALEdit(int cellCount, boolean isReplay)
cellCount
- Pass so can pre-size the WALEdit. Optimization.private Set<byte[]> getOrCreateFamilies()
public Set<byte[]> getFamilies()
getCells()
; may be null.@Deprecated public static boolean isMetaEditFamily(byte[] f)
f
is METAFAMILY
public static boolean isMetaEditFamily(Cell cell)
public boolean isMetaEdit()
METAFAMILY
.public boolean isReplay()
public boolean isEmpty()
public int size()
@InterfaceAudience.Private public void setCells(ArrayList<Cell> cells)
cells
- the list of cells that this WALEdit now contains.public int readFromCells(Codec.Decoder cellDecoder, int expectedCount) throws IOException
cellDecoder
- Cell decoder.expectedCount
- Expected cell count.IOException
public long heapSize()
HeapSize
public long estimatedSerializedSizeOf()
public static WALEdit createFlushWALEdit(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor f)
public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.FlushDescriptor getFlushDescriptor(Cell cell) throws IOException
IOException
public static WALEdit createRegionEventWALEdit(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)
regionEventDesc
serialized and whose row is this region, columnfamily is
METAFAMILY
and qualifier is REGION_EVENT_PREFIX
+
WALProtos.RegionEventDescriptor.getEventType()
; for example
HBASE::REGION_EVENT::REGION_CLOSE.@InterfaceAudience.Private public static WALEdit createRegionEventWALEdit(byte[] rowForRegion, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)
@InterfaceAudience.Private public static byte[] createRegionEventDescriptorQualifier(org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor.EventType t)
public boolean isRegionCloseMarker()
public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.RegionEventDescriptor getRegionEventDescriptor(Cell cell) throws IOException
cell
, IFF the cell
is a RegionEventDescriptor type WALEdit.IOException
public static WALEdit createCompaction(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor c)
c
serialized as its valuepublic static byte[] getRowForRegion(RegionInfo hri)
public static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.CompactionDescriptor getCompaction(Cell kv) throws IOException
kv
- the key valueIOException
public static boolean isCompactionMarker(Cell cell)
WALProtos.CompactionDescriptor
getCompaction(Cell)
public static WALEdit createBulkLoadEvent(RegionInfo hri, org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor)
hri
- The RegionInfo for the region in which we are bulk loadingbulkLoadDescriptor
- The descriptor for the Bulk Loaderpublic static org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos.BulkLoadDescriptor getBulkLoadDescriptor(Cell cell) throws IOException
cell
- the key valueIOException
public void add(Map<byte[],List<Cell>> familyMap)
familyMap
- map of family->editsprivate void addFamily(byte[] family)
Copyright © 2007–2020 The Apache Software Foundation. All rights reserved.