Field Detail

• LOG

  private static final org.apache.commons.logging.Log LOG

• tieLock

  private final Object tieLock

  This lock ties all operations on flushingSequenceIds and lowestUnflushedSequenceIds Maps. lowestUnflushedSequenceIds has the lowest outstanding sequence ids EXCEPT when flushing. When we flush, the current lowest set for the region/column family are moved (atomically because of this lock) to flushingSequenceIds.

  The two Maps are tied by this locking object EXCEPT when we go to update the lowest entry; see #lowest(byte[], Set, Long). In here is a putIfAbsent call on lowestUnflushedSequenceIds. In this latter case, we will add this lowest sequence id if we find that there is no entry for the current column family. There will be no entry only if we just came up OR we have moved aside current set of lowest sequence ids because the current set are being flushed (by putting them into flushingSequenceIds). This is how we pick up the next 'lowest' sequence id per region per column family to be used figuring what is in the next flush.

• lowestUnflushedSequenceIds

  private final ConcurrentMap<byte[],ConcurrentMap<byte[],Long>> lowestUnflushedSequenceIds

  Map of encoded region names and family names to their OLDEST -- i.e. their first, the longest-lived, their 'earliest', the 'lowest' -- sequence id.

  When we flush, the current lowest sequence ids get cleared and added to flushingSequenceIds. The next append that comes in, is then added here to lowestUnflushedSequenceIds as the next lowest sequenceid.

  If flush fails, currently server is aborted so no need to restore previous sequence ids.

  Needs to be concurrent Maps because we use putIfAbsent updating oldest.

• flushingSequenceIds

  private final Map<byte[],Map<byte[],Long>> flushingSequenceIds

  Map of encoded region names and family names to their lowest or OLDEST sequence/edit id currently being flushed out to hfiles. Entries are moved here from lowestUnflushedSequenceIds while the lock tieLock is held (so movement between the Maps is atomic).

• highestSequenceIds

  private Map<byte[],Long> highestSequenceIds

  Map of region encoded names to the latest/highest region sequence id. Updated on each call to append.

  This map uses byte[] as the key, and uses reference equality. It works in our use case as we use HRegionInfo#getEncodedNameAsBytes() as keys. For a given region, it always returns the same array.

Constructor Detail

• SequenceIdAccounting

  SequenceIdAccounting()

Method Detail

• getLowestSequenceId

  long getLowestSequenceId(byte[] encodedRegionName)

  Returns the lowest unflushed sequence id for the region.

  Parameters:

  encodedRegionName -

  Returns:

  Lowest outstanding unflushed sequenceid for encodedRegionName. Will return HConstants.NO_SEQNUM when none.

• getLowestSequenceId

  long getLowestSequenceId(byte[] encodedRegionName,
                         byte[] familyName)

  Parameters:

  encodedRegionName -

  familyName -

  Returns:

  Lowest outstanding unflushed sequenceid for encodedRegionname and familyName. Returned sequenceid may be for an edit currently being flushed.

• resetHighest

  Map<byte[],Long> resetHighest()

  Reset the accounting of highest sequenceid by regionname.

  Returns:

  Return the previous accounting Map of regions to the last sequence id written into each.

• update

  void update(byte[] encodedRegionName,
            Set<byte[]> families,
            long sequenceid,
            boolean lowest)

  We've been passed a new sequenceid for the region. Set it as highest seen for this region and if we are to record oldest, or lowest sequenceids, save it as oldest seen if nothing currently older.

  Parameters:

  encodedRegionName -

  families -

  sequenceid -

  lowest - Whether to keep running account of oldest sequence id.

• getOrCreateLowestSequenceIds

  ConcurrentMap<byte[],Long> getOrCreateLowestSequenceIds(byte[] encodedRegionName)

• getLowestSequenceId

  static long getLowestSequenceId(Map<byte[],Long> sequenceids)

  Parameters:

  sequenceids - Map to search for lowest value.

  Returns:

  Lowest value found in sequenceids.

• flattenToLowestSequenceId

  private <T extends Map<byte[],Long>> Map<byte[],Long> flattenToLowestSequenceId(Map<byte[],T> src)

  Parameters:

  src -

  Returns:

  New Map that has same keys as src but instead of a Map for a value, it instead has found the smallest sequence id and it returns that as the value instead.

• startCacheFlush

  Long startCacheFlush(byte[] encodedRegionName,
                     Set<byte[]> families)

  Parameters:

  encodedRegionName - Region to flush.

  families - Families to flush. May be a subset of all families in the region.

  Returns:

  Returns HConstants.NO_SEQNUM if we are flushing the whole region OR if we are flushing a subset of all families but there are no edits in those families not being flushed; in other words, this is effectively same as a flush of all of the region though we were passed a subset of regions. Otherwise, it returns the sequence id of the oldest/lowest outstanding edit.

• completeCacheFlush

  void completeCacheFlush(byte[] encodedRegionName)

• abortCacheFlush

  void abortCacheFlush(byte[] encodedRegionName)

• areAllLower

  boolean areAllLower(Map<byte[],Long> sequenceids)

  See if passed sequenceids are lower -- i.e. earlier -- than any outstanding sequenceids, sequenceids we are holding on to in this accounting instance.

  Parameters:

  sequenceids - Keyed by encoded region name. Cannot be null (doesn't make sense for it to be null).

  Returns:

  true if all sequenceids are lower, older than, the old sequenceids in this instance.

• findLower

  byte[][] findLower(Map<byte[],Long> sequenceids)

  Iterates over the given Map and compares sequence ids with corresponding entries in #oldestUnflushedRegionSequenceIds. If a region in #oldestUnflushedRegionSequenceIds has a sequence id less than that passed in sequenceids then return it.

  Parameters:

  sequenceids - Sequenceids keyed by encoded region name.

  Returns:

  regions found in this instance with sequence ids less than those passed in.