org.apache.hadoop.hbase.io

Class HFileLink

java.lang.Object

org.apache.hadoop.hbase.io.FileLink

org.apache.hadoop.hbase.io.HFileLink

@InterfaceAudience.Private
public class HFileLink
extends FileLink

HFileLink describes a link to an hfile. An hfile can be served from a region or from the hfile archive directory (/hbase/.archive) HFileLink allows to access the referenced hfile regardless of the location where it is.

Searches for hfiles in the following order and locations:

• /hbase/table/region/cf/hfile
• /hbase/.archive/table/region/cf/hfile

The link checks first in the original path if it is not present it fallbacks to the archived path.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.fs.Path	archivePath
static Pattern	LINK_NAME_PATTERN Define the HFile Link name parser in the form of: table=region-hfile
static String	LINK_NAME_REGEX A non-capture group, for HFileLink, so that this can be embedded.
private static org.slf4j.Logger	LOG
private org.apache.hadoop.fs.Path	mobPath
private org.apache.hadoop.fs.Path	originPath
private static Pattern	REF_OR_HFILE_LINK_PATTERN The pattern should be used for hfile and reference links that can be found in /hbase/table/region/family/
private org.apache.hadoop.fs.Path	tempPath

Fields inherited from class org.apache.hadoop.hbase.io.FileLink

BACK_REFERENCES_DIRECTORY_PREFIX

Constructor Summary

Constructors

Constructor and Description
HFileLink(org.apache.hadoop.fs.Path originPath, org.apache.hadoop.fs.Path tempPath, org.apache.hadoop.fs.Path mobPath, org.apache.hadoop.fs.Path archivePath) Dead simple hfile link constructor

Method Summary

Modifier and Type	Method and Description
static HFileLink	build(org.apache.hadoop.conf.Configuration conf, TableName table, String region, String family, String hfile) Create an HFileLink instance from table/region/family/hfile location
static HFileLink	buildFromHFileLinkPattern(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path hFileLinkPattern)
static HFileLink	buildFromHFileLinkPattern(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path archiveDir, org.apache.hadoop.fs.Path hFileLinkPattern)
static String	create(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dstFamilyPath, RegionInfo hfileRegionInfo, String hfileName) Create a new HFileLink
static String	create(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dstFamilyPath, RegionInfo hfileRegionInfo, String hfileName, boolean createBackRef) Create a new HFileLink
static String	create(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dstFamilyPath, String familyName, String dstTableName, String dstRegionName, TableName linkedTable, String linkedRegion, String hfileName, boolean createBackRef) Create a new HFileLink
static String	create(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dstFamilyPath, TableName linkedTable, String linkedRegion, String hfileName) Create a new HFileLink
static String	create(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dstFamilyPath, TableName linkedTable, String linkedRegion, String hfileName, boolean createBackRef) Create a new HFileLink.
(package private) static String	createBackReferenceName(String tableNameStr, String regionName) Create the back reference name
static String	createFromHFileLink(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dstFamilyPath, String hfileLinkName, boolean createBackRef) Create a new HFileLink starting from a hfileLink name
static String	createHFileLinkName(RegionInfo hfileRegionInfo, String hfileName) Create a new HFileLink name
static String	createHFileLinkName(TableName tableName, String regionName, String hfileName) Create a new HFileLink name
static org.apache.hadoop.fs.Path	createPath(TableName table, String region, String family, String hfile) Create an HFileLink relative path for the table/region/family/hfile location
org.apache.hadoop.fs.Path	getArchivePath() Returns the path of the archived hfile.
static org.apache.hadoop.fs.Path	getHFileFromBackReference(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path linkRefPath) Get the full path of the HFile referenced by the back reference
static org.apache.hadoop.fs.Path	getHFileFromBackReference(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path linkRefPath) Get the full path of the HFile referenced by the back reference
private static org.apache.hadoop.fs.Path	getHFileLinkPatternRelativePath(org.apache.hadoop.fs.Path path) Convert a HFileLink path to a table relative path.
org.apache.hadoop.fs.Path	getMobPath() Returns the path of the mob hfiles.
org.apache.hadoop.fs.Path	getOriginPath() Returns the origin path of the hfile.
static String	getReferencedHFileName(String fileName) Get the HFile name of the referenced link
static String	getReferencedRegionName(String fileName) Get the Region name of the referenced link
static TableName	getReferencedTableName(String fileName) Get the Table name of the referenced link
static boolean	isHFileLink(org.apache.hadoop.fs.Path path)
static boolean	isHFileLink(String fileName)
static Pair<TableName,String>	parseBackReferenceName(String name)

Methods inherited from class org.apache.hadoop.hbase.io.FileLink

equals, exists, getAvailablePath, getBackReferenceFileName, getBackReferencesDir, getFileStatus, getLocations, getUnderlyingFileLinkInputStream, hashCode, isBackReferencesDir, open, open, setLocations, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

LINK_NAME_REGEX

public static final String LINK_NAME_REGEX

A non-capture group, for HFileLink, so that this can be embedded. The HFileLink describe a link to an hfile in a different table/region and the name is in the form: table=region-hfile.

Table name is ([\p{IsAlphabetic}\p{Digit}][\p{IsAlphabetic}\p{Digit}.-]*), so '=' is an invalid character for the table name. Region name is ([a-f0-9]+), so '-' is an invalid character for the region name. HFile is ([0-9a-f]+(?:_SeqId_[0-9]+_)?) covering the plain hfiles (uuid) and the bulk loaded (_SeqId_[0-9]+_) hfiles.

Here is an example name: /hbase/test/0123/cf/testtb=4567-abcd where 'testtb' is table name and '4567' is region name and 'abcd' is filename.

LINK_NAME_PATTERN

public static final Pattern LINK_NAME_PATTERN

Define the HFile Link name parser in the form of: table=region-hfile

REF_OR_HFILE_LINK_PATTERN

private static final Pattern REF_OR_HFILE_LINK_PATTERN

The pattern should be used for hfile and reference links that can be found in /hbase/table/region/family/

archivePath

private final org.apache.hadoop.fs.Path archivePath

originPath

private final org.apache.hadoop.fs.Path originPath

mobPath

private final org.apache.hadoop.fs.Path mobPath

tempPath

private final org.apache.hadoop.fs.Path tempPath

Constructor Detail

HFileLink

public HFileLink(org.apache.hadoop.fs.Path originPath,
                 org.apache.hadoop.fs.Path tempPath,
                 org.apache.hadoop.fs.Path mobPath,
                 org.apache.hadoop.fs.Path archivePath)

Dead simple hfile link constructor

Method Detail

buildFromHFileLinkPattern

public static final HFileLink buildFromHFileLinkPattern(org.apache.hadoop.conf.Configuration conf,
                                                        org.apache.hadoop.fs.Path hFileLinkPattern)
                                                 throws IOException

Parameters:

conf - Configuration from which to extract specific archive locations

hFileLinkPattern - The path ending with a HFileLink pattern. (table=region-hfile)

Throws:

IOException - on unexpected error.

buildFromHFileLinkPattern

public static final HFileLink buildFromHFileLinkPattern(org.apache.hadoop.fs.Path rootDir,
                                                        org.apache.hadoop.fs.Path archiveDir,
                                                        org.apache.hadoop.fs.Path hFileLinkPattern)

Parameters:

rootDir - Path to the root directory where hbase files are stored

archiveDir - Path to the hbase archive directory

hFileLinkPattern - The path of the HFile Link.

createPath

public static org.apache.hadoop.fs.Path createPath(TableName table,
                                                   String region,
                                                   String family,
                                                   String hfile)

Create an HFileLink relative path for the table/region/family/hfile location

Parameters:

table - Table name

region - Region Name

family - Family Name

hfile - HFile Name

Returns:

the relative Path to open the specified table/region/family/hfile link

build

public static HFileLink build(org.apache.hadoop.conf.Configuration conf,
                              TableName table,
                              String region,
                              String family,
                              String hfile)
                       throws IOException

Create an HFileLink instance from table/region/family/hfile location

Parameters:

conf - Configuration from which to extract specific archive locations

table - Table name

region - Region Name

family - Family Name

hfile - HFile Name

Returns:

Link to the file with the specified table/region/family/hfile location

Throws:

IOException - on unexpected error.

getOriginPath

public org.apache.hadoop.fs.Path getOriginPath()

Returns the origin path of the hfile.

getArchivePath

public org.apache.hadoop.fs.Path getArchivePath()

Returns the path of the archived hfile.

getMobPath

public org.apache.hadoop.fs.Path getMobPath()

Returns the path of the mob hfiles.

isHFileLink

public static boolean isHFileLink(org.apache.hadoop.fs.Path path)

Parameters:

path - Path to check.

Returns:

True if the path is a HFileLink.

isHFileLink

public static boolean isHFileLink(String fileName)

Parameters:

fileName - File name to check.

Returns:

True if the path is a HFileLink.

getHFileLinkPatternRelativePath

private static org.apache.hadoop.fs.Path getHFileLinkPatternRelativePath(org.apache.hadoop.fs.Path path)

Convert a HFileLink path to a table relative path. e.g. the link: /hbase/test/0123/cf/testtb=4567-abcd becomes: /hbase/testtb/4567/cf/abcd

Parameters:

path - HFileLink path

Returns:

Relative table path

Throws:

IOException - on unexpected error.

getReferencedHFileName

public static String getReferencedHFileName(String fileName)

Get the HFile name of the referenced link

Parameters:

fileName - HFileLink file name

Returns:

the name of the referenced HFile

getReferencedRegionName

public static String getReferencedRegionName(String fileName)

Get the Region name of the referenced link

Parameters:

fileName - HFileLink file name

Returns:

the name of the referenced Region

getReferencedTableName

public static TableName getReferencedTableName(String fileName)

Get the Table name of the referenced link

Parameters:

fileName - HFileLink file name

Returns:

the name of the referenced Table

createHFileLinkName

public static String createHFileLinkName(RegionInfo hfileRegionInfo,
                                         String hfileName)

Create a new HFileLink name

Parameters:

hfileRegionInfo - - Linked HFile Region Info

hfileName - - Linked HFile name

Returns:

file name of the HFile Link

createHFileLinkName

public static String createHFileLinkName(TableName tableName,
                                         String regionName,
                                         String hfileName)

Create a new HFileLink name

Parameters:

tableName - - Linked HFile table name

regionName - - Linked HFile region name

hfileName - - Linked HFile name

Returns:

file name of the HFile Link

create

public static String create(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.Path dstFamilyPath,
                            RegionInfo hfileRegionInfo,
                            String hfileName)
                     throws IOException

Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:

conf - Configuration to read for the archive directory name

fs - FileSystem on which to write the HFileLink

dstFamilyPath - - Destination path (table/region/cf/)

hfileRegionInfo - - Linked HFile Region Info

hfileName - - Linked HFile name

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

create

public static String create(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.Path dstFamilyPath,
                            RegionInfo hfileRegionInfo,
                            String hfileName,
                            boolean createBackRef)
                     throws IOException

Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:

conf - Configuration to read for the archive directory name

fs - FileSystem on which to write the HFileLink

dstFamilyPath - - Destination path (table/region/cf/)

hfileRegionInfo - - Linked HFile Region Info

hfileName - - Linked HFile name

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

create

public static String create(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.Path dstFamilyPath,
                            TableName linkedTable,
                            String linkedRegion,
                            String hfileName)
                     throws IOException

Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:

conf - Configuration to read for the archive directory name

fs - FileSystem on which to write the HFileLink

dstFamilyPath - - Destination path (table/region/cf/)

linkedTable - - Linked Table Name

linkedRegion - - Linked Region Name

hfileName - - Linked HFile name

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

create

public static String create(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.Path dstFamilyPath,
                            TableName linkedTable,
                            String linkedRegion,
                            String hfileName,
                            boolean createBackRef)
                     throws IOException

Create a new HFileLink. In the event of link creation failure, this method throws an IOException, so that the calling upper laying can decide on how to proceed with this.

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:

conf - Configuration to read for the archive directory name

fs - FileSystem on which to write the HFileLink

dstFamilyPath - - Destination path (table/region/cf/)

linkedTable - - Linked Table Name

linkedRegion - - Linked Region Name

hfileName - - Linked HFile name

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

create

public static String create(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.Path dstFamilyPath,
                            String familyName,
                            String dstTableName,
                            String dstRegionName,
                            TableName linkedTable,
                            String linkedRegion,
                            String hfileName,
                            boolean createBackRef)
                     throws IOException

Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:

conf - Configuration to read for the archive directory name

fs - FileSystem on which to write the HFileLink

dstFamilyPath - - Destination path (table/region/cf/)

dstTableName - - Destination table name

dstRegionName - - Destination region name

linkedTable - - Linked Table Name

linkedRegion - - Linked Region Name

hfileName - - Linked HFile name

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure

createFromHFileLink

public static String createFromHFileLink(org.apache.hadoop.conf.Configuration conf,
                                         org.apache.hadoop.fs.FileSystem fs,
                                         org.apache.hadoop.fs.Path dstFamilyPath,
                                         String hfileLinkName,
                                         boolean createBackRef)
                                  throws IOException

Create a new HFileLink starting from a hfileLink name

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:

conf - Configuration to read for the archive directory name

fs - FileSystem on which to write the HFileLink

dstFamilyPath - - Destination path (table/region/cf/)

hfileLinkName - - HFileLink name (it contains hfile-region-table)

createBackRef - - Whether back reference should be created. Defaults to true.

Returns:

the file link name.

Throws:

IOException - on file or parent directory creation failure.

createBackReferenceName

static String createBackReferenceName(String tableNameStr,
                                      String regionName)

Create the back reference name

getHFileFromBackReference

public static org.apache.hadoop.fs.Path getHFileFromBackReference(org.apache.hadoop.fs.Path rootDir,
                                                                  org.apache.hadoop.fs.Path linkRefPath)

Get the full path of the HFile referenced by the back reference

Parameters:

rootDir - root hbase directory

linkRefPath - Link Back Reference path

Returns:

full path of the referenced hfile

parseBackReferenceName

public static Pair<TableName,String> parseBackReferenceName(String name)

getHFileFromBackReference

public static org.apache.hadoop.fs.Path getHFileFromBackReference(org.apache.hadoop.conf.Configuration conf,
                                                                  org.apache.hadoop.fs.Path linkRefPath)
                                                           throws IOException

Get the full path of the HFile referenced by the back reference

Parameters:

conf - Configuration to read for the archive directory name

linkRefPath - Link Back Reference path

Returns:

full path of the referenced hfile

Throws:

IOException - on unexpected error.