org.apache.hadoop.hbase.regionserver

Class StoreFileInfo

java.lang.Object

org.apache.hadoop.hbase.regionserver.StoreFileInfo

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@InterfaceAudience.Private
public class StoreFileInfo
extends Object
implements org.apache.hadoop.conf.Configurable

Describe a StoreFile (hfile, reference, link)

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
private RegionCoprocessorHost	coprocessorHost
private long	createdTimestamp
static boolean	DEFAULT_STORE_FILE_READER_NO_READAHEAD
private org.apache.hadoop.fs.FileSystem	fs
private HDFSBlocksDistribution	hdfsBlocksDistribution
private static Pattern	HFILE_NAME_PATTERN Regex that will work for hfiles
static String	HFILE_NAME_REGEX A non-capture group, for hfiles, so that this can be embedded.
private HFileInfo	hfileInfo
private org.apache.hadoop.fs.Path	initialPath
private HFileLink	link
private static org.slf4j.Logger	LOG
private boolean	noReadahead
private boolean	primaryReplica
private static Pattern	REF_NAME_PATTERN Regex that will work for straight reference names (<hfile>.<parentEncRegion>) and hfilelink reference names (<table> =<region>-<hfile>.<parentEncRegion>) If reference, then the regex has more than just one group.
(package private) AtomicInteger	refCount
private Reference	reference
private long	size
static String	STORE_FILE_READER_NO_READAHEAD

Constructor Summary

Constructors

Modifier	Constructor and Description
	StoreFileInfo(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus fileStatus) Create a Store File Info
	StoreFileInfo(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus fileStatus, HFileLink link) Create a Store File Info from an HFileLink
private	StoreFileInfo(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus fileStatus, org.apache.hadoop.fs.Path initialPath, boolean primaryReplica)
	StoreFileInfo(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus fileStatus, Reference reference) Create a Store File Info from an HFileLink
	StoreFileInfo(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus fileStatus, Reference reference, HFileLink link) Create a Store File Info from an HFileLink and a Reference
	StoreFileInfo(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path initialPath, boolean primaryReplica) Create a Store File Info

Method Summary

Modifier and Type	Method and Description
HDFSBlocksDistribution	computeHDFSBlocksDistribution(org.apache.hadoop.fs.FileSystem fs) Compute the HDFS Block Distribution for this StoreFile
private HDFSBlocksDistribution	computeHDFSBlocksDistributionInternal(org.apache.hadoop.fs.FileSystem fs)
private static HDFSBlocksDistribution	computeRefFileHDFSBlockDistribution(org.apache.hadoop.fs.FileSystem fs, Reference reference, org.apache.hadoop.fs.FileStatus status) helper function to compute HDFS blocks distribution of a given reference file.For reference file, we don't compute the exact value.
(package private) StoreFileReader	createReader(ReaderContext context, CacheConfig cacheConf)
(package private) ReaderContext	createReaderContext(boolean doDropBehind, long readahead, ReaderContext.ReaderType type)
boolean	equals(Object that)
String	getActiveFileName() Return the active file name that contains the real data.
org.apache.hadoop.conf.Configuration	getConf()
long	getCreatedTimestamp() Returns timestamp when this file was created (as returned by filesystem)
org.apache.hadoop.fs.FileStatus	getFileStatus() Returns The FileStatus of the file
(package private) org.apache.hadoop.fs.FileSystem	getFileSystem()
HDFSBlocksDistribution	getHDFSBlockDistribution() Returns the HDFS block distribution
(package private) HFileInfo	getHFileInfo()
long	getModificationTime() Returns Get the modification time of the file.
org.apache.hadoop.fs.Path	getPath() Returns The Path of the file
Reference	getReference()
org.apache.hadoop.fs.FileStatus	getReferencedFileStatus(org.apache.hadoop.fs.FileSystem fs) Get the FileStatus of the file referenced by this StoreFileInfo
static org.apache.hadoop.fs.Path	getReferredToFile(org.apache.hadoop.fs.Path p)
long	getSize() Size of the Hfile n
int	hashCode()
(package private) void	initHDFSBlocksDistribution()
void	initHFileInfo(ReaderContext context)
static boolean	isHFile(org.apache.hadoop.fs.Path path)
static boolean	isHFile(String fileName)
boolean	isLink() Returns True if the store file is a link
static boolean	isMobFile(org.apache.hadoop.fs.Path path) Checks if the file is a MOB file
static boolean	isMobRefFile(org.apache.hadoop.fs.Path path) Checks if the file is a MOB reference file, created by snapshot
(package private) boolean	isNoReadahead()
boolean	isReference() Returns True if the store file is a Reference
static boolean	isReference(org.apache.hadoop.fs.Path path)
static boolean	isReference(String name)
boolean	isTopReference() Returns True if the store file is a top Reference
static boolean	isValid(org.apache.hadoop.fs.FileStatus fileStatus) Return if the specified file is a valid store file or not.
(package private) StoreFileReader	postStoreFileReaderOpen(ReaderContext context, CacheConfig cacheConf, StoreFileReader reader)
(package private) StoreFileReader	preStoreFileReaderOpen(ReaderContext context, CacheConfig cacheConf)
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setRegionCoprocessorHost(RegionCoprocessorHost coprocessorHost) Sets the region coprocessor env.
String	toString()
static boolean	validateStoreFileName(String fileName) Validate the store file name.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

HFILE_NAME_REGEX

public static final String HFILE_NAME_REGEX

A non-capture group, for hfiles, so that this can be embedded. HFiles are uuid ([0-9a-z]+). Bulk loaded hfiles has (_SeqId_[0-9]+_) has suffix. The mob del file has (_del) as suffix.

See Also:

Constant Field Values

HFILE_NAME_PATTERN

private static final Pattern HFILE_NAME_PATTERN

Regex that will work for hfiles

REF_NAME_PATTERN

private static final Pattern REF_NAME_PATTERN

Regex that will work for straight reference names (<hfile>.<parentEncRegion>) and hfilelink reference names (<table> =<region>-<hfile>.<parentEncRegion>) If reference, then the regex has more than just one group. Group 1, hfile/hfilelink pattern, is this file's id. Group 2 '(.+)' is the reference's parent region name.

STORE_FILE_READER_NO_READAHEAD

public static final String STORE_FILE_READER_NO_READAHEAD

See Also:

Constant Field Values

DEFAULT_STORE_FILE_READER_NO_READAHEAD

public static final boolean DEFAULT_STORE_FILE_READER_NO_READAHEAD

See Also:

Constant Field Values

conf

private org.apache.hadoop.conf.Configuration conf

fs

private final org.apache.hadoop.fs.FileSystem fs

hdfsBlocksDistribution

private HDFSBlocksDistribution hdfsBlocksDistribution

hfileInfo

private HFileInfo hfileInfo

reference

private final Reference reference

link

private final HFileLink link

initialPath

private final org.apache.hadoop.fs.Path initialPath

coprocessorHost

private RegionCoprocessorHost coprocessorHost

createdTimestamp

private long createdTimestamp

size

private long size

primaryReplica

private final boolean primaryReplica

noReadahead

private final boolean noReadahead

refCount

final AtomicInteger refCount

Constructor Detail

StoreFileInfo

public StoreFileInfo(org.apache.hadoop.conf.Configuration conf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.Path initialPath,
                     boolean primaryReplica)
              throws IOException

Create a Store File Info

Parameters:

conf - the Configuration to use

fs - The current file system to use.

initialPath - The Path of the file

primaryReplica - true if this is a store file for primary replica, otherwise false.

Throws:

IOException

StoreFileInfo

private StoreFileInfo(org.apache.hadoop.conf.Configuration conf,
                      org.apache.hadoop.fs.FileSystem fs,
                      org.apache.hadoop.fs.FileStatus fileStatus,
                      org.apache.hadoop.fs.Path initialPath,
                      boolean primaryReplica)
               throws IOException

Throws:

IOException

StoreFileInfo

public StoreFileInfo(org.apache.hadoop.conf.Configuration conf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.FileStatus fileStatus)
              throws IOException

Create a Store File Info

Parameters:

conf - the Configuration to use

fs - The current file system to use.

fileStatus - The FileStatus of the file

Throws:

IOException

StoreFileInfo

public StoreFileInfo(org.apache.hadoop.conf.Configuration conf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.FileStatus fileStatus,
                     HFileLink link)

Create a Store File Info from an HFileLink

Parameters:

conf - The Configuration to use

fs - The current file system to use

fileStatus - The FileStatus of the file

StoreFileInfo

public StoreFileInfo(org.apache.hadoop.conf.Configuration conf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.FileStatus fileStatus,
                     Reference reference)

Create a Store File Info from an HFileLink

Parameters:

conf - The Configuration to use

fs - The current file system to use

fileStatus - The FileStatus of the file

reference - The reference instance

StoreFileInfo

public StoreFileInfo(org.apache.hadoop.conf.Configuration conf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.FileStatus fileStatus,
                     Reference reference,
                     HFileLink link)

Create a Store File Info from an HFileLink and a Reference

Parameters:

conf - The Configuration to use

fs - The current file system to use

fileStatus - The FileStatus of the file

reference - The reference instance

link - The link instance

Method Detail

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getSize

public long getSize()

Size of the Hfile n

setRegionCoprocessorHost

public void setRegionCoprocessorHost(RegionCoprocessorHost coprocessorHost)

Sets the region coprocessor env. n

getReference

public Reference getReference()

isReference

public boolean isReference()

Returns True if the store file is a Reference

isTopReference

public boolean isTopReference()

Returns True if the store file is a top Reference

isLink

public boolean isLink()

Returns True if the store file is a link

getHDFSBlockDistribution

public HDFSBlocksDistribution getHDFSBlockDistribution()

Returns the HDFS block distribution

createReader

StoreFileReader createReader(ReaderContext context,
                             CacheConfig cacheConf)
                      throws IOException

Throws:

IOException

createReaderContext

ReaderContext createReaderContext(boolean doDropBehind,
                                  long readahead,
                                  ReaderContext.ReaderType type)
                           throws IOException

Throws:

IOException

computeHDFSBlocksDistribution

public HDFSBlocksDistribution computeHDFSBlocksDistribution(org.apache.hadoop.fs.FileSystem fs)
                                                     throws IOException

Compute the HDFS Block Distribution for this StoreFile

Throws:

IOException

computeHDFSBlocksDistributionInternal

private HDFSBlocksDistribution computeHDFSBlocksDistributionInternal(org.apache.hadoop.fs.FileSystem fs)
                                                              throws IOException

Throws:

IOException

getReferencedFileStatus

public org.apache.hadoop.fs.FileStatus getReferencedFileStatus(org.apache.hadoop.fs.FileSystem fs)
                                                        throws IOException

Get the FileStatus of the file referenced by this StoreFileInfo

Parameters:

fs - The current file system to use.

Returns:

The FileStatus of the file referenced by this StoreFileInfo

Throws:

IOException

getPath

public org.apache.hadoop.fs.Path getPath()

Returns The Path of the file

getFileStatus

public org.apache.hadoop.fs.FileStatus getFileStatus()
                                              throws IOException

Returns The FileStatus of the file

Throws:

IOException

getModificationTime

public long getModificationTime()
                         throws IOException

Returns Get the modification time of the file.

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

isHFile

public static boolean isHFile(org.apache.hadoop.fs.Path path)

Parameters:

path - Path to check.

Returns:

True if the path has format of a HFile.

isHFile

public static boolean isHFile(String fileName)

isMobFile

public static boolean isMobFile(org.apache.hadoop.fs.Path path)

Checks if the file is a MOB file

Parameters:

path - path to a file

Returns:

true, if - yes, false otherwise

isMobRefFile

public static boolean isMobRefFile(org.apache.hadoop.fs.Path path)

Checks if the file is a MOB reference file, created by snapshot

Parameters:

path - path to a file

Returns:

true, if - yes, false otherwise

isReference

public static boolean isReference(org.apache.hadoop.fs.Path path)

Parameters:

path - Path to check.

Returns:

True if the path has format of a HStoreFile reference.

isReference

public static boolean isReference(String name)

Parameters:

name - file name to check.

Returns:

True if the path has format of a HStoreFile reference.

getCreatedTimestamp

public long getCreatedTimestamp()

Returns timestamp when this file was created (as returned by filesystem)

getReferredToFile

public static org.apache.hadoop.fs.Path getReferredToFile(org.apache.hadoop.fs.Path p)

validateStoreFileName

public static boolean validateStoreFileName(String fileName)

Validate the store file name.

Parameters:

fileName - name of the file to validate

Returns:

true if the file could be a valid store file, false otherwise

isValid

public static boolean isValid(org.apache.hadoop.fs.FileStatus fileStatus)
                       throws IOException

Return if the specified file is a valid store file or not.

Parameters:

fileStatus - The FileStatus of the file

Returns:

true if the file is valid

Throws:

IOException

computeRefFileHDFSBlockDistribution

private static HDFSBlocksDistribution computeRefFileHDFSBlockDistribution(org.apache.hadoop.fs.FileSystem fs,
                                                                          Reference reference,
                                                                          org.apache.hadoop.fs.FileStatus status)
                                                                   throws IOException

helper function to compute HDFS blocks distribution of a given reference file.For reference file, we don't compute the exact value. We use some estimate instead given it might be good enough. we assume bottom part takes the first half of reference file, top part takes the second half of the reference file. This is just estimate, given midkey ofregion != midkey of HFile, also the number and size of keys vary. If this estimate isn't good enough, we can improve it later.

Parameters:

fs - The FileSystem

reference - The reference

status - The reference FileStatus

Returns:

HDFS blocks distribution

Throws:

IOException

equals

public boolean equals(Object that)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

getActiveFileName

public String getActiveFileName()

Return the active file name that contains the real data.

For referenced hfile, we will return the name of the reference file as it will be used to construct the StoreFileReader. And for linked hfile, we will return the name of the file being linked.

getFileSystem

org.apache.hadoop.fs.FileSystem getFileSystem()

isNoReadahead

boolean isNoReadahead()

getHFileInfo

HFileInfo getHFileInfo()

initHDFSBlocksDistribution

void initHDFSBlocksDistribution()
                         throws IOException

Throws:

IOException

preStoreFileReaderOpen

StoreFileReader preStoreFileReaderOpen(ReaderContext context,
                                       CacheConfig cacheConf)
                                throws IOException

Throws:

IOException

postStoreFileReaderOpen

StoreFileReader postStoreFileReaderOpen(ReaderContext context,
                                        CacheConfig cacheConf,
                                        StoreFileReader reader)
                                 throws IOException

Throws:

IOException

initHFileInfo

public void initHFileInfo(ReaderContext context)
                   throws IOException

Throws:

IOException