@InterfaceAudience.Private public class FSTableDescriptors extends Object implements TableDescriptors
TableDescriptors
that reads descriptors from the
passed filesystem. It expects descriptors to be in a file in the
TABLEINFO_DIR
subdir of the table's directory in FS. Can be read-only
-- i.e. does not modify the filesystem or can be read and write.
Also has utility for keeping up the table descriptors tableinfo file.
The table schema file is kept in the TABLEINFO_DIR
subdir
of the table directory in the filesystem.
It has a TABLEINFO_FILE_PREFIX
and then a suffix that is the
edit sequenceid: e.g. .tableinfo.0000000003
. This sequenceid
is always increasing. It starts at zero. The table schema file with the
highest sequenceid has the most recent schema edit. Usually there is one file
only, the most recent but there may be short periods where there are more
than one file. Old files are eventually cleaned. Presumption is that there
will not be lots of concurrent clients making table schema edits. If so,
the below needs a bit of a reworking and perhaps some supporting api in hdfs.
Modifier and Type | Field and Description |
---|---|
private Map<TableName,TableDescriptor> |
cache |
(package private) long |
cachehits |
private org.apache.hadoop.fs.FileSystem |
fs |
private boolean |
fsreadonly |
private boolean |
fsvisited |
(package private) long |
invocations |
private static org.slf4j.Logger |
LOG |
private org.apache.hadoop.fs.Path |
rootdir |
(package private) static String |
TABLEINFO_DIR |
(package private) static String |
TABLEINFO_FILE_PREFIX
The file name prefix used to store HTD in HDFS
|
private static Pattern |
TABLEINFO_FILE_REGEX
Regex to eat up sequenceid suffix on a .tableinfo file.
|
(package private) static Comparator<org.apache.hadoop.fs.FileStatus> |
TABLEINFO_FILESTATUS_COMPARATOR
Compare
FileStatus instances by Path.getName() . |
private static org.apache.hadoop.fs.PathFilter |
TABLEINFO_PATHFILTER |
(package private) static String |
TMP_DIR |
private boolean |
usecache |
(package private) static int |
WIDTH_OF_SEQUENCE_ID
Width of the sequenceid that is a suffix on a tableinfo file.
|
Constructor and Description |
---|
FSTableDescriptors(org.apache.hadoop.conf.Configuration conf)
Construct a FSTableDescriptors instance using the hbase root dir of the given conf and the
filesystem where that root dir lives.
|
FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootdir) |
FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootdir,
boolean fsreadonly,
boolean usecache) |
Modifier and Type | Method and Description |
---|---|
static TableDescriptorBuilder |
createMetaTableDescriptorBuilder(org.apache.hadoop.conf.Configuration conf) |
boolean |
createTableDescriptor(TableDescriptor htd)
Create new TableDescriptor in HDFS.
|
boolean |
createTableDescriptor(TableDescriptor htd,
boolean forceCreation)
Create new TableDescriptor in HDFS.
|
static boolean |
createTableDescriptorForTableDirectory(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir,
TableDescriptor htd,
boolean forceCreation)
Create a new TableDescriptor in HDFS in the specified table directory.
|
boolean |
createTableDescriptorForTableDirectory(org.apache.hadoop.fs.Path tableDir,
TableDescriptor htd,
boolean forceCreation)
Create a new TableDescriptor in HDFS in the specified table directory.
|
private static void |
deleteTableDescriptorFiles(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path dir,
int maxSequenceId)
Deletes files matching the table info file pattern within the given directory
whose sequenceId is at most the given max sequenceId.
|
private static String |
formatTableInfoSequenceId(int number) |
TableDescriptor |
get(TableName tableName)
Get the current table descriptor for the given table, or null if none exists.
|
Map<String,TableDescriptor> |
getAll()
Returns a map from table name to table descriptor for all tables.
|
Map<String,TableDescriptor> |
getByNamespace(String name)
Find descriptors by namespace.
|
private static org.apache.hadoop.fs.FileStatus |
getCurrentTableInfoStatus(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path dir,
boolean removeOldFiles)
Find the most current table info file in the given directory
Looks within the given directory for any table info files and takes the 'current' one - meaning
the one with the highest sequence number if present or no sequence number at all if none exist
(for backward compatibility from before there were sequence numbers).
|
static TableDescriptor |
getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir)
Returns the latest table descriptor for the table located at the given directory
directly from the file system if it exists.
|
static TableDescriptor |
getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path hbaseRootDir,
TableName tableName)
Returns the latest table descriptor for the given table directly from the file system
if it exists, bypassing the local cache.
|
(package private) org.apache.hadoop.fs.Path |
getTableDir(TableName tableName)
Return the table directory in HDFS
|
(package private) static String |
getTableInfoFileName(int sequenceid) |
static org.apache.hadoop.fs.FileStatus |
getTableInfoPath(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir)
Find the most current table info file for the table located in the given table directory.
|
private static org.apache.hadoop.fs.FileStatus |
getTableInfoPath(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir,
boolean removeOldFiles)
Find the most current table info file for the table in the given table directory.
|
private org.apache.hadoop.fs.FileStatus |
getTableInfoPath(org.apache.hadoop.fs.Path tableDir) |
(package private) static int |
getTableInfoSequenceId(org.apache.hadoop.fs.Path p) |
protected boolean |
isUsecache() |
private static TableDescriptor |
readTableDescriptor(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.FileStatus status) |
TableDescriptor |
remove(TableName tablename)
Removes the table descriptor from the local cache and returns it.
|
static TableDescriptor |
tryUpdateAndGetMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootdir) |
static void |
tryUpdateMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf) |
void |
update(TableDescriptor td,
boolean cacheOnly)
Add or update descriptor
|
(package private) org.apache.hadoop.fs.Path |
updateTableDescriptor(TableDescriptor td) |
private static org.apache.hadoop.fs.Path |
writeTableDescriptor(org.apache.hadoop.fs.FileSystem fs,
TableDescriptor htd,
org.apache.hadoop.fs.Path tableDir,
org.apache.hadoop.fs.FileStatus currentDescriptorFile)
Attempts to write a new table descriptor to the given table's directory.
|
private static void |
writeTD(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path p,
TableDescriptor htd) |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
exists, update
private static final org.slf4j.Logger LOG
private final org.apache.hadoop.fs.FileSystem fs
private final org.apache.hadoop.fs.Path rootdir
private final boolean fsreadonly
private final boolean usecache
private volatile boolean fsvisited
long cachehits
long invocations
static final String TABLEINFO_FILE_PREFIX
static final String TABLEINFO_DIR
static final String TMP_DIR
private final Map<TableName,TableDescriptor> cache
static final Comparator<org.apache.hadoop.fs.FileStatus> TABLEINFO_FILESTATUS_COMPARATOR
FileStatus
instances by Path.getName()
. Returns in reverse order.private static final org.apache.hadoop.fs.PathFilter TABLEINFO_PATHFILTER
static final int WIDTH_OF_SEQUENCE_ID
private static final Pattern TABLEINFO_FILE_REGEX
public FSTableDescriptors(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir)
public FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir, boolean fsreadonly, boolean usecache)
public static void tryUpdateMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public static TableDescriptor tryUpdateAndGetMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir) throws IOException
IOException
public static TableDescriptorBuilder createMetaTableDescriptorBuilder(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
protected boolean isUsecache()
@Nullable public TableDescriptor get(TableName tableName)
fsvisited
is not true
, i.e, we haven't done a full scan yet, to see if a newer
file has been created since the cached one was read.get
in interface TableDescriptors
public Map<String,TableDescriptor> getAll() throws IOException
getAll
in interface TableDescriptors
IOException
public Map<String,TableDescriptor> getByNamespace(String name) throws IOException
getByNamespace
in interface TableDescriptors
IOException
get(org.apache.hadoop.hbase.TableName)
public void update(TableDescriptor td, boolean cacheOnly) throws IOException
TableDescriptors
update
in interface TableDescriptors
td
- Descriptor to set into TableDescriptorscacheOnly
- only add the given htd
to cache, without updating the storage. For
example, when creating table, we will write the descriptor to fs when creating the fs
layout, so we do not need to update the fs again.IOException
org.apache.hadoop.fs.Path updateTableDescriptor(TableDescriptor td) throws IOException
IOException
public TableDescriptor remove(TableName tablename) throws IOException
remove
in interface TableDescriptors
IOException
private org.apache.hadoop.fs.FileStatus getTableInfoPath(org.apache.hadoop.fs.Path tableDir) throws IOException
IOException
public static org.apache.hadoop.fs.FileStatus getTableInfoPath(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
TABLEINFO_DIR
subdirectory of the given directory for any table info
files and takes the 'current' one - meaning the one with the highest sequence number if present
or no sequence number at all if none exist (for backward compatibility from before there
were sequence numbers).IOException
private static org.apache.hadoop.fs.FileStatus getTableInfoPath(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, boolean removeOldFiles) throws IOException
TABLEINFO_DIR
subdirectory of the given directory for any table info
files and takes the 'current' one - meaning the one with the highest sequence number if
present or no sequence number at all if none exist (for backward compatibility from before
there were sequence numbers).
If there are multiple table info files found and removeOldFiles is true it also deletes the
older files.IOException
private static org.apache.hadoop.fs.FileStatus getCurrentTableInfoStatus(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, boolean removeOldFiles) throws IOException
IOException
org.apache.hadoop.fs.Path getTableDir(TableName tableName)
private static String formatTableInfoSequenceId(int number)
number
- Number to use as suffix.static int getTableInfoSequenceId(org.apache.hadoop.fs.Path p)
p
- Path to a .tableinfo
file.static String getTableInfoFileName(int sequenceid)
sequenceid
- public static TableDescriptor getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path hbaseRootDir, TableName tableName) throws IOException
IOException
public static TableDescriptor getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
TableInfoMissingException
- if there is no descriptorIOException
private static TableDescriptor readTableDescriptor(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus status) throws IOException
IOException
private static void deleteTableDescriptorFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, int maxSequenceId) throws IOException
IOException
private static org.apache.hadoop.fs.Path writeTableDescriptor(org.apache.hadoop.fs.FileSystem fs, TableDescriptor htd, org.apache.hadoop.fs.Path tableDir, org.apache.hadoop.fs.FileStatus currentDescriptorFile) throws IOException
IOException
private static void writeTD(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, TableDescriptor htd) throws IOException
IOException
public boolean createTableDescriptor(TableDescriptor htd) throws IOException
IOException
public boolean createTableDescriptor(TableDescriptor htd, boolean forceCreation) throws IOException
IOException
public boolean createTableDescriptorForTableDirectory(org.apache.hadoop.fs.Path tableDir, TableDescriptor htd, boolean forceCreation) throws IOException
tableDir
- table directory under which we should write the filehtd
- description of the table to writeforceCreation
- if true,then even if previous table descriptor is present it will
be overwrittenIOException
- if a filesystem error occurspublic static boolean createTableDescriptorForTableDirectory(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, TableDescriptor htd, boolean forceCreation) throws IOException
fs
- Filesystem to use.tableDir
- table directory under which we should write the filehtd
- description of the table to writeforceCreation
- if true,then even if previous table descriptor is present it will
be overwrittenIOException
- if a filesystem error occursCopyright © 2007–2021 The Apache Software Foundation. All rights reserved.