@InterfaceAudience.Private public class FSTableDescriptors extends Object implements TableDescriptors
TableDescriptors
that reads descriptors from the passed filesystem. It
expects descriptors to be in a file in the TABLEINFO_DIR
subdir of the table's directory
in FS. Can be read-only -- i.e. does not modify the filesystem or can be read and write.
Also has utility for keeping up the table descriptors tableinfo file. The table schema file is
kept in the TABLEINFO_DIR
subdir of the table directory in the filesystem. It has a
TABLEINFO_FILE_PREFIX
and then a suffix that is the edit sequenceid: e.g.
.tableinfo.0000000003
. This sequenceid is always increasing. It starts at zero. The
table schema file with the highest sequenceid has the most recent schema edit. Usually there is
one file only, the most recent but there may be short periods where there are more than one file.
Old files are eventually cleaned. Presumption is that there will not be lots of concurrent
clients making table schema edits. If so, the below needs a bit of a reworking and perhaps some
supporting api in hdfs.
Modifier and Type | Class and Description |
---|---|
(package private) static class |
FSTableDescriptors.SequenceIdAndFileLength |
Modifier and Type | Field and Description |
---|---|
private Map<TableName,TableDescriptor> |
cache |
(package private) long |
cachehits |
private org.apache.hadoop.fs.FileSystem |
fs |
private boolean |
fsreadonly |
private boolean |
fsvisited |
(package private) long |
invocations |
private static org.slf4j.Logger |
LOG |
private org.apache.hadoop.fs.Path |
rootdir |
static String |
TABLEINFO_DIR |
(package private) static String |
TABLEINFO_FILE_PREFIX
The file name prefix used to store HTD in HDFS
|
(package private) static Comparator<org.apache.hadoop.fs.FileStatus> |
TABLEINFO_FILESTATUS_COMPARATOR
Compare
FileStatus instances by Path.getName() . |
private static org.apache.hadoop.fs.PathFilter |
TABLEINFO_PATHFILTER |
private boolean |
usecache |
(package private) static int |
WIDTH_OF_SEQUENCE_ID
Width of the sequenceid that is a suffix on a tableinfo file.
|
Constructor and Description |
---|
FSTableDescriptors(org.apache.hadoop.conf.Configuration conf)
Construct a FSTableDescriptors instance using the hbase root dir of the given conf and the
filesystem where that root dir lives.
|
FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootdir) |
FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootdir,
boolean fsreadonly,
boolean usecache) |
Modifier and Type | Method and Description |
---|---|
static TableDescriptorBuilder |
createMetaTableDescriptorBuilder(org.apache.hadoop.conf.Configuration conf) |
boolean |
createTableDescriptor(TableDescriptor htd)
Create new TableDescriptor in HDFS.
|
boolean |
createTableDescriptor(TableDescriptor htd,
boolean forceCreation)
Create new TableDescriptor in HDFS.
|
static boolean |
createTableDescriptorForTableDirectory(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir,
TableDescriptor htd,
boolean forceCreation)
Create a new TableDescriptor in HDFS in the specified table directory.
|
boolean |
createTableDescriptorForTableDirectory(org.apache.hadoop.fs.Path tableDir,
TableDescriptor htd,
boolean forceCreation)
Create a new TableDescriptor in HDFS in the specified table directory.
|
private static void |
deleteMalformedFile(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path file) |
private static void |
deleteTableDescriptorFiles(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path dir,
int maxSequenceId)
Deletes files matching the table info file pattern within the given directory whose sequenceId
is at most the given max sequenceId.
|
static void |
deleteTableDescriptors(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir) |
private static String |
formatTableInfoSequenceId(int number) |
TableDescriptor |
get(TableName tableName)
Get the current table descriptor for the given table, or null if none exists.
|
Map<String,TableDescriptor> |
getAll()
Returns a map from table name to table descriptor for all tables.
|
Map<String,TableDescriptor> |
getByNamespace(String name)
Find descriptors by namespace.
|
static ColumnFamilyDescriptor |
getReplBarrierFamilyDescForMeta() |
static TableDescriptor |
getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir)
Returns the latest table descriptor for the table located at the given directory directly from
the file system if it exists.
|
private static Optional<Pair<org.apache.hadoop.fs.FileStatus,TableDescriptor>> |
getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir,
boolean readonly) |
static TableDescriptor |
getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path hbaseRootDir,
TableName tableName)
Returns the latest table descriptor for the given table directly from the file system if it
exists, bypassing the local cache.
|
private org.apache.hadoop.fs.Path |
getTableDir(TableName tableName)
Return the table directory in HDFS
|
static ColumnFamilyDescriptor |
getTableFamilyDescForMeta(org.apache.hadoop.conf.Configuration conf) |
(package private) static String |
getTableInfoFileName(int sequenceId,
byte[] content)
Returns Name of tableinfo file.
|
(package private) static FSTableDescriptors.SequenceIdAndFileLength |
getTableInfoSequenceIdAndFileLength(org.apache.hadoop.fs.Path p)
Returns the current sequence id and file length or 0 if none found.
|
static boolean |
isTableDir(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path tableDir)
Check whether we have a valid TableDescriptor.
|
protected boolean |
isUsecache() |
TableDescriptor |
remove(TableName tablename)
Removes the table descriptor from the local cache and returns it.
|
static TableDescriptor |
tryUpdateAndGetMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path rootdir) |
static void |
tryUpdateMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf) |
void |
update(TableDescriptor td,
boolean cacheOnly)
Add or update descriptor
|
(package private) org.apache.hadoop.fs.Path |
updateTableDescriptor(TableDescriptor td) |
private static org.apache.hadoop.fs.Path |
writeTableDescriptor(org.apache.hadoop.fs.FileSystem fs,
TableDescriptor td,
org.apache.hadoop.fs.Path tableDir,
org.apache.hadoop.fs.FileStatus currentDescriptorFile)
Attempts to write a new table descriptor to the given table's directory.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
exists, update
private static final org.slf4j.Logger LOG
private final org.apache.hadoop.fs.FileSystem fs
private final org.apache.hadoop.fs.Path rootdir
private final boolean fsreadonly
private final boolean usecache
private volatile boolean fsvisited
long cachehits
long invocations
static final String TABLEINFO_FILE_PREFIX
public static final String TABLEINFO_DIR
private final Map<TableName,TableDescriptor> cache
static final Comparator<org.apache.hadoop.fs.FileStatus> TABLEINFO_FILESTATUS_COMPARATOR
FileStatus
instances by Path.getName()
. Returns in reverse order.private static final org.apache.hadoop.fs.PathFilter TABLEINFO_PATHFILTER
static final int WIDTH_OF_SEQUENCE_ID
public FSTableDescriptors(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir)
public FSTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir, boolean fsreadonly, boolean usecache)
public static void tryUpdateMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public static TableDescriptor tryUpdateAndGetMetaTableDescriptor(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootdir) throws IOException
IOException
public static ColumnFamilyDescriptor getTableFamilyDescForMeta(org.apache.hadoop.conf.Configuration conf)
public static ColumnFamilyDescriptor getReplBarrierFamilyDescForMeta()
public static TableDescriptorBuilder createMetaTableDescriptorBuilder(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
protected boolean isUsecache()
@Nullable public TableDescriptor get(TableName tableName)
fsvisited
is not true
, i.e, we haven't done a full scan yet, to see if a newer
file has been created since the cached one was read.get
in interface TableDescriptors
public Map<String,TableDescriptor> getAll() throws IOException
getAll
in interface TableDescriptors
IOException
public Map<String,TableDescriptor> getByNamespace(String name) throws IOException
getByNamespace
in interface TableDescriptors
IOException
get(org.apache.hadoop.hbase.TableName)
public void update(TableDescriptor td, boolean cacheOnly) throws IOException
TableDescriptors
update
in interface TableDescriptors
td
- Descriptor to set into TableDescriptorscacheOnly
- only add the given htd
to cache, without updating the storage. For
example, when creating table, we will write the descriptor to fs when creating
the fs layout, so we do not need to update the fs again.IOException
org.apache.hadoop.fs.Path updateTableDescriptor(TableDescriptor td) throws IOException
IOException
public TableDescriptor remove(TableName tablename) throws IOException
remove
in interface TableDescriptors
IOException
public static boolean isTableDir(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
IOException
private org.apache.hadoop.fs.Path getTableDir(TableName tableName)
private static String formatTableInfoSequenceId(int number)
number
- Number to use as suffix.static FSTableDescriptors.SequenceIdAndFileLength getTableInfoSequenceIdAndFileLength(org.apache.hadoop.fs.Path p)
p
- Path to a .tableinfo
file.static String getTableInfoFileName(int sequenceId, byte[] content)
public static TableDescriptor getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path hbaseRootDir, TableName tableName) throws IOException
IOException
public static TableDescriptor getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
IOException
private static void deleteMalformedFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file) throws IOException
IOException
private static Optional<Pair<org.apache.hadoop.fs.FileStatus,TableDescriptor>> getTableDescriptorFromFs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, boolean readonly) throws IOException
IOException
public static void deleteTableDescriptors(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir) throws IOException
IOException
private static void deleteTableDescriptorFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir, int maxSequenceId) throws IOException
IOException
private static org.apache.hadoop.fs.Path writeTableDescriptor(org.apache.hadoop.fs.FileSystem fs, TableDescriptor td, org.apache.hadoop.fs.Path tableDir, org.apache.hadoop.fs.FileStatus currentDescriptorFile) throws IOException
IOException
public boolean createTableDescriptor(TableDescriptor htd) throws IOException
IOException
public boolean createTableDescriptor(TableDescriptor htd, boolean forceCreation) throws IOException
IOException
public boolean createTableDescriptorForTableDirectory(org.apache.hadoop.fs.Path tableDir, TableDescriptor htd, boolean forceCreation) throws IOException
tableDir
- table directory under which we should write the filehtd
- description of the table to writeforceCreation
- if true,then even if previous table descriptor is present it will
be overwrittenIOException
- if a filesystem error occurspublic static boolean createTableDescriptorForTableDirectory(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, TableDescriptor htd, boolean forceCreation) throws IOException
fs
- Filesystem to use.tableDir
- table directory under which we should write the filehtd
- description of the table to writeforceCreation
- if true,then even if previous table descriptor is present it will
be overwrittenIOException
- if a filesystem error occursCopyright © 2007–2020 The Apache Software Foundation. All rights reserved.