Package org.apache.hadoop.hbase
Class MetaTableAccessor
java.lang.Object
org.apache.hadoop.hbase.MetaTableAccessor
Read/write operations on
hbase:meta
region as well as assignment information stored
to hbase:meta
.
Some of the methods of this class take ZooKeeperWatcher as a param. The only reason for this is
when this class is used on client-side (e.g. HBaseAdmin), we want to use short-lived connection
(opened before each operation, closed right after), while when used on HM or HRS (like in
AssignmentManager) we want permanent connection.
HBASE-10070 adds a replicaId to HRI, meaning more than one HRI can be defined for the same table
range (table, startKey, endKey). For every range, there will be at least one HRI defined which is
called default replica.
Meta layout
For each table there is single row named for the table with a 'table' column family. The column family currently has one column in it, the 'state' column:table:state => contains table stateFor the catalog family, see the comments of
CatalogFamilyFormat
for more details.
TODO: Add rep_barrier for serial replication explanation. See SerialReplicationChecker.
The actual layout of meta should be encapsulated inside MetaTableAccessor methods, and should not
leak out of it (through Result objects, etc)- See Also:
-
Field Summary
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic Put
addDaughtersToPut
(Put put, RegionInfo splitA, RegionInfo splitB) Adds split daughters to the Putstatic Put
addEmptyLocation
(Put p, int replicaId) static Put
addLocation
(Put p, ServerName sn, long openSeqNum, int replicaId) static Put
addRegionInfo
(Put p, RegionInfo hri) static Put
addRegionStateToPut
(Put put, int replicaId, RegionState.State state) static void
addRegionsToMeta
(Connection connection, List<RegionInfo> regionInfos, int regionReplication) Adds a hbase:meta row for each of the specified new regions.static void
addRegionsToMeta
(Connection connection, List<RegionInfo> regionInfos, int regionReplication, long ts) Adds a hbase:meta row for each of the specified new regions.static void
addSplitsToParent
(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) Adds daughter region infos to hbase:meta row for the specified region.private static void
private static void
debugLogMutations
(List<? extends Mutation> mutations) private static void
deleteFromMetaTable
(Connection connection, List<Delete> deletes) Delete the passeddeletes
from thehbase:meta
table.private static void
deleteFromMetaTable
(Connection connection, Delete d) Delete the passedd
from thehbase:meta
table.static void
deleteTableState
(Connection connection, TableName table) Remove state for table from metafullScan
(Connection connection, ClientMetaTableAccessor.QueryType type) Performs a full scan ofhbase:meta
.static void
fullScanMetaAndPrint
(Connection connection) fullScanRegions
(Connection connection) Performs a full scan ofhbase:meta
for regions.static void
fullScanRegions
(Connection connection, ClientMetaTableAccessor.Visitor visitor) Performs a full scan ofhbase:meta
for regions.static void
fullScanTables
(Connection connection, ClientMetaTableAccessor.Visitor visitor) Performs a full scan ofhbase:meta
for tables.static List<RegionInfo>
getAllRegions
(Connection connection, boolean excludeOfflinedSplitParents) Lists all of the regions currently in META.static Result
getCatalogFamilyRow
(Connection connection, RegionInfo ri) Returns Return theHConstants.CATALOG_FAMILY
row from hbase:meta.private static RegionInfo
getClosestRegionInfo
(Connection connection, TableName tableName, byte[] row) Returns Get closest metatable region row to passedrow
static PairOfSameType<RegionInfo>
getDaughterRegions
(Result data) Returns the daughter regions by reading the corresponding columns of the catalog table Result.private static List<RegionInfo>
getListOfRegionInfos
(List<Pair<RegionInfo, ServerName>> pairs) static Table
getMetaHTable
(Connection connection) Callers should call close on the returnedTable
instance.private static Scan
getMetaScan
(org.apache.hadoop.conf.Configuration conf, int rowUpperLimit) static Pair<RegionInfo,
ServerName> getRegion
(Connection connection, byte[] regionName) Deprecated.static HRegionLocation
getRegionLocation
(Connection connection, byte[] regionName) Returns the HRegionLocation from meta for the given regionstatic HRegionLocation
getRegionLocation
(Connection connection, RegionInfo regionInfo) Returns the HRegionLocation from meta for the given regionstatic Result
getRegionResult
(Connection connection, RegionInfo regionInfo) Gets the result in hbase:meta for the specified region.static Scan
getScanForTableName
(org.apache.hadoop.conf.Configuration conf, TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table.static List<RegionInfo>
getTableRegions
(Connection connection, TableName tableName) Gets all of the regions of the specified table.static List<RegionInfo>
getTableRegions
(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) Gets all of the regions of the specified table.static List<Pair<RegionInfo,
ServerName>> getTableRegionsAndLocations
(Connection connection, TableName tableName) Do not use this method to get meta table regions, use methods in MetaTableLocator instead.static List<Pair<RegionInfo,
ServerName>> getTableRegionsAndLocations
(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) Do not use this method to get meta table regions, use methods in MetaTableLocator instead.static TableState
getTableState
(Connection conn, TableName tableName) Fetch table state for given table from META tablestatic Map<TableName,
TableState> getTableStates
(Connection conn) Fetch table states from META tablestatic ServerName
getTargetServerName
(Result r, int replicaId) Returns theServerName
from catalog tableResult
where the region is transitioning on.static Delete
makeDeleteFromRegionInfo
(RegionInfo regionInfo, long ts) Generates and returns a Delete containing the region info for the catalog tablestatic Put
makePutFromRegionInfo
(RegionInfo regionInfo) Generates and returns aPut
containing theRegionInfo
for the catalog table.static Put
makePutFromRegionInfo
(RegionInfo regionInfo, long ts) Generates and returns aPut
containing theRegionInfo
for the catalog table.static Put
makePutFromTableState
(TableState state, long ts) Construct PUT for given stateprivate static void
static void
putsToMetaTable
(Connection connection, List<Put> ps) Put the passedps
to thehbase:meta
table.private static void
putToMetaTable
(Connection connection, Put p) Put the passedp
to thehbase:meta
table.static Result
scanByRegionEncodedName
(Connection connection, String regionEncodedName) Scans META table for a row whose key contains the specified regionEncodedName, returning a single relatedResult
instance if any row is found, null otherwise.static void
scanMeta
(Connection connection, byte[] startRow, byte[] stopRow, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) Performs a scan of META table.static void
scanMeta
(Connection connection, byte[] startRow, byte[] stopRow, ClientMetaTableAccessor.QueryType type, ClientMetaTableAccessor.Visitor visitor) static void
scanMeta
(Connection connection, byte[] startRow, byte[] stopRow, ClientMetaTableAccessor.QueryType type, Filter filter, int maxRows, ClientMetaTableAccessor.Visitor visitor) static void
scanMeta
(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) Performs a scan of META table for given table starting from given row.private static void
scanMeta
(Connection connection, TableName table, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) static void
scanMetaForTableRegions
(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName) private static void
updateLocation
(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) Updates the location of the specified region to be the specified server.static void
updateRegionLocation
(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.static void
updateRegionState
(Connection connection, RegionInfo ri, RegionState.State state) Update state column in hbase:meta.private static void
updateTableState
(Connection connection, TableState state) Update state of the table in meta.static void
updateTableState
(Connection conn, TableName tableName, TableState.State actual) Updates state in META Do not use.
-
Field Details
-
LOG
-
METALOG
-
-
Constructor Details
-
MetaTableAccessor
private MetaTableAccessor()
-
-
Method Details
-
fullScanRegions
public static void fullScanRegions(Connection connection, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a full scan ofhbase:meta
for regions.- Parameters:
connection
- connection we're usingvisitor
- Visitor invoked against each row in regions family.- Throws:
IOException
-
fullScanRegions
Performs a full scan ofhbase:meta
for regions.- Parameters:
connection
- connection we're using- Throws:
IOException
-
fullScanTables
public static void fullScanTables(Connection connection, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a full scan ofhbase:meta
for tables.- Parameters:
connection
- connection we're usingvisitor
- Visitor invoked against each row in tables family.- Throws:
IOException
-
fullScan
private static List<Result> fullScan(Connection connection, ClientMetaTableAccessor.QueryType type) throws IOException Performs a full scan ofhbase:meta
.- Parameters:
connection
- connection we're usingtype
- scanned part of meta- Returns:
- List of
Result
- Throws:
IOException
-
getMetaHTable
Callers should call close on the returnedTable
instance.- Parameters:
connection
- connection we're using to access Meta- Returns:
- An
Table
forhbase:meta
- Throws:
NullPointerException
- ifconnection
isnull
IOException
-
getRegion
@Deprecated public static Pair<RegionInfo,ServerName> getRegion(Connection connection, byte[] regionName) throws IOException Deprecated.usegetRegionLocation(Connection, byte[])
insteadGets the region info and assignment for the specified region.- Parameters:
connection
- connection we're usingregionName
- Region to lookup.- Returns:
- Location and RegionInfo for
regionName
- Throws:
IOException
-
getRegionLocation
public static HRegionLocation getRegionLocation(Connection connection, byte[] regionName) throws IOException Returns the HRegionLocation from meta for the given region- Parameters:
connection
- connection we're usingregionName
- region we're looking for- Returns:
- HRegionLocation for the given region
- Throws:
IOException
-
getRegionLocation
public static HRegionLocation getRegionLocation(Connection connection, RegionInfo regionInfo) throws IOException Returns the HRegionLocation from meta for the given region- Parameters:
connection
- connection we're usingregionInfo
- region information- Returns:
- HRegionLocation for the given region
- Throws:
IOException
-
getCatalogFamilyRow
Returns Return theHConstants.CATALOG_FAMILY
row from hbase:meta.- Throws:
IOException
-
getRegionResult
public static Result getRegionResult(Connection connection, RegionInfo regionInfo) throws IOException Gets the result in hbase:meta for the specified region.- Parameters:
connection
- connection we're usingregionInfo
- region we're looking for- Returns:
- result of the specified region
- Throws:
IOException
-
scanByRegionEncodedName
public static Result scanByRegionEncodedName(Connection connection, String regionEncodedName) throws IOException Scans META table for a row whose key contains the specified regionEncodedName, returning a single relatedResult
instance if any row is found, null otherwise.- Parameters:
connection
- the connection to query META table.regionEncodedName
- the region encoded name to look for at META.- Returns:
Result
instance with the row related info in META, null otherwise.- Throws:
IOException
- if any errors occur while querying META.
-
getAllRegions
public static List<RegionInfo> getAllRegions(Connection connection, boolean excludeOfflinedSplitParents) throws IOException Lists all of the regions currently in META.- Parameters:
connection
- to connect withexcludeOfflinedSplitParents
- False if we are to include offlined/splitparents regions, true and we'll leave out offlined regions from returned list- Returns:
- List of all user-space regions.
- Throws:
IOException
-
getTableRegions
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName) throws IOException Gets all of the regions of the specified table. Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table we're looking for- Returns:
- Ordered list of
RegionInfo
. - Throws:
IOException
-
getTableRegions
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) throws IOException Gets all of the regions of the specified table. Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table we're looking forexcludeOfflinedSplitParents
- If true, do not include offlined split parents in the return.- Returns:
- Ordered list of
RegionInfo
. - Throws:
IOException
-
getListOfRegionInfos
-
getScanForTableName
public static Scan getScanForTableName(org.apache.hadoop.conf.Configuration conf, TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table. It doesn't specify any columns. This is a better alternative to just using a start row and scan until it hits a new table since that requires parsing the HRI to get the table name.- Parameters:
tableName
- bytes of table's name- Returns:
- configured Scan object
-
getMetaScan
-
getTableRegionsAndLocations
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, TableName tableName) throws IOException Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table we're looking for- Returns:
- Return list of regioninfos and server.
- Throws:
IOException
-
getTableRegionsAndLocations
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, @Nullable TableName tableName, boolean excludeOfflinedSplitParents) throws IOException Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection
- connection we're usingtableName
- table to work with, can be null for getting all regionsexcludeOfflinedSplitParents
- don't return split parents- Returns:
- Return list of regioninfos and server addresses.
- Throws:
IOException
-
fullScanMetaAndPrint
- Throws:
IOException
-
scanMetaForTableRegions
public static void scanMetaForTableRegions(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName) throws IOException - Throws:
IOException
-
scanMeta
private static void scanMeta(Connection connection, TableName table, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) throws IOException - Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, ClientMetaTableAccessor.QueryType type, ClientMetaTableAccessor.Visitor visitor) throws IOException - Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) throws IOException Performs a scan of META table for given table starting from given row.- Parameters:
connection
- connection we're usingvisitor
- visitor to calltableName
- table withing we scanrow
- start scan from this rowrowLimit
- max number of rows to return- Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a scan of META table.- Parameters:
connection
- connection we're usingstartRow
- Where to start the scan. Pass null if want to begin scan at first row.stopRow
- Where to stop the scan. Pass null if want to scan all rows from the start onetype
- scanned part of metamaxRows
- maximum rows to returnvisitor
- Visitor invoked against each row.- Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, ClientMetaTableAccessor.QueryType type, @Nullable Filter filter, int maxRows, ClientMetaTableAccessor.Visitor visitor) throws IOException - Throws:
IOException
-
getClosestRegionInfo
@NonNull private static RegionInfo getClosestRegionInfo(Connection connection, @NonNull TableName tableName, @NonNull byte[] row) throws IOException Returns Get closest metatable region row to passedrow
- Throws:
IOException
-
getTargetServerName
Returns theServerName
from catalog tableResult
where the region is transitioning on. It should be the same asCatalogFamilyFormat.getServerName(Result,int)
if the server is at OPEN state.- Parameters:
r
- Result to pull the transitioning server name from- Returns:
- A ServerName instance or
CatalogFamilyFormat.getServerName(Result,int)
if necessary fields not found or empty.
-
getDaughterRegions
Returns the daughter regions by reading the corresponding columns of the catalog table Result.- Parameters:
data
- a Result object from the catalog table scan- Returns:
- pair of RegionInfo or PairOfSameType(null, null) if region is not a split parent
-
getTableState
@Nullable public static TableState getTableState(Connection conn, TableName tableName) throws IOException Fetch table state for given table from META table- Parameters:
conn
- connection to usetableName
- table to fetch state for- Throws:
IOException
-
getTableStates
Fetch table states from META table- Parameters:
conn
- connection to use- Returns:
- map {tableName -> state}
- Throws:
IOException
-
updateTableState
public static void updateTableState(Connection conn, TableName tableName, TableState.State actual) throws IOException Updates state in META Do not use. For internal use only.- Parameters:
conn
- connection to usetableName
- table to look for- Throws:
IOException
-
makePutFromRegionInfo
Generates and returns aPut
containing theRegionInfo
for the catalog table.- Throws:
IllegalArgumentException
- when the provided RegionInfo is not the default replica.IOException
-
makePutFromRegionInfo
Generates and returns aPut
containing theRegionInfo
for the catalog table.- Throws:
IllegalArgumentException
- when the provided RegionInfo is not the default replica.IOException
-
makeDeleteFromRegionInfo
Generates and returns a Delete containing the region info for the catalog table -
addDaughtersToPut
public static Put addDaughtersToPut(Put put, RegionInfo splitA, RegionInfo splitB) throws IOException Adds split daughters to the Put- Throws:
IOException
-
putToMetaTable
Put the passedp
to thehbase:meta
table.- Parameters:
connection
- connection we're usingp
- Put to add to hbase:meta- Throws:
IOException
-
put
- Parameters:
t
- Table to usep
- put to make- Throws:
IOException
-
putsToMetaTable
Put the passedps
to thehbase:meta
table.- Parameters:
connection
- connection we're usingps
- Put to add to hbase:meta- Throws:
IOException
-
deleteFromMetaTable
Delete the passedd
from thehbase:meta
table.- Parameters:
connection
- connection we're usingd
- Delete to add to hbase:meta- Throws:
IOException
-
deleteFromMetaTable
private static void deleteFromMetaTable(Connection connection, List<Delete> deletes) throws IOException Delete the passeddeletes
from thehbase:meta
table.- Parameters:
connection
- connection we're usingdeletes
- Deletes to add to hbase:meta This list should support #remove.- Throws:
IOException
-
addRegionStateToPut
public static Put addRegionStateToPut(Put put, int replicaId, RegionState.State state) throws IOException Set the column value corresponding to thisreplicaId
'sRegionState
to the providedstate
. Mutates the providedPut
.- Throws:
IOException
-
updateRegionState
public static void updateRegionState(Connection connection, RegionInfo ri, RegionState.State state) throws IOException Update state column in hbase:meta.- Throws:
IOException
-
addSplitsToParent
public static void addSplitsToParent(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) throws IOException Adds daughter region infos to hbase:meta row for the specified region. Note that this does not add its daughter's as different rows, but adds information about the daughters in the same row as the parent. Now only used in snapshot. UseRegionStateStore
if you want to split a region.- Parameters:
connection
- connection we're usingregionInfo
- RegionInfo of parent regionsplitA
- first split daughter of the parent regionInfosplitB
- second split daughter of the parent regionInfo- Throws:
IOException
- if problem connecting or updating meta
-
addRegionsToMeta
public static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication) throws IOException Adds a hbase:meta row for each of the specified new regions. Initial state for new regions is CLOSED.- Parameters:
connection
- connection we're usingregionInfos
- region information list- Throws:
IOException
- if problem connecting or updating meta
-
addRegionsToMeta
public static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication, long ts) throws IOException Adds a hbase:meta row for each of the specified new regions. Initial state for new regions is CLOSED.- Parameters:
connection
- connection we're usingregionInfos
- region information listts
- desired timestamp- Throws:
IOException
- if problem connecting or updating meta
-
updateTableState
Update state of the table in meta.- Parameters:
connection
- what we use for updatestate
- new state- Throws:
IOException
-
makePutFromTableState
Construct PUT for given state- Parameters:
state
- new state
-
deleteTableState
Remove state for table from meta- Parameters:
connection
- to use for deletiontable
- to delete state for- Throws:
IOException
-
updateRegionLocation
public static void updateRegionLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.Uses passed catalog tracker to get a connection to the server hosting hbase:meta and makes edits to that region.
- Parameters:
connection
- connection we're usingregionInfo
- region to update location ofopenSeqNum
- the latest sequence number obtained when the region was opensn
- Server namemasterSystemTime
- wall clock time from master if passed in the open region RPC- Throws:
IOException
-
updateLocation
private static void updateLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException Updates the location of the specified region to be the specified server.Connects to the specified server which should be hosting the specified catalog region name to perform the edit.
- Parameters:
connection
- connection we're usingregionInfo
- region to update location ofsn
- Server nameopenSeqNum
- the latest sequence number obtained when the region was openmasterSystemTime
- wall clock time from master if passed in the open region RPC- Throws:
IOException
- In particular could throwConnectException
if the server is down on other end.
-
addRegionInfo
- Throws:
IOException
-
addLocation
public static Put addLocation(Put p, ServerName sn, long openSeqNum, int replicaId) throws IOException - Throws:
IOException
-
addEmptyLocation
- Throws:
IOException
-
debugLogMutations
- Throws:
IOException
-
debugLogMutation
- Throws:
IOException
-
getRegionLocation(Connection, byte[])
instead