Package org.apache.hadoop.hbase
Class MetaTableAccessor
java.lang.Object
org.apache.hadoop.hbase.MetaTableAccessor
Read/write operations on
hbase:meta region as well as assignment information stored
to hbase:meta.
Some of the methods of this class take ZooKeeperWatcher as a param. The only reason for this is
when this class is used on client-side (e.g. HBaseAdmin), we want to use short-lived connection
(opened before each operation, closed right after), while when used on HM or HRS (like in
AssignmentManager) we want permanent connection.
HBASE-10070 adds a replicaId to HRI, meaning more than one HRI can be defined for the same table
range (table, startKey, endKey). For every range, there will be at least one HRI defined which is
called default replica.
Meta layout
For each table there is single row named for the table with a 'table' column family. The column family currently has one column in it, the 'state' column:table:state => contains table stateFor the catalog family, see the comments of
CatalogFamilyFormat for more details.
TODO: Add rep_barrier for serial replication explanation. See SerialReplicationChecker.
The actual layout of meta should be encapsulated inside MetaTableAccessor methods, and should not
leak out of it (through Result objects, etc)- See Also:
-
Field Summary
Fields -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic PutaddDaughtersToPut(Put put, RegionInfo splitA, RegionInfo splitB) Adds split daughters to the Putstatic PutaddEmptyLocation(Put p, int replicaId) static PutaddLocation(Put p, ServerName sn, long openSeqNum, int replicaId) static PutaddRegionInfo(Put p, RegionInfo hri) static PutaddRegionStateToPut(Put put, int replicaId, RegionState.State state) static voidaddRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication) Adds a hbase:meta row for each of the specified new regions.static voidaddRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication, long ts) Adds a hbase:meta row for each of the specified new regions.static voidaddSplitsToParent(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) Adds daughter region infos to hbase:meta row for the specified region.private static voidprivate static voiddebugLogMutations(List<? extends Mutation> mutations) private static voiddeleteFromMetaTable(Connection connection, List<Delete> deletes) Delete the passeddeletesfrom thehbase:metatable.private static voiddeleteFromMetaTable(Connection connection, Delete d) Delete the passeddfrom thehbase:metatable.static voiddeleteTableState(Connection connection, TableName table) Remove state for table from metafullScan(Connection connection, ClientMetaTableAccessor.QueryType type) Performs a full scan ofhbase:meta.static voidfullScanMetaAndPrint(Connection connection) fullScanRegions(Connection connection) Performs a full scan ofhbase:metafor regions.static voidfullScanRegions(Connection connection, ClientMetaTableAccessor.Visitor visitor) Performs a full scan ofhbase:metafor regions.static voidfullScanTables(Connection connection, ClientMetaTableAccessor.Visitor visitor) Performs a full scan ofhbase:metafor tables.static List<RegionInfo>getAllRegions(Connection connection, boolean excludeOfflinedSplitParents) Lists all of the regions currently in META.static ResultgetCatalogFamilyRow(Connection connection, RegionInfo ri) Returns Return theHConstants.CATALOG_FAMILYrow from hbase:meta.private static RegionInfogetClosestRegionInfo(Connection connection, TableName tableName, byte[] row) Returns Get closest metatable region row to passedrowstatic PairOfSameType<RegionInfo>getDaughterRegions(Result data) Returns the daughter regions by reading the corresponding columns of the catalog table Result.private static List<RegionInfo>getListOfRegionInfos(List<Pair<RegionInfo, ServerName>> pairs) static TablegetMetaHTable(Connection connection) Callers should call close on the returnedTableinstance.private static ScangetMetaScan(org.apache.hadoop.conf.Configuration conf, int rowUpperLimit) static Pair<RegionInfo,ServerName> getRegion(Connection connection, byte[] regionName) Deprecated.static HRegionLocationgetRegionLocation(Connection connection, byte[] regionName) Returns the HRegionLocation from meta for the given regionstatic HRegionLocationgetRegionLocation(Connection connection, RegionInfo regionInfo) Returns the HRegionLocation from meta for the given regionstatic ResultgetRegionResult(Connection connection, RegionInfo regionInfo) Gets the result in hbase:meta for the specified region.static ScangetScanForTableName(org.apache.hadoop.conf.Configuration conf, TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table.static List<RegionInfo>getTableRegions(Connection connection, TableName tableName) Gets all of the regions of the specified table.static List<RegionInfo>getTableRegions(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) Gets all of the regions of the specified table.static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, TableName tableName) Do not use this method to get meta table regions, use methods in MetaTableLocator instead.static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) Do not use this method to get meta table regions, use methods in MetaTableLocator instead.static TableStategetTableState(Connection conn, TableName tableName) Fetch table state for given table from META tablestatic Map<TableName,TableState> getTableStates(Connection conn) Fetch table states from META tablestatic ServerNamegetTargetServerName(Result r, int replicaId) Returns theServerNamefrom catalog tableResultwhere the region is transitioning on.static DeletemakeDeleteFromRegionInfo(RegionInfo regionInfo, long ts) Generates and returns a Delete containing the region info for the catalog tablestatic PutmakePutFromRegionInfo(RegionInfo regionInfo) Generates and returns aPutcontaining theRegionInfofor the catalog table.static PutmakePutFromRegionInfo(RegionInfo regionInfo, long ts) Generates and returns aPutcontaining theRegionInfofor the catalog table.static PutmakePutFromTableState(TableState state, long ts) Construct PUT for given stateprivate static voidstatic voidputsToMetaTable(Connection connection, List<Put> ps) Put the passedpsto thehbase:metatable.private static voidputToMetaTable(Connection connection, Put p) Put the passedpto thehbase:metatable.static ResultscanByRegionEncodedName(Connection connection, String regionEncodedName) Scans META table for a row whose key contains the specified regionEncodedName, returning a single relatedResultinstance if any row is found, null otherwise.static voidscanMeta(Connection connection, byte[] startRow, byte[] stopRow, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) Performs a scan of META table.static voidscanMeta(Connection connection, byte[] startRow, byte[] stopRow, ClientMetaTableAccessor.QueryType type, ClientMetaTableAccessor.Visitor visitor) static voidscanMeta(Connection connection, byte[] startRow, byte[] stopRow, ClientMetaTableAccessor.QueryType type, Filter filter, int maxRows, ClientMetaTableAccessor.Visitor visitor) Performs a scan of META table.static voidscanMeta(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) Performs a scan of META table for given table starting from given row.private static voidscanMeta(Connection connection, TableName table, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) static voidscanMetaForTableRegions(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName) private static voidupdateLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) Updates the location of the specified region to be the specified server.static voidupdateRegionLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.static voidupdateRegionState(Connection connection, RegionInfo ri, RegionState.State state) Update state column in hbase:meta.private static voidupdateTableState(Connection connection, TableState state) Update state of the table in meta.static voidupdateTableState(Connection conn, TableName tableName, TableState.State actual) Updates state in META Do not use.
-
Field Details
-
LOG
-
METALOG
-
-
Constructor Details
-
MetaTableAccessor
private MetaTableAccessor()
-
-
Method Details
-
fullScanRegions
public static void fullScanRegions(Connection connection, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a full scan ofhbase:metafor regions.- Parameters:
connection- connection we're usingvisitor- Visitor invoked against each row in regions family.- Throws:
IOException
-
fullScanRegions
Performs a full scan ofhbase:metafor regions.- Parameters:
connection- connection we're using- Throws:
IOException
-
fullScanTables
public static void fullScanTables(Connection connection, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a full scan ofhbase:metafor tables.- Parameters:
connection- connection we're usingvisitor- Visitor invoked against each row in tables family.- Throws:
IOException
-
fullScan
private static List<Result> fullScan(Connection connection, ClientMetaTableAccessor.QueryType type) throws IOException Performs a full scan ofhbase:meta.- Parameters:
connection- connection we're usingtype- scanned part of meta- Returns:
- List of
Result - Throws:
IOException
-
getMetaHTable
Callers should call close on the returnedTableinstance.- Parameters:
connection- connection we're using to access Meta- Returns:
- An
Tableforhbase:meta - Throws:
NullPointerException- ifconnectionisnullIOException
-
getRegion
@Deprecated public static Pair<RegionInfo,ServerName> getRegion(Connection connection, byte[] regionName) throws IOException Deprecated.usegetRegionLocation(Connection, byte[])insteadGets the region info and assignment for the specified region.- Parameters:
connection- connection we're usingregionName- Region to lookup.- Returns:
- Location and RegionInfo for
regionName - Throws:
IOException
-
getRegionLocation
public static HRegionLocation getRegionLocation(Connection connection, byte[] regionName) throws IOException Returns the HRegionLocation from meta for the given region- Parameters:
connection- connection we're usingregionName- region we're looking for- Returns:
- HRegionLocation for the given region
- Throws:
IOException
-
getRegionLocation
public static HRegionLocation getRegionLocation(Connection connection, RegionInfo regionInfo) throws IOException Returns the HRegionLocation from meta for the given region- Parameters:
connection- connection we're usingregionInfo- region information- Returns:
- HRegionLocation for the given region
- Throws:
IOException
-
getCatalogFamilyRow
Returns Return theHConstants.CATALOG_FAMILYrow from hbase:meta.- Throws:
IOException
-
getRegionResult
public static Result getRegionResult(Connection connection, RegionInfo regionInfo) throws IOException Gets the result in hbase:meta for the specified region.- Parameters:
connection- connection we're usingregionInfo- region we're looking for- Returns:
- result of the specified region
- Throws:
IOException
-
scanByRegionEncodedName
public static Result scanByRegionEncodedName(Connection connection, String regionEncodedName) throws IOException Scans META table for a row whose key contains the specified regionEncodedName, returning a single relatedResultinstance if any row is found, null otherwise.- Parameters:
connection- the connection to query META table.regionEncodedName- the region encoded name to look for at META.- Returns:
Resultinstance with the row related info in META, null otherwise.- Throws:
IOException- if any errors occur while querying META.
-
getAllRegions
public static List<RegionInfo> getAllRegions(Connection connection, boolean excludeOfflinedSplitParents) throws IOException Lists all of the regions currently in META.- Parameters:
connection- to connect withexcludeOfflinedSplitParents- False if we are to include offlined/splitparents regions, true and we'll leave out offlined regions from returned list- Returns:
- List of all user-space regions.
- Throws:
IOException
-
getTableRegions
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName) throws IOException Gets all of the regions of the specified table. Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection- connection we're usingtableName- table we're looking for- Returns:
- Ordered list of
RegionInfo. - Throws:
IOException
-
getTableRegions
public static List<RegionInfo> getTableRegions(Connection connection, TableName tableName, boolean excludeOfflinedSplitParents) throws IOException Gets all of the regions of the specified table. Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection- connection we're usingtableName- table we're looking forexcludeOfflinedSplitParents- If true, do not include offlined split parents in the return.- Returns:
- Ordered list of
RegionInfo. - Throws:
IOException
-
getListOfRegionInfos
-
getScanForTableName
public static Scan getScanForTableName(org.apache.hadoop.conf.Configuration conf, TableName tableName) This method creates a Scan object that will only scan catalog rows that belong to the specified table. It doesn't specify any columns. This is a better alternative to just using a start row and scan until it hits a new table since that requires parsing the HRI to get the table name.- Parameters:
tableName- bytes of table's name- Returns:
- configured Scan object
-
getMetaScan
-
getTableRegionsAndLocations
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, TableName tableName) throws IOException Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection- connection we're usingtableName- table we're looking for- Returns:
- Return list of regioninfos and server.
- Throws:
IOException
-
getTableRegionsAndLocations
public static List<Pair<RegionInfo,ServerName>> getTableRegionsAndLocations(Connection connection, @Nullable TableName tableName, boolean excludeOfflinedSplitParents) throws IOException Do not use this method to get meta table regions, use methods in MetaTableLocator instead.- Parameters:
connection- connection we're usingtableName- table to work with, can be null for getting all regionsexcludeOfflinedSplitParents- don't return split parents- Returns:
- Return list of regioninfos and server addresses.
- Throws:
IOException
-
fullScanMetaAndPrint
- Throws:
IOException
-
scanMetaForTableRegions
public static void scanMetaForTableRegions(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName) throws IOException - Throws:
IOException
-
scanMeta
private static void scanMeta(Connection connection, TableName table, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) throws IOException - Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, ClientMetaTableAccessor.QueryType type, ClientMetaTableAccessor.Visitor visitor) throws IOException - Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, ClientMetaTableAccessor.Visitor visitor, TableName tableName, byte[] row, int rowLimit) throws IOException Performs a scan of META table for given table starting from given row.- Parameters:
connection- connection we're usingvisitor- visitor to calltableName- table withing we scanrow- start scan from this rowrowLimit- max number of rows to return- Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, ClientMetaTableAccessor.QueryType type, int maxRows, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a scan of META table.- Parameters:
connection- connection we're usingstartRow- Where to start the scan. Pass null if want to begin scan at first row.stopRow- Where to stop the scan. Pass null if want to scan all rows from the start onetype- scanned part of metamaxRows- maximum rows to returnvisitor- Visitor invoked against each row.- Throws:
IOException
-
scanMeta
public static void scanMeta(Connection connection, @Nullable byte[] startRow, @Nullable byte[] stopRow, ClientMetaTableAccessor.QueryType type, @Nullable Filter filter, int maxRows, ClientMetaTableAccessor.Visitor visitor) throws IOException Performs a scan of META table.- Parameters:
connection- connection we're usingstartRow- Where to start the scan. Pass null if want to begin scan at first row.stopRow- Where to stop the scan. Pass null if want to scan all rows from the start onetype- scanned part of metamaxRows- maximum rows to returnvisitor- Visitor invoked against each row.- Throws:
IOException
-
getClosestRegionInfo
@NonNull private static RegionInfo getClosestRegionInfo(Connection connection, @NonNull TableName tableName, @NonNull byte[] row) throws IOException Returns Get closest metatable region row to passedrow- Throws:
IOException
-
getTargetServerName
Returns theServerNamefrom catalog tableResultwhere the region is transitioning on. It should be the same asCatalogFamilyFormat.getServerName(Result,int)if the server is at OPEN state.- Parameters:
r- Result to pull the transitioning server name from- Returns:
- A ServerName instance or
CatalogFamilyFormat.getServerName(Result,int)if necessary fields not found or empty.
-
getDaughterRegions
Returns the daughter regions by reading the corresponding columns of the catalog table Result.- Parameters:
data- a Result object from the catalog table scan- Returns:
- pair of RegionInfo or PairOfSameType(null, null) if region is not a split parent
-
getTableState
@Nullable public static TableState getTableState(Connection conn, TableName tableName) throws IOException Fetch table state for given table from META table- Parameters:
conn- connection to usetableName- table to fetch state for- Throws:
IOException
-
getTableStates
Fetch table states from META table- Parameters:
conn- connection to use- Returns:
- map {tableName -> state}
- Throws:
IOException
-
updateTableState
public static void updateTableState(Connection conn, TableName tableName, TableState.State actual) throws IOException Updates state in META Do not use. For internal use only.- Parameters:
conn- connection to usetableName- table to look for- Throws:
IOException
-
makePutFromRegionInfo
Generates and returns aPutcontaining theRegionInfofor the catalog table.- Throws:
IllegalArgumentException- when the provided RegionInfo is not the default replica.IOException
-
makePutFromRegionInfo
Generates and returns aPutcontaining theRegionInfofor the catalog table.- Throws:
IllegalArgumentException- when the provided RegionInfo is not the default replica.IOException
-
makeDeleteFromRegionInfo
Generates and returns a Delete containing the region info for the catalog table -
addDaughtersToPut
public static Put addDaughtersToPut(Put put, RegionInfo splitA, RegionInfo splitB) throws IOException Adds split daughters to the Put- Throws:
IOException
-
putToMetaTable
Put the passedpto thehbase:metatable.- Parameters:
connection- connection we're usingp- Put to add to hbase:meta- Throws:
IOException
-
put
- Parameters:
t- Table to usep- put to make- Throws:
IOException
-
putsToMetaTable
Put the passedpsto thehbase:metatable.- Parameters:
connection- connection we're usingps- Put to add to hbase:meta- Throws:
IOException
-
deleteFromMetaTable
Delete the passeddfrom thehbase:metatable.- Parameters:
connection- connection we're usingd- Delete to add to hbase:meta- Throws:
IOException
-
deleteFromMetaTable
private static void deleteFromMetaTable(Connection connection, List<Delete> deletes) throws IOException Delete the passeddeletesfrom thehbase:metatable.- Parameters:
connection- connection we're usingdeletes- Deletes to add to hbase:meta This list should support #remove.- Throws:
IOException
-
addRegionStateToPut
public static Put addRegionStateToPut(Put put, int replicaId, RegionState.State state) throws IOException Set the column value corresponding to thisreplicaId'sRegionStateto the providedstate. Mutates the providedPut.- Throws:
IOException
-
updateRegionState
public static void updateRegionState(Connection connection, RegionInfo ri, RegionState.State state) throws IOException Update state column in hbase:meta.- Throws:
IOException
-
addSplitsToParent
public static void addSplitsToParent(Connection connection, RegionInfo regionInfo, RegionInfo splitA, RegionInfo splitB) throws IOException Adds daughter region infos to hbase:meta row for the specified region. Note that this does not add its daughter's as different rows, but adds information about the daughters in the same row as the parent. Now only used in snapshot. UseRegionStateStoreif you want to split a region.- Parameters:
connection- connection we're usingregionInfo- RegionInfo of parent regionsplitA- first split daughter of the parent regionInfosplitB- second split daughter of the parent regionInfo- Throws:
IOException- if problem connecting or updating meta
-
addRegionsToMeta
public static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication) throws IOException Adds a hbase:meta row for each of the specified new regions. Initial state for new regions is CLOSED.- Parameters:
connection- connection we're usingregionInfos- region information list- Throws:
IOException- if problem connecting or updating meta
-
addRegionsToMeta
public static void addRegionsToMeta(Connection connection, List<RegionInfo> regionInfos, int regionReplication, long ts) throws IOException Adds a hbase:meta row for each of the specified new regions. Initial state for new regions is CLOSED.- Parameters:
connection- connection we're usingregionInfos- region information listts- desired timestamp- Throws:
IOException- if problem connecting or updating meta
-
updateTableState
Update state of the table in meta.- Parameters:
connection- what we use for updatestate- new state- Throws:
IOException
-
makePutFromTableState
Construct PUT for given state- Parameters:
state- new state
-
deleteTableState
Remove state for table from meta- Parameters:
connection- to use for deletiontable- to delete state for- Throws:
IOException
-
updateRegionLocation
public static void updateRegionLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException Updates the location of the specified region in hbase:meta to be the specified server hostname and startcode.Uses passed catalog tracker to get a connection to the server hosting hbase:meta and makes edits to that region.
- Parameters:
connection- connection we're usingregionInfo- region to update location ofopenSeqNum- the latest sequence number obtained when the region was opensn- Server namemasterSystemTime- wall clock time from master if passed in the open region RPC- Throws:
IOException
-
updateLocation
private static void updateLocation(Connection connection, RegionInfo regionInfo, ServerName sn, long openSeqNum, long masterSystemTime) throws IOException Updates the location of the specified region to be the specified server.Connects to the specified server which should be hosting the specified catalog region name to perform the edit.
- Parameters:
connection- connection we're usingregionInfo- region to update location ofsn- Server nameopenSeqNum- the latest sequence number obtained when the region was openmasterSystemTime- wall clock time from master if passed in the open region RPC- Throws:
IOException- In particular could throwConnectExceptionif the server is down on other end.
-
addRegionInfo
- Throws:
IOException
-
addLocation
public static Put addLocation(Put p, ServerName sn, long openSeqNum, int replicaId) throws IOException - Throws:
IOException
-
addEmptyLocation
- Throws:
IOException
-
debugLogMutations
- Throws:
IOException
-
debugLogMutation
- Throws:
IOException
-
getRegionLocation(Connection, byte[])instead