Package org.apache.hadoop.hbase.quotas
Class QuotaTableUtil
java.lang.Object
org.apache.hadoop.hbase.quotas.QuotaTableUtil
- Direct Known Subclasses:
QuotaUtil
Helper class to interact with the quota table.
| ROW-KEY | FAM/QUAL | DATA | DESC |
|---|---|---|---|
| n.<namespace> | q:s | <global-quotas> | |
| n.<namespace> | u:p | <namespace-quota policy> | |
| n.<namespace> | u:s | <SpaceQuotaSnapshot> | The size of all snapshots against tables in the namespace |
| t.<table> | q:s | <global-quotas> | |
| t.<table> | u:p | <table-quota policy> | |
| t.<table> | u:ss.<snapshot name> | <SpaceQuotaSnapshot> | The size of a snapshot against a table |
| u.<user> | q:s | <global-quotas> | |
| u.<user> | q:s.<table> | <table-quotas> | |
| u.<user> | q:s.<ns> | <namespace-quotas> |
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic interfacestatic interfaceprivate static interfacestatic interfacestatic interface -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate static final org.slf4j.Loggerprivate static final byte[]protected static final byte[]protected static final byte[]protected static final byte[]protected static final Stringprotected static final byte[]protected static final byte[]protected static final byte[]static final Stringprotected static final byte[]protected static final byte[]static final TableNameSystem table for quotasprotected static final byte[]protected static final byte[] -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptioncreateDeletesForExistingNamespaceSnapshotSizes(Set<String> snapshotEntriesToRemove) Returns a list ofDeleteto remove given namespace snapshot entries to removefrom quota tableReturns a list ofDeleteto remove all namespace snapshot entries from quota table.createDeletesForExistingSnapshotsFromScan(Connection connection, Scan scan) Returns a list ofDeleteto remove all entries returned by the passed scanner.createDeletesForExistingTableSnapshotSizes(Connection connection) Returns a list ofDeleteto remove all table snapshot entries from quota table.createDeletesForExistingTableSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotEntriesToRemove) Returns a list ofDeleteto remove given table snapshot entries to remove from quota table(package private) static GetcreateGetNamespaceSnapshotSize(String namespace) Creates aGetto fetch the namespace's total snapshot size.(package private) static PutcreatePutForNamespaceSnapshotSize(String namespace, long size) Creates aPutfor the namespace's total snapshot size.(package private) static PutcreatePutForSnapshotSize(TableName tableName, String snapshot, long size) (package private) static PutcreatePutForSpaceSnapshot(TableName tableName, SpaceQuotaSnapshot snapshot) (package private) static ScanReturns a scanner for all existing namespace snapshot entries.(package private) static ScancreateScanForNamespaceSnapshotSizes(String namespace) Returns a scanner for all namespace snapshot entries of the given namespace(package private) static Scan(package private) static Scan(package private) static voiddeleteTableUsageSnapshotsForNamespace(Connection connection, String namespace) Remove table usage snapshots (u:p columns) for the namespace passedprotected static Result[]doGet(Connection connection, List<Get> gets) protected static ResultdoGet(Connection connection, Get get) static voidextractQuotaSnapshot(Result result, Map<TableName, SpaceQuotaSnapshot> snapshots) Extracts theSpaceViolationPolicyandTableNamefrom the providedResultand adds them to the givenMap.protected static Stringprotected static longextractSnapshotSize(byte[] data, int offset, int length) static SpaceQuotaSnapshotgetCurrentSnapshotFromQuotaTable(Connection conn, TableName tableName) Returns the current space quota snapshot of the giventableNamefromQuotaTableUtil.QUOTA_TABLE_NAMEor null if the no quota information is available for that tableName.protected static byte[]protected static StringgetNamespaceFromRowKey(byte[] key) static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetNamespaceQuota(Connection connection, String namespace) protected static byte[]getNamespaceRowKey(String namespace) protected static StringgetNamespaceRowKeyRegex(String namespace) Returns a set of the names of all namespaces containing snapshot entries.(package private) static longgetNamespaceSnapshotSize(Connection conn, String namespace) Fetches the computed size of all snapshots against tables in a namespace for space quotas.Fetches any persisted HBase snapshot sizes stored in the quota table.protected static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.SpaceQuotaprivate static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetQuotas(Connection connection, byte[] rowKey) private static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetQuotas(Connection connection, byte[] rowKey, byte[] qualifier) protected static StringgetRegionServerFromRowKey(byte[] key) static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetRegionServerQuota(Connection connection, String regionServer) protected static byte[]getRegionServerRowKey(String regionServer) private static StringgetRegionServerRowKeyRegex(String regionServer) private static StringgetRowKeyRegEx(byte[] prefix, String regex) protected static byte[]getSettingsQualifierForUserNamespace(String namespace) protected static byte[]getSettingsQualifierForUserTable(TableName tableName) protected static StringgetSettingsQualifierRegexForUserNamespace(String namespace) protected static Stringstatic Map<TableName,SpaceQuotaSnapshot> getSnapshots(Connection conn) Fetches allSpaceQuotaSnapshotobjects from thehbase:quotatable.protected static byte[]getSnapshotSizeQualifier(String snapshotName) protected static TableNamegetTableFromRowKey(byte[] key) static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetTableQuota(Connection connection, TableName table) protected static byte[]getTableRowKey(TableName table) protected static StringgetTableRowKeyRegex(String table) getTableSnapshots(Connection conn) Returns a multimap for all existing table snapshot entries.protected static StringgetUserFromRowKey(byte[] key) static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetUserQuota(Connection connection, String user) static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetUserQuota(Connection connection, String user, String namespace) static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasgetUserQuota(Connection connection, String user, TableName table) protected static byte[]getUserRowKey(String user) protected static StringgetUserRowKeyRegex(String user) protected static SpaceViolationPolicygetViolationPolicy(org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.SpaceQuota proto) static booleanisEmptyQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas quotas) private static booleanisExceedThrottleQuotaRowKey(byte[] key) protected static booleanisNamespaceRowKey(byte[] key) protected static booleanisRegionServerRowKey(byte[] key) protected static booleanisTableRowKey(byte[] key) protected static booleanisUserRowKey(byte[] key) static FiltermakeFilter(QuotaFilter filter) converts quotafilter to serializeable filterlists.(package private) static GetmakeGetForSnapshotSize(TableName tn, String snapshot) Creates aGetfor the HBase snapshot's size against the given table.static GetCreates aGetwhich returns onlySpaceQuotaSnapshotfrom the quota table for a specific table.static ScanCreates aScanwhich returns only quota snapshots from the quota table.static ScanCreates aScanwhich returns onlySpaceQuotaSnapshotfrom the quota table for a specific table.static ScanmakeScan(QuotaFilter filter) protected static voidparseNamespaceResult(String namespace, Result result, QuotaTableUtil.NamespaceQuotasVisitor visitor) static voidparseNamespaceResult(Result result, QuotaTableUtil.NamespaceQuotasVisitor visitor) private static voidparseRegionServerResult(String regionServer, Result result, QuotaTableUtil.RegionServerQuotasVisitor visitor) private static voidparseRegionServerResult(Result result, QuotaTableUtil.RegionServerQuotasVisitor visitor) static voidparseResult(Result result, QuotaTableUtil.QuotasVisitor visitor) static voidparseResultToCollection(Result result, Collection<QuotaSettings> quotaSettings) (package private) static longParses the snapshot size from the given Cell's value.static voidparseTableResult(Result result, QuotaTableUtil.TableQuotasVisitor visitor) protected static voidparseTableResult(TableName table, Result result, QuotaTableUtil.TableQuotasVisitor visitor) protected static voidparseUserResult(String userName, Result result, QuotaTableUtil.UserQuotasVisitor visitor) static voidparseUserResult(Result result, QuotaTableUtil.UserQuotasVisitor visitor) protected static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasquotasFromData(byte[] data) protected static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.QuotasquotasFromData(byte[] data, int offset, int length) protected static byte[]quotasToData(org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas data)
-
Field Details
-
LOG
-
QUOTA_TABLE_NAME
System table for quotas -
QUOTA_FAMILY_INFO
-
QUOTA_FAMILY_USAGE
-
QUOTA_QUALIFIER_SETTINGS
-
QUOTA_QUALIFIER_SETTINGS_PREFIX
-
QUOTA_QUALIFIER_POLICY
-
QUOTA_SNAPSHOT_SIZE_QUALIFIER
-
QUOTA_POLICY_COLUMN
-
QUOTA_USER_ROW_KEY_PREFIX
-
QUOTA_TABLE_ROW_KEY_PREFIX
-
QUOTA_NAMESPACE_ROW_KEY_PREFIX
-
QUOTA_REGION_SERVER_ROW_KEY_PREFIX
-
QUOTA_EXCEED_THROTTLE_QUOTA_ROW_KEY
-
QUOTA_REGION_SERVER_ROW_KEY
- See Also:
-
-
Constructor Details
-
QuotaTableUtil
public QuotaTableUtil()
-
-
Method Details
-
getTableQuota
public static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getTableQuota(Connection connection, TableName table) throws IOException - Throws:
IOException
-
getNamespaceQuota
public static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getNamespaceQuota(Connection connection, String namespace) throws IOException - Throws:
IOException
-
getUserQuota
public static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getUserQuota(Connection connection, String user) throws IOException - Throws:
IOException
-
getUserQuota
public static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getUserQuota(Connection connection, String user, TableName table) throws IOException - Throws:
IOException
-
getUserQuota
public static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getUserQuota(Connection connection, String user, String namespace) throws IOException - Throws:
IOException
-
getQuotas
private static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getQuotas(Connection connection, byte[] rowKey) throws IOException - Throws:
IOException
-
getRegionServerQuota
public static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getRegionServerQuota(Connection connection, String regionServer) throws IOException - Throws:
IOException
-
getQuotas
private static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas getQuotas(Connection connection, byte[] rowKey, byte[] qualifier) throws IOException - Throws:
IOException
-
makeScan
-
makeFilter
converts quotafilter to serializeable filterlists. -
makeQuotaSnapshotScan
Creates aScanwhich returns only quota snapshots from the quota table. -
getSnapshots
Fetches allSpaceQuotaSnapshotobjects from thehbase:quotatable.- Parameters:
conn- The HBase connection- Returns:
- A map of table names and their computed snapshot.
- Throws:
IOException
-
makeQuotaSnapshotScanForTable
Creates aScanwhich returns onlySpaceQuotaSnapshotfrom the quota table for a specific table.- Parameters:
tn- Optionally, a table name to limit the scan's rowkey space. Can be null.
-
makeQuotaSnapshotGetForTable
Creates aGetwhich returns onlySpaceQuotaSnapshotfrom the quota table for a specific table.- Parameters:
tn- table name to get from. Can't be null.
-
extractQuotaSnapshot
Extracts theSpaceViolationPolicyandTableNamefrom the providedResultand adds them to the givenMap. If the result does not contain the expected information or the serialized policy in the value is invalid, this method will throw anIllegalArgumentException.- Parameters:
result- A row from the quota table.snapshots- A map of snapshots to add the result of this method into.
-
parseResult
public static void parseResult(Result result, QuotaTableUtil.QuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseResultToCollection
public static void parseResultToCollection(Result result, Collection<QuotaSettings> quotaSettings) throws IOException - Throws:
IOException
-
parseNamespaceResult
public static void parseNamespaceResult(Result result, QuotaTableUtil.NamespaceQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseNamespaceResult
protected static void parseNamespaceResult(String namespace, Result result, QuotaTableUtil.NamespaceQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseRegionServerResult
private static void parseRegionServerResult(Result result, QuotaTableUtil.RegionServerQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseRegionServerResult
private static void parseRegionServerResult(String regionServer, Result result, QuotaTableUtil.RegionServerQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseTableResult
public static void parseTableResult(Result result, QuotaTableUtil.TableQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseTableResult
protected static void parseTableResult(TableName table, Result result, QuotaTableUtil.TableQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseUserResult
public static void parseUserResult(Result result, QuotaTableUtil.UserQuotasVisitor visitor) throws IOException - Throws:
IOException
-
parseUserResult
protected static void parseUserResult(String userName, Result result, QuotaTableUtil.UserQuotasVisitor visitor) throws IOException - Throws:
IOException
-
createPutForSpaceSnapshot
-
makeGetForSnapshotSize
Creates aGetfor the HBase snapshot's size against the given table. -
createPutForSnapshotSize
-
createPutForNamespaceSnapshotSize
Creates aPutfor the namespace's total snapshot size. -
createDeletesForExistingTableSnapshotSizes
static List<Delete> createDeletesForExistingTableSnapshotSizes(org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName, String> snapshotEntriesToRemove) Returns a list ofDeleteto remove given table snapshot entries to remove from quota table- Parameters:
snapshotEntriesToRemove- the entries to remove
-
createDeletesForExistingTableSnapshotSizes
static List<Delete> createDeletesForExistingTableSnapshotSizes(Connection connection) throws IOException Returns a list ofDeleteto remove all table snapshot entries from quota table.- Parameters:
connection- connection to re-use- Throws:
IOException
-
createDeletesForExistingNamespaceSnapshotSizes
static List<Delete> createDeletesForExistingNamespaceSnapshotSizes(Set<String> snapshotEntriesToRemove) Returns a list ofDeleteto remove given namespace snapshot entries to removefrom quota table- Parameters:
snapshotEntriesToRemove- the entries to remove
-
createDeletesForExistingNamespaceSnapshotSizes
static List<Delete> createDeletesForExistingNamespaceSnapshotSizes(Connection connection) throws IOException Returns a list ofDeleteto remove all namespace snapshot entries from quota table.- Parameters:
connection- connection to re-use- Throws:
IOException
-
createDeletesForExistingSnapshotsFromScan
static List<Delete> createDeletesForExistingSnapshotsFromScan(Connection connection, Scan scan) throws IOException Returns a list ofDeleteto remove all entries returned by the passed scanner.- Parameters:
connection- connection to re-usescan- the scanner to use to generate the list of deletes- Throws:
IOException
-
deleteTableUsageSnapshotsForNamespace
static void deleteTableUsageSnapshotsForNamespace(Connection connection, String namespace) throws IOException Remove table usage snapshots (u:p columns) for the namespace passed- Parameters:
connection- connection to re-usenamespace- the namespace to fetch the list of table usage snapshots- Throws:
IOException
-
getNamespaceSnapshotSize
Fetches the computed size of all snapshots against tables in a namespace for space quotas.- Throws:
IOException
-
createGetNamespaceSnapshotSize
Creates aGetto fetch the namespace's total snapshot size. -
parseSnapshotSize
static long parseSnapshotSize(Cell c) throws org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException Parses the snapshot size from the given Cell's value.- Throws:
org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException
-
createScanForNamespaceSnapshotSizes
Returns a scanner for all existing namespace snapshot entries. -
createScanForNamespaceSnapshotSizes
Returns a scanner for all namespace snapshot entries of the given namespace- Parameters:
namespace- name of the namespace whose snapshot entries are to be scanned
-
createScanForSpaceSnapshotSizes
-
createScanForSpaceSnapshotSizes
-
getObservedSnapshotSizes
Fetches any persisted HBase snapshot sizes stored in the quota table. The sizes here are computed relative to the table which the snapshot was created from. A snapshot's size will not include the size of files which the table still refers. These sizes, in bytes, are what is used internally to compute quota violation for tables and namespaces.- Returns:
- A map of snapshot name to size in bytes per space quota computations
- Throws:
IOException
-
getTableSnapshots
public static org.apache.hbase.thirdparty.com.google.common.collect.Multimap<TableName,String> getTableSnapshots(Connection conn) throws IOException Returns a multimap for all existing table snapshot entries.- Parameters:
conn- connection to re-use- Throws:
IOException
-
getNamespaceSnapshots
Returns a set of the names of all namespaces containing snapshot entries.- Parameters:
conn- connection to re-use- Throws:
IOException
-
getCurrentSnapshotFromQuotaTable
public static SpaceQuotaSnapshot getCurrentSnapshotFromQuotaTable(Connection conn, TableName tableName) throws IOException Returns the current space quota snapshot of the giventableNamefromQuotaTableUtil.QUOTA_TABLE_NAMEor null if the no quota information is available for that tableName.- Parameters:
conn- connection to re-usetableName- name of the table whose current snapshot is to be retreived- Throws:
IOException
-
quotasFromData
protected static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas quotasFromData(byte[] data) throws IOException - Throws:
IOException
-
quotasFromData
protected static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas quotasFromData(byte[] data, int offset, int length) throws IOException - Throws:
IOException
-
quotasToData
protected static byte[] quotasToData(org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas data) throws IOException - Throws:
IOException
-
isEmptyQuota
public static boolean isEmptyQuota(org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.Quotas quotas) -
doGet
- Throws:
IOException
-
doGet
- Throws:
IOException
-
getUserRowKey
-
getTableRowKey
-
getNamespaceRowKey
-
getRegionServerRowKey
-
getSettingsQualifierForUserTable
-
getSettingsQualifierForUserNamespace
-
getUserRowKeyRegex
-
getTableRowKeyRegex
-
getNamespaceRowKeyRegex
-
getRegionServerRowKeyRegex
-
getExceedThrottleQuotaRowKey
-
getRowKeyRegEx
-
getSettingsQualifierRegexForUserTable
-
getSettingsQualifierRegexForUserNamespace
-
isNamespaceRowKey
-
getNamespaceFromRowKey
-
isRegionServerRowKey
-
isExceedThrottleQuotaRowKey
-
getRegionServerFromRowKey
-
isTableRowKey
-
getTableFromRowKey
-
isUserRowKey
-
getUserFromRowKey
-
getProtoViolationPolicy
protected static org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.SpaceQuota getProtoViolationPolicy(SpaceViolationPolicy policy) -
getViolationPolicy
protected static SpaceViolationPolicy getViolationPolicy(org.apache.hadoop.hbase.shaded.protobuf.generated.QuotaProtos.SpaceQuota proto) -
getSnapshotSizeQualifier
-
extractSnapshotNameFromSizeCell
-
extractSnapshotSize
protected static long extractSnapshotSize(byte[] data, int offset, int length) throws org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException - Throws:
org.apache.hbase.thirdparty.com.google.protobuf.InvalidProtocolBufferException
-