org.apache.hadoop.hbase.master.snapshot

Class MasterSnapshotVerifier

java.lang.Object

org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public final class MasterSnapshotVerifier
extends Object

General snapshot verification on the master.

This is a light-weight verification mechanism for all the files in a snapshot. It doesn't attempt to verify that the files are exact copies (that would be paramount to taking the snapshot again!), but instead just attempts to ensure that the files match the expected files and are the same length.

Taking an online snapshots can race against other operations and this is an last line of defense. For example, if meta changes between when snapshots are taken not all regions of a table may be present. This can be caused by a region split (daughters present on this scan, but snapshot took parent), or move (snapshots only checks lists of region servers, a move could have caused a region to be skipped or done twice).

Current snapshot files checked:

1. SnapshotDescription is readable
2. Table info is readable
3. Regions

• Matching regions in the snapshot as currently in the table
• RegionInfo matches the current and stored regions
• All referenced hfiles have valid names
• All the hfiles are present (either in .archive directory in the region)
• All recovered.edits files are present (by name) and have the correct file size

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.fs.FileSystem	fs
private static org.slf4j.Logger	LOG
private org.apache.hadoop.fs.Path	rootDir
private MasterServices	services
private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription	snapshot
private TableName	tableName

Constructor Summary

Constructors

Constructor and Description
MasterSnapshotVerifier(MasterServices services, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootDir)

Method Summary

Modifier and Type	Method and Description
private void	verifyRegionInfo(RegionInfo region, org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotRegionManifest manifest) Verify that the regionInfo is valid
private void	verifyRegions(SnapshotManifest manifest) Check that all the regions in the snapshot are valid, and accounted for.
void	verifySnapshot(org.apache.hadoop.fs.Path snapshotDir, Set<String> snapshotServers) Verify that the snapshot in the directory is a valid snapshot
private void	verifySnapshotDescription(org.apache.hadoop.fs.Path snapshotDir) Check that the snapshot description written in the filesystem matches the current snapshot
private void	verifyTableInfo(SnapshotManifest manifest) Check that the table descriptor for the snapshot is a valid table descriptor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

snapshot

private org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot

fs

private org.apache.hadoop.fs.FileSystem fs

rootDir

private org.apache.hadoop.fs.Path rootDir

tableName

private TableName tableName

services

private MasterServices services

Constructor Detail

MasterSnapshotVerifier

public MasterSnapshotVerifier(MasterServices services,
                              org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotDescription snapshot,
                              org.apache.hadoop.fs.Path rootDir)

Parameters:

services - services for the master

snapshot - snapshot to check

rootDir - root directory of the hbase installation.

Method Detail

verifySnapshot

public void verifySnapshot(org.apache.hadoop.fs.Path snapshotDir,
                           Set<String> snapshotServers)
                    throws CorruptedSnapshotException,
                           IOException

Verify that the snapshot in the directory is a valid snapshot

Parameters:

snapshotDir - snapshot directory to check

snapshotServers - ServerName of the servers that are involved in the snapshot

Throws:

CorruptedSnapshotException - if the snapshot is invalid

IOException - if there is an unexpected connection issue to the filesystem

verifySnapshotDescription

private void verifySnapshotDescription(org.apache.hadoop.fs.Path snapshotDir)
                                throws CorruptedSnapshotException

Check that the snapshot description written in the filesystem matches the current snapshot

Parameters:

snapshotDir - snapshot directory to check

Throws:

CorruptedSnapshotException

verifyTableInfo

private void verifyTableInfo(SnapshotManifest manifest)
                      throws IOException

Check that the table descriptor for the snapshot is a valid table descriptor

Parameters:

manifest - snapshot manifest to inspect

Throws:

IOException

verifyRegions

private void verifyRegions(SnapshotManifest manifest)
                    throws IOException

Check that all the regions in the snapshot are valid, and accounted for.

Parameters:

manifest - snapshot manifest to inspect

Throws:

IOException - if we can't reach hbase:meta or read the files from the FS

verifyRegionInfo

private void verifyRegionInfo(RegionInfo region,
                              org.apache.hadoop.hbase.shaded.protobuf.generated.SnapshotProtos.SnapshotRegionManifest manifest)
                       throws IOException

Verify that the regionInfo is valid

Parameters:

region - the region to check

manifest - snapshot manifest to inspect

Throws:

IOException