org.apache.hadoop.hbase.client

Class TableSnapshotScanner

java.lang.Object

org.apache.hadoop.hbase.client.AbstractClientScanner

org.apache.hadoop.hbase.client.TableSnapshotScanner

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<Result>, ResultScanner

@InterfaceAudience.Private
public class TableSnapshotScanner
extends AbstractClientScanner

A Scanner which performs a scan over snapshot files. Using this class requires copying the snapshot to a temporary empty directory, which will copy the snapshot reference files into that directory. Actual data files are not copied.

This also allows one to run the scan from an online or offline hbase cluster. The snapshot files can be exported by using the org.apache.hadoop.hbase.snapshot.ExportSnapshot tool, to a pure-hdfs cluster, and this scanner can be used to run the scan directly over the snapshot files. The snapshot should not be deleted while there are open scanners reading from snapshot files.

An internal RegionScanner is used to execute the Scan obtained from the user for each region in the snapshot.

HBase owns all the data and snapshot files on the filesystem. Only the HBase user can read from snapshot files and data files. HBase also enforces security because all the requests are handled by the server layer, and the user cannot read from the data files directly. To read from snapshot files directly from the file system, the user who is running the MR job must have sufficient permissions to access snapshot and reference files. This means that to run mapreduce over snapshot files, the job has to be run as the HBase user or the user must have group or other priviledges in the filesystem (See HBASE-8369). Note that, given other users access to read from snapshot/data files will completely circumvent the access control enforced by HBase. See org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat.

Field Summary

Fields

Modifier and Type	Field and Description
private org.apache.hadoop.conf.Configuration	conf
private int	currentRegion
private ClientSideRegionScanner	currentRegionScanner
private org.apache.hadoop.fs.FileSystem	fs
private TableDescriptor	htd
private static org.slf4j.Logger	LOG
private int	numOfCompleteRows
private ArrayList<RegionInfo>	regions
private org.apache.hadoop.fs.Path	restoreDir
private org.apache.hadoop.fs.Path	rootDir
private Scan	scan
private boolean	snapshotAlreadyRestored
private String	snapshotName

Fields inherited from class org.apache.hadoop.hbase.client.AbstractClientScanner

scanMetrics

Constructor Summary

Constructors

Constructor and Description
TableSnapshotScanner(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path restoreDir, String snapshotName, Scan scan)
TableSnapshotScanner(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path restoreDir, String snapshotName, Scan scan, boolean snapshotAlreadyRestored) Creates a TableSnapshotScanner.
TableSnapshotScanner(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path restoreDir, String snapshotName, Scan scan) Creates a TableSnapshotScanner.

Method Summary

Modifier and Type	Method and Description
private void	cleanup()
void	close() Closes the scanner and releases any resources it has allocated
private boolean	isValidRegion(RegionInfo hri)
Result	next() Grab the next row's worth of values.
private void	openWithoutRestoringSnapshot()
private void	openWithRestoringSnapshot()
boolean	renewLease() Allow the client to renew the scanner's lease on the server.

Methods inherited from class org.apache.hadoop.hbase.client.AbstractClientScanner

getScanMetrics, initScanMetrics

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.client.ResultScanner

iterator, next

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

LOG

private static final org.slf4j.Logger LOG

conf

private org.apache.hadoop.conf.Configuration conf

snapshotName

private String snapshotName

fs

private org.apache.hadoop.fs.FileSystem fs

rootDir

private org.apache.hadoop.fs.Path rootDir

restoreDir

private org.apache.hadoop.fs.Path restoreDir

scan

private Scan scan

regions

private ArrayList<RegionInfo> regions

htd

private TableDescriptor htd

snapshotAlreadyRestored

private final boolean snapshotAlreadyRestored

currentRegionScanner

private ClientSideRegionScanner currentRegionScanner

currentRegion

private int currentRegion

numOfCompleteRows

private int numOfCompleteRows

Constructor Detail

TableSnapshotScanner

public TableSnapshotScanner(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.Path restoreDir,
                            String snapshotName,
                            Scan scan)
                     throws IOException

Creates a TableSnapshotScanner.

Parameters:

conf - the configuration

restoreDir - a temporary directory to copy the snapshot files into. Current user should have write permissions to this directory, and this should not be a subdirectory of rootDir. The scanner deletes the contents of the directory once the scanner is closed.

snapshotName - the name of the snapshot to read from

scan - a Scan representing scan parameters

Throws:

IOException - in case of error

TableSnapshotScanner

public TableSnapshotScanner(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.Path rootDir,
                            org.apache.hadoop.fs.Path restoreDir,
                            String snapshotName,
                            Scan scan)
                     throws IOException

Throws:

IOException

TableSnapshotScanner

public TableSnapshotScanner(org.apache.hadoop.conf.Configuration conf,
                            org.apache.hadoop.fs.Path rootDir,
                            org.apache.hadoop.fs.Path restoreDir,
                            String snapshotName,
                            Scan scan,
                            boolean snapshotAlreadyRestored)
                     throws IOException

Creates a TableSnapshotScanner.

Parameters:

conf - the configuration

rootDir - root directory for HBase.

restoreDir - a temporary directory to copy the snapshot files into. Current user should have write permissions to this directory, and this should not be a subdirectory of rootdir. The scanner deletes the contents of the directory once the scanner is closed.

snapshotName - the name of the snapshot to read from

scan - a Scan representing scan parameters

snapshotAlreadyRestored - true to indicate that snapshot has been restored.

Throws:

IOException - in case of error

Method Detail

openWithoutRestoringSnapshot

private void openWithoutRestoringSnapshot()
                                   throws IOException

Throws:

IOException

isValidRegion

private boolean isValidRegion(RegionInfo hri)

openWithRestoringSnapshot

private void openWithRestoringSnapshot()
                                throws IOException

Throws:

IOException

next

public Result next()
            throws IOException

Description copied from interface: ResultScanner

Grab the next row's worth of values. The scanner will return a Result.

Returns:

Result object if there is another row, null if the scanner is exhausted.

Throws:

IOException - e

cleanup

private void cleanup()

close

public void close()

Description copied from interface: ResultScanner

Closes the scanner and releases any resources it has allocated

renewLease

public boolean renewLease()

Description copied from interface: ResultScanner

Allow the client to renew the scanner's lease on the server.

Returns:

true if the lease was successfully renewed, false otherwise.