SnapshotFileCache (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache

All Implemented Interfaces:

Stoppable
```
@InterfaceAudience.Private
@InterfaceStability.Evolving
public class SnapshotFileCache
extends Object
implements Stoppable
```
Intelligently keep track of all the files for all the snapshots.
A cache of files is kept to avoid querying the FileSystem frequently. If there is a cache miss the directory modification time is used to ensure that we don't rescan directories that we already have in cache. We only check the modification times of the snapshot directories (/hbase/.snapshot/[snapshot_name]) to determine if the files need to be loaded into the cache.
New snapshots will be added to the cache and deleted snapshots will be removed when we refresh the cache. If the files underneath a snapshot directory are changed, but not the snapshot itself, we will ignore updates to that snapshot's files.
This is sufficient because each snapshot has its own directory and is added via an atomic rename once, when the snapshot is created. We don't need to worry about the data in the snapshot being run.
Further, the cache is periodically refreshed ensure that files in snapshots that were deleted are also removed from the cache.
A SnapshotFileCache.SnapshotFileInspector must be passed when creating this to allow extraction of files under /hbase/.snapshot/[snapshot name] directory, for each snapshot. This allows you to only cache files under, for instance, all the logs in the .logs directory or all the files under all the regions.
this also considers all running snapshots (those under /hbase/.snapshot/.tmp) as valid snapshots and will attempt to cache files from those snapshots as well.
Queries about a given file are thread-safe with respect to multiple queries and cache refreshes.

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`class`	`SnapshotFileCache.RefreshCacheTask` Simple helper task that just periodically attempts to refresh the cache
`private static class`	`SnapshotFileCache.SnapshotDirectoryInfo` Information about a snapshot directory
`(package private) static interface`	`SnapshotFileCache.SnapshotFileInspector`

Field Summary

Fields
Modifier and Type	Field and Description
`private Set<String>`	`cache`
`private SnapshotFileCache.SnapshotFileInspector`	`fileInspector`
`private org.apache.hadoop.fs.FileSystem`	`fs`
`private long`	`lastModifiedTime`
`private static org.apache.commons.logging.Log`	`LOG`
`private Timer`	`refreshTimer`
`private org.apache.hadoop.fs.Path`	`snapshotDir`
`private Map<String,SnapshotFileCache.SnapshotDirectoryInfo>`	`snapshots` This is a helper map of information about the snapshot directories so we don't need to rescan them if they haven't changed since the last time we looked.
`private boolean`	`stop`

Constructor Summary

Constructors
Constructor and Description
`SnapshotFileCache(org.apache.hadoop.conf.Configuration conf, long cacheRefreshPeriod, String refreshThreadName, SnapshotFileCache.SnapshotFileInspector inspectSnapshotFiles)` Create a snapshot file cache for all snapshots under the specified [root]/.snapshot on the filesystem.
`SnapshotFileCache(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path rootDir, long cacheRefreshPeriod, long cacheRefreshDelay, String refreshThreadName, SnapshotFileCache.SnapshotFileInspector inspectSnapshotFiles)` Create a snapshot file cache for all snapshots under the specified [root]/.snapshot on the filesystem

Method Summary

Methods
Modifier and Type	Method and Description
`(package private) List<String>`	`getSnapshotsInProgress()`
`Iterable<org.apache.hadoop.fs.FileStatus>`	`getUnreferencedFiles(Iterable<org.apache.hadoop.fs.FileStatus> files, SnapshotManager snapshotManager)` Check to see if any of the passed file names is contained in any of the snapshots.
`boolean`	`isStopped()`
`private void`	`refreshCache()`
`void`	`stop(String why)` Stop this service.
`void`	`triggerCacheRefreshForTesting()` Trigger a cache refresh, even if its before the next cache refresh.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
private static final org.apache.commons.logging.Log LOG
```
  - stop
```
private volatile boolean stop
```
  - fs
```
private final org.apache.hadoop.fs.FileSystem fs
```
  - fileInspector
```
private final SnapshotFileCache.SnapshotFileInspector fileInspector
```
  - snapshotDir
```
private final org.apache.hadoop.fs.Path snapshotDir
```
  - cache
```
private final Set<String> cache
```
  - snapshots
```
private final Map<String,SnapshotFileCache.SnapshotDirectoryInfo> snapshots
```
    This is a helper map of information about the snapshot directories so we don't need to rescan them if they haven't changed since the last time we looked.
  - refreshTimer
```
private final Timer refreshTimer
```
  - lastModifiedTime
```
private long lastModifiedTime
```
- Constructor Detail
  - SnapshotFileCache
```
public SnapshotFileCache(org.apache.hadoop.conf.Configuration conf,
                 long cacheRefreshPeriod,
                 String refreshThreadName,
                 SnapshotFileCache.SnapshotFileInspector inspectSnapshotFiles)
                  throws IOException
```
    Create a snapshot file cache for all snapshots under the specified [root]/.snapshot on the filesystem.
    Immediately loads the file cache.
    
    Parameters:
    conf - to extract the configured FileSystem where the snapshots are stored and hbase root directory
    cacheRefreshPeriod - frequency (ms) with which the cache should be refreshed
    refreshThreadName - name of the cache refresh thread
    inspectSnapshotFiles - Filter to apply to each snapshot to extract the files.
    
    Throws:
    
    IOException - if the FileSystem or root directory cannot be loaded
  - SnapshotFileCache
```
public SnapshotFileCache(org.apache.hadoop.fs.FileSystem fs,
                 org.apache.hadoop.fs.Path rootDir,
                 long cacheRefreshPeriod,
                 long cacheRefreshDelay,
                 String refreshThreadName,
                 SnapshotFileCache.SnapshotFileInspector inspectSnapshotFiles)
```
    Create a snapshot file cache for all snapshots under the specified [root]/.snapshot on the filesystem
    
    Parameters:
    fs - FileSystem where the snapshots are stored
    rootDir - hbase root directory
    cacheRefreshPeriod - period (ms) with which the cache should be refreshed
    cacheRefreshDelay - amount of time to wait for the cache to be refreshed
    refreshThreadName - name of the cache refresh thread
    inspectSnapshotFiles - Filter to apply to each snapshot to extract the files.
- Method Detail
  - triggerCacheRefreshForTesting
```
public void triggerCacheRefreshForTesting()
```
    Trigger a cache refresh, even if its before the next cache refresh. Does not affect pending cache refreshes.
    Blocks until the cache is refreshed.
    Exposed for TESTING.
  - getUnreferencedFiles
```
public Iterable<org.apache.hadoop.fs.FileStatus> getUnreferencedFiles(Iterable<org.apache.hadoop.fs.FileStatus> files,
                                                             SnapshotManager snapshotManager)
                                                               throws IOException
```
    Check to see if any of the passed file names is contained in any of the snapshots. First checks an in-memory cache of the files to keep. If its not in the cache, then the cache is refreshed and the cache checked again for that file. This ensures that we never return files that exist.
    Note this may lead to periodic false positives for the file being referenced. Periodically, the cache is refreshed even if there are no requests to ensure that the false negatives get removed eventually. For instance, suppose you have a file in the snapshot and it gets loaded into the cache. Then at some point later that snapshot is deleted. If the cache has not been refreshed at that point, cache will still think the file system contains that file and return true, even if it is no longer present (false positive). However, if the file never was on the filesystem, we will never find it and always return false.
    
    Parameters:
    files - file to check, NOTE: Relies that files are loaded from hdfs before method is called (NOT LAZY)
    
    Returns:
    unReferencedFiles the collection of files that do not have snapshot references
    
    Throws:
    
    IOException - if there is an unexpected error reaching the filesystem.
  - refreshCache
```
private void refreshCache()
                   throws IOException
```
    Throws:
    
    IOException
  - getSnapshotsInProgress
```
List<String> getSnapshotsInProgress()
                              throws IOException
```
    Throws:
    
    IOException
  - stop
```
public void stop(String why)
```
    Description copied from interface: Stoppable
    
    Stop this service.
    
    Specified by:
    
    stop in interface Stoppable
    
    Parameters:
    why - Why we're stopping.
  - isStopped
```
public boolean isStopped()
```
    Specified by:
    
    isStopped in interface Stoppable
    
    Returns:
    True if Stoppable.stop(String) has been closed.

Class SnapshotFileCache

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

stop

fs

fileInspector

snapshotDir

cache

snapshots

refreshTimer

lastModifiedTime

Constructor Detail

SnapshotFileCache

SnapshotFileCache

Method Detail

triggerCacheRefreshForTesting

getUnreferencedFiles

refreshCache

getSnapshotsInProgress

stop

isStopped