HFileCorruptionChecker (Apache HBase 1.2.12 API)

java.lang.Object
- org.apache.hadoop.hbase.util.hbck.HFileCorruptionChecker

```
@InterfaceAudience.Private
public class HFileCorruptionChecker
extends Object
```
This class marches through all of the region's hfiles and verifies that they are all valid files. One just needs to instantiate the class, use checkTables(List<Path>) and then retrieve the corrupted hfiles (and quarantined files if in quarantining mode) The implementation currently parallelizes at the regionDir level.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

private class HFileCorruptionChecker.RegionDirChecker
An individual work item for parallelized regiondir processing.

Nested Classes
Modifier and Type	Class and Description
`private class`	`HFileCorruptionChecker.RegionDirChecker` An individual work item for parallelized regiondir processing.

Field Summary

Fields
Modifier and Type	Field and Description
`(package private) CacheConfig`	`cacheConf`
`(package private) org.apache.hadoop.conf.Configuration`	`conf`
`(package private) Set<org.apache.hadoop.fs.Path>`	`corrupted`
`(package private) ExecutorService`	`executor`
`(package private) Set<org.apache.hadoop.fs.Path>`	`failures`
`(package private) org.apache.hadoop.fs.FileSystem`	`fs`
`(package private) AtomicInteger`	`hfilesChecked`
`(package private) boolean`	`inQuarantineMode`
`private static org.apache.commons.logging.Log`	`LOG`
`(package private) Set<org.apache.hadoop.fs.Path>`	`missing`
`(package private) Set<org.apache.hadoop.fs.Path>`	`quarantined`

Constructor Summary

Constructors
Constructor and Description
`HFileCorruptionChecker(org.apache.hadoop.conf.Configuration conf, ExecutorService executor, boolean quarantine)`

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`checkColFamDir(org.apache.hadoop.fs.Path cfDir)` Check all files in a column family dir.
`protected void`	`checkHFile(org.apache.hadoop.fs.Path p)` Checks a path to see if it is a valid hfile.
`protected void`	`checkRegionDir(org.apache.hadoop.fs.Path regionDir)` Check all column families in a region dir.
`(package private) void`	`checkTableDir(org.apache.hadoop.fs.Path tableDir)` Check all the regiondirs in the specified tableDir
`void`	`checkTables(Collection<org.apache.hadoop.fs.Path> tables)` Check the specified table dirs for bad hfiles.
`(package private) org.apache.hadoop.fs.Path`	`createQuarantinePath(org.apache.hadoop.fs.Path hFile)` Given a path, generates a new path to where we move a corrupted hfile (bad trailer, no trailer).
`Collection<org.apache.hadoop.fs.Path>`	`getCorrupted()`
`Collection<org.apache.hadoop.fs.Path>`	`getFailures()`
`int`	`getHFilesChecked()`
`Collection<org.apache.hadoop.fs.Path>`	`getMissing()`
`Collection<org.apache.hadoop.fs.Path>`	`getQuarantined()`
`void`	`report(HBaseFsck.ErrorReporter out)` Print a human readable summary of hfile quarantining operations.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.apache.commons.logging.Log LOG

conf

final org.apache.hadoop.conf.Configuration conf

fs

final org.apache.hadoop.fs.FileSystem fs

cacheConf
```
final CacheConfig cacheConf
```

executor
```
final ExecutorService executor
```

corrupted

final Set<org.apache.hadoop.fs.Path> corrupted

failures

final Set<org.apache.hadoop.fs.Path> failures

quarantined

final Set<org.apache.hadoop.fs.Path> quarantined

missing

final Set<org.apache.hadoop.fs.Path> missing

inQuarantineMode
```
final boolean inQuarantineMode
```

hfilesChecked
```
final AtomicInteger hfilesChecked
```

Constructor Detail

HFileCorruptionChecker

public HFileCorruptionChecker(org.apache.hadoop.conf.Configuration conf,
                      ExecutorService executor,
                      boolean quarantine)
                       throws IOException

Throws:: IOException

Method Detail
- checkHFile
```
protected void checkHFile(org.apache.hadoop.fs.Path p)
                   throws IOException
```
  Checks a path to see if it is a valid hfile.
  
  Parameters:
  p - full Path to an HFile
  
  Throws:
  
  IOException - This is a connectivity related exception
- createQuarantinePath
```
org.apache.hadoop.fs.Path createQuarantinePath(org.apache.hadoop.fs.Path hFile)
                                         throws IOException
```
  Given a path, generates a new path to where we move a corrupted hfile (bad trailer, no trailer).
  
  Parameters:
  hFile - Path to a corrupt hfile (assumes that it is HBASE_DIR/ table /region/cf/file)
  
  Returns:
  path to where corrupted files are stored. This should be HBASE_DIR/.corrupt/table/region/cf/file.
  
  Throws:
  
  IOException
- checkColFamDir
```
protected void checkColFamDir(org.apache.hadoop.fs.Path cfDir)
                       throws IOException
```
  Check all files in a column family dir.
  
  Parameters:
  cfDir - column family directory
  
  Throws:
  
  IOException
- checkRegionDir
```
protected void checkRegionDir(org.apache.hadoop.fs.Path regionDir)
                       throws IOException
```
  Check all column families in a region dir.
  
  Parameters:
  regionDir - region directory
  
  Throws:
  
  IOException
- checkTableDir
```
void checkTableDir(org.apache.hadoop.fs.Path tableDir)
             throws IOException
```
  Check all the regiondirs in the specified tableDir
  
  Parameters:
  tableDir - path to a table
  
  Throws:
  
  IOException
- checkTables
```
public void checkTables(Collection<org.apache.hadoop.fs.Path> tables)
                 throws IOException
```
  Check the specified table dirs for bad hfiles.
  
  Throws:
  
  IOException
- getFailures
```
public Collection<org.apache.hadoop.fs.Path> getFailures()
```
  Returns:
  the set of check failure file paths after checkTables is called.
- getCorrupted
```
public Collection<org.apache.hadoop.fs.Path> getCorrupted()
```
  Returns:
  the set of corrupted file paths after checkTables is called.
- getHFilesChecked
```
public int getHFilesChecked()
```
  Returns:
  number of hfiles checked in the last HfileCorruptionChecker run
- getQuarantined
```
public Collection<org.apache.hadoop.fs.Path> getQuarantined()
```
  Returns:
  the set of successfully quarantined paths after checkTables is called.
- getMissing
```
public Collection<org.apache.hadoop.fs.Path> getMissing()
```
  Returns:
  the set of paths that were missing. Likely due to deletion/moves from compaction or flushes.
- report
```
public void report(HBaseFsck.ErrorReporter out)
```
  Print a human readable summary of hfile quarantining operations.
  
  Parameters:
  out -

Class HFileCorruptionChecker

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

conf

fs

cacheConf

executor

corrupted

failures

quarantined

missing

inQuarantineMode

hfilesChecked

Constructor Detail

HFileCorruptionChecker

Method Detail

checkHFile

createQuarantinePath

checkColFamDir

checkRegionDir

checkTableDir

checkTables

getFailures

getCorrupted

getHFilesChecked

getQuarantined

getMissing

report