Field Detail

• LOG

  private static final org.apache.commons.logging.Log LOG

• SEPARATOR

  private static final String SEPARATOR

  See Also:

  Constant Field Values

• DEFAULT_RETRIES_NUMBER

  private static final int DEFAULT_RETRIES_NUMBER

  Number of retries in case of fs operation failure

  See Also:

  Constant Field Values

Constructor Detail

• HFileArchiver

  private HFileArchiver()

Method Detail

• archiveRegion

  public static void archiveRegion(org.apache.hadoop.conf.Configuration conf,
                   org.apache.hadoop.fs.FileSystem fs,
                   HRegionInfo info)
                            throws IOException

  Cleans up all the files for a HRegion by archiving the HFiles to the archive directory

  Parameters:

  conf - the configuration to use

  fs - the file system object

  info - HRegionInfo for region to be deleted

  Throws:

  IOException

• archiveRegion

  public static boolean archiveRegion(org.apache.hadoop.fs.FileSystem fs,
                      org.apache.hadoop.fs.Path rootdir,
                      org.apache.hadoop.fs.Path tableDir,
                      org.apache.hadoop.fs.Path regionDir)
                               throws IOException

  Remove an entire region from the table directory via archiving the region's hfiles.

  Parameters:

  fs - FileSystem from which to remove the region

  rootdir - Path to the root directory where hbase files are stored (for building the archive path)

  tableDir - Path to where the table is being stored (for building the archive path)

  regionDir - Path to where a region is being stored (for building the archive path)

  Returns:

  true if the region was sucessfully deleted. false if the filesystem operations could not complete.

  Throws:

  IOException - if the request cannot be completed

• archiveFamily

  public static void archiveFamily(org.apache.hadoop.fs.FileSystem fs,
                   org.apache.hadoop.conf.Configuration conf,
                   HRegionInfo parent,
                   org.apache.hadoop.fs.Path tableDir,
                   byte[] family)
                            throws IOException

  Remove from the specified region the store files of the specified column family, either by archiving them or outright deletion

  Parameters:

  fs - the filesystem where the store files live

  conf - Configuration to examine to determine the archive directory

  parent - Parent region hosting the store files

  tableDir - Path to where the table is being stored (for building the archive path)

  family - the family hosting the store files

  Throws:

  IOException - if the files could not be correctly disposed.

• archiveStoreFiles

  public static void archiveStoreFiles(org.apache.hadoop.conf.Configuration conf,
                       org.apache.hadoop.fs.FileSystem fs,
                       HRegionInfo regionInfo,
                       org.apache.hadoop.fs.Path tableDir,
                       byte[] family,
                       Collection<StoreFile> compactedFiles)
                                throws IOException

  Remove the store files, either by archiving them or outright deletion

  Parameters:

  conf - Configuration to examine to determine the archive directory

  fs - the filesystem where the store files live

  regionInfo - HRegionInfo of the region hosting the store files

  family - the family hosting the store files

  compactedFiles - files to be disposed of. No further reading of these files should be attempted; otherwise likely to cause an IOException

  Throws:

  IOException - if the files could not be correctly disposed.

• archiveStoreFile

  public static void archiveStoreFile(org.apache.hadoop.conf.Configuration conf,
                      org.apache.hadoop.fs.FileSystem fs,
                      HRegionInfo regionInfo,
                      org.apache.hadoop.fs.Path tableDir,
                      byte[] family,
                      org.apache.hadoop.fs.Path storeFile)
                               throws IOException

  Archive the store file

  Parameters:

  fs - the filesystem where the store files live

  regionInfo - region hosting the store files

  conf - Configuration to examine to determine the archive directory

  tableDir - Path to where the table is being stored (for building the archive path)

  family - the family hosting the store files

  storeFile - file to be archived

  Throws:

  IOException - if the files could not be correctly disposed.

• resolveAndArchive

  private static boolean resolveAndArchive(org.apache.hadoop.fs.FileSystem fs,
                          org.apache.hadoop.fs.Path baseArchiveDir,
                          Collection<HFileArchiver.File> toArchive)
                                    throws IOException

  Archive the given files and resolve any conflicts with existing files via appending the time archiving started (so all conflicts in the same group have the same timestamp appended).

  If any of the passed files to archive are directories, archives all the files under that directory. Archive directory structure for children is the base archive directory name + the parent directory and is built recursively is passed files are directories themselves.

  Parameters:

  fs - FileSystem on which to archive the files

  baseArchiveDir - base archive directory to archive the given files

  toArchive - files to be archived

  Returns:

  true on success, false otherwise

  Throws:

  IOException - on unexpected failure

• resolveAndArchive

  private static List<HFileArchiver.File> resolveAndArchive(org.apache.hadoop.fs.FileSystem fs,
                                           org.apache.hadoop.fs.Path baseArchiveDir,
                                           Collection<HFileArchiver.File> toArchive,
                                           long start)
                                                     throws IOException

  Resolve any conflict with an existing archive file via timestamp-append renaming of the existing file and then archive the passed in files.

  Parameters:

  fs - FileSystem on which to archive the files

  baseArchiveDir - base archive directory to store the files. If any of the files to archive are directories, will append the name of the directory to the base archive directory name, creating a parallel structure.

  toArchive - files/directories that need to be archvied

  start - time the archiving started - used for resolving archive conflicts.

  Returns:

  the list of failed to archive files.

  Throws:

  IOException - if an unexpected file operation exception occured

• resolveAndArchiveFile

  private static boolean resolveAndArchiveFile(org.apache.hadoop.fs.Path archiveDir,
                              HFileArchiver.File currentFile,
                              String archiveStartTime)
                                        throws IOException

  Attempt to archive the passed in file to the archive directory.

  If the same file already exists in the archive, it is moved to a timestamped directory under the archive directory and the new file is put in its place.

  Parameters:

  archiveDir - Path to the directory that stores the archives of the hfiles

  currentFile - Path to the original HFile that will be archived

  archiveStartTime - time the archiving started, to resolve naming conflicts

  Returns:

  true if the file is successfully archived. false if there was a problem, but the operation still completed.

  Throws:

  IOException - on failure to complete FileSystem operations.

• deleteRegionWithoutArchiving

  private static boolean deleteRegionWithoutArchiving(org.apache.hadoop.fs.FileSystem fs,
                                     org.apache.hadoop.fs.Path regionDir)
                                               throws IOException

  Without regard for backup, delete a region. Should be used with caution.

  Parameters:

  regionDir - Path to the region to be deleted.

  fs - FileSystem from which to delete the region

  Returns:

  true on successful deletion, false otherwise

  Throws:

  IOException - on filesystem operation failure

• deleteStoreFilesWithoutArchiving

  private static void deleteStoreFilesWithoutArchiving(Collection<StoreFile> compactedFiles)
                                                throws IOException

  Just do a simple delete of the given store files

  A best effort is made to delete each of the files, rather than bailing on the first failure.

  This method is preferable to #deleteFilesWithoutArchiving(Collection) since it consumes less resources, but is limited in terms of usefulness

  Parameters:

  compactedFiles - store files to delete from the file system.

  Throws:

  IOException - if a file cannot be deleted. All files will be attempted to deleted before throwing the exception, rather than failing at the first file.