Field Detail

• LOG

  private static final org.apache.commons.logging.Log LOG

• fs

  private final org.apache.hadoop.fs.FileSystem fs

• oldFileDir

  private final org.apache.hadoop.fs.Path oldFileDir

• conf

  private final org.apache.hadoop.conf.Configuration conf

• cleanersChain

  protected List<T extends FileCleanerDelegate> cleanersChain

• params

  protected Map<String,Object> params

Constructor Detail

• CleanerChore

  public CleanerChore(String name,
              int sleepPeriod,
              Stoppable s,
              org.apache.hadoop.conf.Configuration conf,
              org.apache.hadoop.fs.FileSystem fs,
              org.apache.hadoop.fs.Path oldFileDir,
              String confKey)

• CleanerChore

  public CleanerChore(String name,
              int sleepPeriod,
              Stoppable s,
              org.apache.hadoop.conf.Configuration conf,
              org.apache.hadoop.fs.FileSystem fs,
              org.apache.hadoop.fs.Path oldFileDir,
              String confKey,
              Map<String,Object> params)

  Parameters:

  name - name of the chore being run

  sleepPeriod - the period of time to sleep between each run

  s - the stopper

  conf - configuration to use

  fs - handle to the FS

  oldFileDir - the path to the archived files

  confKey - configuration key for the classes to instantiate

  params - members could be used in cleaner

Method Detail

• validate

  protected abstract boolean validate(org.apache.hadoop.fs.Path file)

  Validate the file to see if it even belongs in the directory. If it is valid, then the file will go through the cleaner delegates, but otherwise the file is just deleted.

  Parameters:

  file - full Path of the file to be checked

  Returns:

  true if the file is valid, false otherwise

• initCleanerChain

  private void initCleanerChain(String confKey)

  Instantiate and initialize all the file cleaners set in the configuration

  Parameters:

  confKey - key to get the file cleaner classes from the configuration

• newFileCleaner

  private T newFileCleaner(String className,
                 org.apache.hadoop.conf.Configuration conf)

  A utility method to create new instances of LogCleanerDelegate based on the class name of the LogCleanerDelegate.

  Parameters:

  className - fully qualified class name of the LogCleanerDelegate

  conf -

  Returns:

  the new instance

• chore

  protected void chore()

  Description copied from class: ScheduledChore
  The task to execute on each scheduled execution of the Chore

  Specified by:

  chore in class ScheduledChore

• checkAndDeleteEntries

  private boolean checkAndDeleteEntries(org.apache.hadoop.fs.FileStatus[] entries)

  Loop over the given directory entries, and check whether they can be deleted. If an entry is itself a directory it will be recursively checked and deleted itself iff all subentries are deleted (and no new subentries are added in the mean time)

  Parameters:

  entries - directory entries to check

  Returns:

  true if all entries were successfully deleted

• checkAndDeleteDirectory

  boolean checkAndDeleteDirectory(org.apache.hadoop.fs.Path dir)

  Attempt to delete a directory and all files under that directory. Each child file is passed through the delegates to see if it can be deleted. If the directory has no children when the cleaners have finished it is deleted.

  If new children files are added between checks of the directory, the directory will not be deleted.

  Parameters:

  dir - directory to check

  Returns:

  true if the directory was deleted, false otherwise.

• checkAndDeleteFiles

  private boolean checkAndDeleteFiles(List<org.apache.hadoop.fs.FileStatus> files)

  Run the given files through each of the cleaners to see if it should be deleted, deleting it if necessary.

  Parameters:

  files - List of FileStatus for the files to check (and possibly delete)

  Returns:

  true iff successfully deleted all files

• cleanup

  public void cleanup()

  Description copied from class: ScheduledChore
  Override to run cleanup tasks when the Chore encounters an error and must stop running

  Overrides:

  cleanup in class ScheduledChore