Package org.apache.hadoop.hbase

Class ChoreService

java.lang.Object
org.apache.hadoop.hbase.ChoreService
@Private public class ChoreService extends Object
ChoreService is a service that can be used to schedule instances of ScheduledChore to run periodically while sharing threads. The ChoreService is backed by a ScheduledThreadPoolExecutor whose core pool size changes dynamically depending on the number of ScheduledChore scheduled. All of the threads in the core thread pool of the underlying ScheduledThreadPoolExecutor are set to be daemon threads.

The ChoreService provides the ability to schedule, cancel, and trigger instances of ScheduledChore. The ChoreService also provides the ability to check on the status of scheduled chores. The number of threads used by the ChoreService changes based on the scheduling load and whether or not the scheduled chores are executing on time. As more chores are scheduled, there may be a need to increase the number of threads if it is noticed that chores are no longer meeting their scheduled start times. On the other hand, as chores are cancelled, an attempt is made to reduce the number of running threads to see if chores can still meet their start times with a smaller thread pool.

When finished with a ChoreService it is good practice to call shutdown(). Calling this method ensures that all scheduled chores are cancelled and cleaned up properly.

  • Field Details

    • LOG

      private static final org.slf4j.Logger LOG

    • MIN_CORE_POOL_SIZE

      @Private public static final int MIN_CORE_POOL_SIZE
      The minimum number of threads in the core pool of the underlying ScheduledThreadPoolExecutor
      See Also:

    • CHORE_SERVICE_INITIAL_POOL_SIZE

      public static final String CHORE_SERVICE_INITIAL_POOL_SIZE
      The initial number of threads in the core pool for the ChoreService.
      See Also:

    • DEFAULT_CHORE_SERVICE_INITIAL_POOL_SIZE

      public static final int DEFAULT_CHORE_SERVICE_INITIAL_POOL_SIZE
      See Also:

    • scheduler

      private final ScheduledThreadPoolExecutor scheduler
      This thread pool is used to schedule all of the Chores

    • scheduledChores

      private final HashMap<ScheduledChore,ScheduledFuture<?>> scheduledChores
      Maps chores to their futures. Futures are used to control a chore's schedule

    • choresMissingStartTime

      private final HashMap<ScheduledChore,Boolean> choresMissingStartTime
      Maps chores to Booleans which indicate whether or not a chore has caused an increase in the core pool size of the ScheduledThreadPoolExecutor. Each chore should only be allowed to increase the core pool size by 1 (otherwise a single long running chore whose execution is longer than its period would be able to spawn too many threads).

    • coreThreadPoolPrefix

      private final String coreThreadPoolPrefix
      The coreThreadPoolPrefix is the prefix that will be applied to all threads within the ScheduledThreadPoolExecutor. The prefix is typically related to the Server that the service is running on. The prefix is useful because it allows us to monitor how the thread pool of a particular service changes over time VIA thread dumps.

  • Constructor Details

    • ChoreService

      @Private public ChoreService(String coreThreadPoolPrefix)
      Parameters:
      coreThreadPoolPrefix - Prefix that will be applied to the Thread name of all threads spawned by this service

    • ChoreService

      public ChoreService(String coreThreadPoolPrefix, boolean jitter)
      Parameters:
      coreThreadPoolPrefix - Prefix that will be applied to the Thread name of all threads spawned by this service
      jitter - Should chore service add some jitter for all of the scheduled chores. When set to true this will add -10% to 10% jitter.

    • ChoreService

      public ChoreService(String coreThreadPoolPrefix, int corePoolSize, boolean jitter)
      Parameters:
      coreThreadPoolPrefix - Prefix that will be applied to the Thread name of all threads spawned by this service
      corePoolSize - The initial size to set the core pool of the ScheduledThreadPoolExecutor to during initialization. The default size is 1, but specifying a larger size may be beneficial if you know that 1 thread will not be enough.
      jitter - Should chore service add some jitter for all of the scheduled chores. When set to true this will add -10% to 10% jitter.

  • Method Details

    • scheduleChore

      public boolean scheduleChore(ScheduledChore chore)
      Schedule a chore.
      Parameters:
      chore - Chore to be scheduled. If the chore is already scheduled with another ChoreService instance, that schedule will be cancelled (i.e. a Chore can only ever be scheduled with a single ChoreService instance).
      Returns:
      true when the chore was successfully scheduled. false when the scheduling failed (typically occurs when a chore is scheduled during shutdown of service)

    • rescheduleChore

      private void rescheduleChore(ScheduledChore chore, boolean immediately)
      Parameters:
      chore - The Chore to be rescheduled. If the chore is not scheduled with this ChoreService yet then this call is equivalent to a call to scheduleChore.

    • cancelChore

      void cancelChore(ScheduledChore chore)
      Cancel any ongoing schedules that this chore has with the implementer of this interface.

      Call ScheduledChore.cancel() to cancel a ScheduledChore, in ScheduledChore.cancel() method we will call this method to remove the ScheduledChore from this ChoreService.

    • cancelChore

      void cancelChore(ScheduledChore chore, boolean mayInterruptIfRunning)
      Cancel any ongoing schedules that this chore has with the implementer of this interface.

      Call ScheduledChore.cancel(boolean) to cancel a ScheduledChore, in ScheduledChore.cancel(boolean) method we will call this method to remove the ScheduledChore from this ChoreService.

    • isChoreScheduled

      @Private public boolean isChoreScheduled(ScheduledChore chore)
      Returns true when the chore is scheduled with the implementer of this interface

    • triggerNow

      void triggerNow(ScheduledChore chore)
      This method tries to execute the chore immediately. If the chore is executing at the time of this call, the chore will begin another execution as soon as the current execution finishes

    • getNumberOfScheduledChores

      int getNumberOfScheduledChores()
      Returns number of chores that this service currently has scheduled

    • getNumberOfChoresMissingStartTime

      int getNumberOfChoresMissingStartTime()
      Return number of chores that this service currently has scheduled that are missing their scheduled start time

    • getCorePoolSize

      int getCorePoolSize()
      Returns number of threads in the core pool of the underlying ScheduledThreadPoolExecutor

    • requestCorePoolIncrease

      private boolean requestCorePoolIncrease()
      Represents a request to increase the number of core pool threads. Typically a request originates from the fact that the current core pool size is not sufficient to service all of the currently running Chores
      Returns:
      true when the request to increase the core pool size succeeds

    • requestCorePoolDecrease

      private void requestCorePoolDecrease()
      Represents a request to decrease the number of core pool threads. Typically a request originates from the fact that the current core pool size is more than sufficient to service the running Chores.

    • onChoreMissedStartTime

      void onChoreMissedStartTime(ScheduledChore chore)
      A callback that tells the implementer of this interface that one of the scheduled chores is missing its start time. The implication of a chore missing its start time is that the service's current means of scheduling may not be sufficient to handle the number of ongoing chores (the other explanation is that the chore's execution time is greater than its scheduled period). The service should try to increase its concurrency when this callback is received.
      Parameters:
      chore - The chore that missed its start time

    • shutdown

      public void shutdown()
      Shut down the service. Any chores that are scheduled for execution will be cancelled. Any chores in the middle of execution will be interrupted and shutdown. This service will be unusable after this method has been called (i.e. future scheduling attempts will fail).

      Notice that, this will only clean the chore from this ChoreService but you could still schedule the chore with other ChoreService.

    • isShutdown

      public boolean isShutdown()
      Returns true when the service is shutdown and thus cannot be used anymore

    • isTerminated

      public boolean isTerminated()
      Returns true when the service is shutdown and all threads have terminated

    • cancelAllChores

      private void cancelAllChores(boolean mayInterruptIfRunning)

    • printChoreDetails

      private void printChoreDetails(String header, ScheduledChore chore)
      Prints a summary of important details about the chore. Used for debugging purposes

    • printChoreServiceDetails

      private void printChoreServiceDetails(String header)
      Prints a summary of important details about the service. Used for debugging purposes