Executor Service in Java

In the previous article we saw how we can implement basic multithreading in java. However to design efficient and reliable systems using multithreading we need to understand concepts pertaining to the topic of concurrency to actually build systems which can benefit from multithreading. Multithreading at its core allows us to execute tasks simultaneously with the help of multiple threads working at the same time. So if we want to execute a lot of tasks we should be creating as many threads as possible to finish the tasks quickly right? No. Creating threads is also an expensive process for the machine to do. We need to find the right balance between having enough threads so that our tasks get executed quickly and the number of threads we are using should be efficient for that system. ExecutorService is a way for us to manage and execute concurrent tasks by using threads, making it simpler to create and control the thread execution process. Let us look at how we can achieve this.

Imagine the example of the taxis and passengers in the previous article. If we wish to drop all the arriving passengers to their hotels quicker we would have to hire more taxi drivers. Hiring too many taxi drivers is obviously an expensive task. In this if we were to hire an optimal amount of taxi drivers and a manager to manage the scheduling of the rides of the customers and handling the waiting ones then this process can be made faster and efficient in an resourceful manner. This is how a thread pool works.

ExecutorService allows us to create Thread Pools. A thread pool is a group of threads which are managed by the ExecutorService. New tasks can be submitted for their execution to the service itself. Once the tasks are submitted they are queued up in a blocking queue. The blocking queue enqueues the tasks if all the threads in the thread pool are executing existing tasks and dequeues them once a thread gets free and the thread begins to execute the task which is dequeued.

Note: To use the executor service you will have to import the concurrency package by java

import java.util.concurrent;

Fixed Thread Pool

Now if we choose to use a fixed thread pool size we need to be mindful of the size. Usually it is recommended to use a pool size equivalent to or just under the available threads for that machine. A unique possibility is that if we are performing tasks which require the threads to be in a waiting state, then we can have more threads as otherwise there will be no threads to execute new tasks if all the threads are in waiting state and resources will not be used effectively. Cases like this can be observed while using asynchronous programming.

ExecutorService myService = Executors.newFixedThreadPool(10);
    for (int i=0; i < 100; i++) {
        myService.execute(new Tasks());
    }

Cached Thread Pool

A cached thread pool is very similar to a fixed thread pool just the blocking queue is replaced by a synchronous queue. A synchronous queue can hold only one task at a time. When it recieves a new task it looks for any free threads in the thread pool. If it finds a free thread then the task is executed by that particular thread otherwise a new thread is created to execute that task. Since this may lead to the creation of a large number of threads, the newly created threads are terminated if they are found to be idle for a specific amount of time to avoid having too many threads.

ExecutorService myService = Executors.newCachedThreadPool();

Single Thread Executor

A single thread executor is a thread pool with a single thread. It fetches new tasks and executes them from the blocking queue. If the thread is terminated in the process of execution of a task it is recreated. This way there always exists one thread in the thread pool.

Handling Rejections

When all the threads are busy the new tasks are stored in the blocking queue, but the blocking queue also must have its limitations right. What would happen to a new task if all the threads are busy and the blocking queue is also full. If new thread creation is not an option then the task is rejected by the thread pool, this rejection can happen in one of four ways – Abort Policy, Discard Policy, Caller-Runs Policy and Discard Oldest Policy. Let us see how each of these handles the rejection of tasks.

• • Abort Policy: Simply throws a RejectedExecutionException.
  • Discard Policy: Discards the task.
  • Caller-Runs Policy: Executed the task on the caller’s thread itlsef.
  • Discard Oldest Policy: Discards the oldest task in the blocking queue and the new task is added to the queue.

Shutdown Initiations

We have seen how to start an ExecutorService and how it works, but how do we shut it down? Since the threads are executing the tasks wouldn’t the shutdown affect the execution of those tasks? We can shutdown an ExecutorService using the following command.

ExecutorService myService = Executors.newFixedThreadPool(15);
for (int i=0; i<50; i++) {
    myService.execute(new Task());
}

myService.shutdown();

When we use the command myService.shutdown(), it only initiates a shutdown. Which means that all the tasks being executed by the threads and all the tasks in the blocking queue will be completed first and then it will shutdown. In this duration if any new tasks are submitted to the thread pool  a RejectionExecutionException will be thrown.

List<Runnable> queuedTasks = myService.shutdownNow();

The above command is used when we want to shutdown immediately after the threads finish executing their current tasks. The tasks queued up in the blocking queue are returned as a list.

The status of the shutdown can be checked with the two commands – isShutdown() and isTerminated(). isShutdown() is used to check if the shutdown process has been initiated, however there is no way to know if the entire service has been shutdown only using isShutdown(), for that we will have to use the isTerminated() function as it is used to check if all the tasks are completed and hence the service has been shutdown.

// returns true if the shutdown has been initiated.
myService.isShutdown();

// returns true if all the tasks are completed, including the queues ones.
myService.isTerminated();