Beginners Approach to Multithreading in Java Spring Boot

Brien Gillen Software Engineering

Multithreading is similar to multitasking, with the difference that it enables the running of multiple threads simultaneously, rather than processes. In Java, a thread refers to a sequence of programmed instructions which the operating system’s scheduler manages. Threads can be used to perform complex tasks in the background without interrupting the main program. It is worth noting that a thread shares common memory across the application with other threads, whereas a process is independent and doesn’t share memory.

There are many reasons to write a multithreaded application, however some of the most common are:

  • to save time by performing multiple operations at once
  • to not block the user(s) while one operation is performed
  • if an exception occurs in one thread other threads are not affected
  • CPU usage can be maximised by using two or more threads at the same time
  • response time can be reduced by breaking up larger problems into smaller, faster computations

Thread States

The Thread.getState() method can be used to check a Thread’s state. Different states of a Thread are described in the Thread.State enum. They are:

  • NEW – a new thread instance that has not yet started
  • RUNNABLE – a running thread. It is called runnable because at any given time it could be either running or waiting for the next time interval from the scheduler. A thread transitions from NEW to RUNNABLE when the Thread.start() method is called on it
  • BLOCKED – a running thread becomes BLOCKED if it needs to enter a synchronized section but cannot due to another thread holding the monitor of this section
  • WAITING – a thread enters this state if it is waiting for another thread to perform a particular action. For instance if the Object.wait() method is called on the monitor or a Thread.join() method is called on another thread
  • TIMED_WAITING – similar to WAITING but a thread enters this state after calling timed versions of Thread.sleep(), Object.wait() or Thread.join()
  • TERMINATED – a thread has completed the execution of its Runnable.run() method and terminated

Synchronized Methods

The synchronized keyword before a block means that any thread entering this block has to acquire the monitor. If the monitor is already acquired by another thread, the former thread will enter the BLOCKED state and wait until the monitor is released. The monitor for a synchronized method can change depending on the style of method:

  • A simple synchronized block within a method takes the parameter as the monitor
    • synchronized(object) {…}
  • A synchronized instance method takes the instance itself as the monitor
    • synchronized void instanceMethod() {…}
  • A static synchronized method has the Class object act as the monitor
    • static synchronized void staticMethod() {…}

General Java Multithreading

At any given time you can run as many threads as cores available in your CPU(s). You should considered, however, how your code operates before choosing how many threads you want to have available in your application:

  • For CPU Intensive tasks: threads should be limited to at most the number of cores in the system, if other processes also require threads we may need to reduce this number
  • For Network or I/O intensive tasks: if threads are making HTTP calls and waiting on a response or slow reads/writes to a disk we may opt to have more threads than the number of cores available as cores are not limiting the speed

If the code uses more threads than cores available then the OS will schedule these threads accordingly.

Dual Threaded Applications

A single additional thread can be created using an ExecutorService created from the Executors.newSingleThreadExecutor() static method. The ExecutorService contains the execute() method that takes an input which can be any class implementing the Runnable interface or a lambda expression.
The thread can be kicked off using an Executor at any point of the application run. An example might be to load cached data as a Spring application boots up. To do this the Executor call would happen in the main method of the Spring Boot application, prior to the SpringApplication.run(). Dual threading is also useful for preforming a series of steps that must be done sequentially, while also wanting to do another task.

Multithreaded Java Applications

The ExecutorService interface has three standard implementations:

  • ThreadPoolExecutor – used for executing tasks using a pool of threads. Once a thread is finished executing a task it goes back into the pool. If all threads in the pool are busy then the task has to wait for it’s turn.
  • ScheduledThreadPoolExecutor – allows the scheduling of task execution instead of immediately running it when a thread is available. It can also schedule tasks with a fixed rate or delay.
  • ForkJoinPool – a special ExecutorService for dealing with recursive algorithm tasks. If you use a regular ThreadPoolExecutor for a recursive algorithm you will find that all the threads are in use fairly quickly waiting for the lower levels of recursion to finish. ForkJoinPool implements a so-called “work-stealing” algorithm that allows it to use available threads more efficiently.
Thread Pools

When wanting to use more than two threads at the same time Thread Pools come into play. Thread Pools consist of one or more threads and a queue, where the queue holds tasks until a thread is ready to complete them. The key properties that make up a Thread Pool are:

  • Core Pool Size – the number of threads that are always in existence
  • Max Pool Size – the maximum number of threads that can be generated
  • Keep Alive Time – how long idle threads are allowed to exist
  • Allow Core Thread timeout – Usually false, if set to true core threads can be killed after the keep alive time passes

There are a number of types of thread pools that can be used in Java.

Fixed Thread Pool
  • The core and max pool sizes are equal, meaning the pool maintains the same number of threads at all times
  • If one thread fails due to an exception a new one is started
  • Threads exist until the .shutdown() method is called
  • When running a program which could have more tasks than the max pool size, then one of the threads must complete its processing before it can handle the additional tasks
Cached Thread Pool
  • An “unlimited” number of threads can be generated
  • Core pool size is defined as zero and max pool size is Integer.MAX_VALUE
  • If previously constructed threads are available they will be used, if no existing thread is available a new thread will be created and added to the pool.
  • Threads that have not been used for sixty seconds are terminated and removed from the cache
Scheduled Thread Pool
  • Contain a fixed number of threads, which can run tasks in specified regular time intervals
  • Similar to Cached Thread Pools, the max pool size is Integer.MAX_VALUE but the constructor contains the definition of the core pool size
  • Tasks can be performed in a number of ways with the ScheduledExecutorService:
    • .schedule() – runs a one off task after a specified delay
    • .scheduleAtFixedRate() – schedules a task to repeat given some time unit, a period ‘d’ for the fixed rate and a delay. The task will run every ‘d’ time units after the start time of the previous execution. If a task takes longer than the delay interval then the next execution will start late but will not execute concurrently
    • .scheduleWithFixedDelay() – similar to fixed rate but the runnable will execute ‘d’ time units after the end time of the previous task run

CompletableFuture and The Future Interface

CompletableFuture was introduced in Java 8 to provide an easy way to write asynchronous, non-blocking and multithreaded code. In Java 5 the Future interface was introduced to handle asynchronous computations. Future didn’t have any methods to combine multiple async computations and handle all the possible errors. CompletableFuture implements the Future interface and can combine multiple asynchronous computations, handle possible errors and offers much more capabilities.

Simple Future

The most basic implementation of CompletableFuture is a ‘Simple Future’. To implement a ‘Simple Future’ we can create an instance of the class to represent some future result and complete it in the future using the complete method. The consumers may use the get method to block the current thread until this result is provided.

public Future<String> async() throws InterruptedException {
    CompletableFuture<String> completableFuture = new CompletableFuture<>();

    Executors.newCachedThreadPool().submit(() -> {
        Thread.sleep(500);
        completableFuture.complete("Hello");
        return null;
    });

    return completableFuture;
}

Future<String> completableFuture = async();
// ... 
String result = completableFuture.get();
assertEquals("Hello", result);

Encapsulated Computation

Static methods runAsync and supplyAsync allow us to create a CompletableFuture instance out of Runnable and Supplier functional types correspondingly. While similar, there is a key difference between these two interfaces:

  • The Runnable interface is the same interface that is used in threads and it doesn’t allow to return a value.
  • The Supplier interface is a generic functional interface. It has a single method that has no arguments and returns a value of a parameterized type.

This allows us to provide the Supplier as a lambda expression that does the calculation and returns the result. It is as simple as:

CompletableFuture<String> future
 = CompletableFuture.supplyAsync(() -> "Hello");
// ...
assertEquals("Hello", future.get());

Combining Futures

Another feature of the CompletableFuture API is the ability to combine instances in a chain. The result of a chain is another CompletableFuture. The example method below takes a function that returns a CompletableFuture instance. The argument of this function is the result of the previous computation step. This allows us to use this value inside the next CompletableFuture‘s lambda:

CompletableFuture<String> completableFuture 
  = CompletableFuture.supplyAsync(() -> "Hello")
    .thenCompose(s -> CompletableFuture.supplyAsync(() -> s + " World"));
assertEquals("Hello World", completableFuture.get());

If you want to combine the result of two separate Futures then the thenCombine method is used. This method accepts a Future and a Function with two arguments to process both results.

CompletableFuture<String> completableFuture 
  = CompletableFuture.supplyAsync(() -> "Hello")
    .thenCombine(CompletableFuture.supplyAsync(
      () -> " World"), (s1, s2) -> s1 + s2));
assertEquals("Hello World", completableFuture.get());

Multithreading in Spring Boot

Async Configuration

The Spring Async annotation (@Async) is used to run methods in a separate thread. To setup @Async a configuration class must be used to define the thread pool to be used by these methods. This configuration class must have the annotation @EnableAsync present. Spring uses a ThreadPoolTaskExecutor to define the configuration to be used in the Thread Pool e.g.:

@Configuration     
@EnableAsync
     public class AsyncConfig {

         @Bean(name="taskExecutor")
         public Executor taskExecutor() {
             ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
             executor.setCorePoolSize(2);
             executor.setMaxPoolSize(2);
             executor.setQueueCapacity(100);
             executor.setThreadNamePrefix("poolThread-");
             executor.initialize();
             return executor;
         } 
    } 

In the example above, the Core Pool Size and Max Pool Size are equal so the thread pool will always have 2 threads along with a queue of 100. This bean can be accessed programmatically to check the status of the thread pool – this can be done by Autowiring in the Executor bean and accessing this instance in the service.

ThreadPoolTaskExecutor

With ThreadPoolTaskExecutor, even if the core pool size is defined, on start of the process there will be no threads in the pool. New threads will be created every time a task gets executed until the core pool size is met. When core pool size is met the next task will shift to the queue and wait for a free thread. If the load is too high and the queue Capacity is full, new executor threads get created until the max pool size is reached, these additional threads expire as soon as the queue is empty. If the core pool size, queue capacity and max pool size are all exceeded, the task is rejected and an exception will be returned.

When defining a method to be run as Async you can define which executor you would like to use in the arguments for the annotation. If there is no executor defined Spring is intelligent enough to find one automatically from the registered beans. This bean can be used to find information such as the active count (i.e. number of active threads), current pool size and queue size as mentioned previously by Autowiring the Executor.

General Spring Multithreading

It is worth noting with Spring you can’t call an Async method from the same class. Often they will be called through either a Rest controller or some Async Service class. If a service has multiple Async methods defined and only one AsyncConfig then the thread pool will be shared across all Async methods.

Concurrency and Other Pitfalls

Thread-Safe Objects

Generally speaking threads share access to the same object. This can cause issues because as one thread reads an object another may update it, meaning the reading thread may receive a different result than expected. In a worst case scenario concurrently updating an object may leave it in a corrupted or inconsistent state.

The simplest approach to avoid this issue is to use immutable objects. Multiple threads cannot modify the state of an immutable object. When it isn’t possible to use this approach, we have to instead make our mutable objects thread safe.

Thread-Safe Collections

Collections maintain state internally (as do all objects). Multiple threads concurrently accessing a single collection object can all alter this state. One workaround for this is to to synchronize the collection.

Map<String, String> map = Collections.synchronizedMap(new HashMap<>());
List<Integer> list = Collections.synchronizedList(new ArrayList<>());

By synchronizing these collections we are ensuring that only one thread can access them at a time. This, in turn prevents the object from being left in an inconsistent state.

Multithreaded Collections

Using a synchronized approach can introduce more issues. While the approach is useful for stopping multiple updates to the state, if the threads are being used for reading the object then we might encounter performance issues. This is because if two threads want to read the same object, one has to wait until the other is finished. Java provides some concurrent collections to help deal with this, such as ConcurrentHashMap.

Map<String, String> map = new ConcurrentHashMap<>();

ConcurrentHashMap is thread-safe and also more performative that the Collections.synchronizedMap() approach. It is a thread-safe map, made of thread-safe maps. There are partitions within the map which can be locked independently. This allows for different activities to happen simultaneously in the child maps.

Using Non-Thread-Safe Objects

Sometimes we have to work with objects with no thread-safe alternative. When this occurs it’s best to take an approach that will attempt to protect the state of the object from other threads. Some examples of these approaches are:

  • Every time you want to use the object create a new instance
  • Use the Synchronize keyword to control access to the object
  • Use ThreadLocal<Object> to create an instance of Object for each thread

Memory Consistency

Memory Consistency describes the problem when multiple threads have a different value for the same object. This can occur due to the fact that most modern computers have a hierarchy of caches on top of their core memory. Any thread can store a variable in any of these caches. The scenario where this might cause issues is described below:

  1. Two threads access the same object
  2. Both threads store the value of the object in two difference caches
  3. The first thread updates the value of the object
  4. The second thread reads the value from a different cache and gets the original value

While it is possible that the expected behaviour of reading the right value occurs, there is no guarantee.
As has previously been mentioned this can be handled by using synchronization, however at the cost of performance. Another approach to take is to use the volatile keyword. When we use volatile we ensure any change to a variable is visible to every thread. It’s worth noting that volatile does not mean that the object is now thread-safe.

Race Conditions

A race condition occurs when the result of multiple threads executing the same piece of code may differ depending on the sequence in which the threads run. The most common source of a race condition is when one thread is performing a ‘check-then-act’ operation and another thread updates between the check and act. An example of this can be see below:

if(x == 10){ //check
   y = x+20;  //act
}
if(y == 30) 
  \...

If a second thread where to update the value of x between the check and the act then y == 30 would still return true when in reality it shouldn’t.
A common approach to this problem is to use the Lock interface to protect the shared data. Another option would be to use the synchronized keyword to control access to the method. In some cases the lock may be more useful as you can explicitly state what you want to act as the monitor as opposed to the implied monitor from synchronized.

Summary

This post walked through the core concepts required to start working on a Spring Boot multithreaded project. Understanding general Java concepts for multithreading is the most important step in creating this application. Once you can create a Java application using multiple threads moving over to use the Spring Boot configuration is fairly straightforward. The real enhancements on the application come with proper understanding of Futures and how to handle results from running async methods.

Brien GillenBeginners Approach to Multithreading in Java Spring Boot