Using Multiple JDBC Drivers in an Application

For running queries, stored procedures, etc. in a relational/SQL database from a Java application, JDBC drivers are used. These JDBC drivers have been made available by various providers. For a SQL database, all of these providers need to provide an implementation of java.sql.Driver interface. Two such implementations are available as JTDS driver and Microsoft’s JDBC driver.

The application may need access to multiple databases, which could be hosting different versions of MS SQL Server. While one driver type can be a better choice for SQL Server 2012, other drivers could provide access to more features in SQL Server 2016. Each implementation has its own benefits and limitations. It may become apt to use different database drivers for different databases

I encountered one such application; it had JTDS and Microsoft’s JDBC drivers both in the same application. The application was configured to call multiple stored procedures from multiple databases using these two drivers. All was working fine until we made a @Component bean instead of an @Bean, which blocked the application startup. I tried various hypotheses and was able to find that the problem was with loading multiple JDBC drivers concurrently. The application had a multi-threaded set up to load stored procedures’ meta-data.

Although I had found the trigger for the problem, I still wasn’t sure about the root cause of it. After a few hours of code debugging, I found the sequence of events that was blocking application start-up — it was a deadlock keeping it from starting up. Let me explain.

When the following statement is executed, it triggers the creation of a database connection. Note that the jdbcCall compile statement is being submitted to an executor service with a threadpool.

executorService.execute(() -> simpleJdbcCall.compile());

This requires loading the jdbc driver class. If two such statements are executed for different database drivers, they trigger the creation of two database connections using two different database drivers. These two database driver classes are ‘com.microsoft.sqlserver.jdbc.SQLServerDriver’ and ‘net.sourceforge.jtds.jdbc.Driver’. Both implement java.sql.Driver interface.

Class-loading requires initializing static class members and running static blocks. Both of these driver classes have a static block, which must be executed in order to create instances of these drivers.

Static block in com.microsoft.sqlserver.jdbc.SQLServerDriver

static {
        ...
        try {
            DriverManager.registerDriver(new SQLServerDriver());
        } catch (SQLException var1) {
            var1.printStackTrace();
        }
    }

Static block in net.sourceforge.jtds.jdbc.Driver

static {
        try {
            DriverManager.registerDriver(new Driver());
        } catch (SQLException var1) {
            ;
        }
    }

As shown in the code snippet above, both of these classes register themselves to java.sql.DriverManager by calling DriverManager.registerDriver(..). Since the call for both database sproc compile statement is being executed by different threads, let me call JTDS static block calling thread — Thread-1 and MSSQL driver static block executing thread — Thread-2.

Calling DriverManager class’s static method registerDriver triggers DriverManager class loading operation. Now, DriverManager class also has a static block, which needs to be executed as part of class loading routine. Following is a snippet from the static block of this class:

static {
  ...
  AccessController.doPrivileged(new PrivilegedAction() {
            public Void run() {
                ServiceLoader loadedDrivers = ServiceLoader.load(Driver.class);
                Iterator driversIterator = loadedDrivers.iterator();
                try{
                    while(driversIterator.hasNext()) {
                        driversIterator.next();
                    }
                } catch(Throwable t) {
                // Do nothing
                }
                return null;
            }
        });
  ...
}

DriverManager class uses Service Locator pattern to find the implementations of java.sql.Driver interface and then attempts to load those classes in its static block. Suppose Thread-1 attempts to calls DriverManager.registerDriver method first (calls are concurrent and not parallel, remember! 🙂 ). Then, Thread-1 will take care of DriverManager class loading and hence execute its static block before it reaches there it will acquire “initiaization lock” of this class (Thread-2 would wait until the DriverManager class is loaded — class loading op started by Thread-1). In DriverManager’s static block, it will find class names ‘com.microsoft.sqlserver.jdbc.SQLServerDriver’ and ‘net.sourceforge.jtds.jdbc.Driver’ in the following statement:

Once it has class names, it tries to load these classes and create an instance in the following statement.

The class loading operation for both of these is already in progress by Thread-1 and Thread-2 for JTDS and MSSQL driver respectively. So, when Thread-1 events sequence is like JTDS driver class loading -> running its static block -> DriverManager class loading -> executing DriverManager’s static block -> waiting for seeing MSSQL SQLServerDriver class loaded by Thread-2. And Thread-2 events sequence is like MSSQL SQLServerDriver class loading -> running its static block -> waiting for DriverManager class loading to finish by Thread-1.

This sequence of events would look like this:

Events sequence for both threads

From java.sql.DriverManager java documentation

Applications no longer need to explicitly load JDBC drivers using Class.forName(). Existing programs, which currently load JDBC drivers using Class.forName() will continue to work without modification.

When the method getConnection is called, the DriverManager will attempt to locate a suitable driver from amongst those loaded at initialization and those loaded explicitly using the same classloader as the current applet or application.

The problem is that DataSource class loads drivers using Class.forName() method and hence creates the deadlock.

org.apache.tomcat.jdbc.pool.DataSource

Code for demonstrating this deadlock behavior can be seen here.

The Hidden Synchronized Keyword With a Static Block

A static block in Java is a block of code that is executed at the time of loading a class for use in a Java application. It starts with a ‘static {‘ and it is used for initializing static Class members in general — and is also known as a ‘Static Initializer’. The most powerful use of a static block can be realized while performing operations that are required to be executed only once for a Class in an application lifecycle.

Image title

Let me provide a brief on when and how a static block is executed. A Class needs to be loaded by a ClassLoader whenever one of the following events happen:

  1. A static member variable is set by the application (by application, I mean code outside the class containing the static block).
  2. A non-final static member variable is accessed by the application.
  3. A static method is called by the application.
  4. Class.forName(“..”).
  5. Creating an instance using Class.newInstance() or through the new keyword.

Whenever any of those events occur, they trigger class loading activity. For creating a class instance, it must be ‘fully initialized.’ The JVM keeps some internal information about each class — such as Class status that tells if a Class has been loaded properly or not. For a Class to be properly loaded and get a status of ‘fully initialized,’ it must have all its static fields initialized and static blocks run without errors.

As the use of a static block is in the initialization/loading of a class (which should happen only once), the block is treated as a ‘synchronized’ block by the JVM. A thread can reach this block of code only when the class status is ‘Not initialized.’ Since there are multiple aforementioned events that can trigger class loading, the JVM needs to make sure that Class loading/initialization happens only once. To ensure this, there is an ‘initialization lock,’ available for each class, that is acquired by the thread that reaches the static block first, and the lock is released only after the static block code execution finishes.

Other threads are blocked on any activity, like attempting class loading or creating instances, until then. Once the lock has been released, the status of the Class is set to ‘fully initialized,’ leaving no need for any other thread to get to the static block.

The full code to demonstrate a static initializer run blocking other threads can be seen here.

We need to be very careful when using a static initializer, though, because the class loading function depends on it — and hence so does application startup, in most cases. The code in a static block should be such that it doesn’t trigger class loading events for another class that also has a static block that, in turn, depends on this class. This may introduce the possibility of deadlock. I encountered one such incident at work where the application startup was stuck in a deadlock because of static initializers. Here, I have similar code to demonstrate the situation:

An interface, IRequestHandler, is used by a service locator to find all implementing classes on the classpath.

package com.iliyakos;

public interface IRequestHandler {
    void handleRequest();
}

A class, RequestHandlerRegistrar, makes sure that all implementations of IRequestHandler are loaded:

package com.iliyakos;

import java.util.concurrent.CopyOnWriteArrayList;

public class RequestHandlerRegistrar {
    private static String[] handlerNames = new String[]{"com.iliyakos.TcpRequestHandler", "com.iliyakos.UdpRequestHandler"};
    private static CopyOnWriteArrayList registeredHandlers = new CopyOnWriteArrayList();

    static {
        try {
            // through the service locator this class would find handler class names in real world scenario
            final Class aClass1 = Class.forName(handlerNames[1]);
            aClass1.newInstance();
            final Class aClass = Class.forName(handlerNames[0]);
            aClass.newInstance();
        } catch (ClassNotFoundException | IllegalAccessException | InstantiationException e) {
            e.printStackTrace();
        }
    }

    public static synchronized void register(IRequestHandler handler) {
        if (!registeredHandlers.contains(handler)) {
            registeredHandlers.add(handler);
        }
    }
}

Class TcpRequestHandler provides one implementation of IRequestHandler and calls:

package com.iliyakos;

public class TcpRequestHandler implements IRequestHandler {
    static {
        RequestHandlerRegistrar.register(new TcpRequestHandler());
    }

    @Override
    public void handleRequest() {
        System.out.println("Handle request in TCP request handler");
    }
}

Class UdpRequestHandler provides a different implementation of IRequestHandler:

package com.iliyakos;

public class UdpRequestHandler implements IRequestHandler {
    static {
        RequestHandlerRegistrar.register(new UdpRequestHandler());
    }

    @Override
    public void handleRequest() {
        System.out.println("Handle request in UDP request handler");
    }
}

Note that the RequestHandlerRegistrar is attempting to load the classes TcpRequestHandler and UdpRequestHandler. And both implementation classes are attempting to register themselves by calling a register method in their static blocks, initializing RequestHandlerRegistrar’s class loading.

This application’s requirement is to have both the request handlers loaded, so the main method in the class below is trying to load the two classes:

package com.iliyakos;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {

    public static void main(String[] args) {
        ExecutorService executorService = Executors.newFixedThreadPool(1);

        executorService.submit(() -> {
                    try {
                        Class.forName("com.iliyakos.TcpRequestHandler").newInstance();
                    } catch (ClassNotFoundException | IllegalAccessException | InstantiationException e) {
                        e.printStackTrace();
                    }
                }
        );

        try {
            Class.forName("com.iliyakos.UdpRequestHandler").newInstance();
        } catch (ClassNotFoundException | IllegalAccessException | InstantiationException e) {
            e.printStackTrace();
        }

        executorService.shutdown();
    }
}

This results in a deadlock. Here are the series of steps that would take place:

  1. An ExecutorService thread would trigger a class loading event for TcpRequestHandler.
  2. The ExecutorService thread would start by checking if the class has a ‘fully initialized’ state. Because it is not loaded yet, the state would instead be ‘Not initialized.’
  3. The ExecutorService thread would then acquire an ‘initialization lock’ on the Class TcpRequestHandler to run its static block.
  4. The main thread would trigger a class loading event of UdpRequestHandler, and then acquire an ‘initialization lock’ followed by starting the execution of its static block.
  5. The static block in TcPRequestHandler will trigger a class loading event of the RequestHandlerRegistrar class. The ExecutorService thread will then acquire an ‘initialization lock’ of the Class RequestHandlerRegistrar to execute its static block.
  6. To execute a static method of RequestHandlerRegistrar in a static block of UdpRequestHandler, the main thread would wait to acquire the ‘initialization lock’ of the RequestHandlerRegistrar, since the class is not loaded yet.
  7. The static block in RequestHandlerRegistrar will again trigger the class loading event of TcpRequestHandler, and the ExecutorService thread will check the state of the class, which is still ‘Not initialized,’ but it won’t be blocked.
  8. The ExecutorService thread would then execute the statement of loading UdpRequestHandler from the static block of RequestHandlerRegistrar. The UdpRequestHandler is not loaded yet, but the ‘initialization lock’ has been acquired by the main thread.

While the main thread is waiting on initialization to complete for the RequestHandlerRegistrar event (started by the ExecutorService thread — it is still in its static block), the ExecutorService thread is waiting on the UdpRequestHandler initialization completion event — the static block of which is under execution by the main thread, creating a deadlock situation.

Full code to demonstrate how a static initializer in conjuction with a ServiceLocator can cause a deadlock can be seen here.