Time Machine for Java

This article describes a tool developed to support unit-testing of time-dependent logic in Java applications. The tool helps control the quality of trading platforms, and other complex and/or concurrent systems.

It is not always simple to write unit tests for time-dependent functionality. In straightforward situations simply replacing a method which returns the current time with a custom implementation [1][2] will work.

Here is a simple method counting the number of days before the end of the world:

fun daysBeforeDoom() {
    return doomTime - System.currentTimeMillis()) / millisInDay
}

Replacing all invocations of System.currentTimeMillis() with either an existing tool [1][2] or by writing a system specific code transformer using the ASM framework [3] or AspectJ [4] will often suffice.

There are other cases where this approach is insufficient. Imagine we want our code to wake up each day and display “N days left until the doomsday” as in the following code snippet.

while (true) {
    Thread.sleep(ONE_DAY)
    println(“${daysBeforeDoom()} days left till the doomsday”)
}

How can we test this code? How do we assert that it is invoked every day and that it displays the correct message? Using the simple approach described above and replacing System.currentTimeMillis() only allows us to test the correctness of the message. We would need to wait for a whole day to test the time span.

It is practically impossible to test such code without additional tools. Let’s try to create one.

Currently we have two methods that return the current time: System.currentTimeMillis() and System.nanoTime(). We also have several time-dependent methods that wait for an event or may timeout: Thread.sleep(), Object.wait(), and LockSupport.park().

We want to create a new method increaseTime() which can change the current time and wake up any waiting threads with a timeout.

To enable this, all of the existing time-dependent methods will need to be replaced with custom implementations. Let’s see how this may work..

Test example:

increaseTime(ONE_DAY)
checkMessage()

This creates a potential race condition between our check message and the actual time it takes to complete the print operation. Of course, one approach is to add a pause:

Repaired test:

increaseTime(ONE_DAY)
Thread.sleep(500 /*ms*/)
checkMessage()

In regular practice, this test will almost always work but it is not guaranteed that checkMessage() is invoked after the message is printed. This can occur due to the complexity of the test logic or simply having the code executed on an overcommitted server. You might be tempted to increase the timeout but this makes the tests slower and still offers no guarantee of correctness.

Instead we need a special method that waits until all woken up threads have been completed.

Test we would like to write:

increaseTime(ONE_DAY)
waitUntilThreadsAreFrozen(1_000/*ms, timeout*/)
checkMessage()

The inherent challenge is that we want to support all time-dependent methods during testing but we also want a waitUntilThreadsAreFrozen() method and it is not simple to support both.

Our solution is implemented in a special tool at Devexperts for testing time-dependent logic called time-test [5].

Let’s look at how it works.

Time-test is implemented as a Java agent. To utilize it you should add javaagent:timetest.jar and include it in the classpath. The tool transforms byte-code and replaces all time-dependent method invocations with our specific implementations. However, writing a good java agent sometimes is not simple so we have developed a JAgent framework [6] to simplify java agents development.

When creating your time dependent tests you should enable TestTimeProvider. It implements all required time-dependent methods (System.currentTimeMillis(), Thread.sleep(), Object.wait(), LockSupport.park(), …) and overrides their default implementations. In most tests, you do not need to actually manage the underlying time so the tool internally continues to utilize the default time-dependent methods wrapped inside the overloaded method. After starting TestTimeProvider you can use the TestTimeProvider.setTime(), TestTimeProvider.increaseTime() and TestTimeProvider.waitUntilThreadsAreFrozen() methods.

TimeProvider.java:

long timeMillis();
long nanoTime();
void sleep(long millis) throws InterruptedException;
void sleep(long millis, int nanos) throws InterruptedException;
void waitOn(Object monitor, long millis) throws InterruptedException;
void waitOn(Object monitor, long millis, int nanos) throws InterruptedException;
void notifyAll(Object monitor);
void notify(Object monitor);
void park(boolean isAbsolute, long time);
void unpark(Object thread);

As previously described, the primary challenge of the TestTimeProvider implementation is supporting both the time-dependent method along side the additional waitUntilThreadsAreFrozen() method. On every time change all required threads are marked as resumed and only then are woken. At the same time, waitUntilThreadsAreFrozen() waits until all threads are in a waiting state and none of them are marked as resumed. With this approach threads will wake up, reset their resumed mark, perform their task and then return to a waiting state before waitUntilThreadsAreFrozen will recognize it as complete.

Test with TestTimeProvider:

@Before
public void setup() {
    // Use TestTimeProvider for this test
    TestTimeProvider.start(/* initial time could be passed here */);
}

@After
public void reset() {
    // Reset time provider to default after the test execution
    TestTimeProvider.reset();
}

@Test
public void test() {
    runMyConcurrentApplication();
    TestTimeProvider.increaseTime(60_000 /*ms*/);
    TestTimeProvider.waitUntilThreadsAreFrozen(1_000 /*ms*/);
    checkMyApplicationState();
}

We hope that the time-test tool will make your life easier. The tool is open-sourced and available on GitHub [5].

Happy testing!
 
References:
[1] https://github.com/TOPdesk/time-transformer-agent/
[2] https://stackoverflow.com/questions/2001671/override-java-system-currenttimemillis-for-testing-time-sensitive-code
[3] http://asm.ow2.org/
[4] https://www.eclipse.org/aspectj/
[5] https://github.com/Devexperts/time-test
[6] https://github.com/Devexperts/jagent

By Nikita Koval

Research engineer on concurrency and code analysis problems at Devexperts