This content originally appeared on Angular Blog – Medium and was authored by Andrew Scott
In the realm of software testing, the concept of time can be a tricky one to manage, especially when dealing with asynchronous operations and timeouts. We are continually refining our approach to handling time in unit tests, particularly concerning the use of mock clocks. We’ve found that mock clocks, while useful, often lead to headaches in test code and unrealistic timing scenarios. To address these challenges, we are exploring the exciting concept of “auto-advancing” mock clocks, which aim to streamline testing and enhance the accuracy of time-based simulations. In this blog post, we’ll delve into the intricacies of mock clocks, their limitations, and the potential benefits of auto-advancing functionality.
Why use a mock clock
We see a few reasons to install mock clocks in tests.
- To fake the date. This makes it possible to test components which display the current time or date related information, or do some calculation with respect to the current time.
- More frequently, to speed up time. UIs are frequently written with debounces, visible delays for users, timeouts, etc. We want to verify the state of the app after these long-running tasks complete but we don’t want the test to actually wait 30 seconds for a timeout. Instead, mock clocks can fake the progression of time without needing to modify the production code to expose some overridable timeout simply to make the test quicker.
- To prevent timers from leaking into other tests, creating action at a distance errors. Mock clocks can uninstall after each test and either dispose of or flush any pending timers.
- To execute all pending timers or stabilize the test. This allows the test to say, “wait until everything that’s supposed to happen has happened”. This can be useful for several reasons, including to verify that something isn’t going to happen. For example, consider testing Gmail’s “undo” send button. After clicking the button, you would want to verify that the RPC to send the email was never sent. We can also verify that there is no pending work. If the test finishes while there is more asynchronous work pending, we don’t want the pending work to continue executing during the next test case. (It’s hopelessly confusing.) The test should either (1) wait for the work to finish, (2) dispose() of the workers explicitly, or (3, least preferred) ask the fake clock to discard pending timers/frames.
- Much less common is to make time progress in specific increments in a unit test. One example might be a toast which appears after a known delay, and disappears after another known delay. This use of mock clocks, however, is not really a separate point from the second. We already have the ability to tick the clock specific amounts of time with timeouts. It’s just slow and mock clocks provide nice APIs to speed it up. Mock clocks can make these specific increments more predictable in tests. When using real timeouts, the ordering can sometimes depend on how quickly each task can be processed.
These are pretty good reasons to use a fake clock, and pretty broadly useful when async code is in play. We do support the idea of installing a fake clock.
Common pitfalls with mock clocks
While mock clocks offer a degree of control over time in tests, they can also introduce challenges and anti-patterns, and these all apply equally to Angular’s fakeAsync test helper.
Before the widespread availability and adoption of Promises and async/await in javascript, mock clocks helped keep tests manageable by making them synchronous. In modern-day javascript, we no longer need this. Furthermore, native Promises, async-await, resize or mutation observers, RPCs, etc cannot be executed synchronously; it is impossible to write a synchronous test for such logic. Tests with mock clocks are often forced into a situation where they need to both tick the mocked time and wait for real asynchronous code to happen. This can lead to unrealistic execution orders and can mask timing-related bugs. As a tester, keeping track of this can become a real nightmare and as a result, we’ve seen plenty of “big hammer” helper functions in tests:
async function realFlush(delay?: number) {
clock.flush();
await new Promise<void>(resolve => void setTimeout(resolve, delay));
clock.flush();
}Tests also become littered with hard-coded wait times, sometimes with purpose, but often somewhat arbitrary, such as mockClock.tick(1000) or mockClock.flushPendingTimers. Not only do these ticks and flushes often lack clear justification, but they also may need to be adjusted as the codebase evolves. Adjusting a timer in the production code can cause tests to fail, including many which were not intended to depend on or observe the specific timing implementation. In many ways, mock clocks can turn these sections of the tests into change detector tests.
Finally, some test utilities take it upon themselves to tick the mock clock, potentially interfering with the test’s intended timing and causing unexpected behavior. On the flipside, asynchronous test helpers and harnesses need to be informed about any installed mock clocks and how to advance them. Forgetting to provide this (assuming the testing APIs even have an option to do so, and many do not) will cause the test helper to stall, waiting for time that never advances.
Proposal: Allow mock clocks to automatically advance time
In the majority of use cases, mock clocks are used to speed up time. In order to do this, testers are opting into behavior that also stops time. We suspect that the majority of tests don’t want or need this. Clocks don’t stop in real life so it would make sense for mock clocks to advance on their own as well, unless there is a specific desire to pause it.
If mock clocks were able to advance time in a realistic way then we could realize a future where: No one should care whether time is being faked (unless they want to).
This includes business logic, test helpers, and even most tests themselves. It would be easy to migrate to a mock clock implementation with this property. That is, it would be ideal to be able to take a test that is written against real timer APIs and install a mock clock with no other changes to the tests. Auto-advance gets us pretty close.
Use cases
Tests with mock clocks would look identical to those without. Let’s go through some examples:
Testing async functions
When testing async functions, tests only need to enable auto advancing:
// methods_under_test.ts
export async function blackBoxWithLotsOfAsyncStuff() {
// do some work
await new Promise(r => void setTimeout(r, 10));
// do some more work
await new Promise(r => void setTimeout(r, 20));
await Promise.resolve();
// yay, we're done!
return 'done';
}
// methods_under_test.spec.ts
// …
beforeEach(() => clock.setTickMode('auto'));
it('is easy to test async functions with interleaved timers and microtasks', async () => {
const result = await blackBoxWithLotsOfAsyncStuff();
expect(result).toBe('done');
});
// …
Waiting for conditions to be satisfied
The specific timeframe that conditions occur is usually best thought of as an “implementation detail”, rather than part of the public API commitments. It’s fairly easy to imagine changes in the future that alter the timeframe, and which are completely invisible to the user (and the test!).
For example, changing a debounce time on an instant search input from 100ms to 200ms. Tests which use mock clocks today would typically trigger the input, explicitly tick (100 or 200ms), and then make assertions. Instead, tests would use a polling API to explicitly wait for a condition to be satisfied as a result of the input changing, such as the Testing Library’s async methods:
beforeEach(() => clock.setTickMode('auto'));
it('gets results when the search changes', () => {
const input = screen.getByLabelText('search-input');
fireEvent.change(input, {target: {value: 'signals'}});
await waitFor(() => expect(mockAPI).toHaveBeenCalledTimes(1));
await screen.findByText('Computed signals');
fireEvent.change(input, {target: {value: 'control flow'}});
await screen.findByText('Control Flow in Components - @if');
});What’s next
We started working with test framework authors to support automatically advancing time in mock clock implementations. As of Jasmine v5.7, the clock now has the ability to advance automatically and can be enabled with Clock#autoTick. It is also released in @sinonjs/fake-timers version 15, which is used by Jest and Vitest.
// jasmine v5.7
jasmine.clock().install().autoTick();
// @sinonjs/fake-timers v15
const clock = FakeTimers.install();
clock.setTickMode({mode: 'nextAsync'});
// future Jest release
jest.useFakeTimers().setTimerTickMode({mode: 'nextAsync'});
Follow us on social media for updates. We hope that this feature is as exciting to the community as it is to us. An automatically advancing mock clock could help existing async tests be faster and more predictable, regardless of what web framework is being used, since this feature doesn’t require any other changes to the test after it’s enabled.
Handling Time and Mock Clocks in Tests was originally published in Angular Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Angular Blog – Medium and was authored by Andrew Scott