How to Test Complex Logic Containing Lots of I/O

Let's say we are making a desktop or mobile app that downloads a file to your system, and we have the following additional requirements:

• If the device hasn't been granted storage permissions, show an error.
• If the file is already downloaded, then simply attempt to open it.
• If the file exists but for some reason can't be opened, show an error.
• If the user clicks "download" when it's already being downloaded, show a notice saying it's already being downloaded.
• Else, start downloading the file and show a success message.

Pretty complex. We could model this using a flowchart:

If we implement this model, it'll end up having a rather high cyclomatic complexity, so even if we code it correctly, it will be very fragile and may break in the future if another engineer modifies the code. We'd like to have a way to test the paths and branches of the flowchart. In practice, this means writing tests where if a condition is satisfied, then the next node in the flowchart is executed, and so on.

The Difficulty of Testing I/O Code

In the algorithm described above, we can tell that there are at least four different moments where I/O occurs:

• File download.
• Check if a file exists in the device.
• Show a message on the screen.
• Check whether the file is being downloaded (assuming download tasks are stored in a database that manages their current progress and/or download status).

Suppose you want to write unit tests for this. The problem you'll run into is that testing things like storage, content displaying on the screen, and database accesses is tricky. Testing the integration with a database will usually require a fake database with pre-populated data, which may take time to set up (this is relatively straightforward in frameworks like Ruby on Rails.) Verifying that certain content or message is being displayed on the screen is even harder without specialized testing tools that can access the UI.

In some situations, you may want to refactor the logic so it's easier to manage and implement, but this isn't always possible, so I'd like to explore a different solution that works for complex cases as well. Let's assume we have to implement the flowchart as is without simplifications.

Using Dependency Inversion

The dependency inversion principle (DIP) is one of the five SOLID principles. It states that "One should depend upon abstractions, not concretions." A slightly different but closely related concept is dependency injection. According to Wikipedia:

In software engineering, dependency injection is a programming technique in which an object or function receives other objects or functions that it depends on. Dependency injection aims to separate the concerns of constructing objects and using them, leading to loosely coupled programs.

[...]

Dependency injection is often used to keep code in-line with the dependency inversion principle.

Let's first show what the opposite of dependency inversion would look like. I'll use Javascript since many people understand it, but the language itself is irrelevant, as this concept applies to programming in general. We could implement the above logic hardcoding all I/O inside the main function (for simplicity, the await keyword was omitted, even though some function calls should be awaited in practice):

function theProcess(fileId) {
  if (!OS.hasPermission()) {
    alert("No storage permission");
    return;
  }

  const taskStatus = sqlite.query(
    `SELECT status FROM tasks WHERE id = ${fileId}`
  );
  if (taskStatus === DOWNLOADING) {
    alert("Already being downloaded");
    return;
  }

  if (OS.fileExists(fileId)) {
    try {
      OS.openFile(fileId);
    } catch (e) {
      alert("Cannot open file");
    }

    return;
  }

  fetch(`https://my-endpoint.com/downloads/${fileId}`);
  alert("Download started!");
}

Not only this code depends on sqlite and the vanilla alert (Javascript's default dialog, used in browsers), which means it wouldn't work on mobile if we were to port it, but also, it's very inconvenient to test because we can't trigger an alert during testing (at least not easily without doing a lot of hacks). If the mobile version doesn't support fetch and/or sqlite then we'd have to change a lot of things in order to fix it.

Note that most of this is just pseudocode and that the most important thing to keep in mind is that none of the things implemented inside the function are test-friendly.

We can refactor it and use dependency injection. This means passing all the I/O logic from the caller instead of hardcoding the logic inside the function:

function theProcess(
  fileId,
  hasPermission,
  showMessage,
  isTaskBeingDownloaded,
  fileExists,
  openFile,
  downloadFile
) {
  if (!hasPermission()) {
    showMessage("No storage permission");
    return;
  }

  if (isTaskBeingDownloaded(fileId)) {
    showMessage("Already being downloaded");
    return;
  }

  if (fileExists(fileId)) {
    if (openFile(fileId)) {
      // File was opened
    } else {
      showMessage("Cannot open file");
    }

    return;
  }

  downloadFile(fileId);
  showMessage("Download started!");
}

Note that while theProcess keeps the original control flow (i.e., the branches described in the flowchart), it now receives functions as arguments for the I/O logic. They need to be coded independently, which I won't do here.

Testing With Jest

With the refactor above it may look like nothing changed other than passing things from the caller. We still have to implement the I/O somewhere. However, the magic is that now we have control over what those functions are, and this is especially useful when used along testing frameworks because now you can mock functions and achieve the following things:

• Make a function return an arbitrary hardcoded value.
• Keep track of how many times a function has been called.
• Check the order and/or sequence in which one or more functions have been called.

Let's say we want to test that an error message is shown when the user's device has no storage permission. We could simply force hasPermission to return false and then verify showMessage was called once with the argument No storage permission. This would be enough to test the case where the user has no storage permissions. Ideally, you'd also want to assert the order in which they were called and whether no more functions were called, but as far as I know, Jest doesn't support this out of the box.

test("no storage permissions", () => {
  const hasPermission = jest.fn(() => false);
  const showMessage = jest.fn();

  theProcess("some-file-id", hasPermission, showMessage); // Omit some arguments

  expect(hasPermission).toHaveBeenCalledTimes(1);
  expect(showMessage).toHaveBeenCalledTimes(1);

  expect(showMessage).toHaveBeenCalledWith("No storage permission");
});

If we now want to test the full flow, from the start until the download begins, we can write the following test:

test("starts download (full flow)", () => {
  const hasPermission = jest.fn(() => true);
  const showMessage = jest.fn();
  const isTaskBeingDownloaded = jest.fn(() => false);
  const fileExists = jest.fn(() => false);
  const openFile = jest.fn();
  const downloadFile = jest.fn();

  theProcess(
    "some-file-id",
    hasPermission,
    showMessage,
    isTaskBeingDownloaded,
    fileExists,
    openFile,
    downloadFile
  );

  expect(hasPermission).toHaveBeenCalledTimes(1);
  expect(showMessage).toHaveBeenCalledTimes(1);
  expect(isTaskBeingDownloaded).toHaveBeenCalledTimes(1);
  expect(fileExists).toHaveBeenCalledTimes(1);
  expect(openFile).not.toHaveBeenCalled();
  expect(downloadFile).toHaveBeenCalledTimes(1);

  expect(downloadFile).toHaveBeenCalledWith("some-file-id");
  expect(showMessage).toHaveBeenCalledWith("Download started!");
});

With these two tests, we have increased the stability of the control flow logic.

Run tests on CodeSandbox.

Note that we must still test the remaining paths in our logic (or at least the most important ones). While it may take time and code to do so, I usually prefer to have a lot of tests as opposed to a few.

Some libraries/frameworks allow for more sophisticated and compact assertions. Here's a code sample using Dart's package Mockito (from the official readme):

cat.eatFood("Milk");
cat.sound();
cat.eatFood("Fish");

verifyInOrder([
  cat.eatFood("Milk"),
  cat.sound(),
  cat.eatFood("Fish")
]);

verifyNoMoreInteractions(cat);

This is very useful for testing that a control flow path has been executed as expected. I'm currently unaware if something like this can be done in Jest.

Advantages and Disadvantages of This Approach

The main disadvantage of this approach is that you still haven't tested the actual code, but just a mocked version of it. This means that the actual code that will run on production will not be tested entirely, and in most cases, it's still necessary to have end-to-end or acceptance tests to verify it works as expected.

With this approach, you can only test the logic's skeleton and ensure the flow happens in the expected way, but you can't test the actual non-mock code.

In the case presented in this article, we had a complex flowchart that we had to implement and make sure all the conditions and branches happened the same way we expected. This would've been very hard to do if we couldn't mock the functions required by the main process because of the limitations of doing I/O testing.