A Dolt from the blue

I’ve been playing with Dolt lately. It’s a SQL database, but versioned like a Git repository.

I noticed that Dolt can use a Git repository as its backing store. This seems like a fascinating way to store versioned metadata alongside source code. I wondered whether Dolt’s artefacts would be visible in the Git repository and interfere with normal Git operations, but it seems they don’t. So how does it work?

The meat of a Git repository is its object database. Files and directory trees can be added to the database, with Git assigning each object a unique identifier based on its contents. Commits are objects too, tying a tree of files to metadata and, usually, one or more parent commits.

When I configured Dolt to push to a Git remote, it was writing its own commit history into the remote’s Git object database. I next wondered why those commits don’t show up in typical git operations on the repo.

The answer, it seems, lies in refs. Git commit objects point to their parents, forming a graph of history. But there doesn’t have to be just one connected graph: a single object database can contain several disconnected histories. git commands use refs to find interesting traversal points in those graphs, such as the latest commit on a branch.

These pieces explain how Dolt is able to smuggle versioned data into a Git repository. Its commits live in the same object database as the source history, but form a separate graph. Dolt then maintains its own refs/dolt/data ref pointing at the latest commit in that graph. Normal Git branches continue to point into the source-code graph, so most everyday Git operations never encounter the Dolt history.

(Understanding these Git internals better also gave me a new appreciation for git reflog. I’ve used it before to rescue a commit that I’d left stranded after an errant rebase. Now I understand why that works: the commit object hasn’t immediately disappeared just because my branch no longer points to it. The reflog records previous positions of refs, giving me a way back into parts of the commit graph that I’ve otherwise left behind).

In the age of LLMs and coding agents, it’s tempting to blow past all the little details while we’re hurrying along; making software faster and faster. I still get a kick out of understanding how the pieces fit together. And now we have tutors with seemingly endless patience, ready to follow us down whatever rabbit holes we find.

What a time to be curious.

Breaking your promises in Javascript

I love that modern Javascript supports promises, and the await/async syntax. But native promises can’t (currently) be cancelled. This is a great shame, as it can encourage developers to write unresponsive user interfaces.

For example, suppose we’re building a dashboard that is populated with data from an API. Each time the user picks a new date on the dashboard, we want to load in the relevant data. It’s tempting to code this such that the request fires and then our UI re-renders with the latest data. In React, we might write this as shown below. Note that the datePickerEnabled state is used to disable the date picker whilst there’s an API request in flight.

import React, {useEffect, useState} from "react";
import axios from "axios"; // for HTTP requests

function Dashboard() {
    // Initialise state for storing the current date and the current
    // counter value
    const [date, setDate] = useState(new Date);
    const [value, setValue] = useState("-");

    // Initialise state for controlling whether or not to enable
    // the date picker
    const [datePickerEnabled, setDatePickerEnabled] = useState(true);

    // Prepare a function for loading the counter value by hitting
    // an API
    const loadValue = async (date) => {
        const newValue = await axios.get("/counter", {date});
        setValue(newValue);
    };

    // Schedule loading of the counter value for when React mounts
    // this component
    useEffect(() => { loadValue(date) }, []);

    // Prepare a function for updating the date and loading the
    // relevant counter value
    const onDateChange = async (newDate) => {
        setDate(newDate);
        setDatePickerEnabled(false);
        await loadValue(newDate);
        setDatePickerEnabled(true);
    };

    // Render a date picker and the counter
    return (
        <div>
            <DateRangePicker
              date={date}
              onChange={onDateChange}
              enabled={datePickerEnabled}
            />

            <Counter value={value}/>
        </div>
    )
}

This code works but it provides a fairly unresponsive user experience: a user has to wait for the API request to finish and the counter value to update before they can pick a new date. This can be frustrating if you pick the wrong date or if the API is slow to respond.

We could opt not to disable the date picker when there’s an API request in flight, but that can lead to inconsistent results. Our API requests are not guaranteed to return in the order that we send them (e.g., if an earlier request suffers a delay).

Instead, my preferred solution nowadays is to keep the UI responsive, but to cancel the consequences of earlier promises. Here’s a version of the earlier React component that breaks any previous promises:

import React, {useEffect, useState} from "react";
import axios from "axios";

function Dashboard() {
    // Initialise state for storing the current date and the current
    // counter value
    const [date, setDate] = useState(new Date);
    const [value, setValue] = useState("-");

    // Initialise state that stores a function for cancelling any
    // previous promise. Note: functions stored in React's useState
    // must be wrapped (in a function)
    const [cancelPreviousPromise, setCancelPreviousPromise] =
      useState(() => () => {});

    // Prepare a function for loading the counter value by hitting
    // an API
    const loadValue = async (date) => {
        const [cancellablePromise, cancel] =
          cancellable(axios.get("/counter", {date}));

        // Cancel the consequences of any previous promise
        cancelPreviousPromise();

        // Allow subsequent promises to be able to cancel updates
        // Note: functions stored in React's useState must
        // be wrapped (in a function)
        setCancelPreviousPromise(() => cancel);

        try {
            // Wait for the new data to load
            const newValue = await cancellablePromise;

            // Store the new value ready for display in the counter
            setValue(newValue);

        } catch (error) {
            // Ignore any errors from cancelled promises
            if (error instanceof CancelledPromiseError) return;

            // ... other error handling omitted
        }
    };

    // Schedule loading of the counter value for when React mounts
    // this component
    useEffect(() => { loadValue(date) }, []);

    // Prepare a function for updating the date and loading the
    // relevant counter value
    const onDateChange = async (newDate) => {
        setDate(newDate);
        await loadValue(newDate);
    };

    // Render a date picker and the counter
    return (
        <div>
            <DateRangePicker
              date={date}
              onChange={onDateChange}
              enabled
            />

            <Counter value={value}/>
        </div>
    )
}

The key here is to store a function (in the cancelPreviousPromise) state that can be used to cancel the consequences of an earlier promises. If the user selects a date whilst there’s already a request in flight, the code after the await doesn’t run and a CancelledPromiseError is thrown instead. This keeps the UI responsive and consistent!

Here’s the implementation of cancellable:

const cancellable = (original) => {
    let cancel = () => {};

    const cancellation = new Promise(
        (resolve, reject) =>
          cancel = () => reject(new CancelledPromiseError(original))
    );

    const wrapped = Promise.race([original, cancellation]);

    return [wrapped, cancel];
};

class CancelledPromiseError extends Error {
    constructor(promise) {
        super("Promise was cancelled");
        this.name = 'CancelledPromiseError';
        this.promise = promise;
    }
}

cancellable races our original promise with a further promise that only completes (rejects) if the cancel function returned as the second argument from cancellable is called. Thanks to Pho3nixHun for this implementation!

Debugging non-deterministic tests in PHPUnit

You have a test that fails when run as part of your test suite. You re-run the test on its own, and it passes. What gives? More than likely you have a non-deterministic test, and probably a test that lacks isolation.

Here’s a tip for debugging tests that are breaking isolation: a PHPUnit Listener can check which of your test cases is changing the environment and causing a failure of a later test case. For example, suppose your tests rely on the APP_ENV environment variable being set to test. The following PHPUnit listener will check this before every test:

class AppEnvIsTestListener extends PHPUnit_Framework_BaseTestListener
{

  public function startTest(PHPUnit_Framework_Test $test) {
    if (getenv("APP_ENV") !== "test") {
      echo "A prior test has changed APP_ENV!\n";
    }
  }
}

To use the Listener, you’ll need to add it to your phpunit.xml:

<phpunit>
  ...

  <listeners>
    <listener
      class="AppEnvIsTestListener"
      file="app/test/AppEnvIsTestListener.php"
    />
  </listeners>
</phpunit>

Combine this with the --testdox flag to phpunit and you’ll get a very basic glimpse into which of your tests is leaking environment changes, and causing a later test to fail:

$ phpunit --filter EnvTest --testdox
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Acme\EnvTest
 [x] Tests a thing
 [x] Tests another thing
A prior test has changed APP_ENV!
 [x] Tests a thing that does not depend on APP_ENV
A prior test has changed APP_ENV!
 [ ] Tests a thing that depends on APP_ENV
A prior test has changed APP_ENV!
 [ ] Tests another thing that depends on APP_ENV

Above, we can now see that the Tests another thing test case is leaking a change to APP_ENV and is likely causing the last two tests to fail.

Automating Reminders with AppleScript

Occasionally, I need to provision a complete replica of our AWS production to get my work done. As you might imagine, this incurs a fairly hefty hourly charge and we try to avoid running this setup for any longer than necessary. Although I’ve tried to develop the habit of checking the AWS consoles before heading home for the day, I sometimes forget. We then run a completely unused production stack overnight, wasting a whole bunch of money and resources.

Automatically shutting down the replica every evening seemed like the obvious solution. But the trouble is that sometimes I do want to run the stack for a couple of extra hours or even overnight whilst a long-running test finishes.

And then it occurred to me: perhaps I should adapt my provisioning script to also create a task in Apple’s Reminders app.

Here’s the code:

# add_teardown_reminder.applescript

tell application "Reminders"

  # Calculate date time for midnight today
  set currentDay to (current date) - (time of (current date))
  # Calculate date time for 1700 today
  set theDate to currentDay + (17 * hours)

  # Select the relevant list in Reminders.app
  set myList to list "Work"

  tell myList
    # Create the reminder
    set newReminder to make new reminder
    set name of newReminder to "Teardown test servers"
    set remind me date of newReminder to theDate
  end tell
end tell

This script calculates a date time for today at midnight, adds 17 hours to get a due date of “5pm today” and then adds the task to the Work list in Reminders.app.

Running the script from a shell script is simple:

osascript add_teardown_reminder.applescript

I’d not dabbled with AppleScript very much before, but getting this done took around 10 minutes. The “Open Dictionary” tool in Apple’s Script Editor is an excellent way to explore the APIs provided by Reminders.app (and other applications), and I picked up a few tips from Federico Viticci.

Best of all, our AWS account won’t be wasting midnight oil anymore.