Henry Kobutra
← All notes
Notes

A timeout doesn't mean nothing happened

Run a small retry experiment that separates a lost reply, a duplicate command and an edit based on an old version.

A client sends an edit. The server saves it. The reply never arrives.

From the client's side, this can look much like an edit that never reached the server. Retrying might be the right response, but only if the system knows whether the second request is another attempt at the same operation or a new operation altogether.

I don't want an agent to settle that uncertainty by generating a new request ID and trying again. It needs a retry contract. A timeout tells us that the caller stopped waiting, not whether the requested work happened.

Lose the reply on purpose

The companion example uses Python's standard library and an in-memory SQLite database. Download the companion folder, open a terminal in it, and run:

python3 -B demo.py retries
python3 -B -m unittest -v test_retries

You need Python 3 with SQLite support. There is no service to configure, no network request and no credential. Each run starts with disposable synthetic notes.

The demonstration updates note-1, then raises a timeout after the update commits. This is a deliberately placed exception, not a simulation of every way a network can fail. It gives us the particular uncertainty we want to examine: the caller missed the reply, but the operation happened.

An exact retry returns the first receipt. The note reaches version 2 and stays there. Reusing that request ID with a different title produces a request-ID conflict. Sending the old command with a new ID produces a version conflict.

Those outcomes answer different questions. The request ID identifies the logical operation. The expected version says which state the author reviewed before requesting it.

Keep the command together

Before sending a mutation, retain its request ID and complete payload. For a real integration, keep the record somewhere durable and appropriately protected; don't log secrets just to make retrying convenient.

The synthetic payload contains a record ID, an expected version and a title. Changing any of those makes it a different command. The example serializes this fixed set of fields consistently, so changing the order of dictionary keys doesn't change its meaning.

On receipt, the store checks whether it has already recorded that request ID:

  • If the stored command matches, it returns the stored result without applying another edit.
  • If the command differs, it refuses the request.
  • If the ID is new, it checks the expected version, applies the edit and records the receipt in one transaction.

Keeping the edit and receipt in the same transaction matters. Saving the edit first and the receipt later creates another uncertain interval. In the tests, a forced receipt-write failure rolls back the edit too.

This is a teaching model for one operation in one workspace. A production implementation also needs authenticated callers, an explicit scope for its request IDs, durable storage and concurrency handling. External side effects such as sending email don't automatically join a database transaction. A saved receipt cannot undo a second message that another service already sent.

A conflict needs a decision

Suppose another editor changed the note after you read it. Your command still expects version 1, but the note is now at version 2.

A tempting recovery is to fetch version 2, substitute that number into the old command and send it again. That satisfies the mechanical version check while discarding its purpose. The old edit was never reviewed against the new state.

Instead, read the current note and compare it with the intended change. Perhaps the other edit already achieved the result. Perhaps both changes can coexist. Perhaps the request no longer makes sense. If you decide to proceed, create a newly reviewed command with a new request ID and the version you actually inspected.

The example rejects a stale command without changing its payload. The test then makes a separate, explicit edit against the current version. The code can't perform the human judgment between those steps; it can avoid silently skipping it.

The receipt is not today's state

An exact retry can return a receipt for version 2 even after somebody else has moved the note to version 3. That is correct: it describes what the original operation did.

Read the target again before reporting its current state. Keep these statements separate: "the service recorded my edit" and "the note currently contains this title."

For an API you didn't design, check its documented retry behavior before applying this pattern. Find out where keys are scoped, how long receipts survive, which outcomes are recorded and what happens when a key is reused with a different payload. A UUID in the request isn't evidence that the service deduplicates anything.

If the contract is missing, inspect the target or use a documented status lookup before another write. Sometimes the honest result is still uncertain. That's a better place to pause than an accidental second operation.

A conversation starts somewhere

What are you
working on?

If something here connects with what you're working on, email me.

henry@kobutra.com