Skip to content
Commit Detail

Commit ac15659

Author
Milan Miladinovic <milan@cloudflare.com> 2025-10-29 13:07:30 -0400
Parents
df988ee
Tree
b94afa8
Prevent alarm update races with promise chain serialization

`alarmLaterTasks` is a background task queue for updating the alarm
manager when alarms are moved to a later time (including deletion).
Tasks in this queue retry up to 4 times on failure.

Prior to this commit, a race condition could occur:
1. Alarm handler completes, queuing a deferred delete in alarmLaterTasks
2. User calls setAlarm() with a new time, which precommits to alarm
   manager
3. The deferred delete from step 1 retries and overwrites the new alarm

This causes the alarm manager to have no alarm while SRS has the new
alarm time, leading to alarms that never fire (or fire late if the alarm
time was set to run later, not never).

To fix this race, we replace `alarmLaterTasks` (TaskSet) with
`alarmLaterChain` (promise chain) that serializes all "move later"
operations. Additionally, when moving an alarm earlier (precommit
phase), we wait for any pending "move later" operations to complete
first (simply canceling these promises will not guarantee that they're
also canceled on the remote alarm manager in time).

This ensures:
	- "Move later" operations execute sequentially at the alarm manager
	- "Move earlier" operations wait for pending updates before proceeding
	- No races between delete and set operations at the alarm manager

We do not wait on these chained promises in the synchronous Workerd
implementation, but do wait if we're in "async mode" (i.e. production).

Files changed