Skip to content
Commit Detail

Commit d1faebb

Author
Joe Lee <jlee@cloudflare.com> 2024-11-19 14:14:22 -0800
Parents
f072094
Tree
0131932
sqlite-backed DOs: Also reschedule alarm requests when persisted DB alarm has been cleared

In the case where an SRS actor receives a request to run the alarm handler, and
the alarm state is "clean" (i.e. both the local and last-known durably
persisted DB state match), and that alarm state is empty, tell the scheduler to
unschedule the alarm before cancelling.

Should allow the system to clean up old alarm scheduler entries where the
scheduler state is out-of-sync due to a failure occuring between when an alarm
deletion is durably persisted and when the scheduler is informed of the alarm
deletion.

We already reschedule alarms in the case where a runAlarm request has an
earlier time than the "clean" alarm state, but we were avoiding rescheduling in
the case where the clean alarm state was empty, under the assumption that
returning a CancelAlarmHandler result would be sufficient to tell the alarm
scheduler to unschedule the current alarm request, since the ActorCache code
contains similar logic.

It turns out that the reason the ActorCache code works is because the scheduled
time of a runAlarm request always comes from the durably persisted alarm state,
so if the worker sees a mismatch, it's because the alarm scheduler has a stale
view of the scheduled alarm time.  Returning a canceled result doesn't actually
remove the scheduled alarm -- it does nothing, with the expectation that the
alarm scheduler will later make a request with the correct time when it sees
the new persisted state.

In the case of sqlite-backed DOs, the durably persisted alarm state is in
sqlite, separate from the alarm scheduler, so we must explicitly tell the alarm
scheduler to unschedule an alarm run when we know the alarm has been cleared,
in addition to returning a canceled result.

Files changed

2 files changed~2 modified