Author

tewaro <adityaatewari@gmail.com> 2026-04-25 05:08:29 -0500

Parents

cadd1aa

Tree

90d8693

containers: Keep sidecar warm across destroy/start

container-client tests flake on CI because each container.start()
recreates the proxy-everything sidecar, and that path dominates start()
cost under dockerd contention (bridge IPAM lock + iptables in the new
netns). CI's ~1400 concurrent tests on one dockerd socket regularly
push start() past the 10s readiness-probe budget.
The sidecar is workerd-internal; container.capnp makes no promises
about its lifetime. Reusing it across destroy/start within one workerd
process is safe and eliminates the dominant per-start cost.
Scope: localDocker container engine only — local-dev/test, gated behind
--experimental. No production code path affected.

Measured impact:
bazel --jobs=16 --local_test_jobs=16 --runs_per_test=10:
  Before:  @ 10/10 fail · @all-autogates 9/10 timeout · @all-compat-flags 7/10 timeout
  After:   @ 10/10 PASS · @all-autogates 10/10 PASS  · @all-compat-flags 10/10 PASS
Sidecar phase under contention (instrumented, then reverted):
  Cold:  avg 1.48s, max 6.90s
  Warm:  avg 32us,  max 291us
32% of start() calls take the warm path. The new PUT /egress on every
start() is ~1ms.

Changes:
- start() no longer resets containerSidecarStarted; the flag's invariant
  ("true iff a sidecar is running") is correctly maintained by
  ensureSidecarStarted (sets on cold-path success, clears via
  KJ_ON_SCOPE_FAILURE) and status() (re-derives from Docker at DO
  startup). The previous per-start reset was redundant — status()
  already handles workerd-restart orphan recovery.
- destroy() no longer tears down the sidecar.
- start() always pushes the egress config (PUT /egress is synchronous;
  proxy-everything updates an atomic.Pointer before responding), so
  internetEnabled and DNS allow-list changes still take effect.
User-visible Container API (status/start/destroy/monitor/getTcpPort/
exec/setEgress*/snapshot) is unchanged.

Risks:
- `docker ps` shows *-proxy sidecars between destroy() and the next
  start() (or until DO eviction). ~15MB per warm sidecar.
- Orphan recovery unchanged: status() reattaches if app+sidecar both
  running; otherwise next start() cold path clears the orphan.
- TLS-intercept CA cert now stable across destroy/start within one
  workerd process instead of regenerating per start. No documented
  contract about CA rotation.