Retry container-side tap device setup, external process only
A user's log from the real target device showed nsenter'd `ip addr add ... dev eth0` failing with "Cannot find device" immediately after network_tap_relay.h's relay had already created it -- confirmed by hand that retrying the whole session a few times eventually worked. join_one_network() now retries (bounded, ~500ms, quiet until final success/give-up) the three steps that touch the just-created container interface -- IPv4 address, IPv6 address, bringing it up -- via a new run_with_retry() instead of plain run(). A tempting first fix was investigated and ruled out by direct A/B testing, not just reasoned about: having the relay itself self-verify the device is visible (a same-process check, immediately after creating it, before ever reporting success) was tried first, in relay_child_main(). It made things categorically worse: the container-side device became permanently invisible to every external nsenter afterward, 100% reproducibly (confirmed with a 10-second retry budget -- never once became visible), on a mechanism that had otherwise worked correctly and instantly, zero retries needed, on every real session tested earlier the same day -- including a from-scratch self-test reproduction that had passed reliably many times before this one change, and immediately went back to passing once it was reverted. Root cause not fully understood (something about forking a subprocess that inherits the tap fd -- deliberately not O_CLOEXEC -- while still holding it open, immediately after device creation, appears to corrupt the device's external visibility on this kernel specifically), but the fix is unambiguous: never add an internal, same-process/fd-holding self-check to the relay; only the external, separate-process retry is safe. network_tap_relay.cpp ends up completely unchanged -- the actual fix lives entirely in network_join.cpp's own retry. self_test.cpp's own container-visibility check needed the same external retry treatment, for the same underlying reason. Verified as root via the doas rule: three separate real --no-veth sessions all succeeded getting eth0 on the first attempt (no retries triggered), and the self-test passes reliably across repeated runs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
This commit is contained in:
@@ -524,6 +524,40 @@ the `doas` rule: two freshly created `extern` networks got
|
||||
`fdf0:f243:f06f:168::/64` and `fdf0:f243:f06f:169::/64` exactly as
|
||||
expected, correctly paired with `10.168.0.0/24`/`10.168.1.0/24`.
|
||||
|
||||
## Resolved: container-side tap device intermittently not immediately visible
|
||||
|
||||
**Trigger**: a user's own log from the real target device showed
|
||||
`nsenter --net=/proc/<ns_pid>/ns/net -- ip addr add 10.168.0.2/24 dev eth0`
|
||||
failing with `"Cannot find device \"eth0\""` immediately after the relay had
|
||||
already created it — confirmed by hand that retrying the whole session a few
|
||||
times eventually let it succeed.
|
||||
|
||||
**Fix**: `network_join.cpp`'s `join_one_network()` now retries (bounded,
|
||||
~500ms, quiet until final success/give-up) the three steps that touch the
|
||||
just-created container interface — IPv4 address, IPv6 address, bringing it
|
||||
up — via a new `run_with_retry()` instead of the plain `run()` used
|
||||
elsewhere.
|
||||
|
||||
**A tempting "fix" investigated and ruled out by direct A/B testing on the
|
||||
dev machine, not just reasoned about**: the obvious first instinct — have
|
||||
the relay *itself* self-verify the device is visible (a same-process check,
|
||||
immediately after creating it, before ever reporting success back) — was
|
||||
tried first, in `network_tap_relay.cpp`. It made things categorically worse:
|
||||
it made the container-side device **permanently invisible to every external
|
||||
`nsenter` afterward, 100% reproducibly** (confirmed with a 10-second retry
|
||||
budget — never once became visible), on a mechanism that had otherwise
|
||||
worked correctly and instantly, with zero retries needed, on every real
|
||||
session tested earlier the same day — including a from-scratch self-test
|
||||
reproduction (`self_test.cpp`) that had passed reliably many times before
|
||||
this one change, and immediately went back to passing reliably once the
|
||||
change was reverted. Root cause not fully understood (something about
|
||||
forking a subprocess that inherits the tap fd — deliberately not
|
||||
`O_CLOEXEC` — while still holding it open, immediately after device
|
||||
creation, appears to corrupt the device's *external* visibility on this
|
||||
kernel specifically), but the fix that shipped is unambiguous: never add an
|
||||
internal, same-process/fd-holding self-check to the relay; the external,
|
||||
separate-process retry above is safe and sufficient on its own.
|
||||
|
||||
## Explicitly out of scope for now
|
||||
|
||||
- **Rootless networking.** An earlier draft of this design considered a
|
||||
|
||||
Reference in New Issue
Block a user