Document the multi-network join race and its fix
Records the strace-based root-cause investigation for the bug where joining 2+ networks in one -r/--run left every network after the first permanently unreachable, and the retry-on-EIO/ENETDOWN fix applied in network_tap_relay.cpp, matching the level of detail already recorded for the extern-connectivity investigation above it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
This commit is contained in:
@@ -1264,6 +1264,40 @@ Source layout (all under `src/`):
|
||||
`tap-relay create/attach/teardown` case (updated per `self_test.{h,cpp}`'s
|
||||
own entry above) passes reliably across repeated runs.
|
||||
|
||||
**Real bug reported from the real target device: joining 2+ networks in
|
||||
one `-r/--run` left every network after the first permanently
|
||||
unreachable, regardless of extern/intern.** Root-caused via `strace -f`
|
||||
on the real device (the user's own suggestion, after several
|
||||
inconclusive timing-based experiments): the second network's own relay
|
||||
died on its literal first frame — `write(fd_container, ..., 86) = -1
|
||||
EIO`, immediately followed by `exit_group(0)`. `EIO` writing to a tap fd
|
||||
means the device isn't administratively up yet, and it genuinely wasn't:
|
||||
`create_tap_relay()` returns, and this relay starts polling, the instant
|
||||
the container-side tap device is *created*; `join_one_network()`
|
||||
(`network_join.cpp`), a *different* process, still has its own `ip addr
|
||||
add`/`ip link set <if> up` steps left to run afterward for that same
|
||||
device — confirmed via the trace's own timestamps, `ip addr add ... dev
|
||||
eth1` ran *after* the relay's fatal write. For the first network joined
|
||||
this race is narrow enough that no frame ever arrives first; for the
|
||||
second (and any later) network, something reliably delivers a frame
|
||||
before the interface is up, and the previous code treated any `write()`
|
||||
failure as fatal — exiting for good on that single `EIO`, so the network
|
||||
never worked again for the rest of the session. Fixed in the relay's
|
||||
frame-forwarding loop by retrying specifically on `EIO`/`ENETDOWN` (both
|
||||
mean "not up yet", a startup race, not a torn-down namespace) with a
|
||||
short bounded backoff (up to 50 × 20ms = 1s) instead of exiting
|
||||
immediately — generous compared to the ~14ms gap actually observed in
|
||||
the trace. **Methodology note**: an earlier `strace -f` attempt, wrapped
|
||||
in `timeout 30`, produced a misleadingly corrupted trace — GNU `timeout`
|
||||
sends its kill signal to the whole process group by default, and
|
||||
`strace`'s own tracing overhead was large enough (mount steps that
|
||||
normally take seconds took 3+ minutes under trace) that the real
|
||||
wall-clock timeout elapsed mid-setup, killing several traced children
|
||||
prematurely; dropping the `timeout` wrapper entirely produced a clean,
|
||||
complete trace. **Verified end-to-end on the real target device** with
|
||||
both 2 and 3 `intern` networks joined simultaneously in one session, all
|
||||
gateways reachable at 0% packet loss, clean teardown, no leftover state.
|
||||
|
||||
**Crash-orphan sweep**, the direct tap+relay analog of `port_forward.h`'s
|
||||
own (see its own entry below): unlike a veth pair or a session's own
|
||||
bridge/persistent-namespace state, a relay process is host-global state
|
||||
|
||||
Reference in New Issue
Block a user