Retry tap writes on EIO/ENETDOWN: fixes multi-network join race

Joining a container to two or more networks in one -r/--run left every
network after the first permanently unreachable, from the very start of
the session, regardless of extern/intern -- confirmed via strace -f on
the real target device.

create_tap_relay() returns, and its relay starts polling, the instant the
container-side tap device is *created*; join_one_network() (a separate
process) still has its own ip addr add/ip link set <if> up steps left to
run afterward for that same device. For the first network joined this
race is narrow enough that no frame ever arrives first; for the second
(and any later) network, something reliably delivers a frame before the
interface is up, and the previous code treated any write() failure as
fatal -- exiting for good on that single EIO, breaking the network for
the rest of the session.

Fixed by retrying specifically on EIO/ENETDOWN (both mean "not up yet",
a startup race, not a torn-down namespace) with a short bounded backoff
(up to 50 * 20ms = 1s) instead of exiting immediately.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
This commit is contained in:
2026-08-31 16:59:37 +00:00
parent 743e899c3d
commit 244a814e85
+40
View File
@@ -25,6 +25,7 @@
#include <signal.h>
#include <sys/ioctl.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>
#include <algorithm>
@@ -269,12 +270,51 @@ void report_line(int fd, const std::string& line) {
_exit(0);
}
ssize_t written = 0;
// **Real bug found by strace on the real target device, not
// assumed**: joining a container to two or more networks in one
// `-r/--run` always left every network after the first
// unreachable, permanently, from the very start of the session
// -- confirmed via `strace -f` that the *second* network's own
// relay died on its literal first frame:
// `write(fd_container, ..., 86) = -1 EIO`, immediately followed
// by `exit_group(0)`. EIO from writing to a tap fd means the
// device isn't administratively up yet -- and it genuinely
// wasn't: `create_tap_relay()` returns, and this relay starts
// polling, the instant the container-side tap device is
// *created*; `join_one_network()` (network_join.cpp), a
// *different* process, still has its own `ip addr add`/
// `ip link set <if> up` steps left to run afterward for that
// same device. For the first network joined this race is narrow
// enough that no frame ever arrives before those steps finish;
// for the second (and any later) network, something -- possibly
// just scheduling contention from the first network's own
// already-active relay and bridge -- reliably delivers a frame
// (confirmed to be as early as the very first `read()`) before
// the interface is up, and the previous code treated *any*
// write() failure as fatal, silently exiting for good, so that
// network never worked again for the rest of the session.
// Fixed by retrying specifically on EIO/ENETDOWN (both mean "the
// device isn't up yet", a startup race, not a torn-down
// namespace) with a short bounded backoff -- confirmed
// sufficient in practice, since `join_one_network()`'s own
// `ip link set ... up` for this exact device is always only
// milliseconds away by the time this is hit.
int device_not_up_retries = 0;
while (written < n) {
ssize_t w = write(to, buffer.data() + written, static_cast<size_t>(n - written));
if (w < 0) {
if (errno == EINTR) {
continue;
}
if ((errno == EIO || errno == ENETDOWN) && device_not_up_retries < 50) {
++device_not_up_retries;
struct timespec ts {
0, 20L * 1000000L // 20ms
};
while (nanosleep(&ts, &ts) != 0 && errno == EINTR) {
}
continue;
}
_exit(0);
}
written += w;