96efcf37e1
Second step of the tap+relay fallback for veth-less kernels (the real target device supports tun/tap but not veth). Reuses the existing bridge as the switching fabric -- provision_bridge()'s NAT/forwarding setup needs no changes -- and only replaces how a container's namespace gets connected to it: - a host-side tap device, created wherever the network's bridge lives and enslaved to it, playing veth's host-side role - a container-side tap device, created directly inside the container's namespace and named eth<N> from the start (no rename step needed) - a relay process holding both fds open, copying raw Ethernet frames bidirectionally between them -- reproducing a veth pair's kernel wire via one userspace hop Not wired into join_one_network() yet -- this commit only adds create_tap_relay()/stop_tap_relay() and exercises them standalone via a new self-test (throwaway bridge + throwaway namespace). A real synchronization bug turned up while writing that self-test: fork() returning to the parent doesn't mean the child has reached its own unshare(CLONE_NEWNET) yet, so using its pid immediately raced and created the container-side tap in the wrong (host) namespace. Fixed by polling namespace_isolated() first, the same guard network_join.cpp's wait_for_isolated_net_namespace() already uses for a real session. Verified twice as root via the doas rule: host-side tap gets created and attached to the bridge, container-side tap gets created with the right name inside the target namespace, and -- the biggest open assumption from the design doc addendum -- both devices disappear on their own once stop_tap_relay() stops the process, no explicit `ip link del` needed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
252 lines
9.7 KiB
C++
252 lines
9.7 KiB
C++
// Copyright (C) 2026 Viorel Munteanu
|
|
//
|
|
// This program is free software; you can redistribute it and/or modify
|
|
// it under the terms of the GNU General Public License as published by
|
|
// the Free Software Foundation; either version 2 of the License, or
|
|
// (at your option) any later version.
|
|
//
|
|
// This program is distributed in the hope that it will be useful,
|
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
// GNU General Public License for more details.
|
|
//
|
|
// You should have received a copy of the GNU General Public License along
|
|
// with this program; if not, write to the Free Software Foundation, Inc.,
|
|
// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
|
|
|
#include "network_tap_relay.h"
|
|
|
|
#include <fcntl.h>
|
|
#include <linux/if_tun.h>
|
|
#include <net/if.h>
|
|
#include <poll.h>
|
|
#include <sched.h>
|
|
#include <signal.h>
|
|
#include <sys/ioctl.h>
|
|
#include <sys/wait.h>
|
|
#include <unistd.h>
|
|
|
|
#include <cerrno>
|
|
#include <cstring>
|
|
#include <vector>
|
|
|
|
#include <fmt/core.h>
|
|
#include <spdlog/spdlog.h>
|
|
|
|
#include "network_bridge.h"
|
|
#include "persistent_netns.h"
|
|
#include "process.h"
|
|
|
|
namespace {
|
|
|
|
// Opens /dev/net/tun and creates a tap device named `name` in whatever
|
|
// network namespace this process is currently in -- IFF_NO_PI so both ends
|
|
// of a relay agree on raw-frame framing with no extra header, IFF_TAP (not
|
|
// IFF_TUN) so the host-side end can be enslaved to a bridge like any other
|
|
// Ethernet device. Deliberately not IFF_PERSIST: the device should disappear
|
|
// on its own once this fd (the only one ever opened on it) closes, the same
|
|
// property veth already has -- see stop_tap_relay()'s own doc comment.
|
|
// Returns -1 (logging why) on failure.
|
|
int open_tap(const std::string& name) {
|
|
int fd = open("/dev/net/tun", O_RDWR);
|
|
if (fd < 0) {
|
|
spdlog::error("failed to open /dev/net/tun: {}", strerror(errno));
|
|
return -1;
|
|
}
|
|
struct ifreq ifr {};
|
|
ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
|
|
std::strncpy(ifr.ifr_name, name.c_str(), IFNAMSIZ - 1);
|
|
if (ioctl(fd, TUNSETIFF, &ifr) < 0) {
|
|
spdlog::error("failed to create tap device '{}': {}", name, strerror(errno));
|
|
close(fd);
|
|
return -1;
|
|
}
|
|
return fd;
|
|
}
|
|
|
|
// setns(2) into the CLONE_NEWNET namespace named by the open file at `path`
|
|
// (e.g. /proc/<pid>/ns/net, or a persistent_netns.h path) -- the direct
|
|
// syscall equivalent of what every other namespace-crossing operation in
|
|
// this project reaches via shelling out to `nsenter`, needed here because
|
|
// this whole sequence must keep running (and later hold onto live fds) in
|
|
// the *same* process across each namespace switch, not just for the
|
|
// duration of one external command's argv.
|
|
bool enter_namespace(const std::string& path) {
|
|
int fd = open(path.c_str(), O_RDONLY);
|
|
if (fd < 0) {
|
|
spdlog::error("failed to open namespace {}: {}", path, strerror(errno));
|
|
return false;
|
|
}
|
|
bool ok = setns(fd, CLONE_NEWNET) == 0;
|
|
if (!ok) {
|
|
spdlog::error("failed to enter namespace {}: {}", path, strerror(errno));
|
|
}
|
|
close(fd);
|
|
return ok;
|
|
}
|
|
|
|
void report_line(int fd, const std::string& line) {
|
|
// Best-effort: if this write fails, the parent's own read-until-EOF loop
|
|
// still unblocks (with an empty/partial report) once report_fd is closed
|
|
// by this process exiting, which read_relay_report() already treats as
|
|
// failure.
|
|
ssize_t unused = write(fd, line.data(), line.size());
|
|
(void)unused;
|
|
}
|
|
|
|
// The relay child's entire lifetime, from namespace/device setup through the
|
|
// frame-copy loop -- never returns to its caller (always _exit()s, whether
|
|
// setup failed or the loop itself ended). Deliberately _exit(), not exit():
|
|
// this is a forked child, and _exit() skips flushing this process's own
|
|
// (copied-by-fork, possibly stale) buffered stdio state -- the same
|
|
// reasoning bwrap.cpp's kernel_supports_namespace() and this file's own
|
|
// probe_veth_support() (network_bridge.cpp) already rely on for their own
|
|
// throwaway forked children.
|
|
[[noreturn]] void relay_child_main(int report_fd, const NetworkEntry& network, const std::string& bridge,
|
|
const std::string& host_tap_name, pid_t container_ns_pid,
|
|
const std::string& container_if_name) {
|
|
if (network.kind == NetworkKind::intern) {
|
|
if (!enter_namespace(persistent_netns_path(network.name).string())) {
|
|
report_line(report_fd, "ERROR failed to enter network's persistent namespace\n");
|
|
_exit(1);
|
|
}
|
|
}
|
|
|
|
int fd_host = open_tap(host_tap_name);
|
|
if (fd_host < 0) {
|
|
report_line(report_fd, "ERROR failed to create host-side tap device\n");
|
|
_exit(1);
|
|
}
|
|
|
|
// Now running in whichever namespace network's bridge actually lives in
|
|
// (host root for extern, entered above for intern) -- exactly where
|
|
// network_join.cpp's join_one_network() attaches veth's host-side end,
|
|
// just via a direct setns() here instead of wrap_for_network()'s argv
|
|
// wrapping (this whole function needs to keep running across later
|
|
// namespace switches, not just for one external command's duration).
|
|
if (run_process({"ip", "link", "set", host_tap_name, "master", bridge}).exit_code != 0) {
|
|
report_line(report_fd, "ERROR failed to attach host-side tap device to bridge\n");
|
|
_exit(1);
|
|
}
|
|
if (run_process({"ip", "link", "set", host_tap_name, "up"}).exit_code != 0) {
|
|
report_line(report_fd, "ERROR failed to bring host-side tap device up\n");
|
|
_exit(1);
|
|
}
|
|
|
|
// The already-open fd_host stays valid across this switch -- fds aren't
|
|
// namespace-scoped, only their *creation* is, the same property
|
|
// slirp4netns and this project's own wrap_for_root_namespace() (bwrap.cpp)
|
|
// already rely on.
|
|
if (!enter_namespace(fmt::format("/proc/{}/ns/net", container_ns_pid))) {
|
|
report_line(report_fd, "ERROR failed to enter container's network namespace\n");
|
|
_exit(1);
|
|
}
|
|
|
|
int fd_container = open_tap(container_if_name);
|
|
if (fd_container < 0) {
|
|
report_line(report_fd, "ERROR failed to create container-side tap device\n");
|
|
_exit(1);
|
|
}
|
|
|
|
report_line(report_fd, "OK\n");
|
|
close(report_fd);
|
|
|
|
// The actual "veth wire", reimplemented in userspace: whatever arrives
|
|
// on one fd is written verbatim to the other. No SIGTERM handler is
|
|
// installed -- default disposition (terminate) already closes both fds
|
|
// on the way out, which is all that's needed for the "no explicit
|
|
// teardown" property documented on stop_tap_relay() to hold.
|
|
struct pollfd fds[2] = {{fd_host, POLLIN, 0}, {fd_container, POLLIN, 0}};
|
|
std::vector<char> buffer(65536);
|
|
while (true) {
|
|
int ready = poll(fds, 2, -1);
|
|
if (ready < 0) {
|
|
if (errno == EINTR) {
|
|
continue;
|
|
}
|
|
break;
|
|
}
|
|
for (int i = 0; i < 2; ++i) {
|
|
if ((fds[i].revents & (POLLIN | POLLERR | POLLHUP | POLLNVAL)) == 0) {
|
|
continue;
|
|
}
|
|
int from = fds[i].fd;
|
|
int to = fds[1 - i].fd;
|
|
ssize_t n = read(from, buffer.data(), buffer.size());
|
|
if (n <= 0) {
|
|
// A namespace this fd's device lived in was torn down, or a
|
|
// genuine error -- either way, this relay's job is done.
|
|
_exit(0);
|
|
}
|
|
ssize_t written = 0;
|
|
while (written < n) {
|
|
ssize_t w = write(to, buffer.data() + written, static_cast<size_t>(n - written));
|
|
if (w < 0) {
|
|
if (errno == EINTR) {
|
|
continue;
|
|
}
|
|
_exit(0);
|
|
}
|
|
written += w;
|
|
}
|
|
}
|
|
}
|
|
_exit(0);
|
|
}
|
|
|
|
std::string read_relay_report(int fd) {
|
|
std::string report;
|
|
char chunk[256];
|
|
ssize_t n;
|
|
while ((n = read(fd, chunk, sizeof(chunk))) > 0) {
|
|
report.append(chunk, static_cast<size_t>(n));
|
|
}
|
|
return report;
|
|
}
|
|
|
|
} // namespace
|
|
|
|
std::optional<TapRelayHandle> create_tap_relay(const NetworkEntry& network, const std::string& bridge,
|
|
const std::string& host_tap_name, pid_t container_ns_pid,
|
|
const std::string& container_if_name) {
|
|
int report_pipe[2];
|
|
if (pipe2(report_pipe, O_CLOEXEC) != 0) {
|
|
spdlog::error("failed to set up tap-relay report pipe: {}", strerror(errno));
|
|
return std::nullopt;
|
|
}
|
|
|
|
pid_t pid = fork();
|
|
if (pid < 0) {
|
|
spdlog::error("failed to fork tap relay for network '{}': {}", network.name, strerror(errno));
|
|
close(report_pipe[0]);
|
|
close(report_pipe[1]);
|
|
return std::nullopt;
|
|
}
|
|
if (pid == 0) {
|
|
close(report_pipe[0]);
|
|
relay_child_main(report_pipe[1], network, bridge, host_tap_name, container_ns_pid, container_if_name);
|
|
}
|
|
|
|
close(report_pipe[1]);
|
|
std::string report = read_relay_report(report_pipe[0]);
|
|
close(report_pipe[0]);
|
|
|
|
if (report.rfind("OK", 0) != 0) {
|
|
spdlog::error("failed to create tap relay for network '{}': {}", network.name,
|
|
report.empty() ? "no response from relay process" : report);
|
|
int status = 0;
|
|
waitpid(pid, &status, 0); // relay_child_main always _exit()s before this point on failure
|
|
return std::nullopt;
|
|
}
|
|
|
|
return TapRelayHandle{pid, host_tap_name};
|
|
}
|
|
|
|
void stop_tap_relay(const TapRelayHandle& handle) {
|
|
if (kill(handle.relay_pid, SIGTERM) != 0 && errno != ESRCH) {
|
|
spdlog::warn("failed to signal tap relay pid {}: {}", handle.relay_pid, strerror(errno));
|
|
}
|
|
int status = 0;
|
|
waitpid(handle.relay_pid, &status, 0);
|
|
}
|