Document the tap+relay veth fallback in the design doc

Adds a dedicated section to docs/networking-design.md covering the
tap+relay fallback for veth-less kernels: the trigger (real target
device supports tun/tap but not veth), why tap can't 1:1 replace veth,
the confirmed design (two tap devices + a relay reusing the existing
bridge, replacing veth's earlier N-way-switch-daemon sketch once
bridge support was confirmed available), the four-commit
implementation sequence with what testing actually found (the fd-leak
deadlock, the reverted cgroup fix), and an honest writeup of the
unresolved gateway/outside-reachability gap.

README.md's -n/--network row now also flags that gap directly, next
to the existing NAT-hairpinning limitation note for -p.

This closes out the tap+relay fallback work for now: peer-to-peer
connectivity through it is solid and dev-verified; gateway/outside
reachability needs re-verification on the actual veth-less target
device before being relied on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
This commit is contained in:
2026-08-30 16:36:43 +00:00
parent fd234124e1
commit 53b859b7bf
2 changed files with 113 additions and 1 deletions
+1 -1
View File
@@ -92,7 +92,7 @@ slocker-lite -V|--version
| `--list-volumes` | List all named volumes (see `-v/--volume`) with their host directory. |
| `--delete-volume <name>` | Remove a named volume from the config. The host directory is left untouched. |
| `--delete-volume-full <name>` | Like `--delete-volume`, but also recursively deletes the volume's host directory. |
| `-n, --network <name>` | Create/manage a persistent named network: requires exactly one of `--extern` (a real Linux bridge in the host's own namespace, with NAT/forwarding set up so containers on it reach the host's real network) or `--intern` (a bridge inside its own dedicated, routeless namespace, only reachable by other containers on the same network). `--subnet <cidr>` overrides the auto-allocated IPv4 range (`10.168.0.0/24`, incrementing per network); `--no-ipv6` disables (and `--subnet6 <cidr>` overrides) the auto-allocated IPv6 range, on by default. `--no-veth` forces the tap+relay join fallback even on a kernel that supports veth (useful for testing that path; it's otherwise chosen automatically whenever the running kernel lacks veth support). With `--run`, instead joins `<name>` to the container as its own `eth<N>` interface with an address from the network's subnet; repeatable, no membership limit. Root-only for now. See [`docs/networking-design.md`](docs/networking-design.md). |
| `-n, --network <name>` | Create/manage a persistent named network: requires exactly one of `--extern` (a real Linux bridge in the host's own namespace, with NAT/forwarding set up so containers on it reach the host's real network) or `--intern` (a bridge inside its own dedicated, routeless namespace, only reachable by other containers on the same network). `--subnet <cidr>` overrides the auto-allocated IPv4 range (`10.168.0.0/24`, incrementing per network); `--no-ipv6` disables (and `--subnet6 <cidr>` overrides) the auto-allocated IPv6 range, on by default. `--no-veth` forces the tap+relay join fallback even on a kernel that supports veth (useful for testing that path; it's otherwise chosen automatically whenever the running kernel lacks veth support). With `--run`, instead joins `<name>` to the container as its own `eth<N>` interface with an address from the network's subnet; repeatable, no membership limit. Root-only for now. **Known gap**: on a network joined via the tap+relay fallback, peer-to-peer connectivity works, but reaching the network's own gateway (and thus the outside, for `extern`) does not yet — root cause unconfirmed, see [`docs/networking-design.md`](docs/networking-design.md). |
| `--list-networks` | List all named networks (see `-n/--network`) with their kind, IPv4 subnet, and IPv6 subnet (or `(no ipv6)`). |
| `--delete-network <name>` | Remove a named network from the config. |
| `-p, --port-forward [<network>:]<host-port>:<container-port>` | With `--run`, forward a TCP port from the host into the container. `<network>` is optional, defaulting to the container's sole `--extern` network (an error if it joined more than one without specifying). Repeatable. Reachable via the host's real, externally-facing IP; `localhost`/loopback access has a known NAT-hairpinning limitation (see [`docs/networking-design.md`](docs/networking-design.md)). |
+112
View File
@@ -313,6 +313,118 @@ pass.
attempted, and its file cleaned up, while a record matching a real
running session was left untouched.
## TUN/TAP fallback for veth-less kernels
**Trigger**: the real target device's kernel supports `tun`/`tap`
(`CONFIG_TUN` — Android needs this for `VpnService`-based VPN apps) but not
`veth` (`CONFIG_VETH`, commonly stripped from mobile kernels), so `-n
--extern`/`--intern` as designed above (a veth pair per join) simply can't
work there at all — `ip link add ... type veth ...` fails outright. Bridge
support was separately confirmed working on this same device, which rules
out the more complex fallback this section originally considered (see
`git log` on this file for the superseded sketch: a per-network userspace
Ethernet switch with no bridge dependency at all) in favor of a much
smaller design.
**Why tap can't just replace veth 1:1**: a veth pair is two real kernel
netdevices, switched between (or into a bridge) entirely by the kernel with
zero userspace involvement. A tap device only has *one* kernel-side
netdevice — the other "end" is a raw-Ethernet-frame file descriptor that
only a userspace process can read/write, so there's no second kernel
endpoint to attach to a bridge. This is exactly why `slirp4netns`/QEMU's own
tap networking need a userspace process on the fd side at all.
**Design, confirmed and implemented**: per network-join, two tap devices +
one small relay process that copies bytes 1:1 between them — a direct
functional substitute for one veth pair, **reusing the existing bridge as
the switching fabric** so `provision_bridge()` needs no changes at all:
- A **host-side tap device**, created wherever the network's bridge lives
and enslaved to it — exactly veth's host-side role.
- A **container-side tap device**, created directly inside the container's
own namespace, **named `eth<N>` from the start** — no peer-name-then-
rename dance needed, unlike veth.
- A **relay process** holding both fds open, copying raw Ethernet frames
bidirectionally between them for as long as it runs. This *is* the "veth
wire," just implemented once in userspace instead of by the kernel.
Once the container-side tap exists as `eth<N>`, everything downstream —
IP assignment, routes, the address handed to `-p` — is completely
unchanged; only the interface-creation step is swapped. Strategy selection
is per-network, per-join: `should_use_veth(network) = network.veth &&
probe_veth_support()`, mirroring the existing kernel-capability-vs-policy
split `namespace_policy_enabled()` (`bwrap.cpp`) already uses for
`--unshare-xxx`. `--no-veth` at network-creation time forces the fallback
even on a veth-capable kernel — how this was actually tested, since the
real target device wasn't available during development.
### Implementation plan: commit sequence
Landed as four commits (a fifth, this doc update, closes it out) — see
`CLAUDE.md`'s own entries (`network_bridge.{h,cpp}`, `network_tap_relay.{h,cpp}`,
`network_join.{h,cpp}`, `self_test.{h,cpp}`) for full file-by-file detail:
1. **Veth capability probe + `--no-veth` flag, no relay yet.**
`probe_veth_support()` (fork, `unshare(CLONE_NEWNET)` into a throwaway
namespace, try `ip link add ... type veth ...` there — the same
kernel-capability-probing shape `bwrap.cpp`'s own
`kernel_supports_namespace()` already uses); `NetworkEntry::veth` +
YAML round-trip; `--no-veth` CLI flag. Verified: `probe_veth_support()`
returns `true` on this dev machine (a real veth pair is created
successfully); `--no-veth` persists `veth: false`.
2. **`network_tap_relay.{h,cpp}`: relay creation/loop/teardown, standalone.**
Verified via a new `-t/--test` case: a throwaway bridge + throwaway
network namespace, confirming the host-side tap attaches to the bridge,
the container-side tap appears inside the target namespace with the
requested name, and — the biggest assumption going in — both devices
disappear on their own once the relay is stopped, no explicit
`ip link del` needed (neither is created with `IFF_PERSIST`).
3. **Wire into `join_one_network()`/`join_networks()`.** `JoinedNetwork`
gains an optional `relay` handle; `run_container()` collects and stops
them after `run_bwrap()` returns, mirroring `-p`'s own
`active_port_forwards` handling exactly. **Two real bugs found here, not
assumed**: the relay child, unlike every other forked child in this
project, never `exec()`s, so it inherited (and never closed) a live copy
of `daemonize.cpp`'s own report-pipe write end, hanging `-D` combined
with `-n` indefinitely until fixed with an explicit
`close_inherited_fds()`; and a first-attempt fix making `--kill` reach
the relay directly (adding its pid to the session's own cgroup) was
reverted after it caused a *different* bug (the cgroup's own removal,
which happens before `run_container()` gets to stop the relay, started
failing with `EBUSY`) — the ordinary flow already stops the relay
correctly on its own, so the added complexity wasn't worth it. Verified
end-to-end: two containers on a `--no-veth extern` network got distinct
addresses via two tap+relay pairs (no veth at all) and pinged each other
with 0% packet loss, repeatably.
4. **Crash-orphan sweep.** `record_tap_relays()`/`clean_stale_tap_relays()`,
the direct structural analog of `-p`'s own sweep, wired into
`--clean-processes`. Verified the same way the port-forward sweep was: a
real rootless session's pid alongside a hand-written matching record
(left untouched) and a fabricated stale one (correctly swept).
### Known gap: gateway/outside reachability unconfirmed
**Confirmed by testing, not yet root-caused.** Peer-to-peer connectivity
through the tap+relay fallback is solid (verified above). Reaching the
network's own gateway IP — and, in turn, the real outside through NAT — is
**not**: neither ICMP nor a TCP `wget` ever got a response, despite ARP
resolving correctly (the container's own `ip neigh` shows a `REACHABLE`
entry with the gateway's real MAC, ruling out an L2/relay-framing problem).
The identical bridge/subnet/host reached via veth instead of this fallback
works perfectly — ruling out every environment-level explanation (host
firewall, `rp_filter`, this dev sandbox's own networking) that would
otherwise affect both paths equally, since those apply regardless of which
mechanism connects the container. `rp_filter=0` (tried at the host-tap,
bridge, and global `all` scope) did not fix it. Diagnosing further needs
host-level tools (`tcpdump`, direct `iptables`/`sysctl` inspection) this
project's `doas`-scoped root access during development didn't permit
(restricted to running `slocker-lite` itself, no other commands). This
needs re-verification — ideally on the actual veth-less target device,
where the environment differs and this dev sandbox's own unidentified cause
may simply not apply — before extern/outside connectivity through this
fallback is relied on. `intern` networks (peer-to-peer only, no gateway
involved at all) are unaffected by this gap.
## Explicitly out of scope for now
- **Rootless networking.** An earlier draft of this design considered a