slocker-lite/docs/networking-design.md

Network isolation design

Status: all six commits landed (see "Implementation plan: commit sequence" below for what shipped, including corrections found by testing along the way). Rootless networking and an nftables backend remain deliberately out of scope (see below). Captured 2026-08-30 on the networking branch.

Goal

Add persistent, named networks that -r/--run containers can join, similar in spirit to how -v/--volume already creates persistent named volumes:

• extern networks: reachable from/to the host's real network.
• intern networks: only reachable by other containers that joined the same network, never from the host or outside.
• -r/--run takes one or more -n/--network <name> flags to join networks.
• -p [<network>:]<host-port>:<container-port> forwards a host port into a container on one of its extern networks.

Why not slirp4netns

slirp4netns (and pasta) exist specifically to provide networking to an unprivileged network namespace without CAP_NET_ADMIN or host iptables access — a strict 1-to-1 usermode-NAT translator between the host and exactly one namespace. Neither tool bridges multiple namespaces together; Podman's own rootless multi-container networks get inter-container connectivity by putting every container of that network into one shared namespace holding a real Linux bridge, with only that one namespace getting a slirp4netns/ pasta instance for outside access.

This project's target device is Android and normally has root available (slocker-lite-priv-drop's whole reason to exist is making root-mode -r/--run first-class). Root-only for now — rootless networking is explicitly deferred as its own future task. Given root is assumed, the usermode-NAT workaround buys nothing: native Linux bridge/veth/iptables is faster (kernel-native routing/NAT, no usermode packet copy), simpler (no extra long-lived process per network to babysit), and is exactly what Docker/Podman themselves do when running as root. slirp4netns/pasta are dropped from the design entirely; the target device doesn't need either installed for this feature.

bwrap itself has no "join an existing network namespace" flag (confirmed via bwrap --help): it has --userns FD/--pidns FD (join-by-FD) for user/pid, but only --unshare-net (always create new) for networking. To put a container into a specific persistent namespace, the join happens the same way this project already joins the rootless containers-storage mount's namespace: wrap the bwrap invocation in nsenter --net=<path> first, and don't ask bwrap for --unshare-net on that invocation. bwrap.cpp's existing wrap_for_root_namespace() is precisely this pattern already (for mnt/user) and is directly reusable for net.

Mechanism

Both extern and intern networks use the same core mechanism — a Linux bridge, with one veth pair per joined container (container-side end moved into that container's own, still separately---unshare-net'ed namespace; the bridge-side end attached to the network's bridge). The only structural difference is where the bridge lives:

• extern: the bridge lives directly in the host's own root network namespace, so it already has a path outside via the host's real routing. Needs net.ipv4.ip_forward=1 (and the IPv6 forwarding sysctl, if IPv6 is enabled for that network) plus one iptables MASQUERADE (SNAT) rule for the bridge's subnet — the same pattern docker0 uses.
• intern: the bridge lives inside its own dedicated, free-standing namespace (persistent the way ip netns add keeps a namespace alive with no process in it — bind-mounting its ns/net file to a path that outlives any one process). No route out exists at all: a real structural isolation boundary, not merely "extern but without a NAT rule" (which would still be reachable from the host itself).

Each container always keeps its own private net namespace, so its own loopback and interface set never clash with another container's. Joining N networks is just N veth pairs into that one namespace — genuine multi-network membership falls out naturally, no namespace-sharing tricks needed.

Port forwarding (-p)

No slirp4netns hostfwd API to lean on, so this is implemented directly:

• An iptables DNAT rule in PREROUTING: --dport <host-port> -j DNAT --to-destination <container-veth-ip>:<container-port>.
• A FORWARD ACCEPT rule for the bridge's subnet — needed because a default FORWARD DROP policy (plausible on a stock Android kernel) would otherwise silently eat the forwarded traffic, the same gap Docker itself works around.

Syntax: -p [<network>:]<host-port>:<container-port>. <network> is optional — when omitted, resolves to whichever single extern network the container joined; an error (not a silent guess) if the container joined more than one extern network and didn't disambiguate.

Subnet / IP allocation

• IPv4 auto-allocates a /24 starting at 10.168.0.0/24, incrementing per network (10.168.1.0/24, 10.168.2.0/24, ...), with an optional --subnet <cidr> override at -n/--network creation time.
• IPv6 is a per-network on/off option, defaulting to enabled. When on, also auto-allocates a ULA /64 alongside the IPv4 block from a matching incrementing base (proposed: fd00:168:0:1::/64, fd00:168:0:2::/64, ... — mirroring the 168 from the v4 base so the two are visibly paired), with a --subnet6 <cidr> override. --no-ipv6 at creation time skips both the v6 allocation and any ip6tables/IPv6-forwarding setup for that network entirely.

CLI surface

-n/--network, dual-purpose like -v/--volume (alone creates/manages a network; combined with -r joins it) — with a simpler token shape than -v's, since a network join has no equivalent of a volume's container-mount-path second argument:

• Alone: -n <name> --extern|--intern [--subnet <cidr>] [--no-ipv6] [--subnet6 <cidr>] — <name> is -n's own required_argument (a single token, standard getopt_long); kind and the subnet/ipv6 options are ordinary separate flags rather than additional positional tokens.
• With -r: -n <name> is repeatable, one token (just the name) per occurrence — joins that network. No membership limit.
• --list-networks / --delete-network <name> round out the set, mirroring --list-volumes / --delete-volume.

Config file schema

New top-level networks section in config.yaml, parallel to volumes (config_file.{h,cpp}) — per entry: name, kind (extern/intern), subnet, ipv6 (bool), subnet6 (if ipv6). create_volume_command() / list_volumes_command() / delete_volume_command() (commands.cpp) are the direct templates to follow, including the shared tab-aligned listing helper for --list-networks.

A network's config-file entry is the durable source of truth, the same way a volume's directory path is. The live host-side state (namespace file for intern, bridge, veths, iptables rules) doesn't survive a reboot and is reconciled lazily and transparently — checked and, if missing/stale, recreated from the config entry — the first time anything needs it after a reboot (no explicit "start" step; the first -r/--run -n <name> or -p after boot just works).

Tooling

Shell out to iptables/ip6tables (via process.h's existing run_process(), matching how containers-storage/bwrap/fuse-overlayfs are already invoked) for MASQUERADE/DNAT/FORWARD rules, and to ip for bridge/veth/namespace management. Both need an on-device availability check the same way check_required_dependencies() already gates on containers-storage/bwrap.

iptables only, for now — confirmed available on the real target device; nftables is not currently installed there. nft support is real future work (its own task, once it actually matters — e.g. a device that only ships nft), not built preemptively as a dual-backend abstraction here.

Host-global state discipline

Unlike a bwrap session (self-contained via its own namespaces), bridges/veths/iptables rules are host-global, named, persistent resources that outlive any single process. Needs the same create-on-start/ remove-on-stop discipline already used for pid locks (pid_file.{h,cpp}) and cgroups (session_cgroup.{h,cpp}), plus a --clean-processes-style sweep for anything orphaned by a crash, so a dead slocker-lite doesn't leave stray bridges/veths/iptables rules behind forever.

Implementation plan: commit sequence

The whole feature is too large for one commit (unlike, say, --kill, which landed as a single commit despite touching several new files). Split into six commits, each a coherent, independently buildable and manually verifiable unit, in dependency order. Docs (CLAUDE.md/README.md) get updated within each commit, matching this branch's existing practice — not saved for a final pass.

1. Config schema + subnet/IPv6 allocation + -n/--network create/list/delete (config-only, no host-side effects yet)

   • config_file.{h,cpp}: new NetworkEntry {name, kind, subnet, ipv6, subnet6} (kind = extern/intern), a networks section — directly parallel to VolumeEntry/volumes.
   • New network_subnet.{h,cpp}: IPv4 /24 auto-allocation starting at 10.168.0.0/24 incrementing per existing network, --subnet override; paired IPv6 ULA /64 auto-allocation (fd00:168:0:N::/64) when enabled (default), --subnet6 override, --no-ipv6 to skip. Pure allocation logic against the already-loaded config's existing networks — no kernel/ ip/iptables calls in this commit.
   • cli_args.{h,cpp}: Mode::network/list_networks/delete_network, -n/--network (required_argument, single token = name; separate --extern/--intern/--subnet/--no-ipv6/--subnet6 flags for the create case), --list-networks, --delete-network <name> — directly mirroring -v/--volume's existing three-mode shape in the same file.
   • commands.cpp: create_network_command()/list_networks_command()/ delete_network_command() — mirroring create_volume_command()/ list_volumes_command()/delete_volume_command(), including the shared tab-aligned listing helper.
   • Verify: -n mynet --extern, -n other --intern --no-ipv6, --list-networks shows both with correct kind/subnet, --subnet/ --subnet6 overrides land correctly in config.yaml, --delete-network removes an entry. No bridges/namespaces/iptables rules exist yet — purely config bookkeeping, same as a freshly-created volume before it's ever mounted.

2. Persistent network namespace primitives (generic infra for intern networks)

   • New persistent_netns.{h,cpp}: create/find/remove a persistent network namespace kept alive with no process in it, the way ip netns add does (bind-mount a fresh namespace's ns/net onto a path that outlives the creating process) — narrow, reusable infra, no intern/extern branching or bridge logic here (parallels how session_cgroup.{h,cpp} stayed narrowly scoped to cgroup mechanics only).
   • Not wired into -n/--network yet in this commit.
   • Verify: a small manual exercise (or a -t/--test addition) creating a persistent namespace, confirming it survives after the creating process exits, then removing it.

3. Bridge provisioning for a network (idempotent — this is also the reboot- reconciliation mechanism, not a separate later step)

   • New network_bridge.{h,cpp}: given a NetworkEntry, ensure its bridge exists and is configured — creating it if missing (idempotent, so this doubles as "reconcile after reboot" with no separate code path):
     • extern: bridge in the host's own root namespace; assign it the gateway IP from the network's subnet; net.ipv4.ip_forward=1 (+ IPv6 forwarding sysctl if ipv6); one iptables MASQUERADE rule for the subnet (ip6tables too, if ipv6).
     • intern: bridge inside its own dedicated persistent_netns.h namespace (commit 2); gateway IP assigned; no forwarding, no NAT rule — no route out at all.
   • Wire this "ensure provisioned" call into create_network_command() (so creating a network actually stands up its bridge immediately) — later commits also call it lazily before a join, covering the reboot case.
   • check_required_dependencies()-style availability check added for ip/ iptables (and ip6tables when needed), alongside the existing containers-storage/bwrap check.
   • Verify: -n mynet --extern produces a real bridge with the expected gateway IP, ip_forward enabled, and a matching MASQUERADE rule (ip link show, iptables -t nat -L); an intern network's bridge exists in its own namespace with no such rule. Delete/recreate a network's config entry, delete its bridge by hand (ip link del), then trigger provisioning again (e.g. re-running --network create or the first join in commit 4) and confirm it comes back.

4. Joining networks at -r/--run time: veth creation, IP assignment, route

   • Repeatable -n <name> with -r/--run (cli_args.cpp, same repeatable-with--r shape -v/--volume already has).
   • commands.cpp's run_container(): once bwrap's pid (and via resolve_namespace_pid()-style lookup, sandbox_process.h, its actual net namespace) is known — same timing hook on_bwrap_pid_known/ on_start already provides for session locks/cgroups (bwrap.cpp) — for each joined network: ensure it's provisioned (commit 3, covers reboot recreation), create a veth pair, move the container-side end into the container's net namespace, attach the bridge-side end, assign the container's veth an IP from the subnet, and (for an extern join) set it as the default route.
   • This is the core connectivity commit — no veth pairs exist before it, regardless of how many networks are configured/joined.
   • Verify: two containers joined to the same intern network can ping each other and cannot reach the host or outside; a container joined to an extern network can reach the outside (and the host cannot reach it without commit 5's port forwarding); a container joined to both loses neither path (two interfaces, both functional).

5. -p port forwarding

   • cli_args.cpp: -p [<network>:]<host-port>:<container-port>, <network> optional (resolves to the container's sole extern network; error if ambiguous).
   • New port_forward.{h,cpp}: add/remove the iptables DNAT (PREROUTING) + FORWARD ACCEPT rule pair for one mapping, tied to the container's own session lifecycle the same create-on-start/ remove-on-stop way pid_file.{h,cpp}/session_cgroup.{h,cpp} already are.
   • Verify: -p 8080:80 against a container on an extern network answering on port 80 is reachable via curl localhost:8080 from the host; the rule is gone after the container exits.
   • Landed with two real corrections found by testing (see CLAUDE.md's port_forward.{h,cpp} entry for the full detail): the DNAT rule needs both PREROUTING and OUTPUT (locally-generated traffic never traverses PREROUTING); and curl localhost:<port> specifically still doesn't work even so (NAT hairpinning — the container sees an inbound packet claiming a loopback source on a non-loopback interface and drops it as martian) — verified instead via curl <host's real IP>:<port>, the actually-relevant path for real clients. Also surfaced, unrelated to -p itself but found while testing it: -x/--exec didn't join the net namespace (written back when this project never isolated networking at all), so it saw the host's network stack, not a network-isolated session's own — fixed in a follow-up commit (exec_session.cpp, see CLAUDE.md's own entry for that file).

6. Crash-orphan cleanup sweep

   • Extend --clean-processes (or add a dedicated --clean-networks, whichever reads better once this is reached) to find and remove bridges/veths/iptables rules left behind by a slocker-lite that died before its own teardown ran — mirroring clean_stale_sessions() (pid_file.cpp)'s existing stale-pid-file sweep, but for host-global network state instead of pid files.
   • Verify: kill -9 a running -r/--run session mid-flight (bypassing its normal cleanup), confirm the orphaned veth/iptables rule is detected and removed by the sweep, and that a still-running session's state is left untouched.
   • Landed narrower in scope than the bullet above once the actual orphan surface was worked out (see CLAUDE.md's port_forward.{h,cpp} entry for the full detail): veths need no sweep at all (the kernel tears down an entire pair once either end's namespace is destroyed — never survives a crash), and bridges/persistent namespaces are deliberately meant to always outlive any one session (that's the whole point of the reboot-reconciliation design, not something a crash changes). Only -p's iptables rules — host-global, named, with no automatic teardown — can actually outlive a crashed session, so that's the entire sweep: extended --clean-processes (not a separate flag) with clean_stale_port_forwards(), cross-referencing a small per-session port-forward record file against list_sessions()'s own liveness check. Verified via a controlled scratch test rather than a literal kill -9 on a root-owned slocker-lite process (not achievable through this session's scoped doas rule, which only permits running slocker-lite itself, not arbitrary commands like kill): a fabricated stale record was correctly detected, its removal attempted, and its file cleaned up, while a record matching a real running session was left untouched.

TUN/TAP fallback for veth-less kernels

Trigger: the real target device's kernel supports tun/tap (CONFIG_TUN — Android needs this for VpnService-based VPN apps) but not veth (CONFIG_VETH, commonly stripped from mobile kernels), so -n --extern/--intern as designed above (a veth pair per join) simply can't work there at all — ip link add ... type veth ... fails outright. Bridge support was separately confirmed working on this same device, which rules out the more complex fallback this section originally considered (see git log on this file for the superseded sketch: a per-network userspace Ethernet switch with no bridge dependency at all) in favor of a much smaller design.

Why tap can't just replace veth 1:1: a veth pair is two real kernel netdevices, switched between (or into a bridge) entirely by the kernel with zero userspace involvement. A tap device only has one kernel-side netdevice — the other "end" is a raw-Ethernet-frame file descriptor that only a userspace process can read/write, so there's no second kernel endpoint to attach to a bridge. This is exactly why slirp4netns/QEMU's own tap networking need a userspace process on the fd side at all.

Design, confirmed and implemented: per network-join, two tap devices + one small relay process that copies bytes 1:1 between them — a direct functional substitute for one veth pair, reusing the existing bridge as the switching fabric so provision_bridge() needs no changes at all:

• A host-side tap device, created wherever the network's bridge lives and enslaved to it — exactly veth's host-side role.
• A container-side tap device, created directly inside the container's own namespace, named eth<N> from the start — no peer-name-then- rename dance needed, unlike veth.
• A relay process holding both fds open, copying raw Ethernet frames bidirectionally between them for as long as it runs. This is the "veth wire," just implemented once in userspace instead of by the kernel.

Once the container-side tap exists as eth<N>, everything downstream — IP assignment, routes, the address handed to -p — is completely unchanged; only the interface-creation step is swapped. Strategy selection is per-network, per-join: should_use_veth(network) = network.veth && probe_veth_support(), mirroring the existing kernel-capability-vs-policy split namespace_policy_enabled() (bwrap.cpp) already uses for --unshare-xxx. --no-veth at network-creation time forces the fallback even on a veth-capable kernel — how this was actually tested, since the real target device wasn't available during development.

Implementation plan: commit sequence

Landed as four commits (a fifth, this doc update, closes it out) — see CLAUDE.md's own entries (network_bridge.{h,cpp}, network_tap_relay.{h,cpp}, network_join.{h,cpp}, self_test.{h,cpp}) for full file-by-file detail:

1. Veth capability probe + --no-veth flag, no relay yet. probe_veth_support() (fork, unshare(CLONE_NEWNET) into a throwaway namespace, try ip link add ... type veth ... there — the same kernel-capability-probing shape bwrap.cpp's own kernel_supports_namespace() already uses); NetworkEntry::veth + YAML round-trip; --no-veth CLI flag. Verified: probe_veth_support() returns true on this dev machine (a real veth pair is created successfully); --no-veth persists veth: false.
2. network_tap_relay.{h,cpp}: relay creation/loop/teardown, standalone. Verified via a new -t/--test case: a throwaway bridge + throwaway network namespace, confirming the host-side tap attaches to the bridge, the container-side tap appears inside the target namespace with the requested name, and — the biggest assumption going in — both devices disappear on their own once the relay is stopped, no explicit ip link del needed (neither is created with IFF_PERSIST).
3. Wire into join_one_network()/join_networks(). JoinedNetwork gains an optional relay handle; run_container() collects and stops them after run_bwrap() returns, mirroring -p's own active_port_forwards handling exactly. Two real bugs found here, not assumed: the relay child, unlike every other forked child in this project, never exec()s, so it inherited (and never closed) a live copy of daemonize.cpp's own report-pipe write end, hanging -D combined with -n indefinitely until fixed with an explicit close_inherited_fds(); and a first-attempt fix making --kill reach the relay directly (adding its pid to the session's own cgroup) was reverted after it caused a different bug (the cgroup's own removal, which happens before run_container() gets to stop the relay, started failing with EBUSY) — the ordinary flow already stops the relay correctly on its own, so the added complexity wasn't worth it. Verified end-to-end: two containers on a --no-veth extern network got distinct addresses via two tap+relay pairs (no veth at all) and pinged each other with 0% packet loss, repeatably.
4. Crash-orphan sweep. record_tap_relays()/clean_stale_tap_relays(), the direct structural analog of -p's own sweep, wired into --clean-processes. Verified the same way the port-forward sweep was: a real rootless session's pid alongside a hand-written matching record (left untouched) and a fabricated stale one (correctly swept).

Known gap: gateway/outside reachability unconfirmed

Confirmed by testing, not yet root-caused. Peer-to-peer connectivity through the tap+relay fallback is solid (verified above). Reaching the network's own gateway IP — and, in turn, the real outside through NAT — is not: neither ICMP nor a TCP wget ever got a response, despite ARP resolving correctly (the container's own ip neigh shows a REACHABLE entry with the gateway's real MAC, ruling out an L2/relay-framing problem). The identical bridge/subnet/host reached via veth instead of this fallback works perfectly — ruling out every environment-level explanation (host firewall, rp_filter, this dev sandbox's own networking) that would otherwise affect both paths equally, since those apply regardless of which mechanism connects the container. rp_filter=0 (tried at the host-tap, bridge, and global all scope) did not fix it. Diagnosing further needs host-level tools (tcpdump, direct iptables/sysctl inspection) this project's doas-scoped root access during development didn't permit (restricted to running slocker-lite itself, no other commands). This needs re-verification — ideally on the actual veth-less target device, where the environment differs and this dev sandbox's own unidentified cause may simply not apply — before extern/outside connectivity through this fallback is relied on. intern networks (peer-to-peer only, no gateway involved at all) are unaffected by this gap.

Explicitly out of scope for now

• Rootless networking. An earlier draft of this design considered a hybrid strategy (real bridge+veth when root, a simpler shared-network- namespace fallback when rootless, mirroring --kill's multi-strategy pattern). Shelved: root-only is sufficient for the actual target device today, and the rootless fallback has real limitations (single-network membership only, no per-container port isolation on it) not worth building before there's an actual rootless use case.
• nftables backend. iptables only, see above.