ceamac/slocker-lite
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix extern network connectivity: private namespace + uplink to host root

Browse Source 
extern networks had no connectivity at all on the real Android target
device -- confirmed by direct on-device testing (see
docs/networking-design.md's own writeup) that this wasn't a code bug in
the tap+relay mechanism itself, but the bridge's own location: extern's
bridge lived directly in the host's root network namespace, while
intern's already lived in its own dedicated persistent namespace and
always worked correctly. Relocating extern's bridge into the same kind
of private namespace fixed gateway reachability (both IPv4 and IPv6)
immediately, most likely because Android's own netd-managed
iptables/routing policy applies only to the root namespace and never
touches a network that's genuinely isolated in its own.

Doing that alone loses outside connectivity by construction -- a
genuinely isolated namespace has no uplink at all. Restoring it needed a
second, narrow mechanism: a point-to-point tap+relay link (reusing
network_tap_relay.h's existing primitive with a new
attach_host_side_to_bridge=false mode -- a plain routed link, not another
bridge port) between the private namespace and the host's root namespace,
on its own small deterministic transit subnet, with NAT applied only in
host root.

Three further pieces, each found and confirmed by direct real-device
testing (not assumed), were independently necessary for that uplink to
actually carry traffic -- dropping any one reproduces the original "no
connectivity to anything" symptom:
  1. The iptables FORWARD accept rule for the uplink must be *inserted at
     the front* of the chain (`-I FORWARD 1`), not appended. Android's own
     FORWARD chain unconditionally jumps through several subordinate
     chains first, one of which (tetherctrl_FORWARD, its tethering
     control chain) contains an unconditional DROP with no match criteria
     at all -- an appended rule is structurally unreachable, since DROP is
     already a terminal verdict long before a packet gets that far.
  2. An outbound `ip rule`, since a genuinely *forwarded* packet doesn't
     get the same routing treatment an interactive command does on this
     device: every `ip rule` landing in a table with a real route requires
     `iif lo` (locally-generated traffic only), so forwarded traffic falls
     through to a generic catch-all landing in a routeless table and never
     even reaches the FORWARD chain. Fixed by discovering, dynamically
     (via the same `ip route get` trick, not hardcoded), whichever table
     the host is actually using for its own real traffic right now, and
     routing the uplink's own traffic into it.
  3. A **return-path** `ip rule`, mirroring #2 for the reverse direction --
     confirmed via live /proc/net/nf_conntrack inspection during a real
     request that the outbound leg was already fully working (a genuine,
     tracked reply, not just a locally-generated packet succeeding), but
     the reply -- arriving back on the real interface and correctly
     de-MASQUERADEd to the transit-subnet address by conntrack -- still had
     nowhere to go: same "falls into a routeless table" failure, just for
     the destination address on the way back in.

IPv6 outside connectivity remains local-only (same-bridge reachability),
same as intern's IPv6 side already was -- deliberately, not a bug:
confirmed on the real device that neither ip6tables nor nftables can even
create an IPv6 NAT table on that kernel at all ("Not supported"), and the
device's own global IPv6 prefix rotates every ~10 minutes, too short-lived
to build stable addressing on top of via the alternative (NDP proxying).

Verified end-to-end on the real target device, from a clean state:
gateway IPv4 0% loss, gateway IPv6 0% loss, and a real outside destination
(8.8.8.8) 0% loss (3/3 replies) through a container on a freshly created
extern network. Also verified on this dev machine: self-test, a plain
veth-capable join, the --no-veth tap+relay fallback, and intern (still
unaffected -- no uplink, "Network unreachable" for outside as intended).

self_test.cpp's own tap-relay test needed a small matching update: it
constructs its own throwaway extern NetworkEntry and calls
create_tap_relay() directly, which now unconditionally enters the
network's persistent namespace first (both kinds, matching
wrap_for_network()'s own change) -- the test now provisions one for its
own throwaway network the same way a real network would be.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
This commit is contained in:
Viorel Munteanu
2026-08-31 12:19:48 +00:00
parent bebf559039
commit fd3f25b0b2
5 changed files with 470 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/network_bridge.cpp
+281 -21
View File
@@ -22,6 +22,8 @@
#include <array>
#include <cstdint>
#include <fstream>
#include <optional>
#include <string>
#include <string_view>
#include <vector>
@@ -30,7 +32,9 @@
#include <spdlog/spdlog.h>
#include "network_subnet.h"
#include "network_tap_relay.h"
#include "persistent_netns.h"
#include "pid_file.h"
#include "process.h"
namespace {
@@ -51,9 +55,6 @@ uint32_t fnv1a(std::string_view s) {
std::string bridge_name(const std::string& network_name) { return fmt::format("slk{:08x}", fnv1a(network_name)); }
std::vector<std::string> wrap_for_network(const NetworkEntry& network, std::vector<std::string> argv) {
if (network.kind == NetworkKind::extern_) {
return argv;
}
std::vector<std::string> wrapped = {"nsenter", fmt::format("--net={}", persistent_netns_path(network.name).string()),
"--"};
wrapped.insert(wrapped.end(), argv.begin(), argv.end());
@@ -140,9 +141,13 @@ bool provision_bridge(const NetworkEntry& network, const std::string& bridge) {
// prefix, which this project doesn't do). So `extern`'s IPv6
// side behaves the same as `intern`'s already does: real
// same-bridge reachability between containers, no route to the
// actual internet. Forwarding is still enabled (harmless, global,
// symmetric with the IPv4 case) in case it's ever useful for
// routing between networks some other way.
// actual internet. Confirmed on the real target device, not just
// theorized: neither `ip6tables` nor `nftables` can even create
// an IPv6 NAT table on that kernel at all ("Not supported"),
// so this isn't achievable there regardless. Forwarding is still
// enabled (harmless, global, symmetric with the IPv4 case) in
// case it's ever useful for routing between networks some other
// way.
if (!run_admin_command(network, {"sysctl", "-w", "net.ipv6.conf.all.forwarding=1"},
"enable IPv6 forwarding")) {
return false;
@@ -153,18 +158,261 @@ bool provision_bridge(const NetworkEntry& network, const std::string& bridge) {
return true;
}
// Everything below stands up (and tears down) `extern`'s uplink -- a
// point-to-point tap+relay pair (network_tap_relay.h, reused with its
// `attach_host_side_to_bridge=false` mode: a plain routed link, not another
// bridge port) connecting the network's private namespace to the host's own
// root namespace, needed because relocating the bridge into a private
// namespace above -- the fix for the connectivity bug -- also removes its
// only path to the real network, by construction: a genuinely isolated
// namespace has no uplink at all otherwise.
//
// Names and addresses are deterministic functions of network.name, the same
// fnv1a()-based convention bridge_name() already uses, so
// teardown_uplink_state() (a possibly separate process invocation, e.g. a
// later --delete-network-full) can recompute them without needing to read
// them back from anywhere.
std::string uplink_root_tap_name(const std::string& network_name) {
return fmt::format("uph{:08x}", fnv1a(network_name));
}
std::string uplink_netns_tap_name(const std::string& network_name) {
return fmt::format("upn{:08x}", fnv1a(network_name));
}
// Derives a small, deterministic /30 transit pair in 169.254.0.0/16
// (link-local, never globally routed) from network_name -- one address for
// each end of the uplink.
std::pair<std::string, std::string> uplink_transit_addresses(const std::string& network_name) {
uint32_t hash = fnv1a("uplink-" + network_name);
uint8_t byte2 = static_cast<uint8_t>((hash >> 8) & 0x3f);
uint8_t byte3 = static_cast<uint8_t>(hash & 0xfc); // multiple of 4 -> a /30 block
std::string netns_addr = fmt::format("169.254.{}.{}/30", byte2, byte3 + 1);
std::string root_addr = fmt::format("169.254.{}.{}/30", byte2, byte3 + 2);
return {netns_addr, root_addr};
}
// The /30 block itself (the ".0" base address), for the MASQUERADE rule's
// -s match and the return-path ip rule's -to match -- same derivation as
// uplink_transit_addresses() above.
std::string uplink_transit_subnet(const std::string& network_name) {
uint32_t hash = fnv1a("uplink-" + network_name);
uint8_t byte2 = static_cast<uint8_t>((hash >> 8) & 0x3f);
uint8_t byte3 = static_cast<uint8_t>(hash & 0xfc);
return fmt::format("169.254.{}.{}/30", byte2, byte3);
}
std::string strip_prefix(const std::string& cidr) { return cidr.substr(0, cidr.find('/')); }
// $XDG_STATE_HOME/slocker-lite/network-uplinks/<sanitized-network-name> --
// just the relay's pid as text, since both tap device names are already
// deterministically recomputable from network_name alone (see above). Lets
// teardown_uplink_state() find and stop a relay that was created by a
// completely different, earlier process invocation --
// ensure_network_provisioned() runs at network-creation time, potentially
// long before any --delete-network-full.
std::filesystem::path uplink_state_path(const std::string& network_name) {
return xdg_state_dir() / "network-uplinks" / sanitize_for_filename(network_name);
}
// Runs `ip route get <a well-known public address>` in host root and parses
// out the `table <N>` it names -- the routing table Android is *actually*
// using for real traffic right now. Needed because a genuinely forwarded
// packet (arriving via the uplink's own host-root-side tap, not locally
// generated) doesn't get the same routing treatment plain interactive
// commands do: confirmed on the real target device via `ip rule show` --
// every rule landing in a table with a real route requires `iif lo`
// (locally-generated traffic only, which is why an interactive `ip route
// get` always looked fine on its own); a forwarded packet instead falls
// through to a generic `fwmark 0/0x10000` catch-all landing in an unrelated,
// routeless table, and gets silently dropped before ever reaching the
// FORWARD chain at all -- no route, no forwarding decision to make. The
// table number itself is Android-version/network-specific (varies with
// which real network -- WiFi, cellular -- is currently active), so it's
// discovered dynamically here rather than hardcoded. Returns nullopt if no
// explicit table is named (the plain `main` table already applies then, no
// extra rule needed) or the lookup itself fails.
std::optional<std::string> discover_default_table() {
auto result = run_process({"ip", "route", "get", "8.8.8.8"});
if (result.exit_code != 0) {
return std::nullopt;
}
auto pos = result.stdout_output.find(" table ");
if (pos == std::string::npos) {
return std::nullopt;
}
pos += 7;
auto end = result.stdout_output.find(' ', pos);
return result.stdout_output.substr(pos, end == std::string::npos ? std::string::npos : end - pos);
}
bool uplink_provisioned(const std::string& root_tap) {
return run_process({"ip", "link", "show", root_tap}).exit_code == 0;
}
// Idempotent (see uplink_provisioned() above): a no-op if the host-root-side
// device already exists. Three distinct pieces are needed beyond the
// tap+relay link and its addresses/route/NAT to actually carry traffic to
// and from the real internet -- each found and confirmed by direct
// real-device testing, not assumed, and each independently necessary
// (dropping any one reproduces the original "no connectivity to anything"
// symptom):
// 1. An iptables FORWARD accept rule for the uplink's own interface,
// **inserted at the very front** of the chain (`-I FORWARD 1`), not
// appended. Android's own FORWARD chain unconditionally jumps through
// several of its own subordinate chains before reaching anything else
// -- `tetherctrl_FORWARD` (its tethering-control chain) among them
// contains one unconditional `DROP` with no match criteria at all, so
// every forwarded packet reaches it and dies there regardless of
// interface; appending our own accept rule after that point is
// structurally guaranteed to never be reached, since DROP is already a
// terminal verdict. Inserting at the front pre-empts it entirely.
// 2. An outbound `ip rule`, routing the uplink's own traffic into
// whichever table discover_default_table() names -- see that
// function's own doc comment.
// 3. A **return-path** `ip rule`, mirroring #2 for the reverse direction:
// confirmed via live testing (reading /proc/net/nf_conntrack during a
// real request showed the outbound leg fully working -- a genuine,
// tracked reply, not merely a locally-generated packet succeeding) that
// the *reply*, arriving back on whichever real interface is active and
// correctly de-MASQUERADEd by conntrack back to this uplink's own
// transit-subnet address, still had nowhere to go: `ip route get
// <transit-addr> from <remote> iif <real-interface>` returned "Network
// unreachable", the exact same "falls into a routeless table" failure
// as #2, just for a packet whose *destination* (not source) is now the
// transit subnet, arriving on a real interface instead of the uplink's
// own. Routes by destination into the plain `main` table, which
// already has the directly-connected route to this subnet, regardless
// of which real interface a reply happens to arrive on.
bool ensure_uplink_provisioned(const NetworkEntry& network) {
std::string root_tap = uplink_root_tap_name(network.name);
if (uplink_provisioned(root_tap)) {
return true;
}
std::string netns_tap = uplink_netns_tap_name(network.name);
auto relay = create_tap_relay(network, /*bridge=*/"", netns_tap, /*container_ns_pid=*/getpid(), root_tap,
/*attach_host_side_to_bridge=*/false);
if (!relay) {
spdlog::error("failed to create uplink relay for network '{}'", network.name);
return false;
}
relay->root_side_tap_name = root_tap;
auto [netns_addr, root_addr] = uplink_transit_addresses(network.name);
std::string transit_subnet = uplink_transit_subnet(network.name);
bool ok = run_admin_command(network, {"ip", "addr", "add", netns_addr, "dev", netns_tap},
"assign uplink address inside the private namespace");
// Host root from here on -- unwrapped, always host root by construction
// (the relay's own second setns() already put its two devices there).
ok = ok && run_process({"ip", "addr", "add", root_addr, "dev", root_tap}).exit_code == 0;
ok = ok && run_process({"ip", "link", "set", root_tap, "up"}).exit_code == 0;
ok = ok && run_admin_command(network,
{"ip", "route", "replace", "default", "via", strip_prefix(root_addr), "dev", netns_tap},
"set the private namespace's default route via the uplink");
ok = ok && run_process({"sysctl", "-w", "net.ipv4.ip_forward=1"}).exit_code == 0;
ok = ok && run_process({"iptables", "-t", "nat", "-A", "POSTROUTING", "-s", transit_subnet, "!", "-o", root_tap,
"-j", "MASQUERADE"})
.exit_code == 0;
ok = ok && run_process({"iptables", "-I", "FORWARD", "1", "-o", root_tap, "-j", "ACCEPT"}).exit_code == 0;
ok = ok && run_process({"iptables", "-I", "FORWARD", "1", "-i", root_tap, "-j", "ACCEPT"}).exit_code == 0;
if (auto table = discover_default_table()) {
ok = ok &&
run_process({"ip", "rule", "add", "priority", "100", "iif", root_tap, "lookup", *table}).exit_code == 0;
} else {
spdlog::warn(
"could not determine host root's own default routing table for network '{}' -- forwarded traffic "
"may not find a route",
network.name);
}
ok = ok &&
run_process({"ip", "rule", "add", "priority", "100", "to", transit_subnet, "lookup", "main"}).exit_code == 0;
if (!ok) {
spdlog::error("failed to configure uplink for network '{}'", network.name);
stop_tap_relay(*relay);
return false;
}
std::error_code ec;
auto path = uplink_state_path(network.name);
std::filesystem::create_directories(path.parent_path(), ec);
std::ofstream out(path);
if (out) {
out << relay->relay_pid << '\n';
} else {
spdlog::warn("failed to record uplink relay pid for network '{}' -- a later teardown won't find it",
network.name);
}
return true;
}
// Reverses ensure_uplink_provisioned() above, one step per piece it added
// (best-effort throughout, same warn-and-continue policy as teardown_step()
// -- a step failing because that piece was already gone by hand is the
// expected common case, not an error). discover_default_table() is called
// again here rather than the table number being stored, since the same call
// already needs to run once at provisioning time anyway and the two should
// almost always agree; a network change in between (the host switching from
// WiFi to cellular, say) could mean this guesses a different table than the
// one actually added, in which case the `ip rule del` for it just fails
// harmlessly like any other already-gone piece.
void teardown_uplink_state(const NetworkEntry& network) {
std::string root_tap = uplink_root_tap_name(network.name);
std::string transit_subnet = uplink_transit_subnet(network.name);
if (run_process({"ip", "rule", "del", "priority", "100", "to", transit_subnet, "lookup", "main"}).exit_code !=
0) {
spdlog::warn("failed to remove uplink return-path ip rule for network '{}' (already gone, or never existed)",
network.name);
}
if (auto table = discover_default_table()) {
if (run_process({"ip", "rule", "del", "priority", "100", "iif", root_tap, "lookup", *table}).exit_code != 0) {
spdlog::warn("failed to remove uplink ip rule for network '{}' (already gone, or never existed)",
network.name);
}
}
if (run_process({"iptables", "-D", "FORWARD", "-o", root_tap, "-j", "ACCEPT"}).exit_code != 0) {
spdlog::warn("failed to remove uplink FORWARD accept rule (outbound) for network '{}'", network.name);
}
if (run_process({"iptables", "-D", "FORWARD", "-i", root_tap, "-j", "ACCEPT"}).exit_code != 0) {
spdlog::warn("failed to remove uplink FORWARD accept rule (inbound) for network '{}'", network.name);
}
if (run_process({"iptables", "-t", "nat", "-D", "POSTROUTING", "-s", transit_subnet, "!", "-o", root_tap, "-j",
"MASQUERADE"})
.exit_code != 0) {
spdlog::warn("failed to remove uplink MASQUERADE rule for network '{}' (already gone, or never existed)",
network.name);
}
auto path = uplink_state_path(network.name);
std::ifstream in(path);
pid_t relay_pid = 0;
if (in && (in >> relay_pid) && relay_pid > 0) {
TapRelayHandle handle{relay_pid, uplink_netns_tap_name(network.name), network, root_tap};
stop_tap_relay(handle);
} else {
// No record (never provisioned, or already torn down) -- still try
// to remove the host-root-side device directly, best-effort, in
// case a record was lost without the device itself going away.
run_process({"ip", "link", "del", root_tap});
}
std::error_code ec;
std::filesystem::remove(path, ec);
}
} // namespace
bool check_network_dependencies(const NetworkEntry& network) {
std::vector<std::string_view> tools = {"ip"};
std::vector<std::string_view> tools = {"ip", "nsenter"};
if (network.kind == NetworkKind::extern_) {
// `ip6tables` deliberately not required even when `ipv6` -- no
// ip6tables call is ever made (see provision_bridge()'s own
// MASQUERADE comment above for why).
tools.push_back("iptables");
tools.push_back("sysctl");
} else {
tools.push_back("nsenter");
}
bool all_found = true;
@@ -211,24 +459,36 @@ bool ensure_network_provisioned(const NetworkEntry& network) {
return false;
}
if (network.kind == NetworkKind::intern && !persistent_netns_exists(network.name)) {
if (!persistent_netns_exists(network.name)) {
if (!create_persistent_netns(network.name)) {
return false;
}
}
std::string bridge = bridge_name(network.name);
if (bridge_exists(network, bridge)) {
return true; // already provisioned
if (!bridge_exists(network, bridge) && !provision_bridge(network, bridge)) {
return false;
}
return provision_bridge(network, bridge);
// The uplink step must run even when the bridge already existed (an
// early `return true` here would skip straight past it) --
// ensure_uplink_provisioned() has its own device-existence idempotency
// check, so calling it on every already-provisioned extern network is
// cheap, and this is also what lets a network created before the uplink
// mechanism existed pick it up.
if (network.kind == NetworkKind::extern_ && !ensure_uplink_provisioned(network)) {
return false;
}
return true;
}
void teardown_network_state(const NetworkEntry& network) {
std::string bridge = bridge_name(network.name);
if (network.kind == NetworkKind::extern_) {
teardown_uplink_state(network);
// Remove the MASQUERADE rule before the bridge itself -- purely for
// tidiness, iptables doesn't require the referenced interface to
// still exist for the rule to be removable. No IPv6 counterpart:
@@ -237,13 +497,13 @@ void teardown_network_state(const NetworkEntry& network) {
{"iptables", "-t", "nat", "-D", "POSTROUTING", "-s", network.subnet, "!", "-o", bridge, "-j",
"MASQUERADE"},
"remove IPv4 MASQUERADE rule");
teardown_step(network, {"ip", "link", "del", bridge}, "delete bridge");
} else {
// Removing the whole persistent namespace destroys everything
// inside it -- the bridge included -- in one step; no separate
// `ip link del` needed.
if (persistent_netns_exists(network.name) && !remove_persistent_netns(network.name)) {
spdlog::warn("failed to remove persistent namespace for network '{}'", network.name);
}
}
// Removing the whole persistent namespace destroys everything inside it
// -- the bridge, and for extern the uplink's own private-namespace-side
// tap device, both included -- in one step; no separate `ip link del`
// needed for those.
if (persistent_netns_exists(network.name) && !remove_persistent_netns(network.name)) {
spdlog::warn("failed to remove persistent namespace for network '{}'", network.name);
}
}
src/network_bridge.h
+78 -47
View File
@@ -31,22 +31,32 @@
// bridge this file provisioned.
std::string bridge_name(const std::string& network_name);
// Wraps `argv` so it runs wherever `network`'s bridge actually lives: as-is
// for `extern` (the host's own root namespace -- this whole feature is
// root-only for now, so slocker-lite's own current namespace already is the
// right one); through `nsenter --net=<persistent path>` for `intern` (its
// own dedicated namespace, persistent_netns.h). Exported so network_join.h
// can run its own veth-setup commands (creating the pair, attaching the
// bridge-side end) in that same place, not just this file's own
// provisioning commands.
// Wraps `argv` so it runs wherever `network`'s bridge actually lives: through
// `nsenter --net=<persistent path>` into its own dedicated persistent
// namespace (persistent_netns.h) -- both kinds, `extern` included. Exported
// so network_join.h can run its own veth-setup commands (creating the pair,
// attaching the bridge-side end) in that same place, not just this file's
// own provisioning commands.
//
// **`extern`'s bridge used to live directly in the host's own root
// namespace** -- confirmed by real on-device testing to be the actual cause
// of `extern` having no connectivity at all on the real Android target
// device (gateway included, both IPv4 and IPv6), almost certainly Android's
// own `netd`-managed iptables/routing policy applying only in the root
// namespace and never to a network genuinely isolated in its own namespace
// (exactly why `intern`, unaffected by any of this, always worked). Giving
// `extern` the same private-namespace treatment `intern` already had fixed
// gateway reachability immediately; restoring real outside connectivity on
// top of that needed the uplink mechanism in ensure_network_provisioned()'s
// own doc comment below.
std::vector<std::string> wrap_for_network(const NetworkEntry& network, std::vector<std::string> argv);
// Checks that the external tools provisioning `network` needs are found in
// PATH, logging which are missing: always `ip`; `iptables`/`sysctl` for
// `extern` (no `ip6tables`, even when `network.ipv6` -- see
// ensure_network_provisioned()'s own comment on why no ip6tables call is
// ever made); `nsenter` for `intern` (to reach its dedicated persistent
// namespace). Same shape/spirit as commands.cpp's own
// PATH, logging which are missing: always `ip` and `nsenter` (both kinds now
// reach their bridge through a private namespace, see wrap_for_network()
// above); `iptables`/`sysctl` for `extern` (no `ip6tables`, even when
// `network.ipv6` -- see ensure_network_provisioned()'s own comment on why no
// ip6tables call is ever made). Same shape/spirit as commands.cpp's own
// check_required_dependencies(), kept separate since this file's tool set
// depends on the network's own kind/ipv6 setting.
bool check_network_dependencies(const NetworkEntry& network);
@@ -72,43 +82,64 @@ bool probe_veth_support();
// namespace_policy_enabled() (bwrap.cpp) already uses for --unshare-xxx.
bool should_use_veth(const NetworkEntry& network);
// Ensures `network`'s bridge (and, for `intern`, its dedicated persistent
// namespace -- persistent_netns.h) exists and is configured, creating
// whatever's missing:
// - extern: bridge in the host's own root namespace (this whole feature is
// root-only for now, see docs/networking-design.md, so no nsenter
// wrapping is needed to reach it); net.ipv4.ip_forward=1 and one iptables
// MASQUERADE rule for the subnet; if network.ipv6, also the IPv6
// forwarding sysctl, but deliberately **no** ip6tables MASQUERADE rule --
// the fd00::/8 ULA addresses network_subnet.h allocates are
// non-globally-routable by design (RFC 4193), so NAT66 for them isn't
// correct IPv6 practice to begin with (also confirmed not universally
// supported: some ip6tables builds lack a MASQUERADE target at all).
// extern's IPv6 side is thus same-bridge reachability only, same as
// intern's IPv6 already is.
// - intern: bridge inside network's own dedicated persistent namespace
// (created here if it doesn't exist yet); no forwarding sysctl, no NAT
// rule -- no route out at all.
// Idempotent: a no-op (true) if the bridge already exists. This doubles as
// the mechanism that transparently recreates a network's host-side state
// after a reboot (nothing about it survives one except the config.yaml
// entry) -- there's no separate "reconcile" path; calling this again just
// recreates whatever's missing.
// Ensures `network`'s bridge, its dedicated persistent namespace
// (persistent_netns.h -- both kinds now, see wrap_for_network()'s own
// comment above), and, for `extern`, its uplink out to the host's real
// network all exist and are configured, creating whatever's missing:
// - Both kinds: bridge inside the network's own dedicated persistent
// namespace (created here if it doesn't exist yet).
// - extern only, inside that same private namespace: net.ipv4.ip_forward=1
// and one iptables MASQUERADE rule for the container subnet; if
// network.ipv6, also the IPv6 forwarding sysctl, but deliberately **no**
// ip6tables MASQUERADE rule -- the fd00::/8 ULA addresses
// network_subnet.h allocates are non-globally-routable by design
// (RFC 4193), so NAT66 for them isn't correct IPv6 practice to begin
// with; also confirmed on the real target device that ip6tables/nftables
// both lack any IPv6 NAT support at the kernel level at all, so this
// isn't even achievable there regardless. extern's IPv6 side is thus
// same-bridge reachability only, same as intern's IPv6 already is -- a
// deliberate, confirmed limitation, not a bug (see
// docs/networking-design.md for the full investigation).
// - extern only, additionally: an **uplink** out of that private namespace
// to the host's own root namespace -- ensure_uplink_provisioned()
// (`.cpp`-local), a point-to-point tap+relay pair (network_tap_relay.h,
// reused with a new non-bridged mode) on its own small deterministic
// transit subnet, with NAT and routing set up in host root. Needed
// because relocating the bridge into a private namespace (the fix for
// the connectivity bug above) also removes its only path to the real
// network by construction -- a genuinely isolated namespace has no
// uplink at all otherwise. See its own doc comment for the three
// distinct, real-device-confirmed pieces this needs (a plain default
// route alone is not sufficient on Android): an inserted-at-the-front
// iptables FORWARD accept rule, and two `ip rule`s (outbound and
// return-path) routing this traffic into whichever policy-routing table
// the host is actually using for its own real traffic right now.
// Idempotent: a no-op (true) if the bridge already exists -- **except** the
// uplink step for extern, which still runs (with its own, separate
// idempotency check) even when the bridge already existed, so an
// already-provisioned network from before this uplink mechanism existed
// still picks it up. This doubles as the mechanism that transparently
// recreates a network's host-side state after a reboot (nothing about it
// survives one except the config.yaml entry) -- there's no separate
// "reconcile" path; calling this again just recreates whatever's missing.
bool ensure_network_provisioned(const NetworkEntry& network);
// Tears down `network`'s live host-side state -- the reverse of
// ensure_network_provisioned(). For `extern`: removes the IPv4 MASQUERADE
// rule (purely for tidiness, since iptables doesn't require the interface a
// rule references to still exist -- no IPv6 counterpart, since none is ever
// added, see ensure_network_provisioned()'s own comment), then deletes the
// bridge itself. For `intern`: removes
// the whole persistent namespace (persistent_netns.h) in one step, which
// destroys everything inside it -- the bridge included -- with no separate
// `ip link del` needed. Deliberately does **not** touch the IPv4/IPv6
// forwarding sysctls `provision_bridge()` enables for `extern` -- those are
// global host state shared across every `extern` network, not per-network,
// so disabling them here could break others still relying on them.
// Best-effort, like every other host-global teardown in this project
// ensure_network_provisioned(). For `extern`: first tears down the uplink
// (teardown_uplink_state(), `.cpp`-local -- stops the relay, removes its
// host-root-side device, and removes the FORWARD/MASQUERADE/ip-rule state
// added for it), then removes the IPv4 MASQUERADE rule for the container
// subnet (purely for tidiness, since iptables doesn't require the interface
// a rule references to still exist -- no IPv6 counterpart, since none is
// ever added, see ensure_network_provisioned()'s own comment). Both kinds
// then remove the whole persistent namespace (persistent_netns.h) in one
// step, which destroys everything left inside it -- the bridge and the
// uplink's own private-namespace-side tap device included -- with no
// separate `ip link del` needed for those. Deliberately does **not** touch
// the IPv4/IPv6 forwarding sysctls `provision_bridge()` enables for `extern`
// -- those are global host state shared across every `extern` network, not
// per-network, so disabling them here could break others still relying on
// them. Best-effort, like every other host-global teardown in this project
// (`remove_session_cgroup()`, `remove_port_forward()`): logs a warning and
// keeps going past any individual step that fails, rather than stopping --
// a step failing because that piece was already removed by hand (see
src/network_tap_relay.cpp
+32 -15
View File
@@ -179,14 +179,14 @@ void report_line(int fd, const std::string& line) {
// throwaway forked children.
[[noreturn]] void relay_child_main(int report_fd, const NetworkEntry& network, const std::string& bridge,
const std::string& host_tap_name, pid_t container_ns_pid,
const std::string& container_if_name) {
const std::string& container_if_name, bool attach_host_side_to_bridge) {
close_inherited_fds(report_fd);
if (network.kind == NetworkKind::intern) {
if (!enter_namespace(persistent_netns_path(network.name).string())) {
report_line(report_fd, "ERROR failed to enter network's persistent namespace\n");
_exit(1);
}
// Both kinds now -- network_bridge.cpp's wrap_for_network() reaches the
// bridge the same way, through its own dedicated persistent namespace.
if (!enter_namespace(persistent_netns_path(network.name).string())) {
report_line(report_fd, "ERROR failed to enter network's persistent namespace\n");
_exit(1);
}
if (!create_persistent_tap(host_tap_name)) {
@@ -200,12 +200,17 @@ void report_line(int fd, const std::string& line) {
}
// Now running in whichever namespace network's bridge actually lives in
// (host root for extern, entered above for intern) -- exactly where
// network_join.cpp's join_one_network() attaches veth's host-side end,
// just via a direct setns() here instead of wrap_for_network()'s argv
// wrapping (this whole function needs to keep running across later
// namespace switches, not just for one external command's duration).
if (run_process({"ip", "link", "set", host_tap_name, "master", bridge}).exit_code != 0) {
// -- exactly where network_join.cpp's join_one_network() attaches veth's
// host-side end, just via a direct setns() here instead of
// wrap_for_network()'s argv wrapping (this whole function needs to keep
// running across later namespace switches, not just for one external
// command's duration). attach_host_side_to_bridge=false skips the
// bridge-join entirely -- used for the point-to-point uplink between a
// network's private namespace and the host's root namespace, which is
// deliberately not a bridge port (the caller assigns it an address and a
// route instead, see network_bridge.cpp's ensure_uplink_provisioned()).
if (attach_host_side_to_bridge &&
run_process({"ip", "link", "set", host_tap_name, "master", bridge}).exit_code != 0) {
report_line(report_fd, "ERROR failed to attach host-side tap device to bridge\n");
_exit(1);
}
@@ -293,7 +298,8 @@ std::string read_relay_report(int fd) {
std::optional<TapRelayHandle> create_tap_relay(const NetworkEntry& network, const std::string& bridge,
const std::string& host_tap_name, pid_t container_ns_pid,
const std::string& container_if_name) {
const std::string& container_if_name,
bool attach_host_side_to_bridge) {
int report_pipe[2];
if (pipe2(report_pipe, O_CLOEXEC) != 0) {
spdlog::error("failed to set up tap-relay report pipe: {}", strerror(errno));
@@ -309,7 +315,8 @@ std::optional<TapRelayHandle> create_tap_relay(const NetworkEntry& network, cons
}
if (pid == 0) {
close(report_pipe[0]);
relay_child_main(report_pipe[1], network, bridge, host_tap_name, container_ns_pid, container_if_name);
relay_child_main(report_pipe[1], network, bridge, host_tap_name, container_ns_pid, container_if_name,
attach_host_side_to_bridge);
}
close(report_pipe[1]);
@@ -324,7 +331,7 @@ std::optional<TapRelayHandle> create_tap_relay(const NetworkEntry& network, cons
return std::nullopt;
}
return TapRelayHandle{pid, host_tap_name, network};
return TapRelayHandle{pid, host_tap_name, network, std::nullopt};
}
void stop_tap_relay(const TapRelayHandle& handle) {
@@ -341,6 +348,16 @@ void stop_tap_relay(const TapRelayHandle& handle) {
spdlog::warn("failed to remove host-side tap device '{}' for network '{}'", handle.host_tap_name,
handle.network.name);
}
// The uplink's second tap lives directly in the host's root namespace
// (unlike a real container join's, which is inside the container's own
// ephemeral namespace and disappears on its own) -- remove it too,
// unwrapped (always host root by construction), when set.
if (handle.root_side_tap_name &&
run_process({"ip", "link", "del", *handle.root_side_tap_name}).exit_code != 0) {
spdlog::warn("failed to remove uplink's host-root-side tap device '{}' for network '{}'",
*handle.root_side_tap_name, handle.network.name);
}
}
std::filesystem::path tap_relay_state_path(std::string_view container_name, pid_t pid) {
src/network_tap_relay.h
+30 -6
View File
@@ -36,11 +36,20 @@ struct TapRelayHandle {
pid_t relay_pid;
std::string host_tap_name;
// Needed so stop_tap_relay() can wrap_for_network() (network_bridge.h)
// to reach wherever the host-side device actually lives (host root for
// extern, the network's own persistent namespace for intern) when
// explicitly removing it -- see stop_tap_relay()'s own doc comment for
// why that's needed now.
// to reach wherever the host-side device actually lives -- the
// network's own persistent namespace, both kinds -- when explicitly
// removing it, see stop_tap_relay()'s own doc comment for why that's
// needed now.
NetworkEntry network;
// Set only for the point-to-point uplink (network_bridge.cpp's
// ensure_uplink_provisioned()) -- names the second tap device, which for
// a real container join lives inside the container's own ephemeral
// namespace (torn down for free once the session ends, per
// create_tap_relay()'s own doc comment below) but for the uplink lives
// directly in the host's root namespace, which never goes away on its
// own. stop_tap_relay() removes it too (unwrapped -- always host root by
// construction) when set.
std::optional<std::string> root_side_tap_name;
};
// Creates a tap-backed substitute for one veth pair between `network`'s
@@ -65,12 +74,23 @@ struct TapRelayHandle {
// ordinary interface from the container's own point of view -- IP
// assignment, routes, etc. all work exactly as they do for a veth-created
// interface (join_one_network()'s existing code, unchanged).
//
// `attach_host_side_to_bridge` (default true, so every existing call site
// needs no change) lets a caller skip the "master <bridge>" step and just
// bring the host-side tap up plain instead -- used by the point-to-point
// uplink between a network's private namespace and the host's root
// namespace (network_bridge.cpp's ensure_uplink_provisioned()), which is
// deliberately *not* a bridge port: it's a plain routed link out, not
// another switched port on the same L2 domain. `bridge` is ignored when
// false (pass "").
//
// Returns nullopt (logging why) on any setup failure; the relay process, if
// one was forked, is guaranteed to have already exited (and been reaped) by
// the time this returns in that case.
std::optional<TapRelayHandle> create_tap_relay(const NetworkEntry& network, const std::string& bridge,
const std::string& host_tap_name, pid_t container_ns_pid,
const std::string& container_if_name);
const std::string& container_if_name,
bool attach_host_side_to_bridge = true);
// Stops a relay started by create_tap_relay(): sends SIGTERM, reaps it, then
// explicitly removes the host-side tap device (`ip link del`, wrapped via
@@ -91,7 +111,11 @@ std::optional<TapRelayHandle> create_tap_relay(const NetworkEntry& network, cons
// anymore, so this explicit removal step is now required. The
// container-side device needs no such step: it lives inside the container's
// own network namespace, torn down (taking every interface inside it,
// persistent or not, along with it) once the session itself ends.
// persistent or not, along with it) once the session itself ends -- with one
// exception: `handle.root_side_tap_name`, set only for the uplink
// (create_tap_relay()'s own doc comment above), lives directly in the host's
// root namespace instead, which never goes away on its own -- removed here
// too (unwrapped, always host root by construction) when set.
void stop_tap_relay(const TapRelayHandle& handle);
// $XDG_STATE_HOME/slocker-lite/tap-relays/<container_name>-<pid> -- same
src/self_test.cpp
+49 -24
View File
@@ -73,32 +73,54 @@ bool test_persistent_netns() {
}
// Exercises network_tap_relay.h's create/verify/teardown cycle end to end: a
// throwaway bridge stands in for a real network's bridge (network_bridge.h's
// own provisioning isn't needed here -- a tap device only cares that *some*
// bridge interface exists to attach to), and a throwaway network namespace
// (kept alive by a child blocked in pause()) stands in for a real -r/--run
// session's isolated net namespace. Confirms: the host-side tap gets created
// and attached to the bridge; the container-side tap gets created, with the
// requested name, inside the target namespace; and, once stop_tap_relay()
// stops the relay, the host-side device is explicitly removed (`ip link
// del`, since it's now created via `ip tuntap add` and no longer disappears
// on its own just because its one-and-only fd closes -- see
// stop_tap_relay()'s own doc comment) while the container-side device
// deliberately survives, unaffected by the relay stopping: it only goes away
// once the container's own network namespace itself is torn down (below,
// when this test kills container_pid).
// throwaway bridge stands in for a real network's bridge, inside a throwaway
// persistent namespace (persistent_netns.h) standing in for a real network's
// own dedicated one -- create_tap_relay() now always enters that namespace
// first (network_bridge.cpp's wrap_for_network() reaches a real network's
// bridge the same way, both kinds), so this test needs one too, even though
// network_bridge.h's own provisioning isn't otherwise exercised here (a tap
// device only cares that *some* bridge interface exists to attach to). A
// throwaway network namespace (kept alive by a child blocked in pause())
// stands in for a real -r/--run session's isolated net namespace. Confirms:
// the host-side tap gets created and attached to the bridge; the
// container-side tap gets created, with the requested name, inside the
// target namespace; and, once stop_tap_relay() stops the relay, the
// host-side device is explicitly removed (`ip link del`, since it's now
// created via `ip tuntap add` and no longer disappears on its own just
// because its one-and-only fd closes -- see stop_tap_relay()'s own doc
// comment) while the container-side device deliberately survives, unaffected
// by the relay stopping: it only goes away once the container's own network
// namespace itself is torn down (below, when this test kills container_pid).
bool test_tap_relay() {
const std::string test_network_name = "selftest-tap-relay";
const std::string test_bridge = "slkselftest0";
const std::string host_tap = "thselftest0";
const std::string container_if = "ethselftest";
if (run_process({"ip", "link", "add", test_bridge, "type", "bridge"}).exit_code != 0) {
spdlog::error("self-test: failed to create throwaway test bridge");
if (persistent_netns_exists(test_network_name)) {
remove_persistent_netns(test_network_name);
}
if (!create_persistent_netns(test_network_name)) {
spdlog::error("self-test: failed to create persistent namespace for tap-relay test");
return false;
}
if (run_process({"ip", "link", "set", test_bridge, "up"}).exit_code != 0) {
std::vector<std::string> netns_wrap = {
"nsenter", fmt::format("--net={}", persistent_netns_path(test_network_name).string()), "--"};
auto in_netns = [&netns_wrap](std::vector<std::string> argv) {
std::vector<std::string> wrapped = netns_wrap;
wrapped.insert(wrapped.end(), argv.begin(), argv.end());
return wrapped;
};
if (run_process(in_netns({"ip", "link", "add", test_bridge, "type", "bridge"})).exit_code != 0) {
spdlog::error("self-test: failed to create throwaway test bridge");
remove_persistent_netns(test_network_name);
return false;
}
if (run_process(in_netns({"ip", "link", "set", test_bridge, "up"})).exit_code != 0) {
spdlog::error("self-test: failed to bring up throwaway test bridge");
run_process({"ip", "link", "del", test_bridge});
run_process(in_netns({"ip", "link", "del", test_bridge}));
remove_persistent_netns(test_network_name);
return false;
}
@@ -109,7 +131,8 @@ bool test_tap_relay() {
pid_t container_pid = fork();
if (container_pid < 0) {
spdlog::error("self-test: failed to fork throwaway container namespace holder");
run_process({"ip", "link", "del", test_bridge});
run_process(in_netns({"ip", "link", "del", test_bridge}));
remove_persistent_netns(test_network_name);
return false;
}
if (container_pid == 0) {
@@ -143,12 +166,13 @@ bool test_tap_relay() {
kill(container_pid, SIGKILL);
int reap_status = 0;
waitpid(container_pid, &reap_status, 0);
run_process({"ip", "link", "del", test_bridge});
run_process(in_netns({"ip", "link", "del", test_bridge}));
remove_persistent_netns(test_network_name);
return false;
}
bool ok = true;
NetworkEntry network{"selftest-tap-relay", NetworkKind::extern_, "", false, "", true};
NetworkEntry network{test_network_name, NetworkKind::extern_, "", false, "", true};
auto relay = create_tap_relay(network, test_bridge, host_tap, container_pid, container_if);
if (!relay) {
spdlog::error("self-test: failed to create tap relay");
@@ -156,7 +180,7 @@ bool test_tap_relay() {
}
if (ok) {
auto host_check = run_process({"ip", "link", "show", host_tap});
auto host_check = run_process(in_netns({"ip", "link", "show", host_tap}));
if (host_check.exit_code != 0 || host_check.stdout_output.find("master " + test_bridge) == std::string::npos) {
spdlog::error("self-test: host-side tap device missing or not attached to the bridge");
ok = false;
@@ -175,7 +199,7 @@ bool test_tap_relay() {
if (relay) {
stop_tap_relay(*relay);
if (run_process({"ip", "link", "show", host_tap}).exit_code == 0) {
if (run_process(in_netns({"ip", "link", "show", host_tap})).exit_code == 0) {
spdlog::error("self-test: host-side tap device still exists after stopping the relay");
ok = false;
}
@@ -198,7 +222,8 @@ bool test_tap_relay() {
kill(container_pid, SIGKILL);
int status = 0;
waitpid(container_pid, &status, 0);
run_process({"ip", "link", "del", test_bridge});
run_process(in_netns({"ip", "link", "del", test_bridge}));
remove_persistent_netns(test_network_name);
if (ok) {
fmt::print("tap-relay create/attach/teardown: OK\n");