Fix -p/--port-forward unreachable on extern networks
A server listening on an extern network wasn't reachable via -p at all, from the host or from a real outside client -- confirmed by direct testing: since extern's bridge moved into a private namespace (the earlier connectivity fix), host root had no route to the container subnet whatsoever (ip route get fell through to the LAN default gateway instead), so -p's own DNAT rule, which targets the container's real IP directly, had nowhere to route the rewritten packet. Fixed with two pieces in ensure_uplink_provisioned(), both confirmed necessary by direct testing -- the same "route alone isn't enough on Android" lesson the uplink's own outbound/return-path ip rules already learned: a host-root route to the container subnet through the uplink, plus a matching ip rule routing traffic to that subnet into main (without it, Android's own lower-priority-number policy routing -- a generic fwmark 0/0x10000 catch-all among them -- intercepts the packet into an unrelated table before rule evaluation ever reaches main, so the route alone is silently never consulted). Also fixes a related robustness bug found while testing this: a failed ensure_uplink_provisioned() only stopped the tap relay, leaving every already-added ip rule/iptables piece (deterministic, hash-derived names) live on the host -- a first failed attempt then made every later attempt for the same network name fail identically and permanently, until a manual fix or a full device reboot. Fixed by recording the relay's pid to the uplink state file as soon as it's known, so a failure can roll back via the same teardown_uplink_state() a real --delete-network-full would use, instead of a partial, drifting copy of its cleanup logic. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gv3s5jckJKzh6JkMoi2Akz
This commit is contained in:
+83
-12
@@ -250,6 +250,12 @@ bool uplink_provisioned(const std::string& root_tap) {
|
||||
return run_process({"ip", "link", "show", root_tap}).exit_code == 0;
|
||||
}
|
||||
|
||||
// Forward declaration: ensure_uplink_provisioned() below calls this on its
|
||||
// own failure path to fully roll back whatever it already added, and the
|
||||
// two are defined in this order for readability (provision before
|
||||
// teardown) despite the dependency running the other way.
|
||||
void teardown_uplink_state(const NetworkEntry& network);
|
||||
|
||||
// Idempotent (see uplink_provisioned() above): a no-op if the host-root-side
|
||||
// device already exists. Three distinct pieces are needed beyond the
|
||||
// tap+relay link and its addresses/route/NAT to actually carry traffic to
|
||||
@@ -299,6 +305,25 @@ bool ensure_uplink_provisioned(const NetworkEntry& network) {
|
||||
}
|
||||
relay->root_side_tap_name = root_tap;
|
||||
|
||||
// Recorded now, before any of the steps below that can fail -- so that
|
||||
// a failure partway through can roll back via the exact same
|
||||
// teardown_uplink_state() a later --delete-network-full would use,
|
||||
// rather than needing a second, drifting copy of its cleanup logic
|
||||
// here. Harmless on the success path too: uplink_state_path() only
|
||||
// needs to name the relay's pid, already known at this point.
|
||||
{
|
||||
std::error_code ec;
|
||||
auto path = uplink_state_path(network.name);
|
||||
std::filesystem::create_directories(path.parent_path(), ec);
|
||||
std::ofstream out(path);
|
||||
if (out) {
|
||||
out << relay->relay_pid << '\n';
|
||||
} else {
|
||||
spdlog::warn("failed to record uplink relay pid for network '{}' -- a later teardown won't find it",
|
||||
network.name);
|
||||
}
|
||||
}
|
||||
|
||||
auto [netns_addr, root_addr] = uplink_transit_addresses(network.name);
|
||||
std::string transit_subnet = uplink_transit_subnet(network.name);
|
||||
|
||||
@@ -328,24 +353,56 @@ bool ensure_uplink_provisioned(const NetworkEntry& network) {
|
||||
}
|
||||
ok = ok &&
|
||||
run_process({"ip", "rule", "add", "priority", "100", "to", transit_subnet, "lookup", "main"}).exit_code == 0;
|
||||
// Without these two, host root has no way to route to the container
|
||||
// subnet at all -- needed for -p/--port-forward, whose DNAT rule (added
|
||||
// in host root, targeting the container's real IP directly --
|
||||
// port_forward.cpp) otherwise has nowhere to send the rewritten packet.
|
||||
// Two distinct pieces, both confirmed necessary by direct testing, not
|
||||
// assumed -- the same "route alone isn't enough on Android" lesson the
|
||||
// uplink's own outbound/return-path ip rules above already learned:
|
||||
// 1. The route itself, through the uplink's own netns-side address --
|
||||
// the same way any next-hop router would reach it. From there, the
|
||||
// private namespace already forwards arriving traffic to the
|
||||
// bridge on its own (a fresh namespace's FORWARD policy is ACCEPT
|
||||
// by default, and ip_forward is already enabled above), and the
|
||||
// reply's return path is already covered by the private
|
||||
// namespace's own default route out through this same uplink (set
|
||||
// above) plus host root's own conntrack correctly reversing the
|
||||
// original DNAT on the way back out -- no second DNAT stage
|
||||
// needed.
|
||||
// 2. An `ip rule` routing traffic *to* the container subnet into
|
||||
// `main` (where the route above lives), the same shape as the
|
||||
// return-path rule just above for the transit subnet. Without it,
|
||||
// the route added in step 1 is simply never consulted: confirmed
|
||||
// via `ip rule show` on the real device that Android's own
|
||||
// lower-priority-number policy rules (a generic `fwmark
|
||||
// 0/0x10000 lookup 99` catch-all among them, matching any
|
||||
// untouched/forwarded packet) intercept the packet and route it
|
||||
// into an unrelated table -- with no route to the container
|
||||
// subnet there -- long before rule evaluation ever reaches `main`.
|
||||
ok = ok &&
|
||||
run_process({"ip", "route", "add", network.subnet, "via", strip_prefix(netns_addr), "dev", root_tap})
|
||||
.exit_code == 0;
|
||||
ok = ok &&
|
||||
run_process({"ip", "rule", "add", "priority", "100", "to", network.subnet, "lookup", "main"}).exit_code == 0;
|
||||
|
||||
if (!ok) {
|
||||
spdlog::error("failed to configure uplink for network '{}'", network.name);
|
||||
stop_tap_relay(*relay);
|
||||
// Real bug found by testing, not assumed: an earlier version only
|
||||
// called stop_tap_relay() here, leaving every ip rule/iptables piece
|
||||
// already added above (all deterministic, hash-derived names) live
|
||||
// on the host -- a first failed attempt then made every later
|
||||
// attempt for the *same* network name fail identically and
|
||||
// permanently ("File exists" on an `ip rule add` that was never
|
||||
// removed), since nothing ever cleaned it up short of a manual
|
||||
// by-hand fix or a full device reboot. teardown_uplink_state()
|
||||
// reverses every piece unconditionally and best-effort, so calling
|
||||
// it here is exactly as safe as a real --delete-network-full would
|
||||
// be against this same partially-provisioned state.
|
||||
teardown_uplink_state(network);
|
||||
return false;
|
||||
}
|
||||
|
||||
std::error_code ec;
|
||||
auto path = uplink_state_path(network.name);
|
||||
std::filesystem::create_directories(path.parent_path(), ec);
|
||||
std::ofstream out(path);
|
||||
if (out) {
|
||||
out << relay->relay_pid << '\n';
|
||||
} else {
|
||||
spdlog::warn("failed to record uplink relay pid for network '{}' -- a later teardown won't find it",
|
||||
network.name);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -362,7 +419,21 @@ bool ensure_uplink_provisioned(const NetworkEntry& network) {
|
||||
void teardown_uplink_state(const NetworkEntry& network) {
|
||||
std::string root_tap = uplink_root_tap_name(network.name);
|
||||
std::string transit_subnet = uplink_transit_subnet(network.name);
|
||||
auto [netns_addr, root_addr] = uplink_transit_addresses(network.name);
|
||||
|
||||
// Purely for tidiness -- the kernel already removes this route on its
|
||||
// own once root_tap itself is deleted below, same as any route through a
|
||||
// deleted device.
|
||||
if (run_process({"ip", "route", "del", network.subnet, "via", strip_prefix(netns_addr), "dev", root_tap})
|
||||
.exit_code != 0) {
|
||||
spdlog::warn("failed to remove host-root route to network '{}' (already gone, or never existed)",
|
||||
network.name);
|
||||
}
|
||||
if (run_process({"ip", "rule", "del", "priority", "100", "to", network.subnet, "lookup", "main"}).exit_code !=
|
||||
0) {
|
||||
spdlog::warn("failed to remove host-root container-subnet ip rule for network '{}' (already gone, or never existed)",
|
||||
network.name);
|
||||
}
|
||||
if (run_process({"ip", "rule", "del", "priority", "100", "to", transit_subnet, "lookup", "main"}).exit_code !=
|
||||
0) {
|
||||
spdlog::warn("failed to remove uplink return-path ip rule for network '{}' (already gone, or never existed)",
|
||||
|
||||
@@ -113,7 +113,12 @@ bool should_use_veth(const NetworkEntry& network);
|
||||
// route alone is not sufficient on Android): an inserted-at-the-front
|
||||
// iptables FORWARD accept rule, and two `ip rule`s (outbound and
|
||||
// return-path) routing this traffic into whichever policy-routing table
|
||||
// the host is actually using for its own real traffic right now.
|
||||
// the host is actually using for its own real traffic right now. Also
|
||||
// adds a host-root route to the container subnet itself (via the
|
||||
// uplink) plus its own matching `ip rule` -- needed for
|
||||
// -p/--port-forward, whose DNAT rule targets the container's real IP
|
||||
// directly; without both pieces (route *and* rule, the same lesson as
|
||||
// the two above) host root has no way to route to the container at all.
|
||||
// Idempotent: a no-op (true) if the bridge already exists -- **except** the
|
||||
// uplink step for extern, which still runs (with its own, separate
|
||||
// idempotency check) even when the bridge already existed, so an
|
||||
|
||||
Reference in New Issue
Block a user