Environment

• 4× Mac mini (M4, 16 GB), macOS 26.5.1 (all identical)
• exo main @ b5375f8c, run via uv run exo
• All nodes on one flat 192.168.5.0/24 Ethernet switch, macOS firewall off
• IPv6: link-local only (fe80:: on en0; no IPv6 router on the LAN)

Symptom

Nodes never discover each other. Every node logs only its own ID and elects itself Master:

discovered: Some("<own zid>")
Node elected Master - maintaining self

A raw sniffer joined to the discovery group ff12::e0a1:de89 on UDP 52413 sees zero Hello packets on the wire, even with 3–4 nodes running simultaneously. Meanwhile a hand-rolled Python sender on the same host delivers multicast to that exact group/port to all peers fine, so the network path itself is good.

Root cause

Discovery::announce() treats EHOSTUNREACH as permanent and removes the address from the announce list (rust/networking/src/discovery.rs):

Err(e) if e.kind() == io::ErrorKind::HostUnreachable => {
    debug!("disabling discovery address {addr}: {e}");
    _ = self.ifaces.lock().swap_remove(i);
}

On macOS this errno is transient (observed right after boot/wake/link changes). Instrumented timeline from one of the Macs (log level raised so debug!/trace! in the discovery module are visible):

announce n=7                       # all 7 ifaces present (en0 = scope 7)
tick 1: OK sent 20 to [ff12::e0a1:de89%7]:52413   (and the other 6 ifaces)
tick 2: HOSTUNREACH %7 ... %1 ... %18 ... (all 7, "No route to host (os error 65)")
tick 3+: announcing Hello(..) to []               # list drained; discovery dead

After that the node never announces again (the tick loop keeps running but the address list is empty), so clusters silently fail to form. The failure is invisible at default log levels.

Removing addresses here is also redundant: interfaces that actually go away are already pruned by the netwatcher callback via update.diff.removed.

Fix / verification

Treating the error as transient (log + keep the address) and rebuilding: Hello beacons then egress en0 continuously (verified with tcpdump -ni en0 udp port 52413 — steady 1/s from the node's link-local address), and subsequent sends report OK (11/11 in a 14 s window on the same machine that previously drained its list).