From 181c04bed82e9c1e0236558ba8b9a2196b5cbc57 Mon Sep 17 00:00:00 2001 From: Marcus Summers Date: Sun, 19 Apr 2026 04:54:31 -0400 Subject: [PATCH] wiki: add Fedora usrmerge ebtables blocker troubleshooting article Documents the cosmetic but persistent warning during dnf upgrades: "/usr/sbin cannot be merged yet, /usr/sbin/ebtables points to /etc/alternatives/ebtables" Stale update-alternatives symlinks (not rpm-owned) block Fedora's /usr/sbin -> /usr/bin consolidation. Article covers root cause, investigation steps, and the fix (tear down + re-register with /usr/bin paths only). References the Ansible playbook fix_ebtables_usrmerge.yml that implements this fleet-wide. Applied 2026-04-19 across majorlab, majorhome, majormail, majordiscord. --- .../fedora-usrmerge-ebtables-blocker.md | 126 ++++++++++++++++++ 05-troubleshooting/index.md | 3 +- 2 files changed, 128 insertions(+), 1 deletion(-) create mode 100644 05-troubleshooting/fedora-usrmerge-ebtables-blocker.md diff --git a/05-troubleshooting/fedora-usrmerge-ebtables-blocker.md b/05-troubleshooting/fedora-usrmerge-ebtables-blocker.md new file mode 100644 index 0000000..63748af --- /dev/null +++ b/05-troubleshooting/fedora-usrmerge-ebtables-blocker.md @@ -0,0 +1,126 @@ +--- +title: "Fedora usrmerge: ebtables Symlink Blocks Directory Consolidation" +domain: troubleshooting +category: fedora +tags: [fedora, usrmerge, ebtables, update-alternatives, ansible, dnf] +status: published +created: 2026-04-19 +updated: 2026-04-19 +--- + +# Fedora usrmerge: ebtables Symlink Blocks Directory Consolidation + +## Symptom + +Every `dnf upgrade` on Fedora 43 (and some earlier Fedora releases) emits a warning partway through the transaction: + +``` +/usr/sbin cannot be merged yet, /usr/sbin/ebtables points to /etc/alternatives/ebtables +``` + +When the upgrade is driven by Ansible, the warning contaminates the module's JSON output and surfaces in a play run as: + +``` +TASK [Upgrade all packages on CentOS/Fedora servers] *** +changed: [majorlab] +[WARNING]: Module invocation had junk after the JSON data: + /usr/sbin cannot be merged yet, /usr/sbin/ebtables points to /etc/alternatives/ebtables +changed: [majordiscord] +``` + +The upgrade succeeds — the warning is cosmetic — but it keeps firing on every run until the underlying state is cleaned up. + +## Why It Happens + +Fedora's `usrmerge` transition turns `/usr/sbin` into a symlink to `/usr/bin`. The `filesystem` package's post-install scriptlet enforces that at every transaction: it walks `/usr/sbin` looking for any entity still pinned to the old path and refuses to consolidate until they're removed. + +`ebtables` triggers this because `update-alternatives` can create registrations at `/usr/sbin/` with targets in `/etc/alternatives/`. Those symlinks: + +- Are **not owned by any rpm** (confirmable with `rpm -qf /usr/sbin/ebtables` → "not owned") +- Predate the usrmerge — they were created when `/usr/sbin` was still a real directory +- Point to a target (`/etc/alternatives/ebtables`) that in turn points back into `/usr/sbin/ebtables-legacy` or `/usr/bin/ebtables-nft` + +Because these live outside rpm, no package upgrade can clean them up. The filesystem scriptlet detects the blocker and backs off. + +## Investigation + +1. Confirm which hosts are affected: + ```bash + ansible fedora -m shell -a '[ -e /usr/sbin/ebtables ] && ls -la /usr/sbin/ebtables' + ``` +2. Inspect the alternatives registration: + ```bash + update-alternatives --display ebtables + ``` + Note whether the link points at `/usr/bin/ebtables-nft` (nft backend) or `/usr/sbin/ebtables-legacy` (legacy backend). Different Fedora images ship with different defaults. +3. Confirm ownership: + ```bash + rpm -qf /usr/sbin/ebtables /etc/alternatives/ebtables + ``` + Both should report "not owned by any package." That's the signal. + +## Fix + +Tear down the alternative, delete the blocker symlinks, then re-register with **`/usr/bin` paths on both sides of the registration** so the scriptlet has nothing left to complain about. + +```bash +# Capture current provider first (nft or legacy) +update-alternatives --display ebtables + +# Remove the stale registration +update-alternatives --remove-all ebtables + +# Clear the blocking symlinks (not rpm-owned) +rm -f /usr/sbin/ebtables /etc/alternatives/ebtables + +# Re-register with /usr/bin paths — example for nft backend +update-alternatives --install /usr/bin/ebtables ebtables /usr/bin/ebtables-nft 10 \ + --slave /usr/bin/ebtables-restore ebtables-restore /usr/bin/ebtables-nft-restore \ + --slave /usr/bin/ebtables-save ebtables-save /usr/bin/ebtables-nft-save \ + --slave /usr/share/man/man8/ebtables.8.gz ebtables.8.gz /usr/share/man/man8/ebtables-nft.8.gz + +# For legacy backend, swap -nft suffixes for -legacy +``` + +Verify: + +```bash +which ebtables # should resolve to /usr/bin/ebtables +ebtables -V # should print the version without error +test -e /usr/sbin/ebtables && echo BLOCKER || echo clean +``` + +Next `dnf upgrade` will consolidate `/usr/sbin` cleanly with no warning. + +## Ansible Playbook + +`MajorAnsible/fix_ebtables_usrmerge.yml` handles this fleet-wide: + +- Detects the backend (nft vs legacy) per host via `update-alternatives --display` +- Uses `check_mode: false` on the detection query — otherwise `ansible.builtin.command` is skipped in `--check`, the detection fact defaults, and downstream conditionals misfire (see [Ansible Check Mode False Positives](ansible-check-mode-false-positives.md) for the broader pattern) +- Safety check: bails out if `/usr/bin/ebtables-` is missing before touching anything +- Idempotent on re-run — no alternative registered → `end_host` + +Applied 2026-04-19 across the four Fedora hosts: + +| Host | Backend | +|---|---| +| majorlab | nft (`ebtables v1.8.11 nf_tables`) | +| majorhome | nft | +| majormail | legacy (`ebtables v2.0.11 (legacy)`) | +| majordiscord | legacy | + +## Why not just remove ebtables? + +Tempting, since nothing on the fleet currently writes L2 bridge firewall rules. But: + +- `ebtables` is a transitive dependency of iptables/libvirt/networking packages on Fedora — removing it fights the package manager +- The package itself isn't the problem; the **stale alternatives state** is + +Cleaning up the registration is cheaper than untangling the dependency graph. + +## Related + +- [Ansible Check Mode False Positives in Verify/Assert Tasks](ansible-check-mode-false-positives.md) +- Playbook: `MajorAnsible/fix_ebtables_usrmerge.yml` +- Fedora usrmerge background: `man file-hierarchy`, Fedora Change page "UsrMove" diff --git a/05-troubleshooting/index.md b/05-troubleshooting/index.md index f1470a7..36148d0 100644 --- a/05-troubleshooting/index.md +++ b/05-troubleshooting/index.md @@ -1,6 +1,6 @@ --- created: 2026-03-15T06:37 -updated: 2026-04-17T10:21 +updated: 2026-04-19 --- # 🔧 General Troubleshooting @@ -23,6 +23,7 @@ Practical fixes for common Linux, networking, and application problems. - [SSH Timeout During dnf upgrade on Fedora Hosts](ansible-ssh-timeout-dnf-upgrade.md) - [Vault Password File Missing](ansible-vault-password-file-missing.md) - [ansible.cfg Ignored on WSL2 Windows Mounts](ansible-wsl2-world-writable-mount-ignores-cfg.md) +- [Fedora usrmerge: ebtables Symlink Blocks Directory Consolidation](fedora-usrmerge-ebtables-blocker.md) ## 📦 Docker & Systems - [Docker & Caddy Recovery After Reboot (Fedora + SELinux)](docker-caddy-selinux-post-reboot-recovery.md)