majorwiki/05-troubleshooting/logwatch-wrong-hostname-after-migration.md

title	domain	category	tags	status	created	updated
Logwatch Reports the Wrong Hostname (`<host>-hetzner`) After a Migration	troubleshooting	monitoring	logwatch hostname hetzner migration monitoring provisioning fail2ban	published	2026-06-12	2026-06-14

Logwatch Reports the Wrong Hostname (<host>-hetzner) After a Migration

Symptom

Daily Logwatch emails from a recently migrated server arrive titled with the provisioning label instead of the real hostname:

Logwatch for tttpod-hetzner (Linux)
Logwatch for dcaprod-hetzner (Linux)

Everything else works — the report is generated, mailed, and delivered. Only the name in the title is wrong, which makes reports harder to scan and breaks any filter or rule that keys on the expected hostname.

Cause

Logwatch titles each report with the box's live system hostname (hostnamectl --static / /etc/hostname) read at runtime — it does not keep its own copy of the name.

Hetzner Cloud servers are provisioned with a temporary node label as the system hostname — <host>-hetzner (e.g. tttpod-hetzner). The migration runbook renames the Tailscale node back to the bare name and sets Postfix myhostname, but the OS hostname itself is easy to miss because nothing surfaces it day to day. It stays <host>-hetzner until something reads hostname — Logwatch is usually the first thing to do so, weeks later.

Confirm the box is actually mislabelled:

ssh root@<host> 'hostnamectl --static; cat /etc/hostname; grep 127.0.1.1 /etc/hosts'
# static: tttpod-hetzner
# /etc/hostname: tttpod-hetzner
# 127.0.1.1 tttpod-hetzner tttpod-hetzner

Fix

Set the real hostname and fix the matching /etc/hosts loopback line:

ssh root@<host> '
  hostnamectl set-hostname <host>
  sed -i "s/127.0.1.1.*/127.0.1.1 <host> <host>/" /etc/hosts
  hostnamectl --static          # verify -> <host>
'

That's it. Logwatch has no hardcoded hostname override — verify with:

grep -ri hostname /etc/logwatch/ /etc/cron.daily/0logwatch /etc/cron.daily/logwatch 2>/dev/null
cat /etc/mailname 2>/dev/null

If those are empty (the normal case), Logwatch reads the live hostname on its next run, so the next daily report self-corrects — no service restart, no logwatch config change needed.

[!note] If grep does find a hostname pinned in /etc/logwatch/conf/logwatch.conf (e.g. a HostLimit/MailFrom line baked in by Ansible), update it there too — the override file wins over the live hostname.

Sweep the whole fleet

This is a per-box provisioning leftover, so check every migrated host at once — more than one is usually affected:

for ip in 100.98.223.93 100.95.137.38 100.64.169.62 100.112.127.0 100.73.85.46; do
  echo -n "$ip -> "
  ssh -o ConnectTimeout=8 -o BatchMode=yes root@$ip 'hostnamectl --static' 2>/dev/null \
    || echo '(unreachable)'
done

Any value ending in -hetzner (or your provider's build label) needs the fix above. In the 2026-06 sweep, tttpod and dcaprod were still *-hetzner at the OS level; majortoot, majormail, and majorlinux had the correct system hostname — but see the variant below: majormail's configs were still stale even though its hostname wasn't.

Variant: hostname is correct, but a config has the old name baked in

A second, sneakier form of this drift: the system hostname is already right, so the sweep above passes and the Logwatch report title is correct — yet mail still arrives from <host>-hetzner because the old label is hardcoded in a service's From/sender field. These fields are static text, not derived from the live hostname, so fixing hostnamectl does nothing for them.

Seen on majormail (2026-06-14): system hostname was majormail, but Logwatch@majormail-hetzner... was still the sender. Two configs held it:

# sweep a box for the old provisioning label in any send-related config
ssh root@<host> 'grep -rsn "<host>-hetzner" /etc/logwatch/ /etc/fail2ban/ \
  /etc/postfix/ /etc/aliases /etc/mailname 2>/dev/null'
# /etc/logwatch/conf/logwatch.conf:MailFrom = Logwatch@<host>-hetzner.majorshouse.com
# /etc/fail2ban/jail.local:sender         = fail2ban@<host>-hetzner.majorshouse.com

Fix in place (no restart needed for Logwatch; reload fail2ban for its change):

ssh root@<host> '
  sed -i "s/<host>-hetzner/<host>/g" /etc/logwatch/conf/logwatch.conf /etc/fail2ban/jail.local
  systemctl reload fail2ban
'

[!warning] Check the Ansible source, or it comes back A live sed is undone by the next playbook run if the repo still carries the old value. Distinguish two cases:

• Templated (safe): e.g. logwatch.yml sets MailFrom = Logwatch@{{ inventory_hostname }}.... If the inventory host is named correctly, a run regenerates the right value — it even self-heals a stale box.
• Static file (will regress): e.g. roles/fail2ban/files/hosts/<host>/jail.local with the literal sender = ...@<host>-hetzner.... Grep the repo (grep -rn "<host>-hetzner" .) and fix the file too, or every deploy re-pushes the stale sender.

Inert backups (jail.local.bak*, *~) may still contain the old string — they don't send mail, so leave them.

Prevention

Fold "set the system hostname" into the migration bootstrap so it never drifts:

hostnamectl set-hostname <host>
sed -i "s/127.0.1.1.*/127.0.1.1 <host> <host>/" /etc/hosts

Do this in the same step that renames the Tailscale node and sets Postfix myhostname — all three read from the provisioning label and all three must be corrected together. See the VPS Migration Baseline Checklist.

Related

• Logwatch Fleet Setup — Surviving Package Upgrades — the broader "logwatch went silent / wrong-source" class, including the Packer myhostname variant of this same drift
• VPS Migration Baseline Checklist — the full post-migration verification list
• Ansible UNREACHABLE: Host Key Verification Failed After a Host Rebuild or Migration — another IP/identity-drift gotcha from the same Hetzner migration