Marcus Summers
0d1697c0d6
New troubleshooting runbook for Logwatch reports titled with the Hetzner provisioning label instead of the real hostname; cross-linked from the logwatch fleet-setup and VPS migration baseline articles, plus a new 'set system hostname' step in the post-migration checklist.
111 lines
4.1 KiB
Markdown
111 lines
4.1 KiB
Markdown
---
|
|
title: "Logwatch Reports the Wrong Hostname (`<host>-hetzner`) After a Migration"
|
|
domain: troubleshooting
|
|
category: monitoring
|
|
tags: [logwatch, hostname, hetzner, migration, monitoring, provisioning]
|
|
status: published
|
|
created: 2026-06-12
|
|
updated: 2026-06-12
|
|
---
|
|
|
|
# Logwatch Reports the Wrong Hostname (`<host>-hetzner`) After a Migration
|
|
|
|
## Symptom
|
|
|
|
Daily Logwatch emails from a recently migrated server arrive titled with the
|
|
provisioning label instead of the real hostname:
|
|
|
|
```
|
|
Logwatch for tttpod-hetzner (Linux)
|
|
Logwatch for dcaprod-hetzner (Linux)
|
|
```
|
|
|
|
Everything else works — the report is generated, mailed, and delivered. Only the
|
|
**name in the title is wrong**, which makes reports harder to scan and breaks any
|
|
filter or rule that keys on the expected hostname.
|
|
|
|
## Cause
|
|
|
|
Logwatch titles each report with the box's **live system hostname**
|
|
(`hostnamectl --static` / `/etc/hostname`) read at runtime — it does *not* keep
|
|
its own copy of the name.
|
|
|
|
Hetzner Cloud servers are provisioned with a temporary node label as the system
|
|
hostname — `<host>-hetzner` (e.g. `tttpod-hetzner`). The migration runbook renames
|
|
the **Tailscale node** back to the bare name and sets Postfix `myhostname`, but the
|
|
**OS hostname** itself is easy to miss because nothing surfaces it day to day. It
|
|
stays `<host>-hetzner` until something reads `hostname` — Logwatch is usually the
|
|
first thing to do so, weeks later.
|
|
|
|
Confirm the box is actually mislabelled:
|
|
|
|
```bash
|
|
ssh root@<host> 'hostnamectl --static; cat /etc/hostname; grep 127.0.1.1 /etc/hosts'
|
|
# static: tttpod-hetzner
|
|
# /etc/hostname: tttpod-hetzner
|
|
# 127.0.1.1 tttpod-hetzner tttpod-hetzner
|
|
```
|
|
|
|
## Fix
|
|
|
|
Set the real hostname and fix the matching `/etc/hosts` loopback line:
|
|
|
|
```bash
|
|
ssh root@<host> '
|
|
hostnamectl set-hostname <host>
|
|
sed -i "s/127.0.1.1.*/127.0.1.1 <host> <host>/" /etc/hosts
|
|
hostnamectl --static # verify -> <host>
|
|
'
|
|
```
|
|
|
|
That's it. **Logwatch has no hardcoded hostname override** — verify with:
|
|
|
|
```bash
|
|
grep -ri hostname /etc/logwatch/ /etc/cron.daily/0logwatch /etc/cron.daily/logwatch 2>/dev/null
|
|
cat /etc/mailname 2>/dev/null
|
|
```
|
|
|
|
If those are empty (the normal case), Logwatch reads the live hostname on its next
|
|
run, so the **next daily report self-corrects** — no service restart, no logwatch
|
|
config change needed.
|
|
|
|
> [!note] If `grep` *does* find a hostname pinned in `/etc/logwatch/conf/logwatch.conf`
|
|
> (e.g. a `HostLimit`/`MailFrom` line baked in by Ansible), update it there too —
|
|
> the override file wins over the live hostname.
|
|
|
|
## Sweep the whole fleet
|
|
|
|
This is a per-box provisioning leftover, so check every migrated host at once —
|
|
more than one is usually affected:
|
|
|
|
```bash
|
|
for ip in 100.98.223.93 100.95.137.38 100.64.169.62 100.112.127.0 100.73.85.46; do
|
|
echo -n "$ip -> "
|
|
ssh -o ConnectTimeout=8 -o BatchMode=yes root@$ip 'hostnamectl --static' 2>/dev/null \
|
|
|| echo '(unreachable)'
|
|
done
|
|
```
|
|
|
|
Any value ending in `-hetzner` (or your provider's build label) needs the fix above.
|
|
In the 2026-06 sweep, `tttpod` and `dcaprod` were still `*-hetzner`;
|
|
`majortoot`, `majormail`, and `majorlinux` were already correct.
|
|
|
|
## Prevention
|
|
|
|
Fold "set the system hostname" into the migration bootstrap so it never drifts:
|
|
|
|
```bash
|
|
hostnamectl set-hostname <host>
|
|
sed -i "s/127.0.1.1.*/127.0.1.1 <host> <host>/" /etc/hosts
|
|
```
|
|
|
|
Do this in the **same step** that renames the Tailscale node and sets Postfix
|
|
`myhostname` — all three read from the provisioning label and all three must be
|
|
corrected together. See the
|
|
[VPS Migration Baseline Checklist](../02-selfhosting/cloud/vps-migration-baseline-checklist.md).
|
|
|
|
## Related
|
|
|
|
- [Logwatch Fleet Setup — Surviving Package Upgrades](../02-selfhosting/monitoring/logwatch-fleet-setup.md) — the broader "logwatch went silent / wrong-source" class, including the Packer `myhostname` variant of this same drift
|
|
- [VPS Migration Baseline Checklist](../02-selfhosting/cloud/vps-migration-baseline-checklist.md) — the full post-migration verification list
|
|
- [Ansible UNREACHABLE: Host Key Verification Failed After a Host Rebuild or Migration](networking/ansible-host-key-verification-failed-rebuilt-host.md) — another IP/identity-drift gotcha from the same Hetzner migration
|