Compare commits

..

19 commits

Author SHA1 Message Date
7e422ee332 githooks: mark pre-commit executable
The pre-commit hook (which enforces SUMMARY.md links for new articles)
was tracked at mode 100644, so even with `core.hooksPath=.githooks`
configured, git silently skipped it. Bump tracked mode to 100755 so
fresh clones get the working hook without a manual chmod step.

Discovered while installing the wiki-commit/hooks setup on MajorMac.
No content change; .githooks/ is outside the MkDocs source so this
will not alter the rendered notes.majorshouse.com site.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 09:42:06 -04:00
3c4cc74aef Merge: wiki — Ansible regex_search set_fact gotcha 2026-05-06 08:28:22 -04:00
ca123b0312 wiki: add troubleshooting article — Ansible regex_search capture group fails in set_fact
Documents the gotcha hit during the 2026-05-06 update.yml refactor:
the second-positional-argument back-reference form of regex_search
('\1') doesn't reliably select capture groups when used inside
set_fact. The fix is to match the broader substring and use
.split()[0] (or [-1], etc.) to peel off the value, with a default()
bridge for the no-match case.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 08:28:21 -04:00
488268ccd1 Merge branch 'cowork/majorair/rescue-stash-keep-files-may02' 2026-05-02 17:50:28 -04:00
213a84ed79 wiki: add .keep files for 04-streaming and 05-troubleshooting subdirs 2026-05-02 17:50:22 -04:00
ae864452f8 wiki: add Fail2Ban Digest Mode nav entry to SUMMARY.md 2026-05-02 17:17:04 -04:00
49a1173dfc Merge cowork/majorair/index-refresh-may02 — full index refresh (106 articles) 2026-05-02 16:45:37 -04:00
c5b4de4184 wiki: full index refresh — 106 articles, 17 new since Apr 18
Updated article count (89 → 106), domain counts, per-section
listings, and Recently Updated table. Added all articles published
since 2026-04-18 including Pi-hole, Mastodon, fail2ban digest,
LoRA GGUF, Tailscale iOS, and more.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-02 16:45:30 -04:00
021c7f6539 Merge cowork/majorair/wiki-updates-may02 — fail2ban digest + netdata docker health + 3 new articles 2026-05-02 16:28:48 -04:00
4126656c05 wiki: update fail2ban digest + netdata docker health + 3 new articles
- fail2ban-digest-mode-fleet: recidive-only email model, sshd now silent,
  defaults-debian.conf gotcha added
- netdata-docker-health-alarm-tuning: 30m/10m config, tuning history table
- New: wp-fail2ban-logpath-debian-ubuntu, lora-adapter-gguf-conversion-fails,
  tailscale-status-json-hostname-localhost-ios
- Various article updates and nav index refreshes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-02 14:58:07 -04:00
264f1f64c3 Merge: wiki — 2026-04-30 SNI filter article update 2026-04-30 13:08:37 -04:00
74c4ed9959 wiki: 2026-04-30 update to ISP SNI filtering article
Re-diagnoses today's notes.majorshouse.com outage. Original framing
was "ISP filter expanded to include 'notes'" — but the actual root
cause was a stale A record pointing at 136.54.3.248 (not majorlab's
current home IP). Corrects the comparison table to show CNAMEs to
apex resolve to 136.56.0.55, and recommends a Cloudflare-proxied
CNAME as the durable shape so the apex follows home IP automatically
and ISP-level SNI weirdness is bypassed at the same time.

Includes the working CF API payload used to flip the record, and an
audit checklist for any new *.majorshouse.com subdomain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 13:08:36 -04:00
34cc5c3d0b Merge: wiki — LoRA→GGUF troubleshooting article 2026-04-30 11:24:38 -04:00
6e7a0ca21f wiki: add troubleshooting article — LoRA adapter GGUF conversion fails
Documents the gotcha where convert_hf_to_gguf.py crashes with
'config.json not found' because the training output directory holds
only the LoRA adapter, not a merged HF model. Includes inline
save_pretrained_merged() fix snippet, verification checklist, and
resume-pipeline-without-retraining pattern.

Discovered today during the MajorTwin v8c pipeline failure (Step 4).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 11:22:59 -04:00
85f8a5df2d Merge pull request 'wiki: add troubleshooting article — iOS Tailscale clients report HostName="localhost"' (#1) from code/majormac/tailscale-ios-hostname-fix into main
Reviewed-on: #1
2026-04-30 10:00:01 -04:00
9de839066c wiki: add troubleshooting article — iOS Tailscale clients report HostName="localhost"
Documents the non-obvious failure mode where /etc/hosts generator scripts
using `tailscale status --json | jq '.HostName'` get poisoned by iOS
peers, which always report HostName as the literal string "localhost"
because iOS doesn't expose the device name to apps.

Includes the buggy and fixed jq filter (use .DNSName first label
instead), a real-world Postfix outage example, and a verification
checklist. Linked from troubleshooting index and SUMMARY.

Discovered while diagnosing a 24h Postfix outage on majordiscord.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 09:57:40 -04:00
ad2f12c16e wiki: document wiki-commit wrapper + pre-commit hook setup
Adds a 'One-liner wrapper' tip and a 'Pre-Commit Hook (in repo)' section
to MajorWiki-Deploy-Status.md describing the per-clone setup needed on
each workstation:

  git config core.hooksPath .githooks
  git config pull.rebase true

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 05:30:45 -04:00
c4fba631e4 wiki: add pre-commit hook — block new articles missing from SUMMARY.md
The hook fails any commit that adds (or renames) a .md article without a
matching SUMMARY.md entry, addressing the recurring 'article exists but
isn't navigable' drift. Excludes meta files (README/index/SUMMARY,
category index.md, MajorWiki-Deploy-Status). Bypass with --no-verify.

Hook lives in .githooks/ (tracked). Each clone needs:
  git config core.hooksPath .githooks

Companion wrapper ~/bin/wiki-commit (workstation-only, not in repo) does
pull --rebase --autostash + add -A + commit + push so cowork pushes
don't surprise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 05:30:05 -04:00
9e96ebb110 wiki: add wp-fail2ban logpath on Debian/Ubuntu (auth.log not syslog)
Documents the gotcha discovered during the 2026-04-30 DCAProd XML-RPC
outage triage: wp-fail2ban plugin emits via PHP syslog(LOG_AUTH) which
lands in /var/log/auth.log on Debian/Ubuntu, not /var/log/syslog.
wordpress-{hard,soft,extra} jails configured with logpath=/var/log/syslog
(common in tutorials and ansible roles) silently catch zero events.

Article includes diagnostic steps, the fix, fail2ban-regex verification,
distro cheat sheet (Debian/Ubuntu vs RHEL/Fedora vs systemd-journal-only),
and a note on why wordpress-login is unaffected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 05:21:50 -04:00
29 changed files with 847 additions and 193 deletions

36
.githooks/pre-commit Executable file
View file

@ -0,0 +1,36 @@
#!/usr/bin/env bash
# pre-commit — fail if a newly-added article is not linked from SUMMARY.md.
# Bypass with `git commit --no-verify` if you genuinely need to.
set -euo pipefail
# Articles being added or renamed in this commit (excludes meta/index/README/SUMMARY/MajorWiki-Deploy-Status, and any */index.md).
added=$(git diff --cached --name-only --diff-filter=AR -- '*.md' \
| grep -vE '^(README|index|SUMMARY|MajorWiki-Deploy-Status)\.md$|/index\.md$' \
|| true)
[ -z "$added" ] && exit 0
# Read the staged SUMMARY.md if it's part of the commit; otherwise the working-tree copy.
if git diff --cached --name-only | grep -q '^SUMMARY\.md$'; then
summary=$(git show :SUMMARY.md)
else
summary=$(cat SUMMARY.md)
fi
missing=()
while IFS= read -r article; do
[ -z "$article" ] && continue
if ! grep -qF -- "$article" <<<"$summary"; then
missing+=("$article")
fi
done <<<"$added"
if [ ${#missing[@]} -gt 0 ]; then
echo "✗ pre-commit: new article(s) not linked from SUMMARY.md:" >&2
printf ' %s\n' "${missing[@]}" >&2
echo "" >&2
echo "Add a SUMMARY.md entry for each, or use 'git commit --no-verify' to bypass." >&2
exit 1
fi
exit 0

View file

@ -10,7 +10,7 @@ tags:
- majorrig
status: published
created: 2026-03-16
updated: 2026-04-29T22:45
updated: 2026-04-30T05:21
---
# WSL2 Backup via PowerShell Scheduled Task

View file

@ -10,7 +10,7 @@ tags:
- remote-access
status: published
created: 2026-03-08
updated: 2026-04-22T09:20
updated: 2026-04-30T05:21
---
# SSH Config and Key Management

View file

@ -7,7 +7,7 @@ tags:
- asus
- ssh
created: 2026-04-19
updated: 2026-04-29T22:45
updated: 2026-04-30T05:21
---
# Wake-on-LAN via Router SSH

View file

@ -1,6 +1,6 @@
---
created: 2026-04-13T10:15
updated: 2026-04-29T22:45
updated: 2026-04-30T05:21
---
# 🏠 Self-Hosting & Homelab

View file

@ -1,11 +1,17 @@
---
title: "Tuning Netdata Docker Health Alarms to Prevent Update Flapping"
title: Tuning Netdata Docker Health Alarms to Prevent Update Flapping
domain: selfhosting
category: monitoring
tags: [netdata, docker, nextcloud, alarms, health, monitoring]
tags:
- netdata
- docker
- nextcloud
- alarms
- health
- monitoring
status: published
created: 2026-03-18
updated: 2026-03-28
updated: 2026-05-02T11:04
---
# Tuning Netdata Docker Health Alarms to Prevent Update Flapping
@ -61,9 +67,9 @@ chart labels: container_name=!nextcloud-aio-nextcloud *
### Dedicated Nextcloud AIO Alarm
Added 2026-03-23, updated 2026-03-28. The `nextcloud-aio-nextcloud` container needs a more lenient window than other containers. Its healthcheck (`/healthcheck.sh`) verifies PostgreSQL connectivity (port 5432) and PHP-FPM (port 9000). PHP-FPM takes ~90 seconds to warm up after a normal restart — but during nightly AIO update cycles, the full startup (occ upgrade, app updates, migrations) can take 5+ minutes. On 2026-03-27, a startup hung and left the container unhealthy for 20 hours until the next nightly cycle replaced it.
Added 2026-03-23, updated 2026-05-02. The `nextcloud-aio-nextcloud` container needs a more lenient window than other containers. Its healthcheck (`/healthcheck.sh`) verifies PostgreSQL connectivity (port 5432) and PHP-FPM (port 9000). PHP-FPM takes ~90 seconds to warm up after a normal restart — but during nightly AIO update cycles, the full startup (occ upgrade, app updates, migrations) can take 5+ minutes. On 2026-03-27, a startup hung and left the container unhealthy for 20 hours until the next nightly cycle replaced it.
The dedicated alarm uses a 10-minute lookup window and 10-minute delay to absorb normal startup, while still catching sustained failures:
The dedicated alarm uses a 30-minute lookup window and 10-minute delay to absorb normal startup and update cycles (~40 minutes total grace), while still catching sustained failures:
```ini
# Dedicated alarm for nextcloud-aio-nextcloud — lenient window to absorb nightly update cycle
@ -76,15 +82,23 @@ template: docker_nextcloud_unhealthy
component: Docker
units: status
every: 30s
lookup: average -10m of unhealthy
lookup: average -30m of unhealthy
chart labels: container_name=nextcloud-aio-nextcloud
warn: $this > 0
warn: $this >= 1
delay: up 10m down 5m multiplier 1.5 max 30m
summary: Nextcloud container health sustained
info: nextcloud-aio-nextcloud has been unhealthy for a sustained period — not a transient update blip
info: nextcloud-aio-nextcloud has been continuously unhealthy for 30+ minutes — not a transient update blip
to: sysadmin
```
**Tuning history:**
| Date | Lookup | Delay | Trigger | Notes |
|---|---|---|---|---|
| 2026-03-23 | 35m | 35m | Initial split from general alarm | Absorbed PHP-FPM warm-up |
| 2026-04-29 | 15m | 5m | Backup blip (~6m) never triggered | Tightened after stability |
| 2026-05-02 | 30m | 10m | 15m still too aggressive for update cycles | ~40m total grace; catches real outages |
## Watchdog Cron: Auto-Restart on Sustained Unhealthy
If the Nextcloud container stays unhealthy for more than 1 hour (well past any normal startup window), a cron watchdog on majorlab auto-restarts it and logs the event. This was added 2026-03-28 after an incident where the container sat unhealthy for 20 hours until the next nightly backup cycle replaced it.

View file

@ -11,7 +11,7 @@ tags:
- cron
status: published
created: 2026-04-18
updated: 2026-04-18T11:13
updated: 2026-04-30T05:21
---
# ClamAV Fleet Deployment with Ansible

View file

@ -1,11 +1,18 @@
---
title: "Fail2Ban Digest Mode — Fleet-Wide Quiet Alerts"
title: Fail2Ban Digest Mode — Fleet-Wide Quiet Alerts
domain: selfhosting
category: security
tags: [fail2ban, security, email, ansible, fleet, cron, digest]
tags:
- fail2ban
- security
- email
- ansible
- fleet
- cron
- digest
status: published
created: 2026-04-22
updated: 2026-04-22
updated: 2026-05-02T14:56
---
# Fail2Ban Digest Mode — Fleet-Wide Quiet Alerts
@ -21,11 +28,11 @@ Three tiers replace the firehose:
| Tier | Jails | Action | Why |
|------|-------|--------|-----|
| **Immediate email** | `sshd`, `recidive` | `action_mwl` | Security-critical — someone is actively targeting auth or is a repeat offender |
| **Immediate email** | `recidive` | `action_mwl` | Repeat offenders only — someone has been banned multiple times across jails |
| **Silent ban** | Everything else | `action_` (default) | Ban happens, firewall rule applied, no email sent |
| **Daily digest** | All jails | Cron script at 08:00 UTC | One summary email per host with ban counts across all jails |
This reduces email volume from hundreds per day to ~10 (one digest per host + occasional sshd/recidive alerts).
This reduces email volume from hundreds per day to ~10 (one digest per host + occasional recidive alerts).
## jail.local Configuration
@ -40,18 +47,20 @@ action = %(action_)s
This overrides the stock `action_mwl` for all jails. Bans still happen — the firewall rule is applied — but no email is sent.
### Keep immediate alerts for critical jails
### Keep immediate alerts for recidive only
```ini
[sshd]
enabled = true
action = %(action_mwl)s
action = %(action_)s
[recidive]
enabled = true
action = %(action_mwl)s
```
> **Updated 2026-05-02:** sshd was moved to silent (`action_`). Only recidive (repeat offenders) now triggers immediate email. sshd bans are captured in the daily digest.
### Clean up email subjects with fq-hostname
By default, fail2ban uses the system FQDN in email subjects. On Tailscale hosts, this produces ugly subjects like `[Fail2Ban] sshd: banned 1.2.3.4 on MajorToot.tail7f2d9.ts.net`. Override it in `[DEFAULT]`:
@ -91,8 +100,9 @@ The playbook `configure_fail2ban_digest.yml` deploys the full digest model fleet
### What it does
1. Deploys a Python helper script that performs **section-aware editing** of `jail.local` (see gotchas below)
2. Sets `action = %(action_)s` in `[DEFAULT]`
3. Sets `action = %(action_mwl)s` in `[sshd]` and `[recidive]`
2. Sets `action = %(action_)s` in `[DEFAULT]` and `[sshd]`
3. Sets `action = %(action_mwl)s` in `[recidive]`
4. Removes stale `action = %(action_mwl)s` from `defaults-debian.conf` if present
4. Sets `fq-hostname` per host using an override dict
5. Deploys the digest script from a Jinja2 template
6. Creates the cron job via `ansible.builtin.cron`
@ -143,6 +153,14 @@ option 'action' in section 'DEFAULT' already exists
The Python editor script handles this by replacing existing keys rather than appending.
### defaults-debian.conf overrides jail.local
On Debian/Ubuntu, `/etc/fail2ban/jail.d/defaults-debian.conf` is loaded **after** `jail.local`. If it contains `action = %(action_mwl)s`, it silently overrides your silent default — every jail sends email on every ban. The Ansible playbook now removes this line automatically. If you see per-ban emails after deploying digest mode, check this file first:
```bash
grep action /etc/fail2ban/jail.d/defaults-debian.conf
```
### fq-hostname scope
Setting `fq-hostname` in `[DEFAULT]` affects all action templates that use the `<fq-hostname>` tag — including both immediate emails and the digest subject. This is the desired behavior, but be aware that it overrides the system hostname globally within fail2ban.

View file

@ -0,0 +1,151 @@
---
title: "wp-fail2ban Plugin Logpath on Debian/Ubuntu (auth.log, not syslog)"
domain: selfhosting
category: security
tags: [fail2ban, wordpress, wp-fail2ban, debugging, gotcha, debian, ubuntu]
status: published
created: 2026-04-30
updated: 2026-04-30
---
# wp-fail2ban Plugin Logpath on Debian/Ubuntu (auth.log, not syslog)
## The Problem
You install the [WP fail2ban](https://wordpress.org/plugins/wp-fail2ban/) WordPress plugin, configure the fleet-standard `wordpress-hard`, `wordpress-soft`, and `wordpress-extra` jails, and… nothing. Weeks pass. `fail2ban-client status wordpress-hard` reports `Total failed: 0, Total banned: 0`. Your site is being attacked, but the jails are dead.
Meanwhile the `wordpress-login` jail (which reads Apache access logs for `POST /wp-login.php` directly) is happily catching brute-forcers. So the problem isn't fail2ban itself — it's specifically the wp-fail2ban-plugin-derived jails.
## The Cause
The wp-fail2ban plugin emits events via PHP's `syslog()` call with facility `LOG_AUTH`. On Debian/Ubuntu, rsyslog routes the `auth` facility to **`/var/log/auth.log`**, NOT `/var/log/syslog`. On RHEL/Fedora it's `/var/log/secure`.
A lot of tutorials, ansible-galaxy roles, and copy-paste config snippets specify:
```ini
logpath = /var/log/syslog
```
That's wrong on Debian/Ubuntu. The events never land there, so the filter regex has nothing to match, so the jail catches zero events forever. Silently.
## Diagnostic Steps
If a `wordpress-{hard,soft,extra}` jail shows `Total failed: 0` over a long window despite the plugin being active and the site getting attacked:
**1. Check what the jail thinks it's watching:**
```bash
sudo fail2ban-client status wordpress-hard | grep "File list"
```
**2. Check where wp-fail2ban events actually land:**
```bash
sudo grep -c "wordpress(" /var/log/auth.log /var/log/syslog /var/log/secure 2>/dev/null
```
You'll see something like:
```
/var/log/auth.log:314
/var/log/syslog:0
```
**3. If the jail's `File list` ≠ the file with events, fix the `logpath`.**
A real event line on Debian/Ubuntu looks like:
```
2026-04-18T23:28:21.027004-04:00 hostname wordpress(example.com)[719989]: XML-RPC authentication failure for someone from 1.2.3.4
```
The `wordpress(domain)[pid]` syslog tag is the giveaway — those are wp-fail2ban events.
## The Fix
Edit the jail blocks in `/etc/fail2ban/jail.local` (or your Ansible source for the jail) and set:
```ini
[wordpress-hard]
enabled = true
port = http,https
filter = wordpress-hard
logpath = /var/log/auth.log
maxretry = 1
findtime = 60
bantime = 30d
backend = polling
[wordpress-soft]
enabled = true
port = http,https
filter = wordpress-soft
logpath = /var/log/auth.log
maxretry = 5
findtime = 60
bantime = 30d
backend = polling
[wordpress-extra]
enabled = true
port = http,https
filter = wordpress-extra
logpath = /var/log/auth.log
maxretry = 5
findtime = 60
bantime = 30d
backend = polling
```
Then:
```bash
sudo fail2ban-client -t # validate
sudo fail2ban-client reload
sudo fail2ban-client status wordpress-hard | grep "File list"
# should now show /var/log/auth.log
```
## Verification
You can prove the filter regex actually matches your real events without waiting for an attack — run `fail2ban-regex` against the rotated log:
```bash
sudo fail2ban-regex /var/log/auth.log.1 /etc/fail2ban/filter.d/wordpress-hard.conf | grep -E "Failregex:|Lines:"
```
Healthy output looks like:
```
Failregex: 81 total
Lines: 13008 lines, 0 ignored, 81 matched, 12927 missed
```
If you see `Failregex: 0 total`, the filter regex doesn't match what the plugin actually emits — which is a different bug (filter version skew vs. plugin version), not the logpath gotcha. Investigate `/etc/fail2ban/filter.d/wordpress-{hard,soft}.conf` against actual event lines.
> **Note:** On a freshly-fixed jail, counters will sit at `Total failed: 0` for a while — the `polling` backend starts at the file's current EOF, so old events aren't retroactively counted. New events from the moment of `reload` onward will accumulate. Allow a few days of normal attack traffic before declaring the fix broken.
## Distribution Cheat Sheet
| Distro family | wp-fail2ban events land in |
|---|---|
| Debian / Ubuntu | `/var/log/auth.log` |
| RHEL / CentOS / Fedora | `/var/log/secure` |
| systemd-journal-only systems | `journalctl SYSLOG_FACILITY=4` (use `backend = systemd` + `journalmatch = SYSLOG_FACILITY=4`) |
If you have a mixed fleet, parameterize the path:
```yaml
# Ansible vars
wp_fail2ban_log_path: "{{ '/var/log/auth.log' if ansible_os_family == 'Debian' else '/var/log/secure' }}"
```
## Why wordpress-login Is Unaffected
The `wordpress-login` jail is a different beast — it reads `/var/log/apache2/access.log` directly and matches `^<HOST> -.*"POST /wp-login.php` via the `wordpress-login` filter. No plugin involved, no syslog facility involved. So a host can have `wordpress-login` working perfectly while `wordpress-{hard,soft,extra}` are silently dead. Don't let a healthy `wordpress-login` reassure you that the rest of the wp-fail2ban stack is also fine.
## Related
- [[fail2ban-wordpress-login-jail]] — the access-log layer that catches WP brute force without any plugin dependency
- [[fail2ban-apache-bad-request-jail]]
- [[fail2ban-apache-php-probe-jail]]
- [[clamav-fleet-deployment]]

View file

@ -10,7 +10,7 @@ tags:
- docker
status: published
created: 2026-04-02
updated: 2026-04-29T22:45
updated: 2026-04-30T05:21
---
# Mastodon Instance Tuning

0
04-streaming/audio/.keep Normal file
View file

View file

View file

View file

View file

@ -11,7 +11,7 @@ tags:
- troubleshooting
status: published
created: 2026-04-18
updated: 2026-04-29T22:45
updated: 2026-04-30T05:21
---
# Ansible Check Mode False Positives in Verify/Assert Tasks

View file

@ -0,0 +1,72 @@
---
title: "Ansible regex_search — capture-group argument doesn't work in set_fact"
domain: troubleshooting
category: general
tags: [ansible, jinja, regex, set_fact, gotcha]
status: published
created: 2026-05-06
updated: 2026-05-06
---
# Ansible `regex_search` — capture-group argument doesn't work in `set_fact`
## Problem
You want to extract a number from a registered command's stdout — e.g. the package count from a dnf or apt upgrade — and stash it in a fact. The natural-looking `regex_search('pattern', '\1')` form fails or produces an empty string when used inside `set_fact`:
```yaml
- name: Capture package count # ❌ does not behave as expected
ansible.builtin.set_fact:
pkg_count: "{{ apt_upgrade_result.stdout | regex_search('([0-9]+) upgraded', '\\1') }}"
```
You'll see one of:
- An empty `pkg_count` (the filter ran but the back-reference returned nothing in this context)
- A Jinja error about argument arity if the syntax is slightly off
- The whole matched substring instead of just the captured group
## Root cause
In `set_fact` templating, the second-positional-argument form of `regex_search` (the back-reference `'\1'` you've seen in tutorials) doesn't reliably select capture groups. The filter is happiest returning the full match. Capture-group selection works in some contexts (e.g. `vars:` blocks, certain Jinja invocations) but not consistently inside `set_fact`, which makes "copy this snippet from the docs" fail intermittently.
## Fix — match the broader pattern, then split
Stop fighting the back-reference. Use `regex_search` to grab a string that *contains* the value you want, then peel it apart with plain Python string ops:
```yaml
- name: Capture package count # ✅ works in set_fact
ansible.builtin.set_fact:
pkg_count: "{{ (apt_upgrade_result.stdout | regex_search('[0-9]+ upgraded') | default('0')).split()[0] }}"
```
What this does:
1. `regex_search('[0-9]+ upgraded')` returns the matching substring (e.g. `"7 upgraded"`) or `None` on no match.
2. `default('0')` turns the `None` case into the string `"0"` so the next step always has something to operate on.
3. `.split()[0]` keeps just the number.
The result (`"7"`) is a string — cast with `| int` if you need arithmetic.
## Where this comes up in MajorAnsible
The `update.yml` executive-summary task uses this pattern to pull package counts out of `apt_upgrade_result.stdout` and `dnf_upgrade_result.stdout` so each host can print one tidy line:
```
majorhome: 7 pkg(s) upgraded | No reboot needed | 2 active screen(s)
majormail: 14 pkg(s) upgraded | REBOOT REQUIRED | Snapshot taken
majorlab: 0 pkg(s) upgraded | No reboot needed
```
The summary line is built with a Jinja `parts` array joined with `' | '` so segments that don't apply (no snapshot, no screens) drop out cleanly without leaving trailing separators.
## Quick checks if this still misbehaves
- **Confirm the source variable.** Ansible 2.x sometimes returns stdout as `result.stdout` and sometimes as `result.stdout_lines`; the `regex_search` filter wants a string, not a list. Use `.stdout` (or `.stdout | join('\n')` for a multi-line list).
- **Escape your backslashes.** In YAML strings, `\d` needs to be written `\\d` or wrapped in single quotes: `'(\d+) upgraded'`.
- **Always provide a default.** `regex_search` returns `None` on miss, which will explode `.split()[0]`. The `| default('0')` bridge is mandatory in production playbooks where some hosts will legitimately have zero upgrades.
## Related
- [[ansible-vault-password-file-missing]] — another set_fact / vault interaction quirk
- [[ansible-ssh-timeout-dnf-upgrade]] — companion gotcha when running `update.yml`

View file

View file

@ -0,0 +1,119 @@
---
title: "LoRA adapter — GGUF conversion fails with 'config.json not found'"
domain: troubleshooting
category: gpu-display
tags: [lora, qlora, gguf, llama.cpp, unsloth, fine-tuning, qwen]
status: published
created: 2026-04-30
updated: 2026-04-30
---
# LoRA adapter — GGUF conversion fails with 'config.json not found'
## Problem
After a QLoRA fine-tune, you point `llama.cpp/convert_hf_to_gguf.py` at the training output directory and it crashes immediately:
```
FileNotFoundError: [Errno 2] No such file or directory:
'/path/to/training-runs/<run>/final/config.json'
```
The output directory looks fine — it contains:
```
adapter_config.json
adapter_model.safetensors (~150 MB for a 7B base)
chat_template.jinja
tokenizer_config.json
tokenizer.json
```
But no `config.json`, and `adapter_model.safetensors` is 150 MB — way smaller than the ~14 GB you'd expect for a full Qwen2.5-7B 16-bit checkpoint.
## Root cause
`model.save_pretrained()` after a LoRA/QLoRA train saves **only the adapter weights**, not a merged full-precision model. `convert_hf_to_gguf.py` expects a full HuggingFace model directory — it reads `config.json` to identify the architecture. Adapter-only directories don't have one.
You need to merge the LoRA adapter into the base model first, then point the GGUF converter at the merged dir.
## Solution
### Quick fix — inline merge step
Insert this block between training completion and `convert_hf_to_gguf.py`:
```python
from unsloth import FastLanguageModel
adapter = "/path/to/training-runs/<run>/final"
merged = "/path/to/training-runs/<run>/merged"
model, tok = FastLanguageModel.from_pretrained(
model_name=adapter,
max_seq_length=2048,
load_in_4bit=True,
)
model.save_pretrained_merged(merged, tok, save_method="merged_16bit")
```
Then run the GGUF converter against the **merged** dir, not the adapter dir:
```bash
python3 llama.cpp/convert_hf_to_gguf.py /path/to/training-runs/<run>/merged \
--outfile model-f16.gguf --outtype f16
```
The merged dir will contain `config.json`, `model-00001-of-00004.safetensors` (multiple shards totaling the full base model size), `generation_config.json`, etc.
### Cleaner fix — use a wrapper
If you do this often, encapsulate it:
1. Wrapper Python script accepts `--adapter`, `--output`, `--skip-merge`, `--all-quants`
2. Step 1: load adapter via `FastLanguageModel.from_pretrained()`, call `save_pretrained_merged()`
3. Step 2: subprocess `convert_hf_to_gguf.py` on the merged dir
4. Step 3: subprocess `llama-quantize` for each requested quant
This is what `~/corpus/scripts/convert_gguf.py` does on MajorRig (rewritten 2026-04-09 for the MajorTwin v7b cycle).
## Why this trips people up
- Unsloth and PEFT both save adapter-only by default after `trainer.save_model()` or `model.save_pretrained()`. There's no warning that downstream tools expect a merged model.
- The training output **looks** complete — there's a `tokenizer.json`, a `chat_template.jinja`, and a non-trivial `.safetensors`. It feels like a checkpoint.
- A pipeline that uses `convert_gguf.py` (with merge) once and then someone reimplements Step 4 inline (skipping the wrapper) will silently lose the merge step. This is what happened in MajorTwin v8c (Apr 30, 2026) — see [[majortwin-v8b-plan#Pipeline Bug + Fix (2026-04-30)]].
## Verification checklist
After training, before running the GGUF converter, verify the directory you're pointing at:
| File | Adapter-only dir | Merged dir |
|---|---|---|
| `adapter_config.json` | ✅ | ❌ |
| `adapter_model.safetensors` | ✅ (~150 MB / 7B) | ❌ |
| `config.json` | ❌ | ✅ |
| `model-*.safetensors` (sharded) | ❌ | ✅ (~14 GB / 7B) |
| `generation_config.json` | ❌ | ✅ |
| `tokenizer.json` | ✅ | ✅ |
If you see only the left column, you need to merge before converting.
## Resuming a failed pipeline without re-training
The adapter is small and self-contained. If your pipeline crashes at the GGUF step, you do NOT need to retrain — the LoRA adapter at `<run>/final/` is intact. Write a resume wrapper that runs only:
1. Merge (`save_pretrained_merged`)
2. F16 conversion (`convert_hf_to_gguf.py`)
3. Quantization (`llama-quantize`)
4. Deploy
This saves the cost of however many GPU-hours the training took. See `~/corpus/scripts/resume_v8c_step4.sh` on MajorRig for an example.
## Related
- [[qwen-14b-oom-3080ti]] — base model size choice on a 12GB GPU
- [[majortwin-v8b-plan]] — v8c pipeline architecture and resume
## Maintenance
- 2026-04-30 — Created after MajorTwin v8c pipeline failed Step 4. Root-caused, patched, resumed.

View file

@ -1,6 +1,6 @@
---
created: 2026-03-15T06:37
updated: 2026-04-29T22:45
updated: 2026-05-02T17:50
---
# 🔧 General Troubleshooting
@ -8,12 +8,14 @@ Practical fixes for common Linux, networking, and application problems.
## 🖥️ GPU & AI
- [Qwen2.5-14B OOM on RTX 3080 Ti (12GB)](gpu-display/qwen-14b-oom-3080ti.md)
- [LoRA adapter — GGUF conversion fails with 'config.json not found'](gpu-display/lora-adapter-gguf-conversion-fails.md)
## 🌐 Networking & Web
- [Apache Outage: Fail2ban Self-Ban + Missing iptables Rules](networking/fail2ban-self-ban-apache-outage.md)
- [Mail Client Stops Receiving: Fail2ban IMAP Self-Ban](networking/fail2ban-imap-self-ban-mail-client.md)
- [firewalld: Mail Ports Wiped After Reload](networking/firewalld-mail-ports-reset.md)
- [Tailscale SSH: Unexpected Re-Authentication Prompt](networking/tailscale-ssh-reauth-prompt.md)
- [iOS Tailscale Clients Report HostName="localhost" — Breaks /etc/hosts Generators](networking/tailscale-status-json-hostname-localhost-ios.md)
- [rsync over Tailscale: Hung in TCP Teardown After Transfer Completes](networking/rsync-tailscale-teardown-stall.md)
- [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md)
- [Pi-hole AI Blocklist Blocks Claude Desktop (ERR_CONNECTION_REFUSED)](networking/pihole-blocks-claude-desktop.md)
@ -25,6 +27,7 @@ Practical fixes for common Linux, networking, and application problems.
- [SSH Timeout During dnf upgrade on Fedora Hosts](ansible-ssh-timeout-dnf-upgrade.md)
- [Vault Password File Missing](ansible-vault-password-file-missing.md)
- [ansible.cfg Ignored on WSL2 Windows Mounts](ansible-wsl2-world-writable-mount-ignores-cfg.md)
- [regex_search — capture-group argument doesn't work in set_fact](ansible-regex-search-set-fact-capture-group.md)
## 📦 Docker & Systems
- [Docker & Caddy Recovery After Reboot (Fedora + SELinux)](docker-caddy-selinux-post-reboot-recovery.md)

View file

@ -1,11 +1,17 @@
---
title: "ISP SNI Filtering & Caddy Troubleshooting"
title: ISP SNI Filtering & Caddy Troubleshooting
domain: troubleshooting
category: general
tags: [isp, sni, caddy, tls, dns, cloudflare]
tags:
- isp
- sni
- caddy
- tls
- dns
- cloudflare
status: published
created: 2026-04-02
updated: 2026-04-02
updated: 2026-04-30T13:07
---
# ISP SNI Filtering & Caddy Troubleshooting
@ -29,3 +35,89 @@ notes.majorshouse.com {
```
Once the hostname was changed to one without the "wiki" keyword, the TLS handshake completed successfully.
---
## 🔁 2026-04-30 Update — Stale A Record + Cloudflare Proxy Fix
The hostname rename held for ~4 weeks. On 2026-04-30 the wiki went down with a TLS handshake failure on `notes.majorshouse.com`. The on-the-spot framing was "ISP filter expanded to include 'notes'" — but Cloudflare DNS audit showed a different (and arguably worse) root cause: **the `notes` A record was pointing at `136.54.3.248`, an IP that is not majorlab's current home IP.** Whichever host responds at that address either does not run Caddy or does not know about the `notes.majorshouse.com` SNI, so the TLS handshake was rejected with `internal_error 80`.
### Re-diagnosis
```bash
# Cert + Caddy + mkdocs all healthy on majorlab
$ ssh majorlab 'systemctl is-active caddy; ss -tlnp | grep :443'
active
LISTEN 0 4096 *:443 users:(("caddy",pid=1549,fd=7))
# Loopback-served TLS works fine — cert valid Mar 11 → Jun 9 2026
$ ssh majorlab 'curl -sS -o /dev/null -w "%{http_code}\n" --resolve notes.majorshouse.com:443:127.0.0.1 https://notes.majorshouse.com/'
200
# External TLS handshake gets rejected with internal_error
$ openssl s_client -servername notes.majorshouse.com -connect 136.54.3.248:443
… SSL alert number 80 (internal_error) …
```
### The smoking-gun comparison
Other `*.majorshouse.com` services worked because they were CNAMEs to the apex, which resolves to majorlab's actual home IP:
| Subdomain | DNS shape | Final IP | Status |
|---|---|---|---|
| `notes.majorshouse.com` | **A → `136.54.3.248`** (stale) | `136.54.3.248` (wrong host) | ❌ TLS rejected |
| `git.majorshouse.com` | CNAME → `majorshouse.com.` | `136.56.0.55` (majorlab) | ✅ |
| `n8n.majorshouse.com` | CNAME → `majorshouse.com.` | `136.56.0.55` (majorlab) | ✅ |
| `matrix.majorshouse.com` | CNAME → `majorshouse.com.` | `136.56.0.55` (majorlab) | ✅ |
None of the working subdomains were proxied through Cloudflare (`proxied=false` on all of them); they simply had the right IP. The `notes` A record was the only one pointing somewhere wrong — most likely a stale value from a prior ISP / IP change that never got cleaned up.
### ✅ Fix — switch `notes` to a Cloudflare-proxied CNAME
Rather than just correcting the A record (which would silently break again the next time the home IP changes), the fix is a CNAME to the apex with proxy on. That gives two protections in one move: it always tracks the apex (so home IP changes propagate automatically) and it puts the wiki behind Cloudflare's edge (so any future ISP-side weirdness like the original `wiki` SNI filter is also bypassed).
```bash
# via Cloudflare API (token from ansible-vault: vault_cloudflare_api_token)
PUT /zones/{ZONE_ID}/dns_records/{NOTES_RECORD_ID}
{
"type": "CNAME",
"name": "notes.majorshouse.com",
"content": "majorshouse.com",
"ttl": 1,
"proxied": true,
"comment": "switched A→CNAME proxied to bypass stale IP / ISP SNI filter"
}
```
Or via the dashboard:
1. Cloudflare → `majorshouse.com` zone → DNS → Records
2. Edit the `notes` record: Type `CNAME`, Target `majorshouse.com`, Proxy `Proxied` (orange cloud)
3. Save
External clients now hit Cloudflare edge IPs (`104.21.x.x` / `172.67.x.x`) which TLS-terminate at the edge and tunnel back to majorlab's apex IP. ACME on majorlab keeps working — Cloudflare passes the HTTP-01 challenge through on port 80. Caddy's `notes.majorshouse.com {}` block needs no change.
Verify (response should show `server: cloudflare` and `via: 1.0 Caddy`):
```bash
curl -sSI https://notes.majorshouse.com/
```
### Why a Cloudflare-proxied CNAME is the durable shape
- **Apex follows the home IP automatically.** Update the apex A record once when the ISP changes; every subdomain inherits it without per-record fixes.
- **TLS handshake is offloaded to CF.** Any ISP-level SNI weirdness (the original `wiki` ban; theoretical future bans) becomes irrelevant — external clients SNI=`notes.majorshouse.com` to Cloudflare, which the ISP doesn't filter.
- **Free.** Cloudflare's free tier covers proxy + TLS termination.
### Audit checklist for any home-hosted `*.majorshouse.com` subdomain
- [ ] DNS record is a **CNAME** to `majorshouse.com.`, not an A record to a literal home IP.
- [ ] Cloudflare proxy (orange cloud, `proxied=true`) enabled on the record — at minimum for any subdomain where TLS reachability matters.
- [ ] Caddy entry on majorlab references the public hostname; `reverse_proxy` stays on the localhost port.
- [ ] HTTPS verified from outside the LAN (phone on cellular is sufficient) within the first hour after the change.
- [ ] If an A record is genuinely required (e.g. it must NOT go through CF), document why in the deploy notes for that service.
### Related
- [[majwiki-setup-and-pipeline]] — full wiki deploy pipeline; the DNS step there should reference this fix
- [[Network-Overview]] — fleet IP table

View file

@ -0,0 +1,116 @@
---
title: iOS Tailscale Clients Report HostName="localhost" — Breaks /etc/hosts Generators
domain: troubleshooting
category: networking
tags:
- tailscale
- ios
- postfix
- etc-hosts
- jq
status: published
created: 2026-04-29
updated: 2026-04-29
---
# iOS Tailscale Clients Report HostName="localhost" — Breaks /etc/hosts Generators
## Problem
A homegrown script that builds an `/etc/hosts` block from `tailscale status --json` silently corrupted the file the moment any iOS device joined the tailnet. After the next run, services bound to `localhost` started failing.
On the affected host (`majordiscord`), Postfix refused to start with:
```
postfix: fatal: parameter inet_interfaces: no local interface found for 100.127.114.10
```
`/etc/hosts` looked fine at the top — `127.0.0.1 localhost` was still present — but inside the Tailscale-managed block:
```
# TAILSCALE_START
100.84.42.102 tttpod
100.110.197.17 majortoot
100.95.55.40 localhost <-- WRONG (this is an iPhone)
100.84.165.52 majormail
...
100.127.114.10 localhost <-- WRONG (this is an iPad)
# TAILSCALE_END
```
When Postfix resolved `localhost` (because `inet_interfaces = localhost` in `main.cf`), the **last matching entry** in `/etc/hosts` won — a Tailscale IP that doesn't exist on this host — and the daemon died on bind.
## Root Cause
The script used `.HostName` from the Tailscale JSON:
```bash
tailscale status --json \
| jq -r '.Peer[] | "\(.TailscaleIPs[0]) \(.HostName)"' \
>> "$TEMP_HOSTS"
```
iOS Tailscale clients (iPhone, iPad) **always report `HostName: "localhost"`** in the JSON. iOS doesn't expose the real device name to apps the way macOS/Linux/Windows do, so the Tailscale client falls back to the literal string `localhost`.
Inspect it directly:
```bash
$ tailscale status --json | jq '.Peer[] | select(.OS == "iOS") | {DNSName, HostName, OS}'
{
"DNSName": "iphone171.tail7f2d9.ts.net.",
"HostName": "localhost",
"OS": "iOS"
}
{
"DNSName": "ipad166.tail7f2d9.ts.net.",
"HostName": "localhost",
"OS": "iOS"
}
```
Every iOS device contributes a line `<tailscale-ip> localhost` to `/etc/hosts`, hijacking the `localhost` lookup.
## Fix
Use `.DNSName` (the unique tailnet DNS name) and take the first dotted component instead of `.HostName`:
```bash
tailscale status --json \
| jq -r '.Peer[] | "\(.TailscaleIPs[0]) \(.DNSName | rtrimstr(".") | split(".")[0])"' \
>> "$TEMP_HOSTS"
```
`DNSName` is always set, always unique, and produces clean labels like `iphone171`, `ipad166`, `majorlab`, etc.
After patching the script and re-running it:
```bash
$ bash /root/update_tailscale_hosts.sh
$ systemctl restart postfix
$ systemctl is-active postfix
active
```
## Why It's Hard to Spot
- The corruption only triggers when an iOS device is in the tailnet — so the script "worked" for months.
- `/etc/hosts` files are commonly skimmed top-down. The bogus `localhost` line is buried in the Tailscale block, well below the legitimate `127.0.0.1 localhost` line, and looks superficially like a normal Tailscale entry.
- Postfix's error message names the IP, not `localhost`, so the connection to `/etc/hosts` isn't obvious.
- `getent hosts localhost` shows the *first* match (`127.0.0.1`), not the one Postfix's resolver actually picks for `inet_interfaces` lookup.
## Verification Checklist
If you suspect this on any host using a similar generator script:
```bash
# Any non-loopback "localhost" entries are bugs
grep -nE '^[0-9]+\..* localhost\s*$' /etc/hosts
# Look at iOS peers' HostName field
tailscale status --json | jq '.Peer[] | select(.OS == "iOS") | .HostName'
```
## Related
- [[majordiscord]] — affected host (incident logged 2026-04-29)
- [[Network Overview]] — Tailscale fleet topology

View file

@ -11,7 +11,7 @@ tags:
- powershell
status: published
created: 2026-04-03
updated: 2026-04-22T09:20
updated: 2026-04-30T05:21
---
# Windows OpenSSH: WSL as Default Shell Breaks Remote Commands

View file

@ -10,7 +10,7 @@ tags:
- majorrig
status: published
created: 2026-04-02
updated: 2026-04-22T09:20
updated: 2026-04-30T05:21
---
# Windows OpenSSH Server (sshd) Stops After Reboot

View file

View file

@ -10,7 +10,7 @@ tags:
- deno
status: published
created: 2026-04-02
updated: 2026-04-22T11:33
updated: 2026-04-30T05:21
---
# yt-dlp YouTube JS Challenge Fix (Fedora)

View file

@ -2,7 +2,7 @@
title: MajorWiki Deployment Status
status: deployed
project: MajorTwin
updated: 2026-04-07T10:48
updated: 2026-04-30T05:30
created: 2026-04-02T16:10
---
@ -79,6 +79,23 @@ git push
Gitea receives the push → fires webhook → majorlab pulls → MkDocs rebuilds → `notes.majorshouse.com` updates automatically.
> [!tip] One-liner wrapper
> On MajorRig, the `~/bin/wiki-commit "msg"` helper runs `git pull --rebase --autostash``git add -A``git commit``git push` in one shot. Sidesteps fast-forward rejections from cowork pushes (e.g. MajorAir pushing in parallel) and the empty-credentials issue with HTTPS.
## 🔒 Pre-Commit Hook (in repo)
`.githooks/pre-commit` (tracked) blocks any commit that adds or renames a `*.md` article without a corresponding entry in `SUMMARY.md`. Bypass with `git commit --no-verify` if you genuinely need to.
**Per-clone setup** (one-time, per workstation that uses the repo):
```bash
cd <wiki-repo>
git config core.hooksPath .githooks
git config pull.rebase true
```
The hooksPath line is required — git doesn't run hooks from a tracked directory by default. The `pull.rebase true` makes plain `git pull` always rebase locally, matching the `wiki-commit` wrapper's behavior.
## 📋 Wiki Maintenance Protocol
Every time a new article is added, the following **MUST** be updated to maintain index integrity:

View file

@ -1,6 +1,6 @@
---
created: 2026-04-06T09:52
updated: 2026-04-29T22:46
updated: 2026-04-30T05:21
---
# MajorLinux Tech Wiki — Index

View file

@ -1,6 +1,6 @@
---
created: 2026-04-02T16:03
updated: 2026-04-29T22:45
updated: 2026-05-05T23:39
---
* [Home](index.md)
* [Linux & Sysadmin](01-linux/index.md)
@ -43,6 +43,7 @@ updated: 2026-04-29T22:45
* [Fail2ban Custom Jail: Apache 404 Scanner Detection](02-selfhosting/security/fail2ban-apache-404-scanner-jail.md)
* [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md)
* [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md)
* [wp-fail2ban Plugin Logpath on Debian/Ubuntu (auth.log not syslog)](02-selfhosting/security/wp-fail2ban-logpath-debian-ubuntu.md)
* [SELinux: Fixing Fail2ban grep execmem Denial](02-selfhosting/security/selinux-fail2ban-execmem-fix.md)
* [UFW Firewall Management](02-selfhosting/security/ufw-firewall-management.md)
* [Firewall Hardening with firewalld on Fedora Fleet](02-selfhosting/security/firewalld-fleet-hardening.md)
@ -50,6 +51,8 @@ updated: 2026-04-29T22:45
* [Fail2ban Custom Jail: Apache Bad Request Detection](02-selfhosting/security/fail2ban-apache-bad-request-jail.md)
* [SSH Hardening Fleet-Wide with Ansible](02-selfhosting/security/ssh-hardening-ansible-fleet.md)
* [ClamAV Fleet Deployment with Ansible](02-selfhosting/security/clamav-fleet-deployment.md)
* [Fail2Ban Digest Mode — Fleet-Wide Quiet Alerts](02-selfhosting/security/fail2ban-digest-mode-fleet.md)
* [Apache CVE-2026-23918 — HTTP/2 Double Free Mitigation](02-selfhosting/security/apache-cve-2026-23918-http2-mitigation.md)
* [Open Source & Alternatives](03-opensource/index.md)
* [SearXNG: Private Self-Hosted Search](03-opensource/alternatives/searxng.md)
* [FreshRSS: Self-Hosted RSS Reader](03-opensource/alternatives/freshrss.md)
@ -77,6 +80,7 @@ updated: 2026-04-29T22:45
* [ISP SNI Filtering with Caddy](05-troubleshooting/isp-sni-filtering-caddy.md)
* [Obsidian Vault Recovery — Loading Cache Hang](05-troubleshooting/obsidian-cache-hang-recovery.md)
* [Qwen2.5-14B OOM on RTX 3080 Ti (12GB)](05-troubleshooting/gpu-display/qwen-14b-oom-3080ti.md)
* [LoRA adapter — GGUF conversion fails with 'config.json not found'](05-troubleshooting/gpu-display/lora-adapter-gguf-conversion-fails.md)
* [yt-dlp YouTube JS Challenge Fix on Fedora](05-troubleshooting/yt-dlp-fedora-js-challenge.md)
* [Gemini CLI Manual Update](05-troubleshooting/gemini-cli-manual-update.md)
* [MajorWiki Setup & Publishing Pipeline](05-troubleshooting/majwiki-setup-and-pipeline.md)
@ -90,11 +94,13 @@ updated: 2026-04-29T22:45
* [Ollama Drops Off Tailscale When Mac Sleeps](05-troubleshooting/ollama-macos-sleep-tailscale-disconnect.md)
* [Ollama: `ollama run` with Piped Stdin Bypasses Chat Template + SYSTEM Prompt](05-troubleshooting/ollama-chat-template-pipe-stdin-bypass.md)
* [rsync over Tailscale: Hung in TCP Teardown After Transfer Completes](05-troubleshooting/networking/rsync-tailscale-teardown-stall.md)
* [iOS Tailscale Clients Report HostName="localhost" — Breaks /etc/hosts Generators](05-troubleshooting/networking/tailscale-status-json-hostname-localhost-ios.md)
* [macOS: Repeating Alert Tone from Mirrored iPhone Notification](05-troubleshooting/macos-mirrored-notification-alert-loop.md)
* [ClamAV CPU Spike: Safe Scheduling with nice/ionice](05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md)
* [Ansible: Vault Password File Not Found](05-troubleshooting/ansible-vault-password-file-missing.md)
* [Ansible: ansible.cfg Ignored on WSL2 Windows Mounts](05-troubleshooting/ansible-wsl2-world-writable-mount-ignores-cfg.md)
* [Ansible: SSH Timeout During dnf upgrade on Fedora Hosts](05-troubleshooting/ansible-ssh-timeout-dnf-upgrade.md)
* [Ansible: regex_search Capture-Group Argument Fails in set_fact](05-troubleshooting/ansible-regex-search-set-fact-capture-group.md)
* [Fedora Networking & Kernel Troubleshooting](05-troubleshooting/fedora-networking-kernel-recovery.md)
* [Systemd Session Scope Fails at Login](05-troubleshooting/systemd/session-scope-failure-at-login.md)
* [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md)

328
index.md
View file

@ -1,179 +1,210 @@
---
created: 2026-04-06T09:52
updated: 2026-04-29T22:45
updated: 2026-05-02T16:45
---
# MajorLinux Tech Wiki — Index
> A growing reference of Linux, self-hosting, open source, streaming, and troubleshooting guides. Written by MajorLinux. Used by MajorTwin.
>
> **Last updated:** 2026-04-18
> **Article count:** 89
> **Last updated:** 2026-05-02
> **Article count:** 106
## Domains
| Domain | Folder | Articles |
|---|---|---|
| 🐧 Linux & Sysadmin | `01-linux/` | 12 |
| 🏠 Self-Hosting & Homelab | `02-selfhosting/` | 32 |
| 🏠 Self-Hosting & Homelab | `02-selfhosting/` | 39 |
| 🔓 Open Source Tools | `03-opensource/` | 10 |
| 🎙️ Streaming & Podcasting | `04-streaming/` | 2 |
| 🔧 General Troubleshooting | `05-troubleshooting/` | 34 |
| 🔧 General Troubleshooting | `05-troubleshooting/` | 43 |
---
## 🐧 Linux & Sysadmin
### Files & Permissions
- [Linux File Permissions](01-linux/files-permissions/linux-file-permissions.md) — chmod, chown, special bits, finding permission problems
### Distro-Specific
- [Linux Distro Guide for Beginners](01-linux/distro-specific/linux-distro-guide-beginners.md)
- [WSL2 Backup via PowerShell Scheduled Task](01-linux/distro-specific/wsl2-backup-powershell.md)
- [WSL2 Instance Migration (Fedora 43)](01-linux/distro-specific/wsl2-instance-migration-fedora43.md)
- [Wsl2 Rebuild Fedora43 Training Env](01-linux/distro-specific/wsl2-rebuild-fedora43-training-env.md)
### Process Management
- [Managing Linux Services with systemd](01-linux/process-management/managing-linux-services-systemd-ansible.md) — systemctl, journalctl, writing service files, Ansible service management
### Files & Permissions
- [Linux File Permissions and Ownership](01-linux/files-permissions/linux-file-permissions.md)
### Networking
- [SSH Config & Key Management](01-linux/networking/ssh-config-key-management.md) — key generation, ssh-copy-id, ~/.ssh/config, managing multiple keys, Windows OpenSSH admin key auth
- [SSH Config and Key Management](01-linux/networking/ssh-config-key-management.md)
### Package Management
- [Package Management Reference](01-linux/packages/package-management-reference.md) — apt, dnf, pacman side-by-side reference, Flatpak/Snap
- [Linux Package Management Reference: apt, dnf, pacman](01-linux/packages/package-management-reference.md)
### Process Management
- [Managing Linux Services: systemd and Ansible](01-linux/process-management/managing-linux-services-systemd-ansible.md)
### Shell & Scripting
- [Ansible Getting Started](01-linux/shell-scripting/ansible-getting-started.md) — inventory, ad-hoc commands, playbooks, handlers, roles
- [Bash Scripting Patterns](01-linux/shell-scripting/bash-scripting-patterns.md) — set -euo pipefail, logging, error handling, argument parsing, common patterns
- [Ansible Getting Started: Inventory, Playbooks, and Ad-Hoc Commands](01-linux/shell-scripting/ansible-getting-started.md)
- [Bash Scripting Patterns for Sysadmins](01-linux/shell-scripting/bash-scripting-patterns.md)
### Storage
- [SnapRAID & MergerFS Storage Setup](01-linux/storage/snapraid-mergerfs-setup.md) — Pooling mismatched drives and adding parity on Linux
- [mdadm — Rebuilding a RAID Array After Reinstall](01-linux/storage/mdadm-raid-rebuild.md) — reassembling and recovering mdadm arrays after OS reinstall
- [SnapRAID & MergerFS Storage Setup](01-linux/storage/snapraid-mergerfs-setup.md)
- [mdadm — Rebuilding a RAID Array After Reinstall](01-linux/storage/mdadm-raid-rebuild.md)
### Distro-Specific
- [Linux Distro Guide for Beginners](01-linux/distro-specific/linux-distro-guide-beginners.md) — Ubuntu recommendation, distro comparison, desktop environments
- [WSL2 Instance Migration to Fedora 43](01-linux/distro-specific/wsl2-instance-migration-fedora43.md) — moving WSL2 VHDX from C: to another drive
- [WSL2 Training Environment Rebuild (Fedora 43)](01-linux/distro-specific/wsl2-rebuild-fedora43-training-env.md) — rebuilding the MajorTwin training env in WSL2 from scratch
- [WSL2 Backup via PowerShell Scheduled Task](01-linux/distro-specific/wsl2-backup-powershell.md) — automating WSL2 exports on a schedule using PowerShell
---
## 🏠 Self-Hosting & Homelab
### Docker & Containers
- [Self-Hosting Starter Guide](02-selfhosting/docker/self-hosting-starter-guide.md) — hardware options, Docker install, first services, networking basics
- [Docker vs VMs for the Homelab](02-selfhosting/docker/docker-vs-vms-homelab.md) — when to use containers vs VMs, KVM setup, how to run both
- [Debugging Broken Docker Containers](02-selfhosting/docker/debugging-broken-docker-containers.md) — logs, inspect, exec, port conflicts, permission errors
- [Docker Healthchecks](02-selfhosting/docker/docker-healthchecks.md) — writing and debugging HEALTHCHECK instructions in Docker containers
- [Watchtower SMTP via Localhost Postfix Relay](02-selfhosting/docker/watchtower-smtp-localhost-relay.md) — credential-free container update notifications by routing through a local Postfix relay
### Reverse Proxies
- [Setting Up Caddy as a Reverse Proxy](02-selfhosting/reverse-proxy/setting-up-caddy-reverse-proxy.md) — Caddyfile basics, automatic HTTPS, local TLS, DNS challenge
### Cloud
- [AWS S3 Cost Management](02-selfhosting/cloud/aws-s3-cost-management.md)
### DNS & Networking
- [Tailscale for Homelab Remote Access](02-selfhosting/dns-networking/tailscale-homelab-remote-access.md) — installation, MagicDNS, making services accessible, subnet router, ACLs
- [Network Overview](02-selfhosting/dns-networking/network-overview.md) — MajorsHouse network topology, Tailscale IPs, and connectivity map
- [Wake-on-LAN via Router SSH](02-selfhosting/dns-networking/wake-on-lan-router-ssh.md) — send WOL magic packets through an Asus router over SSH, with Ansible vault integration
- [Network Overview](02-selfhosting/dns-networking/network-overview.md)
- [Pi-hole DoH / DoT Bypass Defense](02-selfhosting/dns-networking/pihole-doh-dot-bypass-defense.md)
- [Pi-hole v6 Adlist Management via SQL](02-selfhosting/dns-networking/pihole-v6-adlist-management.md)
- [Pi-hole v6 Group Management: Per-Client DNS Rules](02-selfhosting/dns-networking/pihole-v6-group-management.md)
- [Tailscale for Homelab Remote Access](02-selfhosting/dns-networking/tailscale-homelab-remote-access.md)
- [Wake-on-LAN via Router SSH](02-selfhosting/dns-networking/wake-on-lan-router-ssh.md)
### Cloud
- [AWS S3 Cost Management](02-selfhosting/cloud/aws-s3-cost-management.md) — identify and control S3 costs: lifecycle rules, storage class selection, bucket inventory, unexpected-growth investigation
### Storage & Backup
- [rsync Backup Patterns](02-selfhosting/storage-backup/rsync-backup-patterns.md) — flags reference, remote backup, incremental with hard links, cron/systemd
### Docker & Containers
- [Debugging Broken Docker Containers](02-selfhosting/docker/debugging-broken-docker-containers.md)
- [Docker Healthchecks](02-selfhosting/docker/docker-healthchecks.md)
- [Docker vs VMs in the Homelab: Why Not Both?](02-selfhosting/docker/docker-vs-vms-homelab.md)
- [Self-Hosting Starter Guide](02-selfhosting/docker/self-hosting-starter-guide.md)
- [Watchtower SMTP via Localhost Postfix Relay](02-selfhosting/docker/watchtower-smtp-localhost-relay.md)
### Monitoring
- [Tuning Netdata Web Log Alerts](02-selfhosting/monitoring/tuning-netdata-web-log-alerts.md) — tuning web_log_1m_redirects threshold for HTTPS-forcing servers
- [Tuning Netdata Docker Health Alarms](02-selfhosting/monitoring/netdata-docker-health-alarm-tuning.md) — preventing false alerts during nightly Nextcloud AIO container update cycles
- [Deploying Netdata to a New Server](02-selfhosting/monitoring/netdata-new-server-setup.md) — install, email notifications, and Netdata Cloud claim for Ubuntu/Debian servers
- [Netdata + n8n Enriched Alert Emails](02-selfhosting/monitoring/netdata-n8n-enriched-alerts.md) — rich HTML alert emails with remediation steps and wiki links via n8n
- [Netdata SELinux AVC Denial Monitoring](02-selfhosting/monitoring/netdata-selinux-avc-chart.md) — custom Netdata chart for tracking SELinux AVC denials
- [Deploying Netdata to a New Server](02-selfhosting/monitoring/netdata-new-server-setup.md)
- [Netdata SELinux AVC Denial Monitoring](02-selfhosting/monitoring/netdata-selinux-avc-chart.md)
- [Netdata n8n Enriched Alert Emails](02-selfhosting/monitoring/netdata-n8n-enriched-alerts.md)
- [Tuning Netdata Docker Health Alarms to Prevent Update Flapping](02-selfhosting/monitoring/netdata-docker-health-alarm-tuning.md)
- [Tuning Netdata Web Log Alerts](02-selfhosting/monitoring/tuning-netdata-web-log-alerts.md)
### Reverse Proxies
- [Setting Up a Reverse Proxy with Caddy](02-selfhosting/reverse-proxy/setting-up-caddy-reverse-proxy.md)
### Security
- [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md) — non-root user, SSH key auth, sshd_config, firewall, fail2ban, SpamAssassin
- [Standardizing unattended-upgrades with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md) — fleet-wide automatic security updates across Ubuntu servers
- [Fail2ban Custom Jail: Apache 404 Scanner Detection](02-selfhosting/security/fail2ban-apache-404-scanner-jail.md) — custom filter and jail for blocking 404 scanners
- [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md) — catching PHP webshell/backdoor probes that return 301 on HTTPS-redirecting servers
- [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md) — access-log-based wp-login.php brute force detection without plugins
- [SELinux: Fixing Fail2ban grep execmem Denial](02-selfhosting/security/selinux-fail2ban-execmem-fix.md) — resolving execmem AVC denials from Fail2ban's grep on Fedora
- [UFW Firewall Management](02-selfhosting/security/ufw-firewall-management.md) — managing UFW rules, common patterns, troubleshooting
- [Firewall Hardening with firewalld on Fedora Fleet](02-selfhosting/security/firewalld-fleet-hardening.md) — audit-and-harden pattern for Fedora fleet hosts using Ansible; flush stale rules, rebuild minimal whitelists
- [Fail2ban Custom Jail: Nginx Bad Request Detection](02-selfhosting/security/fail2ban-nginx-bad-request-jail.md) — wiring the stock nginx-bad-request filter to a jail to catch malformed-request scanners
- [Fail2ban Custom Jail: Apache Bad Request Detection](02-selfhosting/security/fail2ban-apache-bad-request-jail.md) — custom filter for Apache 400 Bad Request responses (no stock equivalent exists)
- [SSH Hardening Fleet-Wide with Ansible](02-selfhosting/security/ssh-hardening-ansible-fleet.md) — drop-in sshd config hardening across mixed Ubuntu/Fedora fleets
- [ClamAV Fleet Deployment with Ansible](02-selfhosting/security/clamav-fleet-deployment.md) — deploy ClamAV with nice/ionice throttling, freshclam, and quarantine to internet-facing hosts
- [ClamAV Fleet Deployment with Ansible](02-selfhosting/security/clamav-fleet-deployment.md)
- [Fail2Ban Digest Mode — Fleet-Wide Quiet Alerts](02-selfhosting/security/fail2ban-digest-mode-fleet.md)
- [Fail2ban Custom Jail: Apache 404 Scanner Detection](02-selfhosting/security/fail2ban-apache-404-scanner-jail.md)
- [Fail2ban Custom Jail: Apache Bad Request Detection](02-selfhosting/security/fail2ban-apache-bad-request-jail.md)
- [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md)
- [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md)
- [Fail2ban: Enable the nginx-bad-request Jail](02-selfhosting/security/fail2ban-nginx-bad-request-jail.md)
- [Firewall Hardening with firewalld on Fedora Fleet](02-selfhosting/security/firewalld-fleet-hardening.md)
- [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md)
- [SELinux: Fixing Fail2ban grep execmem Denial on Fedora](02-selfhosting/security/selinux-fail2ban-execmem-fix.md)
- [SSH Hardening Fleet-Wide with Ansible](02-selfhosting/security/ssh-hardening-ansible-fleet.md)
- [Standardizing unattended-upgrades Across Ubuntu Fleet with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md)
- [UFW Firewall Management](02-selfhosting/security/ufw-firewall-management.md)
- [wp-fail2ban Plugin Logpath on Debian/Ubuntu (auth.log, not syslog)](02-selfhosting/security/wp-fail2ban-logpath-debian-ubuntu.md)
### Services
- [Updating n8n Running in Docker](02-selfhosting/services/updating-n8n-docker.md) — pinned version updates, password reset, Arcane timing gaps
- [Mastodon Instance Tuning](02-selfhosting/services/mastodon-instance-tuning.md) — character limit increase, media cache management for self-hosted Mastodon
- [Ghost Email Configuration with Mailgun](02-selfhosting/services/ghost-smtp-mailgun-setup.md) — configuring Ghost's two independent mail systems (newsletter API + transactional SMTP) with Mailgun
- [Claude Code Remote Control — Mobile Access to a Persistent Host Session](02-selfhosting/services/claude-code-remote-control.md) — running `claude remote-control` on a host so `claude.ai` and the Claude mobile app can drive the CLI, with vault + MCPs intact
- [Claude Code Remote Control — Mobile Access to a Persistent Host Session](02-selfhosting/services/claude-code-remote-control.md)
- [Ghost Email Configuration with Mailgun](02-selfhosting/services/ghost-smtp-mailgun-setup.md)
- [Mastodon DB Maintenance — Statuses, Accounts, and VACUUM](02-selfhosting/services/mastodon-db-maintenance.md)
- [Mastodon Federation — Domain Blocks, Silencing, and FediSeer](02-selfhosting/services/mastodon-federation.md)
- [Mastodon Instance Tuning](02-selfhosting/services/mastodon-instance-tuning.md)
- [Updating n8n Running in Docker](02-selfhosting/services/updating-n8n-docker.md)
### Storage & Backup
- [rsync Backup Patterns](02-selfhosting/storage-backup/rsync-backup-patterns.md)
---
## 🔓 Open Source Tools
### Alternatives
- [SearXNG: Private Self-Hosted Search](03-opensource/alternatives/searxng.md) — metasearch engine that queries multiple engines without exposing your identity
- [FreshRSS: Self-Hosted RSS Reader](03-opensource/alternatives/freshrss.md) — algorithm-free feed aggregator with mobile app sync
- [Gitea: Self-Hosted Git](03-opensource/alternatives/gitea.md) — lightweight GitHub alternative, webhooks, single Docker container
### Productivity
- [rmlint: Duplicate File Scanning](03-opensource/productivity/rmlint-duplicate-scanning.md) — extremely fast duplicate file finding and storage reclamation
- [FreshRSS — Self-Hosted RSS Reader](03-opensource/alternatives/freshrss.md)
- [Gitea — Self-Hosted Git](03-opensource/alternatives/gitea.md)
- [SearXNG — Private Self-Hosted Search](03-opensource/alternatives/searxng.md)
### Development Tools
- [tmux: Persistent Terminal Sessions](03-opensource/dev-tools/tmux.md) — detachable sessions for long-running jobs over SSH
- [screen: Simple Persistent Sessions](03-opensource/dev-tools/screen.md) — lightweight terminal multiplexer, universally available
- [rsync: Fast, Resumable File Transfers](03-opensource/dev-tools/rsync.md) — incremental file sync locally and over SSH, survives interruptions
- [Ventoy: Multi-Boot USB Tool](03-opensource/dev-tools/ventoy.md) — drop ISOs on a USB drive and boot any of them, no reflashing
### Privacy & Security
- [Vaultwarden: Self-Hosted Password Manager](03-opensource/privacy-security/vaultwarden.md) — Bitwarden-compatible server in a single Docker container, passwords stay on your hardware
- [Ventoy — Multi-Boot USB Tool](03-opensource/dev-tools/ventoy.md)
- [rsync — Fast, Resumable File Transfers](03-opensource/dev-tools/rsync.md)
- [screen — Simple Persistent Terminal Sessions](03-opensource/dev-tools/screen.md)
- [tmux — Persistent Terminal Sessions](03-opensource/dev-tools/tmux.md)
### Media & Creative
- [yt-dlp: Video Downloading](03-opensource/media-creative/yt-dlp.md) — download from YouTube and hundreds of other sites, Plex-optimized format selection
- [yt-dlp — Video Downloading](03-opensource/media-creative/yt-dlp.md)
### Privacy & Security
- [Vaultwarden — Self-Hosted Password Manager](03-opensource/privacy-security/vaultwarden.md)
### Productivity
- [rmlint — Extreme Duplicate File Scanning](03-opensource/productivity/rmlint-duplicate-scanning.md)
---
## 🎙️ Streaming & Podcasting
### OBS Studio
- [OBS Studio Setup & Encoding](04-streaming/obs/obs-studio-setup-encoding.md) — installation, NVENC/x264 settings, scene setup, audio filters, Linux Wayland notes
- [OBS Studio Setup and Encoding Settings](04-streaming/obs/obs-studio-setup-encoding.md)
### Plex
- [Plex 4K Codec Compatibility (Apple TV)](04-streaming/plex/plex-4k-codec-compatibility.md) — AV1/VP9 vs HEVC, batch conversion script, yt-dlp auto-convert hook
- [Plex 4K Codec Compatibility (Apple TV)](04-streaming/plex/plex-4k-codec-compatibility.md)
---
## 🔧 General Troubleshooting
- [Apache Outage: Fail2ban Self-Ban + Missing iptables Rules](05-troubleshooting/networking/fail2ban-self-ban-apache-outage.md) — diagnosing and fixing Apache outages caused by missing firewall rules and Fail2ban self-bans
- [Mail Client Stops Receiving: Fail2ban IMAP Self-Ban](05-troubleshooting/networking/fail2ban-imap-self-ban-mail-client.md) — diagnosing why one device stops receiving email when the mail server is healthy
- [firewalld: Mail Ports Wiped After Reload](05-troubleshooting/networking/firewalld-mail-ports-reset.md) — recovering IMAP and webmail after firewalld reload drops all mail service rules
- [Fail2ban & UFW Rule Bloat: 30k Rules Slowing Down a VPS](05-troubleshooting/networking/fail2ban-ufw-rule-bloat-cleanup.md) — diagnosing and cleaning up massive nftables/UFW rule accumulation
- [Tailscale SSH: Unexpected Re-Authentication Prompt](05-troubleshooting/networking/tailscale-ssh-reauth-prompt.md) — resolving unexpected re-auth prompts on Tailscale SSH connections
- [Docker & Caddy Recovery After Reboot (Fedora + SELinux)](05-troubleshooting/docker-caddy-selinux-post-reboot-recovery.md) — fixing docker.socket, SELinux port blocks, and httpd_can_network_connect after reboot
- [n8n Behind Reverse Proxy: X-Forwarded-For Trust Fix](05-troubleshooting/docker/n8n-proxy-trust-x-forwarded-for.md) — fixing webhook failures caused by missing proxy trust configuration
- [Nextcloud AIO Container Unhealthy for 20 Hours](05-troubleshooting/docker/nextcloud-aio-unhealthy-20h-stuck.md) — diagnosing stuck Nextcloud AIO containers after nightly update cycles
- [ISP SNI Filtering with Caddy](05-troubleshooting/isp-sni-filtering-caddy.md) — troubleshooting why wiki.majorshouse.com was blocked by Google Fiber
- [Obsidian Cache Hang Recovery](05-troubleshooting/obsidian-cache-hang-recovery.md) — resolving "Loading cache" hang in Obsidian by cleaning Electron app data and ML artifacts
- [macOS Repeating Alert Tone from Mirrored Notification](05-troubleshooting/macos-mirrored-notification-alert-loop.md) — stopping alert tone loops from mirrored iPhone notifications on Mac
- [Qwen2.5-14B OOM on RTX 3080 Ti (12GB)](05-troubleshooting/gpu-display/qwen-14b-oom-3080ti.md) — fixes and alternatives when hitting VRAM limits during fine-tuning
- [yt-dlp YouTube JS Challenge Fix on Fedora](05-troubleshooting/yt-dlp-fedora-js-challenge.md) — fixing YouTube JS challenge solver errors and missing formats on Fedora
- [Gemini CLI Manual Update](05-troubleshooting/gemini-cli-manual-update.md) — how to manually update the Gemini CLI when automatic updates fail
- [MajorWiki Setup & Pipeline](05-troubleshooting/majwiki-setup-and-pipeline.md) — setting up MajorWiki and the Obsidian → Gitea → MkDocs publishing pipeline
- [Gitea Actions Runner: Boot Race Condition Fix](05-troubleshooting/gitea-runner-boot-race-network-target.md) — fixing act_runner crash loop on boot caused by DNS not ready at startup
- [Cron Heartbeat False Alarm: /var/run Cleared by Reboot](05-troubleshooting/cron-heartbeat-tmpfs-reboot-false-alarm.md) — why `/run` is tmpfs and how a reboot wipes cron heartbeat files, and where to put them instead
- [SELinux: Fixing Dovecot Mail Spool Context (/var/vmail)](05-troubleshooting/selinux-dovecot-vmail-context.md) — fixing thousands of AVC denials when /var/vmail has wrong SELinux context
- [mdadm RAID Recovery After USB Hub Disconnect](05-troubleshooting/storage/mdadm-usb-hub-disconnect-recovery.md) — diagnosing and recovering a failed mdadm array caused by a USB hub dropout
- [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md) — fixing sshd not running after reboot due to Manual startup type
- [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) — fixing remote SSH command failures when wsl.exe is the default shell
- [Ollama Drops Off Tailscale When Mac Sleeps](05-troubleshooting/ollama-macos-sleep-tailscale-disconnect.md) — keeping Ollama reachable over Tailscale by disabling macOS sleep on AC power
- [Ansible: Vault Password File Not Found](05-troubleshooting/ansible-vault-password-file-missing.md) — fixing the missing vault_pass file error when running ansible-playbook
- [Ansible: ansible.cfg Ignored on WSL2 Windows Mounts](05-troubleshooting/ansible-wsl2-world-writable-mount-ignores-cfg.md) — fixing silent config ignore due to world-writable /mnt/d/ permissions
- [Ansible SSH Timeout During dnf upgrade](05-troubleshooting/ansible-ssh-timeout-dnf-upgrade.md) — preventing SSH timeouts during long-running dnf upgrades on Fedora
- [Fedora Networking & Kernel Troubleshooting](05-troubleshooting/fedora-networking-kernel-recovery.md) — nmcli quick fix, GRUB kernel rollback, and recovery for Fedora fleet
- [Custom Fail2ban Jail: Apache Directory Scanning](05-troubleshooting/security/apache-dirscan-fail2ban-jail.md) — blocking directory scanners and junk HTTP methods
- [ClamAV Safe Scheduling on Live Servers](05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md) — preventing clamscan CPU spikes with nice and ionice
- [Systemd Session Scope Fails at Login](05-troubleshooting/systemd/session-scope-failure-at-login.md) — fixing session-cN.scope failures during login
- [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) — fixing broken downloads caused by unquoted URLs with &, ?, # characters
- [Ansible: Check Mode False Positives in Verify/Assert Tasks](05-troubleshooting/ansible-check-mode-false-positives.md) — guarding verify/assert tasks with `when: not ansible_check_mode` to prevent false failures in dry runs
- [Ansible Fails with Permission Denied While `ssh <alias>` Works (Host Alias Bypass)](05-troubleshooting/ansible-ssh-host-alias-bypass.md) — SSH Host blocks match on literal pattern; `ansible_host: <IP>` bypasses the alias and the IdentityFile never gets applied
- [Ghost EmailAnalytics Lag Warning — What It Means and When to Worry](05-troubleshooting/ghost-emailanalytics-lag-warning.md) — explaining the lag counter, `submitted` status, and `fetchMissing end == begin` skip
- [claude-mem: --setting-sources Empty Arg Bug (Claude Code 2.1.x)](05-troubleshooting/claude-mem-setting-sources-empty-arg.md) — fixing silent pipeline failure when claude-mem 12.1.x spawns Claude Code 2.1.112+
- [Ansible Check Mode False Positives in Verify/Assert Tasks](05-troubleshooting/ansible-check-mode-false-positives.md)
- [Ansible Fails with Permission Denied While `ssh <alias>` Works (Host Alias Bypass)](05-troubleshooting/ansible-ssh-host-alias-bypass.md)
- [Ansible SSH Timeout During dnf upgrade on Fedora Hosts](05-troubleshooting/ansible-ssh-timeout-dnf-upgrade.md)
- [Ansible: Vault Password File Not Found](05-troubleshooting/ansible-vault-password-file-missing.md)
- [Ansible Ignores ansible.cfg on WSL2 Windows Mounts](05-troubleshooting/ansible-wsl2-world-writable-mount-ignores-cfg.md)
- [claude-mem Silently Fails with Claude Code 2.1+ (Empty --setting-sources)](05-troubleshooting/claude-mem-setting-sources-empty-arg.md)
- [Cron Heartbeat False Alarm: /var/run Cleared by Reboot](05-troubleshooting/cron-heartbeat-tmpfs-reboot-false-alarm.md)
- [Docker & Caddy Recovery After Reboot (Fedora + SELinux)](05-troubleshooting/docker-caddy-selinux-post-reboot-recovery.md)
- [Fantastical Google Sync Error Flood — Phantom Calendars Fixed via syncselect](05-troubleshooting/fantastical-google-phantom-calendar-syncselect.md)
- [Fantastical MCP Server: Permission Denied on Launch (macOS Quarantine)](05-troubleshooting/fantastical-mcp-permission-denied.md)
- [Fedora Networking & Kernel Troubleshooting](05-troubleshooting/fedora-networking-kernel-recovery.md)
- [Gemini CLI: Manual Update Guide](05-troubleshooting/gemini-cli-manual-update.md)
- [Ghost EmailAnalytics Lag Warning — What It Means and When to Worry](05-troubleshooting/ghost-emailanalytics-lag-warning.md)
- [Gitea Actions Runner: Boot Race Condition Fix](05-troubleshooting/gitea-runner-boot-race-network-target.md)
- [ISP SNI Filtering & Caddy Troubleshooting](05-troubleshooting/isp-sni-filtering-caddy.md)
- [macOS Repeating Alert Tone from Mirrored iPhone Notification](05-troubleshooting/macos-mirrored-notification-alert-loop.md)
- [MajorWiki Setup & Publishing Pipeline](05-troubleshooting/majwiki-setup-and-pipeline.md)
- [Obsidian Vault Recovery — Loading Cache Hang](05-troubleshooting/obsidian-cache-hang-recovery.md)
- [Ollama: `ollama run` with Piped Stdin Bypasses Chat Template + SYSTEM Prompt](05-troubleshooting/ollama-chat-template-pipe-stdin-bypass.md)
- [Ollama Drops Off Tailscale When Mac Sleeps](05-troubleshooting/ollama-macos-sleep-tailscale-disconnect.md)
- [Python smtplib: Missing Date/Message-ID Headers Break Mail Clients](05-troubleshooting/python-smtplib-missing-rfc-headers.md)
- [SELinux: Fixing Dovecot Mail Spool Context (/var/vmail)](05-troubleshooting/selinux-dovecot-vmail-context.md)
- [Ubuntu dist-upgrade Quarantines Third-Party Repos](05-troubleshooting/ubuntu-dist-upgrade-repo-quarantine.md)
- [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md)
- [yt-dlp YouTube JS Challenge Fix (Fedora)](05-troubleshooting/yt-dlp-fedora-js-challenge.md)
### Docker & Containers
- [Nextcloud AIO Container Unhealthy for 20 Hours After Nightly Update](05-troubleshooting/docker/nextcloud-aio-unhealthy-20h-stuck.md)
- [n8n Behind Reverse Proxy: X-Forwarded-For Trust Fix](05-troubleshooting/docker/n8n-proxy-trust-x-forwarded-for.md)
### GPU & Display
- [LoRA adapter — GGUF conversion fails with 'config.json not found](05-troubleshooting/gpu-display/lora-adapter-gguf-conversion-fails.md)
- [Qwen2.5-14B OOM on RTX 3080 Ti (12GB)](05-troubleshooting/gpu-display/qwen-14b-oom-3080ti.md)
### Networking
- [Apache Outage: Fail2ban Self-Ban + Missing iptables Rules](05-troubleshooting/networking/fail2ban-self-ban-apache-outage.md)
- [Fail2ban & UFW Rule Bloat: 30k Rules Slowing Down a VPS](05-troubleshooting/networking/fail2ban-ufw-rule-bloat-cleanup.md)
- [Mail Client Stops Receiving: Fail2ban IMAP Self-Ban](05-troubleshooting/networking/fail2ban-imap-self-ban-mail-client.md)
- [Pi-hole AI Blocklist Blocks Claude Desktop (ERR_CONNECTION_REFUSED)](05-troubleshooting/networking/pihole-blocks-claude-desktop.md)
- [Tailscale SSH: Unexpected Re-Authentication Prompt](05-troubleshooting/networking/tailscale-ssh-reauth-prompt.md)
- [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md)
- [Windows OpenSSH: WSL as Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md)
- [firewalld: Mail Ports Wiped After Reload (IMAP + Webmail Outage)](05-troubleshooting/networking/firewalld-mail-ports-reset.md)
- [iOS Tailscale Clients Report HostName="localhost" — Breaks /etc/hosts Generators](05-troubleshooting/networking/tailscale-status-json-hostname-localhost-ios.md)
- [rsync over Tailscale: Hung in TCP Teardown After Transfer Completes](05-troubleshooting/networking/rsync-tailscale-teardown-stall.md)
### Security
- [ClamAV Safe Scheduling on Live Servers](05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md)
- [Custom Fail2ban Jail: Apache Directory Scanning & Junk Methods](05-troubleshooting/security/apache-dirscan-fail2ban-jail.md)
### Storage
- [mdadm RAID Recovery After USB Hub Disconnect](05-troubleshooting/storage/mdadm-usb-hub-disconnect-recovery.md)
### Systemd
- [Systemd Session Scope Fails at Login (session-cN.scope)](05-troubleshooting/systemd/session-scope-failure-at-login.md)
---
@ -182,57 +213,36 @@ updated: 2026-04-29T22:45
| Date | Article | Domain |
|---|---|---|
| 2026-04-19 | [Wake-on-LAN via Router SSH](02-selfhosting/dns-networking/wake-on-lan-router-ssh.md) | Self-Hosting |
| 2026-04-18 | [Ghost Email Configuration with Mailgun](02-selfhosting/services/ghost-smtp-mailgun-setup.md) | Self-Hosting |
| 2026-04-18 | [Firewall Hardening with firewalld on Fedora Fleet](02-selfhosting/security/firewalld-fleet-hardening.md) | Self-Hosting |
| 2026-04-18 | [ClamAV Fleet Deployment with Ansible](02-selfhosting/security/clamav-fleet-deployment.md) | Self-Hosting |
| 2026-04-18 | [Ansible: Check Mode False Positives in Verify/Assert Tasks](05-troubleshooting/ansible-check-mode-false-positives.md) | Troubleshooting |
| 2026-04-18 | [Ghost EmailAnalytics Lag Warning](05-troubleshooting/ghost-emailanalytics-lag-warning.md) | Troubleshooting |
| 2026-04-17 | [Watchtower SMTP via Localhost Postfix Relay](02-selfhosting/docker/watchtower-smtp-localhost-relay.md) | Self-Hosting |
| 2026-04-17 | [Fail2ban Custom Jail: Nginx Bad Request Detection](02-selfhosting/security/fail2ban-nginx-bad-request-jail.md) | Self-Hosting |
| 2026-04-17 | [Fail2ban Custom Jail: Apache Bad Request Detection](02-selfhosting/security/fail2ban-apache-bad-request-jail.md) | Self-Hosting |
| 2026-04-17 | [SSH Hardening Fleet-Wide with Ansible](02-selfhosting/security/ssh-hardening-ansible-fleet.md) | Self-Hosting |
| 2026-04-17 | [claude-mem: --setting-sources Empty Arg Bug](05-troubleshooting/claude-mem-setting-sources-empty-arg.md) | Troubleshooting |
| 2026-04-13 | [Cron Heartbeat False Alarm: /var/run Cleared by Reboot](05-troubleshooting/cron-heartbeat-tmpfs-reboot-false-alarm.md) | Troubleshooting |
| 2026-04-09 | [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md) | Self-Hosting |
| 2026-04-08 | [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) | Troubleshooting |
| 2026-04-07 | [SSH Config & Key Management](01-linux/networking/ssh-config-key-management.md) | Linux |
| 2026-04-07 | [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) | Troubleshooting |
| 2026-04-07 | [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md) | Troubleshooting |
| 2026-04-03 | [Ansible: ansible.cfg Ignored on WSL2 Windows Mounts](05-troubleshooting/ansible-wsl2-world-writable-mount-ignores-cfg.md) | Troubleshooting |
| 2026-04-02 | [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md) | Self-Hosting |
| 2026-04-02 | [Mastodon Instance Tuning](02-selfhosting/services/mastodon-instance-tuning.md) | Self-Hosting |
| 2026-04-02 | [mdadm — Rebuilding a RAID Array After Reinstall](01-linux/storage/mdadm-raid-rebuild.md) | Linux |
| 2026-04-02 | [Fedora Networking & Kernel Troubleshooting](05-troubleshooting/fedora-networking-kernel-recovery.md) | Troubleshooting |
| 2026-04-02 | [Ventoy: Multi-Boot USB Tool](03-opensource/dev-tools/ventoy.md) | Open Source |
| 2026-04-02 | [rsync Backup Patterns](02-selfhosting/storage-backup/rsync-backup-patterns.md) (updated — Glacier Deep Archive) | Self-Hosting |
| 2026-04-02 | [yt-dlp: Video Downloading](03-opensource/media-creative/yt-dlp.md) (updated — subtitles, temp fix) | Open Source |
| 2026-04-02 | [OBS Studio Setup & Encoding](04-streaming/obs/obs-studio-setup-encoding.md) (updated — captions plugin, VLC capture) | Streaming |
| 2026-04-02 | [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md) (updated — SpamAssassin) | Self-Hosting |
| 2026-03-23 | [Ansible: Vault Password File Not Found](05-troubleshooting/ansible-vault-password-file-missing.md) | Troubleshooting |
| 2026-03-18 | [Deploying Netdata to a New Server](02-selfhosting/monitoring/netdata-new-server-setup.md) | Self-Hosting |
| 2026-03-18 | [Tuning Netdata Docker Health Alarms](02-selfhosting/monitoring/netdata-docker-health-alarm-tuning.md) | Self-Hosting |
| 2026-03-17 | [Ollama Drops Off Tailscale When Mac Sleeps](05-troubleshooting/ollama-macos-sleep-tailscale-disconnect.md) | Troubleshooting |
| 2026-03-17 | [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md) | Troubleshooting |
| 2026-03-16 | [Standardizing unattended-upgrades with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md) | Self-Hosting |
| 2026-03-16 | [WSL2 Training Environment Rebuild (Fedora 43)](01-linux/distro-specific/wsl2-rebuild-fedora43-training-env.md) | Linux |
| 2026-03-16 | [WSL2 Backup via PowerShell Scheduled Task](01-linux/distro-specific/wsl2-backup-powershell.md) | Linux |
| 2026-03-15 | [firewalld: Mail Ports Wiped After Reload](05-troubleshooting/networking/firewalld-mail-ports-reset.md) | Troubleshooting |
| 2026-03-15 | [Plex 4K Codec Compatibility (Apple TV)](04-streaming/plex/plex-4k-codec-compatibility.md) | Streaming |
| 2026-03-15 | [mdadm RAID Recovery After USB Hub Disconnect](05-troubleshooting/storage/mdadm-usb-hub-disconnect-recovery.md) | Troubleshooting |
| 2026-03-15 | [yt-dlp: Video Downloading](03-opensource/media-creative/yt-dlp.md) | Open Source |
| 2026-03-14 | [SELinux: Fixing Dovecot Mail Spool Context (/var/vmail)](05-troubleshooting/selinux-dovecot-vmail-context.md) | Troubleshooting |
| 2026-03-14 | [Gitea Actions Runner: Boot Race Condition Fix](05-troubleshooting/gitea-runner-boot-race-network-target.md) | Troubleshooting |
| 2026-03-14 | [Mail Client Stops Receiving: Fail2ban IMAP Self-Ban](05-troubleshooting/networking/fail2ban-imap-self-ban-mail-client.md) | Troubleshooting |
| 2026-03-14 | [SearXNG: Private Self-Hosted Search](03-opensource/alternatives/searxng.md) | Open Source |
| 2026-03-14 | [FreshRSS: Self-Hosted RSS Reader](03-opensource/alternatives/freshrss.md) | Open Source |
| 2026-03-14 | [Gitea: Self-Hosted Git](03-opensource/alternatives/gitea.md) | Open Source |
| 2026-03-14 | [yt-dlp: Video Downloading](03-opensource/media-creative/yt-dlp.md) | Open Source |
| 2026-03-13 | [Vaultwarden: Self-Hosted Password Manager](03-opensource/privacy-security/vaultwarden.md) | Open Source |
| 2026-03-13 | [Gemini CLI Manual Update](05-troubleshooting/gemini-cli-manual-update.md) | Troubleshooting |
| 2026-03-13 | [rmlint: Duplicate File Scanning](03-opensource/productivity/rmlint-duplicate-scanning.md) | Open Source |
| 2026-03-13 | [SnapRAID & MergerFS Storage Setup](01-linux/storage/snapraid-mergerfs-setup.md) | Linux |
| 2026-03-13 | [Qwen2.5-14B OOM on RTX 3080 Ti (12GB)](05-troubleshooting/gpu-display/qwen-14b-oom-3080ti.md) | Troubleshooting |
| 2026-05-02 | [WSL2 Backup via PowerShell Scheduled Task](01-linux/distro-specific/wsl2-backup-powershell.md) | Linux |
| 2026-05-02 | [SSH Config and Key Management](01-linux/networking/ssh-config-key-management.md) | Linux |
| 2026-05-02 | [Wake-on-LAN via Router SSH](02-selfhosting/dns-networking/wake-on-lan-router-ssh.md) | Self-Hosting |
| 2026-05-02 | [Tuning Netdata Docker Health Alarms to Prevent Update Flapping](02-selfhosting/monitoring/netdata-docker-health-alarm-tuning.md) | Self-Hosting |
| 2026-05-02 | [ClamAV Fleet Deployment with Ansible](02-selfhosting/security/clamav-fleet-deployment.md) | Self-Hosting |
| 2026-05-02 | [Fail2Ban Digest Mode — Fleet-Wide Quiet Alerts](02-selfhosting/security/fail2ban-digest-mode-fleet.md) | Self-Hosting |
| 2026-05-02 | [Mastodon Instance Tuning](02-selfhosting/services/mastodon-instance-tuning.md) | Self-Hosting |
| 2026-05-02 | [Ansible Check Mode False Positives in Verify/Assert Tasks](05-troubleshooting/ansible-check-mode-false-positives.md) | Troubleshooting |
| 2026-05-02 | [ISP SNI Filtering & Caddy Troubleshooting](05-troubleshooting/isp-sni-filtering-caddy.md) | Troubleshooting |
| 2026-05-02 | [Windows OpenSSH: WSL as Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) | Troubleshooting |
| 2026-05-02 | [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md) | Troubleshooting |
| 2026-05-02 | [yt-dlp YouTube JS Challenge Fix (Fedora)](05-troubleshooting/yt-dlp-fedora-js-challenge.md) | Troubleshooting |
| 2026-04-30 | [wp-fail2ban Plugin Logpath on Debian/Ubuntu (auth.log, not syslog)](02-selfhosting/security/wp-fail2ban-logpath-debian-ubuntu.md) | Self-Hosting |
| 2026-04-30 | [LoRA adapter — GGUF conversion fails with 'config.json not found](05-troubleshooting/gpu-display/lora-adapter-gguf-conversion-fails.md) | Troubleshooting |
| 2026-04-29 | [iOS Tailscale Clients Report HostName="localhost" — Breaks /etc/hosts Generators](05-troubleshooting/networking/tailscale-status-json-hostname-localhost-ios.md) | Troubleshooting |
| 2026-04-29 | [Python smtplib: Missing Date/Message-ID Headers Break Mail Clients](05-troubleshooting/python-smtplib-missing-rfc-headers.md) | Troubleshooting |
| 2026-04-28 | [Ubuntu dist-upgrade Quarantines Third-Party Repos](05-troubleshooting/ubuntu-dist-upgrade-repo-quarantine.md) | Troubleshooting |
| 2026-04-26 | [Fantastical MCP Server: Permission Denied on Launch (macOS Quarantine)](05-troubleshooting/fantastical-mcp-permission-denied.md) | Troubleshooting |
| 2026-04-25 | [rsync over Tailscale: Hung in TCP Teardown After Transfer Completes](05-troubleshooting/networking/rsync-tailscale-teardown-stall.md) | Troubleshooting |
| 2026-04-25 | [Ollama: `ollama run` with Piped Stdin Bypasses Chat Template + SYSTEM Prompt](05-troubleshooting/ollama-chat-template-pipe-stdin-bypass.md) | Troubleshooting |
| 2026-04-24 | [Fantastical Google Sync Error Flood — Phantom Calendars Fixed via syncselect](05-troubleshooting/fantastical-google-phantom-calendar-syncselect.md) | Troubleshooting |
| 2026-04-23 | [Pi-hole DoH / DoT Bypass Defense](02-selfhosting/dns-networking/pihole-doh-dot-bypass-defense.md) | Self-Hosting |
| 2026-04-22 | [Pi-hole v6 Adlist Management via SQL](02-selfhosting/dns-networking/pihole-v6-adlist-management.md) | Self-Hosting |
| 2026-04-22 | [Pi-hole v6 Group Management: Per-Client DNS Rules](02-selfhosting/dns-networking/pihole-v6-group-management.md) | Self-Hosting |
| 2026-04-22 | [Mastodon DB Maintenance — Statuses, Accounts, and VACUUM](02-selfhosting/services/mastodon-db-maintenance.md) | Self-Hosting |
| 2026-04-22 | [Mastodon Federation — Domain Blocks, Silencing, and FediSeer](02-selfhosting/services/mastodon-federation.md) | Self-Hosting |
| 2026-04-22 | [Pi-hole AI Blocklist Blocks Claude Desktop (ERR_CONNECTION_REFUSED)](05-troubleshooting/networking/pihole-blocks-claude-desktop.md) | Troubleshooting |
| 2026-04-21 | [Ansible Fails with Permission Denied While `ssh <alias>` Works (Host Alias Bypass)](05-troubleshooting/ansible-ssh-host-alias-bypass.md) | Troubleshooting |
| 2026-04-20 | [Claude Code Remote Control — Mobile Access to a Persistent Host Session](02-selfhosting/services/claude-code-remote-control.md) | Self-Hosting |
| 2026-04-19 | [AWS S3 Cost Management](02-selfhosting/cloud/aws-s3-cost-management.md) | Self-Hosting |
---