No description
MajorLinux
155651c373
Two related additions covering the 2026-05-31 cutover-night incidents on majorlinux and majortoot-hetzner. ssh-socket-tailscale-race-condition.md (update Race 1 fix): - After=tailscaled.service Requires=tailscaled.service orders against the service becoming active, not against tailscale0 having an IPv4 — hosts kept losing SSH intermittently after reboots (incident: majorlinux + majortoot-hetzner 2026-05-31, during cutover-night Ansible reboot). - Canonical fix: a oneshot tailscale-wait-ready.service that polls `ip -4 -o addr show tailscale0` until an address is present, with ssh.socket After=/Requires= that service. Document the full evolution (2026-05-19 BindsTo → 2026-05-23 Requires → 2026-05-31 wait-ready) so future readers don't try the half-fixes thinking they're sufficient. - Add majortoot-hetzner to affected hosts. mastodon-post-install-hardening.md (new): Four upstream-install gaps that bit during the majortoot-hetzner cutover: 1. /home/mastodon at 0750 (useradd default) → nginx www-data can't traverse → every static asset 403s → unstyled "purple screen" in the browser while API/HTML still work through the puma proxy. 2. .env.production at 0644 (mastodon-setup default) → DB_PASS, SECRET_KEY_BASE, OTP_SECRET world-readable once gap (1) is fixed. 3. mastodon user shell at /usr/sbin/nologin → `su - mastodon` blocked. 4. rbenv init in .bashrc only → login shells don't source .bashrc; even when chained, Ubuntu's .bashrc returns early for non-interactive shells. Fix: .bash_profile sets up rbenv BEFORE sourcing .profile + .bashrc, so it works for both interactive and non-interactive logins. All four codified in MajorAnsible configure_mastodon_permissions.yml with self-asserting verification steps. 02-selfhosting/index.md + SUMMARY.md: Add a "Services" section to the selfhosting index linking the mastodon-post-install-hardening article (and the other orphaned services/ entries while there). SUMMARY.md gains one new entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .githooks | ||
| 01-linux | ||
| 02-selfhosting | ||
| 03-opensource | ||
| 04-streaming | ||
| 05-troubleshooting | ||
| .gitattributes | ||
| index.md | ||
| MajorWiki-Deploy-Status.md | ||
| README.md | ||
| SUMMARY.md | ||
| created | updated |
|---|---|
| 2026-04-06T09:52 | 2026-04-30T05:21 |
MajorLinux Tech Wiki — Index
A growing reference of Linux, self-hosting, open source, streaming, and troubleshooting guides. Written by MajorLinux. Used by MajorTwin.
Last updated: 2026-04-14 Article count: 76
Domains
| Domain | Folder | Articles |
|---|---|---|
| 🐧 Linux & Sysadmin | 01-linux/ |
12 |
| 🏠 Self-Hosting & Homelab | 02-selfhosting/ |
22 |
| 🔓 Open Source Tools | 03-opensource/ |
10 |
| 🎙️ Streaming & Podcasting | 04-streaming/ |
2 |
| 🔧 General Troubleshooting | 05-troubleshooting/ |
30 |
🐧 Linux & Sysadmin
Files & Permissions
- Linux File Permissions — chmod, chown, special bits, finding permission problems
Process Management
- Managing Linux Services with systemd — systemctl, journalctl, writing service files, Ansible service management
Networking
- SSH Config & Key Management — key generation, ssh-copy-id, ~/.ssh/config, managing multiple keys, Windows OpenSSH admin key auth
Package Management
- Package Management Reference — apt, dnf, pacman side-by-side reference, Flatpak/Snap
Shell & Scripting
- Ansible Getting Started — inventory, ad-hoc commands, playbooks, handlers, roles
- Bash Scripting Patterns — set -euo pipefail, logging, error handling, argument parsing, common patterns
Storage
- SnapRAID & MergerFS Storage Setup — Pooling mismatched drives and adding parity on Linux
- mdadm — Rebuilding a RAID Array After Reinstall — reassembling and recovering mdadm arrays after OS reinstall
Distro-Specific
- Linux Distro Guide for Beginners — Ubuntu recommendation, distro comparison, desktop environments
- WSL2 Instance Migration to Fedora 43 — moving WSL2 VHDX from C: to another drive
- WSL2 Training Environment Rebuild (Fedora 43) — rebuilding the MajorTwin training env in WSL2 from scratch
- WSL2 Backup via PowerShell Scheduled Task — automating WSL2 exports on a schedule using PowerShell
🏠 Self-Hosting & Homelab
Docker & Containers
- Self-Hosting Starter Guide — hardware options, Docker install, first services, networking basics
- Docker vs VMs for the Homelab — when to use containers vs VMs, KVM setup, how to run both
- Debugging Broken Docker Containers — logs, inspect, exec, port conflicts, permission errors
- Docker Healthchecks — writing and debugging HEALTHCHECK instructions in Docker containers
Reverse Proxies
- Setting Up Caddy as a Reverse Proxy — Caddyfile basics, automatic HTTPS, local TLS, DNS challenge
DNS & Networking
- Tailscale for Homelab Remote Access — installation, MagicDNS, making services accessible, subnet router, ACLs
- Network Overview — MajorsHouse network topology, Tailscale IPs, and connectivity map
Storage & Backup
- rsync Backup Patterns — flags reference, remote backup, incremental with hard links, Glacier Deep Archive
Monitoring
- Tuning Netdata Web Log Alerts — tuning web_log_1m_redirects threshold for HTTPS-forcing servers
- Tuning Netdata Docker Health Alarms — preventing false alerts during nightly Nextcloud AIO container update cycles
- Deploying Netdata to a New Server — install, email notifications, and Netdata Cloud claim for Ubuntu/Debian servers
- Netdata + n8n Enriched Alert Emails — rich HTML alert emails with remediation steps and wiki links via n8n
- Netdata SELinux AVC Denial Monitoring — custom Netdata chart for tracking SELinux AVC denials
Security
- Linux Server Hardening Checklist — non-root user, SSH key auth, sshd_config, firewall, fail2ban, SpamAssassin
- Standardizing unattended-upgrades with Ansible — fleet-wide automatic security updates across Ubuntu servers
- Fail2ban Custom Jail: Apache 404 Scanner Detection — custom filter and jail for blocking 404 scanners
- Fail2ban Custom Jail: Apache PHP Webshell Probe Detection — catching PHP webshell/backdoor probes that return 301 on HTTPS-redirecting servers
- Fail2ban Custom Jail: WordPress Login Brute Force — access-log-based wp-login.php brute force detection without plugins
- SELinux: Fixing Fail2ban grep execmem Denial — resolving execmem AVC denials from Fail2ban's grep on Fedora
- UFW Firewall Management — managing UFW rules, common patterns, troubleshooting
Services
- Updating n8n Running in Docker — pinned version updates, password reset, Arcane timing gaps
- Mastodon Instance Tuning — character limit increase, media cache management for self-hosted Mastodon
🔓 Open Source Tools
Alternatives
- SearXNG: Private Self-Hosted Search — metasearch engine that queries multiple engines without exposing your identity
- FreshRSS: Self-Hosted RSS Reader — algorithm-free feed aggregator with mobile app sync
- Gitea: Self-Hosted Git — lightweight GitHub alternative, webhooks, single Docker container
Productivity
- rmlint: Duplicate File Scanning — extremely fast duplicate file finding and storage reclamation
Development Tools
- tmux: Persistent Terminal Sessions — detachable sessions for long-running jobs over SSH
- screen: Simple Persistent Sessions — lightweight terminal multiplexer, universally available
- rsync: Fast, Resumable File Transfers — incremental file sync locally and over SSH, survives interruptions
- Ventoy: Multi-Boot USB Tool — drop ISOs on a USB drive and boot any of them, no reflashing
Privacy & Security
- Vaultwarden: Self-Hosted Password Manager — Bitwarden-compatible server in a single Docker container, passwords stay on your hardware
Media & Creative
- yt-dlp: Video Downloading — download from YouTube and hundreds of other sites, Plex-optimized format selection
🎙️ Streaming & Podcasting
OBS Studio
- OBS Studio Setup & Encoding — installation, NVENC/x264 settings, scene setup, audio filters, Linux Wayland notes
Plex
- Plex 4K Codec Compatibility (Apple TV) — AV1/VP9 vs HEVC, batch conversion script, yt-dlp auto-convert hook
🔧 General Troubleshooting
- Apache Outage: Fail2ban Self-Ban + Missing iptables Rules — diagnosing and fixing Apache outages caused by missing firewall rules and Fail2ban self-bans
- Mail Client Stops Receiving: Fail2ban IMAP Self-Ban — diagnosing why one device stops receiving email when the mail server is healthy
- firewalld: Mail Ports Wiped After Reload — recovering IMAP and webmail after firewalld reload drops all mail service rules
- Fail2ban & UFW Rule Bloat: 30k Rules Slowing Down a VPS — diagnosing and cleaning up massive nftables/UFW rule accumulation
- Tailscale SSH: Unexpected Re-Authentication Prompt — resolving unexpected re-auth prompts on Tailscale SSH connections
- Docker & Caddy Recovery After Reboot (Fedora + SELinux) — fixing docker.socket, SELinux port blocks, and httpd_can_network_connect after reboot
- n8n Behind Reverse Proxy: X-Forwarded-For Trust Fix — fixing webhook failures caused by missing proxy trust configuration
- Nextcloud AIO Container Unhealthy for 20 Hours — diagnosing stuck Nextcloud AIO containers after nightly update cycles
- ISP SNI Filtering with Caddy — troubleshooting why wiki.majorshouse.com was blocked by Google Fiber
- Obsidian Cache Hang Recovery — resolving "Loading cache" hang in Obsidian by cleaning Electron app data and ML artifacts
- macOS Repeating Alert Tone from Mirrored Notification — stopping alert tone loops from mirrored iPhone notifications on Mac
- Qwen2.5-14B OOM on RTX 3080 Ti (12GB) — fixes and alternatives when hitting VRAM limits during fine-tuning
- yt-dlp YouTube JS Challenge Fix on Fedora — fixing YouTube JS challenge solver errors and missing formats on Fedora
- Gemini CLI Manual Update — how to manually update the Gemini CLI when automatic updates fail
- MajorWiki Setup & Pipeline — setting up MajorWiki and the Obsidian → Gitea → MkDocs publishing pipeline
- Gitea Actions Runner: Boot Race Condition Fix — fixing act_runner crash loop on boot caused by DNS not ready at startup
- Cron Heartbeat False Alarm: /var/run Cleared by Reboot — why
/runis tmpfs and how a reboot wipes cron heartbeat files, and where to put them instead - SELinux: Fixing Dovecot Mail Spool Context (/var/vmail) — fixing thousands of AVC denials when /var/vmail has wrong SELinux context
- mdadm RAID Recovery After USB Hub Disconnect — diagnosing and recovering a failed mdadm array caused by a USB hub dropout
- Windows OpenSSH Server (sshd) Stops After Reboot — fixing sshd not running after reboot due to Manual startup type
- Ollama Drops Off Tailscale When Mac Sleeps — keeping Ollama reachable over Tailscale by disabling macOS sleep on AC power
- Ansible: Vault Password File Not Found — fixing the missing vault_pass file error when running ansible-playbook
- Ansible SSH Timeout During dnf upgrade — preventing SSH timeouts during long-running dnf upgrades on Fedora
- Fedora Networking & Kernel Troubleshooting — nmcli quick fix, GRUB kernel rollback, and recovery for Fedora fleet
- Custom Fail2ban Jail: Apache Directory Scanning — blocking directory scanners and junk HTTP methods
- ClamAV Safe Scheduling on Live Servers — preventing clamscan CPU spikes with nice and ionice
- Systemd Session Scope Fails at Login — fixing session-cN.scope failures during login
- wget/curl: URLs with Special Characters Fail in Bash — fixing broken downloads caused by unquoted URLs with &, ?, # characters
Recently Updated
Writing Backlog
| Topic | Domain | Priority | From Gap? |
|---|---|---|---|
| Docker Compose networking deep dive | Self-Hosting | High | No |
| Troubleshooting NVIDIA on Linux | Troubleshooting | Medium | No |
| Pi-hole setup and local DNS | Self-Hosting | Medium | No |
| OBS audio routing on Linux (PipeWire) | Streaming | Medium | No |
| Nextcloud setup with Docker | Self-Hosting | Medium | No |
Related
- MajorWiki Deploy Status — deployment status and update workflow