majorlinux/MajorWiki
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

wiki: add WSL2 backup, Fedora43 training env, Ansible upgrades, firewalld mail ports articles; update indexes

Browse Source 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
MajorLinux
2026-03-16 16:40:18 -04:00
parent 6a4681dc4b
commit e8598cfac8
5 changed files with 401 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

86
01-linux/distro-specific/wsl2-backup-powershell.md Normal file
View File

@@ -0,0 +1,86 @@
---
title: WSL2 Backup via PowerShell Scheduled Task
domain: linux
category: distro-specific
tags:
- wsl2
- windows
- backup
- powershell
- majorrig
status: published
created: '2026-03-16'
updated: '2026-03-16'
---
# WSL2 Backup via PowerShell Scheduled Task
WSL2 distributions are stored as a VHDX file on disk. Unlike traditional VMs, there's no built-in snapshot or backup mechanism. This article covers a simple weekly backup strategy using `wsl --export` and a PowerShell scheduled task.
## The Short Answer
Save this as `C:\Users\majli\Scripts\backup-wsl.ps1` and register it as a weekly scheduled task.
## Backup Script
```powershell
$BackupDir = "D:\WSL\Backups"
$Date = Get-Date -Format "yyyy-MM-dd"
$BackupFile = "$BackupDir\FedoraLinux-43-$Date.tar"
$MaxBackups = 3
New-Item -ItemType Directory -Force -Path $BackupDir | Out-Null
# Must shut down WSL first — export fails if VHDX is locked
Write-Host "Shutting down WSL2..."
wsl --shutdown
Start-Sleep -Seconds 5
Write-Host "Backing up FedoraLinux-43 to $BackupFile..."
wsl --export FedoraLinux-43 $BackupFile
if ($LASTEXITCODE -eq 0) {
Write-Host "Backup complete: $BackupFile"
Get-ChildItem "$BackupDir\FedoraLinux-43-*.tar" |
Sort-Object LastWriteTime -Descending |
Select-Object -Skip $MaxBackups |
Remove-Item -Force
Write-Host "Cleanup done. Keeping last $MaxBackups backups."
} else {
Write-Host "ERROR: Backup failed!"
}
```
## Register the Scheduled Task
Run in PowerShell as Administrator:
```powershell
$Action = New-ScheduledTaskAction -Execute "PowerShell.exe" `
-Argument "-NonInteractive -File C:\Users\majli\Scripts\backup-wsl.ps1"
$Trigger = New-ScheduledTaskTrigger -Weekly -DaysOfWeek Sunday -At 2am
$Settings = New-ScheduledTaskSettingsSet -StartWhenAvailable -RunOnlyIfNetworkAvailable:$false
Register-ScheduledTask -TaskName "WSL2 Backup - FedoraLinux43" `
-Action $Action -Trigger $Trigger -Settings $Settings `
-RunLevel Highest -Force
```
## Restore from Backup
```powershell
wsl --unregister FedoraLinux-43
wsl --import FedoraLinux-43 D:\WSL\Fedora43 D:\WSL\Backups\FedoraLinux-43-YYYY-MM-DD.tar
```
Then fix the default user — after import WSL resets to root. See [[wsl2-instance-migration-fedora43|WSL2 Instance Migration]] for the `/etc/wsl.conf` fix.
## Gotchas
- **`wsl --export` fails with `ERROR_SHARING_VIOLATION` if WSL is running.** The script includes `wsl --shutdown` before export to handle this. Any active WSL sessions will be terminated — schedule the task for a time when WSL is idle (2am works well).
- **Backblaze picks up D:\WSL\Backups\ automatically** if D: drive is in scope — provides offsite backup without extra config.
- **Each backup tar is ~500MB–1GB** depending on what's installed. Keep MaxBackups at 3 to balance retention vs disk usage.
## See Also
- [[wsl2-instance-migration-fedora43|WSL2 Instance Migration]]
- [[wsl2-rebuild-fedora43-training-env|WSL2 Training Environment Rebuild]]

203
01-linux/distro-specific/wsl2-rebuild-fedora43-training-env.md Normal file
View File

@@ -0,0 +1,203 @@
---
title: WSL2 Fedora 43 Training Environment Rebuild
domain: linux
category: distro-specific
tags:
- wsl2
- fedora
- unsloth
- pytorch
- cuda
- majorrig
- majortwin
status: published
created: '2026-03-16'
updated: '2026-03-16'
---
# WSL2 Fedora 43 Training Environment Rebuild
How to rebuild the MajorTwin training environment from scratch on MajorRig after a WSL2 loss. Covers Fedora 43 install, Python 3.11 via pyenv, PyTorch with CUDA, Unsloth, and llama.cpp for GGUF conversion.
## The Short Answer
```bash
# 1. Install Fedora 43 and move to D:
wsl --install -d FedoraLinux-43 --no-launch
wsl --export FedoraLinux-43 D:\WSL\fedora43.tar
wsl --unregister FedoraLinux-43
wsl --import FedoraLinux-43 D:\WSL\Fedora43 D:\WSL\fedora43.tar
# 2. Set default user
echo -e "[boot]\nsystemd=true\n[user]\ndefault=majorlinux" | sudo tee /etc/wsl.conf
useradd -m -G wheel majorlinux && passwd majorlinux
echo "%wheel ALL=(ALL) ALL" | sudo tee /etc/sudoers.d/wheel
# 3. Install Python 3.11 via pyenv, PyTorch, Unsloth
# See full steps below
```
## Step 1 — System Packages
```bash
sudo dnf update -y
sudo dnf install -y git curl wget tmux screen htop rsync unzip \
python3 python3-pip python3-devel gcc gcc-c++ make cmake \
ninja-build pkg-config openssl-devel libffi-devel \
gawk patch readline-devel sqlite-devel
```
## Step 2 — Python 3.11 via pyenv
Fedora 43 ships Python 3.13. Unsloth requires 3.11. Use pyenv:
```bash
curl https://pyenv.run | bash
# Add to ~/.bashrc
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init - bash)"
source ~/.bashrc
pyenv install 3.11.9
pyenv global 3.11.9
```
The tkinter warning during install is harmless — it's not needed for training.
## Step 3 — Training Virtualenv + PyTorch
```bash
mkdir -p ~/majortwin/{staging,datasets,outputs,scripts}
python -m venv ~/majortwin/venv
source ~/majortwin/venv/bin/activate
pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
# Verify GPU
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
```
Expected output: `True NVIDIA GeForce RTX 3080 Ti`
## Step 4 — Unsloth + Training Stack
```bash
source ~/majortwin/venv/bin/activate
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install transformers datasets accelerate peft trl bitsandbytes \
sentencepiece protobuf scipy einops
# Pin transformers for unsloth-zoo compatibility
pip install "transformers<=5.2.0"
# Verify
python -c "import unsloth; print('Unsloth OK')"
```
> [!warning] Never run `pip install -r requirements.txt` from inside llama.cpp while the training venv is active. It installs CPU-only PyTorch and downgrades transformers, breaking the CUDA setup.
## Step 5 — llama.cpp (CPU-only for GGUF conversion)
CUDA 12.8 is incompatible with Fedora 43's glibc for compiling llama.cpp (math function conflicts in `/usr/include/bits/mathcalls.h`). Build CPU-only — it's sufficient for GGUF conversion, which doesn't need GPU:
```bash
# Install GCC 14 (CUDA 12.8 doesn't support GCC 15 which Fedora 43 ships)
sudo dnf install -y gcc14 gcc14-c++
cd ~/majortwin
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build \
-DGGML_CUDA=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=/usr/bin/gcc-14 \
-DCMAKE_CXX_COMPILER=/usr/bin/g++-14
cmake --build build --config Release -j$(nproc) 2>&1 | tee /tmp/llama_build.log &
tail -f /tmp/llama_build.log
```
Verify:
```bash
ls ~/majortwin/llama.cpp/build/bin/llama-quantize && echo "OK"
ls ~/majortwin/llama.cpp/build/bin/llama-cli && echo "OK"
```
## Step 6 — Shell Environment
```bash
cat >> ~/.bashrc << 'EOF'
# MajorInfrastructure Paths
export VAULT="/mnt/c/Users/majli/Documents/MajorVault"
export MAJORANSIBLE="/mnt/d/MajorAnsible"
export MAJORTWIN_D="/mnt/d/MajorTwin"
export MAJORTWIN_WSL="$HOME/majortwin"
export LLAMA_CPP="$HOME/majortwin/llama.cpp"
# Venv
alias mtwin='source $MAJORTWIN_WSL/venv/bin/activate && cd $MAJORTWIN_WSL'
alias vault='cd $VAULT'
alias ll='ls -lah --color=auto'
# SSH Fleet Aliases
alias majorhome='ssh majorlinux@100.120.209.106'
alias dca='ssh root@100.104.11.146'
alias majortoot='ssh root@100.110.197.17'
alias majorlinuxvm='ssh root@100.87.200.5'
alias majordiscord='ssh root@100.122.240.83'
alias majorlab='ssh root@100.86.14.126'
alias majormail='ssh root@100.84.165.52'
alias teelia='ssh root@100.120.32.69'
alias tttpod='ssh root@100.84.42.102'
alias majorrig='ssh -p 2222 majorlinux@100.98.47.29'
# DNF5
alias update='sudo dnf upgrade --refresh'
alias install='sudo dnf install'
alias clean='sudo dnf clean all'
# MajorTwin helpers
stage_dataset() {
cp "$VAULT/20-Projects/MajorTwin/03-Datasets/$1" "$MAJORTWIN_WSL/datasets/"
echo "Staged: $1"
}
export_gguf() {
cp "$MAJORTWIN_WSL/outputs/$1" "$MAJORTWIN_D/models/"
echo "Exported: $1 → $MAJORTWIN_D/models/"
}
EOF
source ~/.bashrc
```
## Key Rules
- **Always activate venv before pip installs:** `source ~/majortwin/venv/bin/activate`
- **Never train from /mnt/c or /mnt/d** — stage files in `~/majortwin/staging/` first
- **Never put ML artifacts inside MajorVault** — models, venvs, artifacts go on D: drive
- **Max viable training model:** 7B at QLoRA 4-bit (RTX 3080 Ti, 12GB VRAM)
- **Current base model:** Qwen2.5-7B-Instruct (ChatML format — stop token: `<|im_end|>` only)
- **Transformers must be pinned:** `pip install "transformers<=5.2.0"` for unsloth-zoo compatibility
## D: Drive Layout
```
D:\MajorTwin\
models\ ← finished GGUFs
datasets\ ← dataset archives
artifacts\ ← training run artifacts
training-runs\ ← logs, checkpoints
D:\WSL\
Fedora43\ ← WSL2 VHDX
Backups\ ← weekly WSL2 backup tars
```
## See Also
- [[wsl2-instance-migration-fedora43|WSL2 Instance Migration]]
- [[wsl2-backup-powershell|WSL2 Backup via PowerShell]]

94
02-selfhosting/security/ansible-unattended-upgrades-fleet.md Normal file
View File

@@ -0,0 +1,94 @@
---
title: Standardizing unattended-upgrades Across Ubuntu Fleet with Ansible
domain: selfhosting
category: security
tags:
- ansible
- ubuntu
- apt
- unattended-upgrades
- fleet-management
status: published
created: '2026-03-16'
updated: '2026-03-16'
---
# Standardizing unattended-upgrades Across Ubuntu Fleet with Ansible
When some Ubuntu hosts in a fleet self-update via `unattended-upgrades` and others don't, they drift apart over time — different kernel versions, different reboot states, inconsistent behavior. This article covers how to diagnose the drift and enforce uniform auto-update config across all Ubuntu hosts using Ansible.
## Diagnosing the Problem
If only some Ubuntu hosts are flagging for reboot, check:
```bash
# What triggered the reboot flag?
cat /var/run/reboot-required.pkgs
# Is unattended-upgrades installed and active?
systemctl status unattended-upgrades
cat /etc/apt/apt.conf.d/20auto-upgrades
# When did apt last run?
ls -lt /var/log/apt/history.log*
```
The reboot flag is written to `/var/run/reboot-required` by `update-notifier-common` when packages like the kernel, glibc, or systemd are updated. If some hosts have `unattended-upgrades` running and others don't, the ones that self-updated will flag for reboot while the others lag behind.
## The Fix — Ansible Playbook
Add these tasks to your update playbook **before** the apt cache update step:
```yaml
- name: Ensure unattended-upgrades is installed on Ubuntu servers
ansible.builtin.apt:
name:
- unattended-upgrades
- update-notifier-common
state: present
update_cache: true
when: ansible_facts['os_family'] == "Debian"
- name: Enforce uniform auto-update config on Ubuntu servers
ansible.builtin.copy:
dest: /etc/apt/apt.conf.d/20auto-upgrades
content: |
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
owner: root
group: root
mode: '0644'
when: ansible_facts['os_family'] == "Debian"
- name: Ensure unattended-upgrades service is enabled and running
ansible.builtin.systemd:
name: unattended-upgrades
enabled: true
state: started
when: ansible_facts['os_family'] == "Debian"
```
Running this across the `ubuntu` group ensures every host has the same config on every Ansible run — idempotent and safe.
## Rebooting Flagged Hosts
Once identified, reboot specific hosts without touching the rest:
```bash
# Reboot just the flagging hosts
ansible-playbook reboot.yml -l teelia,tttpod
# Run full update on remaining hosts to bring them up to the same kernel
ansible-playbook update.yml -l dca,majorlinux,majortoot
```
## Notes
- `unattended-upgrades` runs daily on its own schedule — hosts that haven't checked yet will lag behind but catch up within 24 hours
- The other hosts showing `ok` (not `changed`) on the config tasks means they were already correctly configured
- After a kernel update is pulled, only an actual reboot clears the `/var/run/reboot-required` flag — Ansible reporting the flag is informational only
## See Also
- [[ansible-getting-started|Ansible Getting Started]]
- [[linux-server-hardening-checklist|Linux Server Hardening Checklist]]

15
MajorWiki-Deploy-Status.md
View File

@@ -102,3 +102,18 @@ Every time a new article is added, the following **MUST** be updated to maintain
- [[MajorRig|MajorRig]] — alternative git push host (WSL2 path documented)
- [[03-11-2026|Status Update 2026-03-11]] — deployment date journal entry
- [[03-13-2026|Status Update 2026-03-13]] — content expansion and SUMMARY.md sync
---
## Session Update — 2026-03-16
**Article count:** 45 (was 42)
**New articles added:**
- `01-linux/distro-specific/wsl2-rebuild-fedora43-training-env.md` — full MajorTwin training env rebuild guide
- `01-linux/distro-specific/wsl2-backup-powershell.md` — WSL2 backup via PowerShell scheduled task
- `02-selfhosting/security/ansible-unattended-upgrades-fleet.md` — standardizing unattended-upgrades across Ubuntu fleet
**SUMMARY.md:** Updated to include all 3 new articles. Run SUMMARY.md dedup script if duplicate content appears (see board file cleanup pattern).
**Updated:** `updated: 2026-03-16`

3
SUMMARY.md
View File

@@ -9,6 +9,8 @@
* [SnapRAID & MergerFS Storage Setup](01-linux/storage/snapraid-mergerfs-setup.md)
* [Linux Distro Guide for Beginners](01-linux/distro-specific/linux-distro-guide-beginners.md)
* [WSL2 Instance Migration to Fedora 43](01-linux/distro-specific/wsl2-instance-migration-fedora43.md)
* [WSL2 Training Environment Rebuild](01-linux/distro-specific/wsl2-rebuild-fedora43-training-env.md)
* [WSL2 Backup via PowerShell](01-linux/distro-specific/wsl2-backup-powershell.md)
* [Self-Hosting & Homelab](02-selfhosting/index.md)
* [Self-Hosting Starter Guide](02-selfhosting/docker/self-hosting-starter-guide.md)
* [Docker vs VMs for the Homelab](02-selfhosting/docker/docker-vs-vms-homelab.md)
@@ -18,6 +20,7 @@
* [rsync Backup Patterns](02-selfhosting/storage-backup/rsync-backup-patterns.md)
* [Tuning Netdata Web Log Alerts](02-selfhosting/monitoring/tuning-netdata-web-log-alerts.md)
* [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md)
* [Standardizing unattended-upgrades with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md)
* [Open Source & Alternatives](03-opensource/index.md)
* [SearXNG: Private Self-Hosted Search](03-opensource/alternatives/searxng.md)
* [FreshRSS: Self-Hosted RSS Reader](03-opensource/alternatives/freshrss.md)