- Fixed 4 broken markdown links (bad relative paths in See Also sections) - Corrected n8n port binding to 127.0.0.1:5678 (matches actual deployment) - Updated SnapRAID article with actual majorhome paths (/majorRAID, disk1-3) - Converted 67 Obsidian wikilinks to relative markdown links or plain text - Added YAML frontmatter to 35 articles missing it entirely - Completed frontmatter on 8 articles with missing fields Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
204 lines
5.9 KiB
Markdown
204 lines
5.9 KiB
Markdown
---
|
|
title: WSL2 Fedora 43 Training Environment Rebuild
|
|
domain: linux
|
|
category: distro-specific
|
|
tags:
|
|
- wsl2
|
|
- fedora
|
|
- unsloth
|
|
- pytorch
|
|
- cuda
|
|
- majorrig
|
|
- majortwin
|
|
status: published
|
|
created: '2026-03-16'
|
|
updated: '2026-03-16'
|
|
---
|
|
|
|
# WSL2 Fedora 43 Training Environment Rebuild
|
|
|
|
How to rebuild the MajorTwin training environment from scratch on MajorRig after a WSL2 loss. Covers Fedora 43 install, Python 3.11 via pyenv, PyTorch with CUDA, Unsloth, and llama.cpp for GGUF conversion.
|
|
|
|
## The Short Answer
|
|
|
|
```bash
|
|
# 1. Install Fedora 43 and move to D:
|
|
wsl --install -d FedoraLinux-43 --no-launch
|
|
wsl --export FedoraLinux-43 D:\WSL\fedora43.tar
|
|
wsl --unregister FedoraLinux-43
|
|
wsl --import FedoraLinux-43 D:\WSL\Fedora43 D:\WSL\fedora43.tar
|
|
|
|
# 2. Set default user
|
|
echo -e "[boot]\nsystemd=true\n[user]\ndefault=majorlinux" | sudo tee /etc/wsl.conf
|
|
useradd -m -G wheel majorlinux && passwd majorlinux
|
|
echo "%wheel ALL=(ALL) ALL" | sudo tee /etc/sudoers.d/wheel
|
|
|
|
# 3. Install Python 3.11 via pyenv, PyTorch, Unsloth
|
|
# See full steps below
|
|
```
|
|
|
|
## Step 1 — System Packages
|
|
|
|
```bash
|
|
sudo dnf update -y
|
|
sudo dnf install -y git curl wget tmux screen htop rsync unzip \
|
|
python3 python3-pip python3-devel gcc gcc-c++ make cmake \
|
|
ninja-build pkg-config openssl-devel libffi-devel \
|
|
gawk patch readline-devel sqlite-devel
|
|
```
|
|
|
|
## Step 2 — Python 3.11 via pyenv
|
|
|
|
Fedora 43 ships Python 3.13. Unsloth requires 3.11. Use pyenv:
|
|
|
|
```bash
|
|
curl https://pyenv.run | bash
|
|
|
|
# Add to ~/.bashrc
|
|
export PYENV_ROOT="$HOME/.pyenv"
|
|
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
|
|
eval "$(pyenv init - bash)"
|
|
|
|
source ~/.bashrc
|
|
pyenv install 3.11.9
|
|
pyenv global 3.11.9
|
|
```
|
|
|
|
The tkinter warning during install is harmless — it's not needed for training.
|
|
|
|
## Step 3 — Training Virtualenv + PyTorch
|
|
|
|
```bash
|
|
mkdir -p ~/majortwin/{staging,datasets,outputs,scripts}
|
|
python -m venv ~/majortwin/venv
|
|
source ~/majortwin/venv/bin/activate
|
|
|
|
pip install --upgrade pip
|
|
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
|
|
|
|
# Verify GPU
|
|
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
|
|
```
|
|
|
|
Expected output: `True NVIDIA GeForce RTX 3080 Ti`
|
|
|
|
## Step 4 — Unsloth + Training Stack
|
|
|
|
```bash
|
|
source ~/majortwin/venv/bin/activate
|
|
|
|
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
|
|
pip install transformers datasets accelerate peft trl bitsandbytes \
|
|
sentencepiece protobuf scipy einops
|
|
|
|
# Pin transformers for unsloth-zoo compatibility
|
|
pip install "transformers<=5.2.0"
|
|
|
|
# Verify
|
|
python -c "import unsloth; print('Unsloth OK')"
|
|
```
|
|
|
|
> [!warning] Never run `pip install -r requirements.txt` from inside llama.cpp while the training venv is active. It installs CPU-only PyTorch and downgrades transformers, breaking the CUDA setup.
|
|
|
|
## Step 5 — llama.cpp (CPU-only for GGUF conversion)
|
|
|
|
CUDA 12.8 is incompatible with Fedora 43's glibc for compiling llama.cpp (math function conflicts in `/usr/include/bits/mathcalls.h`). Build CPU-only — it's sufficient for GGUF conversion, which doesn't need GPU:
|
|
|
|
```bash
|
|
# Install GCC 14 (CUDA 12.8 doesn't support GCC 15 which Fedora 43 ships)
|
|
sudo dnf install -y gcc14 gcc14-c++
|
|
|
|
cd ~/majortwin
|
|
git clone https://github.com/ggerganov/llama.cpp.git
|
|
cd llama.cpp
|
|
|
|
cmake -B build \
|
|
-DGGML_CUDA=OFF \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-DCMAKE_C_COMPILER=/usr/bin/gcc-14 \
|
|
-DCMAKE_CXX_COMPILER=/usr/bin/g++-14
|
|
|
|
cmake --build build --config Release -j$(nproc) 2>&1 | tee /tmp/llama_build.log &
|
|
tail -f /tmp/llama_build.log
|
|
```
|
|
|
|
Verify:
|
|
```bash
|
|
ls ~/majortwin/llama.cpp/build/bin/llama-quantize && echo "OK"
|
|
ls ~/majortwin/llama.cpp/build/bin/llama-cli && echo "OK"
|
|
```
|
|
|
|
## Step 6 — Shell Environment
|
|
|
|
```bash
|
|
cat >> ~/.bashrc << 'EOF'
|
|
# MajorInfrastructure Paths
|
|
export VAULT="/mnt/c/Users/majli/Documents/MajorVault"
|
|
export MAJORANSIBLE="/mnt/d/MajorAnsible"
|
|
export MAJORTWIN_D="/mnt/d/MajorTwin"
|
|
export MAJORTWIN_WSL="$HOME/majortwin"
|
|
export LLAMA_CPP="$HOME/majortwin/llama.cpp"
|
|
|
|
# Venv
|
|
alias mtwin='source $MAJORTWIN_WSL/venv/bin/activate && cd $MAJORTWIN_WSL'
|
|
alias vault='cd $VAULT'
|
|
alias ll='ls -lah --color=auto'
|
|
|
|
# SSH Fleet Aliases
|
|
alias majorhome='ssh majorlinux@100.120.209.106'
|
|
alias dca='ssh root@100.104.11.146'
|
|
alias majortoot='ssh root@100.110.197.17'
|
|
alias majorlinuxvm='ssh root@100.87.200.5'
|
|
alias majordiscord='ssh root@100.122.240.83'
|
|
alias majorlab='ssh root@100.86.14.126'
|
|
alias majormail='ssh root@100.84.165.52'
|
|
alias teelia='ssh root@100.120.32.69'
|
|
alias tttpod='ssh root@100.84.42.102'
|
|
alias majorrig='ssh majorlinux@100.98.47.29' # port 2222 retired 2026-03-25, fleet uses port 22
|
|
|
|
# DNF5
|
|
alias update='sudo dnf upgrade --refresh'
|
|
alias install='sudo dnf install'
|
|
alias clean='sudo dnf clean all'
|
|
|
|
# MajorTwin helpers
|
|
stage_dataset() {
|
|
cp "$VAULT/20-Projects/MajorTwin/03-Datasets/$1" "$MAJORTWIN_WSL/datasets/"
|
|
echo "Staged: $1"
|
|
}
|
|
export_gguf() {
|
|
cp "$MAJORTWIN_WSL/outputs/$1" "$MAJORTWIN_D/models/"
|
|
echo "Exported: $1 → $MAJORTWIN_D/models/"
|
|
}
|
|
EOF
|
|
|
|
source ~/.bashrc
|
|
```
|
|
|
|
## Key Rules
|
|
|
|
- **Always activate venv before pip installs:** `source ~/majortwin/venv/bin/activate`
|
|
- **Never train from /mnt/c or /mnt/d** — stage files in `~/majortwin/staging/` first
|
|
- **Never put ML artifacts inside MajorVault** — models, venvs, artifacts go on D: drive
|
|
- **Max viable training model:** 7B at QLoRA 4-bit (RTX 3080 Ti, 12GB VRAM)
|
|
- **Current base model:** Qwen2.5-7B-Instruct (ChatML format — stop token: `<|im_end|>` only)
|
|
- **Transformers must be pinned:** `pip install "transformers<=5.2.0"` for unsloth-zoo compatibility
|
|
|
|
## D: Drive Layout
|
|
|
|
```
|
|
D:\MajorTwin\
|
|
models\ ← finished GGUFs
|
|
datasets\ ← dataset archives
|
|
artifacts\ ← training run artifacts
|
|
training-runs\ ← logs, checkpoints
|
|
D:\WSL\
|
|
Fedora43\ ← WSL2 VHDX
|
|
Backups\ ← weekly WSL2 backup tars
|
|
```
|
|
|
|
## See Also
|
|
|
|
- [WSL2 Instance Migration](wsl2-instance-migration-fedora43.md)
|
|
- [WSL2 Backup via PowerShell](wsl2-backup-powershell.md)
|