wiki: add ClamAV daemonless mode section + HEVC VAAPI article link

This commit is contained in:
Marcus Summers 2026-05-15 09:02:24 -04:00
parent 28518e403e
commit 64ac418a36
3 changed files with 206 additions and 2 deletions

View file

@ -11,7 +11,7 @@ tags:
- cron
status: published
created: 2026-04-18
updated: 2026-05-10T01:50
updated: 2026-05-15T03:00
---
# ClamAV Fleet Deployment with Ansible
@ -226,6 +226,41 @@ The "polite CPU is invisible to DO" trick stops working once the box is small en
**Alternative considered: switch to `clamdscan`** — uses a resident `clamd` daemon, signatures stay loaded, scan finishes ~10× faster with much less CPU/RAM. Better long-term answer, but requires running `clamd` continuously (memory cost on small boxes is ~250 MB resident vs the cron approach which only holds RAM during scan). Trade-off, not strictly better.
## Daemonless Mode on Memory-Constrained Hosts
On hosts with ≤2 GB RAM, running `clamd` continuously is often counterproductive. The daemon loads its full signature database (~950 MB RSS) into memory and keeps it resident. On small VMs this crowds out MySQL, PHP-FPM, and other services — often pushing the whole system into swap rather than preventing anything.
**Affected hosts (fleet history):**
| Host | RAM | Incident | Resolution |
|------|-----|----------|------------|
| teelia | 1.9 GB | 2026-04-27 — clamd 728 MB RSS, 94% RAM alert | daemonless |
| dcaprod | 3.8 GB | 2026-04-30 — clamd OOM thrash after 512M cgroup cap | daemonless |
| majorlinux | 2.0 GB | 2026-05-15 — clamd 980 MB swap, mysqld swapping 293 MB | daemonless |
**The fix: `clamav_use_daemon: false` host_var**
`configure_clamav.yml` supports a per-host override. Add to the host's `host_vars/<hostname>/vars.yml`:
```yaml
clamav_use_daemon: false
```
Then re-run the playbook:
```bash
ansible-playbook configure_clamav.yml --limit <hostname>
```
This will:
- Stop and disable `clamav-daemon.service` and `clamav-daemon.socket`
- Deploy the weekly scan template using `clamscan` (daemonless, loads DB per run)
- Leave `clamav-freshclam` active so definitions stay current
**Trade-off:** Each weekly scan loads the signature DB fresh (~950 MB peak RAM for the scan duration, then freed). The scan takes longer than `clamdscan` (~35× on a warm daemon), but this is acceptable for a weekly background job. The `systemd-run MemoryMax` cgroup wrapper in the scan template caps peak usage so the scan can't OOM the host.
**Rule of thumb:** Use daemon mode (`clamav_use_daemon: true` or unset) on hosts with ≥4 GB RAM where scan speed matters (mail servers, upload handlers). Use daemonless on webservers and small VMs where continuous memory residency is the bigger risk.
## See Also
- [clamscan-cpu-spike-nice-ionice](../../05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md) — troubleshooting CPU spikes from unthrottled scans

View file

@ -0,0 +1,168 @@
---
title: "HEVC Batch Re-Encode for Plex Using VAAPI (AMD GPU)"
domain: streaming
category: plex
tags: [plex, ffmpeg, hevc, vaapi, amd, gpu, encode, storage, rx480]
status: published
created: 2026-05-15
updated: 2026-05-15
---
# HEVC Batch Re-Encode for Plex Using VAAPI (AMD GPU)
## Problem
Plex NVMe storage is filling up from a large library of H.264-encoded video files (YouTube downloads, stream archives, etc.). Re-encoding to HEVC (H.265) reclaims 3050% of disk space. The catch: Plex tracks each file's "date added" in a SQLite database, and that order matters for playback queues. Naive re-encode-and-replace approaches can corrupt or reset that metadata.
## Solution
Use `ffmpeg` with `hevc_vaapi` (AMD GPU hardware encoder) to batch re-encode files in-place using an atomic rename swap that preserves the Plex database record — including `added_at` — without any Plex downtime or database editing.
---
## How Plex Stores "Date Added"
Plex does **not** use file modification time (`mtime`) for "date added." It stores a Unix timestamp in its SQLite database:
```sql
-- Plex DB location (override via systemd unit may differ — check):
-- /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/
-- Plug-in Support/Databases/com.plexapp.plugins.library.db
-- (or wherever PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR points)
SELECT mi.added_at, datetime(mi.added_at, 'unixepoch'), mp.file
FROM metadata_items mi
JOIN media_items me ON me.metadata_item_id = mi.id
JOIN media_parts mp ON mp.media_item_id = me.id
WHERE mp.file LIKE '%your-file%';
```
> **Note:** If the default path returns 0 rows, check your actual data directory:
> ```bash
> systemctl cat plexmediaserver | grep APPLICATION_SUPPORT
> ```
The `added_at` field is keyed to the **file path** in `media_parts`. As long as the file path doesn't change, the database record — including `added_at` — is untouched even after the file's content is replaced.
---
## Why VAAPI Instead of libx265
On a host with an AMD RX 480/580 (or similar Polaris GPU), hardware HEVC encoding via VAAPI is roughly **9× faster** than software libx265 at comparable quality:
| Encoder | Speed (1080p) | Notes |
|---|---|---|
| libx265 -preset medium | ~21 fps / 0.35× | Best quality/size ratio |
| hevc_vaapi QP 28 | ~186 fps / 3.1× | Sufficient for streaming content |
For 1080p streaming content (game streams, podcasts, YouTube archival), the quality difference is imperceptible. libx265 is preferable only for archival encodes where absolute quality matters.
### Verify VAAPI is working
```bash
vainfo 2>&1 | grep -E "vaapi|HEVC|hevc|Driver"
ls /dev/dri/renderD128
```
You need `VAProfileHEVCMain : VAEntrypointEncSlice` in the output. If missing, install `mesa-va-drivers-freeworld` (RPM Fusion) for AMD hardware.
---
## The Atomic Swap Strategy
The key insight: `mv file.tmp file` on the **same filesystem** is an atomic inode rename at the kernel level. Plex sees the same path still present — it never fires a "file removed" event, so the `metadata_items` record (including `added_at`) is preserved.
**Safe sequence:**
1. Encode source → `.hevc.tmp.mp4` alongside the original
2. Verify the output with `ffprobe`
3. `touch -r original.mp4 temp.mp4` — copy mtime (cosmetic, not required)
4. `mv temp.mp4 original.mp4` — atomic replace
**The one pitfall:** if the original file is deleted *before* the `mv`, Plex orphans the DB record (removes `metadata_items` entry on next scan) and re-indexes the new file with a fresh `added_at`. The original must still exist at swap time.
---
## The Batch Script
Script lives at `~/hevc_batch.sh` on majorhome.
```bash
# Dry run — scan and report what would be encoded, no changes
bash ~/hevc_batch.sh --dry-run
# Full run (default: files >1GB, QP 28)
tmux new-session -d -s hevc_batch 'bash ~/hevc_batch.sh'
# Custom options
bash ~/hevc_batch.sh --min-size-gb 2 --qp 26
```
### Queue and resume
The script writes a queue file at `~/hevc_queue.txt` on first run (scanning all files with ffprobe — takes ~10 min for a large library). On subsequent runs it resumes from where it left off. Completed files are logged to `~/hevc_done.txt`. Failed files go to `~/hevc_failed.txt`.
To restart from scratch: `rm ~/hevc_queue.txt ~/hevc_done.txt`
### Log output
```bash
# Structured log lines only (skip ffmpeg progress noise)
grep '^\[20' ~/hevc_batch.log
# Watch live progress
tail -f ~/hevc_batch.log | grep '^\[20'
```
Each file logs:
- Source size and codec
- `Plex added_at before: <unix timestamp>`
- ffmpeg exit code and elapsed time
- Output size and savings
- `DB check: added_at PRESERVED ✓` (or WARN if changed)
### Space guard
The script aborts if free space on the Plex volume drops below 20GB (`MIN_FREE_GB`). Worst-case headroom needed is `source_size + tmp_size` simultaneously — on a 4GB source file that's ~8GB peak.
---
## ffmpeg Command
```bash
ffmpeg \
-vaapi_device /dev/dri/renderD128 \
-i "input.mp4" \
-vf 'format=nv12,hwupload' \
-c:v hevc_vaapi -rc_mode CQP -qp 28 \
-c:a copy \
-movflags +faststart \
-y "output.tmp.mp4"
```
- `-rc_mode CQP -qp 28` — constant quantizer; higher value = smaller file / lower quality. QP 24 is high quality, QP 28 is good for streaming content.
- `-vf 'format=nv12,hwupload'` — required to move frames to GPU memory for VAAPI encoding.
- `-c:a copy` — passes audio through untouched.
- `hevc_vaapi` does not support 10-bit output on Polaris (RX 480/580). For 10-bit HDR sources, fall back to `libx265` with color signaling flags.
---
## Plex Data Directory Override
On majorhome, the Plex data directory is overridden in the systemd unit — the default path `/var/lib/plexmediaserver/` is empty:
```bash
systemctl cat plexmediaserver | grep APPLICATION_SUPPORT
# Environment=PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/plex/plexdata/Library/Application Support
```
The actual DB path is therefore:
```
/plex/plexdata/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db
```
---
## Related
- [[plex-4k-codec-compatibility]] — Apple TV Direct Play compatibility, HEVC HDR notes
- [[snapraid-mergerfs-setup]] — MajorRAID storage pool setup
- [[SnapRAID-Majorhome]] — majorhome SnapRAID project

View file

@ -1,6 +1,6 @@
---
created: 2026-04-02T16:03
updated: 2026-05-11T07:35
updated: 2026-05-15T09:00
---
* [Home](index.md)
* [Linux & Sysadmin](01-linux/index.md)
@ -70,6 +70,7 @@ updated: 2026-05-11T07:35
* [Streaming & Podcasting](04-streaming/index.md)
* [OBS Studio Setup & Encoding](04-streaming/obs/obs-studio-setup-encoding.md)
* [Plex 4K Codec Compatibility (Apple TV)](04-streaming/plex/plex-4k-codec-compatibility.md)
* [HEVC Batch Re-Encode for Plex Using VAAPI (AMD GPU)](04-streaming/plex/hevc-vaapi-batch-encode.md)
* [Troubleshooting](05-troubleshooting/index.md)
* [Apache Outage: Fail2ban Self-Ban + Missing iptables Rules](05-troubleshooting/networking/fail2ban-self-ban-apache-outage.md)
* [Mail Client Stops Receiving: Fail2ban IMAP Self-Ban](05-troubleshooting/networking/fail2ban-imap-self-ban-mail-client.md)