wiki: add ClamAV daemonless mode section + HEVC VAAPI article link
This commit is contained in:
parent
28518e403e
commit
64ac418a36
3 changed files with 206 additions and 2 deletions
|
|
@ -11,7 +11,7 @@ tags:
|
|||
- cron
|
||||
status: published
|
||||
created: 2026-04-18
|
||||
updated: 2026-05-10T01:50
|
||||
updated: 2026-05-15T03:00
|
||||
---
|
||||
# ClamAV Fleet Deployment with Ansible
|
||||
|
||||
|
|
@ -226,6 +226,41 @@ The "polite CPU is invisible to DO" trick stops working once the box is small en
|
|||
|
||||
**Alternative considered: switch to `clamdscan`** — uses a resident `clamd` daemon, signatures stay loaded, scan finishes ~10× faster with much less CPU/RAM. Better long-term answer, but requires running `clamd` continuously (memory cost on small boxes is ~250 MB resident vs the cron approach which only holds RAM during scan). Trade-off, not strictly better.
|
||||
|
||||
## Daemonless Mode on Memory-Constrained Hosts
|
||||
|
||||
On hosts with ≤2 GB RAM, running `clamd` continuously is often counterproductive. The daemon loads its full signature database (~950 MB RSS) into memory and keeps it resident. On small VMs this crowds out MySQL, PHP-FPM, and other services — often pushing the whole system into swap rather than preventing anything.
|
||||
|
||||
**Affected hosts (fleet history):**
|
||||
|
||||
| Host | RAM | Incident | Resolution |
|
||||
|------|-----|----------|------------|
|
||||
| teelia | 1.9 GB | 2026-04-27 — clamd 728 MB RSS, 94% RAM alert | daemonless |
|
||||
| dcaprod | 3.8 GB | 2026-04-30 — clamd OOM thrash after 512M cgroup cap | daemonless |
|
||||
| majorlinux | 2.0 GB | 2026-05-15 — clamd 980 MB swap, mysqld swapping 293 MB | daemonless |
|
||||
|
||||
**The fix: `clamav_use_daemon: false` host_var**
|
||||
|
||||
`configure_clamav.yml` supports a per-host override. Add to the host's `host_vars/<hostname>/vars.yml`:
|
||||
|
||||
```yaml
|
||||
clamav_use_daemon: false
|
||||
```
|
||||
|
||||
Then re-run the playbook:
|
||||
|
||||
```bash
|
||||
ansible-playbook configure_clamav.yml --limit <hostname>
|
||||
```
|
||||
|
||||
This will:
|
||||
- Stop and disable `clamav-daemon.service` and `clamav-daemon.socket`
|
||||
- Deploy the weekly scan template using `clamscan` (daemonless, loads DB per run)
|
||||
- Leave `clamav-freshclam` active so definitions stay current
|
||||
|
||||
**Trade-off:** Each weekly scan loads the signature DB fresh (~950 MB peak RAM for the scan duration, then freed). The scan takes longer than `clamdscan` (~3–5× on a warm daemon), but this is acceptable for a weekly background job. The `systemd-run MemoryMax` cgroup wrapper in the scan template caps peak usage so the scan can't OOM the host.
|
||||
|
||||
**Rule of thumb:** Use daemon mode (`clamav_use_daemon: true` or unset) on hosts with ≥4 GB RAM where scan speed matters (mail servers, upload handlers). Use daemonless on webservers and small VMs where continuous memory residency is the bigger risk.
|
||||
|
||||
## See Also
|
||||
|
||||
- [clamscan-cpu-spike-nice-ionice](../../05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md) — troubleshooting CPU spikes from unthrottled scans
|
||||
|
|
|
|||
168
04-streaming/plex/hevc-vaapi-batch-encode.md
Normal file
168
04-streaming/plex/hevc-vaapi-batch-encode.md
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
---
|
||||
title: "HEVC Batch Re-Encode for Plex Using VAAPI (AMD GPU)"
|
||||
domain: streaming
|
||||
category: plex
|
||||
tags: [plex, ffmpeg, hevc, vaapi, amd, gpu, encode, storage, rx480]
|
||||
status: published
|
||||
created: 2026-05-15
|
||||
updated: 2026-05-15
|
||||
---
|
||||
# HEVC Batch Re-Encode for Plex Using VAAPI (AMD GPU)
|
||||
|
||||
## Problem
|
||||
|
||||
Plex NVMe storage is filling up from a large library of H.264-encoded video files (YouTube downloads, stream archives, etc.). Re-encoding to HEVC (H.265) reclaims 30–50% of disk space. The catch: Plex tracks each file's "date added" in a SQLite database, and that order matters for playback queues. Naive re-encode-and-replace approaches can corrupt or reset that metadata.
|
||||
|
||||
## Solution
|
||||
|
||||
Use `ffmpeg` with `hevc_vaapi` (AMD GPU hardware encoder) to batch re-encode files in-place using an atomic rename swap that preserves the Plex database record — including `added_at` — without any Plex downtime or database editing.
|
||||
|
||||
---
|
||||
|
||||
## How Plex Stores "Date Added"
|
||||
|
||||
Plex does **not** use file modification time (`mtime`) for "date added." It stores a Unix timestamp in its SQLite database:
|
||||
|
||||
```sql
|
||||
-- Plex DB location (override via systemd unit may differ — check):
|
||||
-- /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/
|
||||
-- Plug-in Support/Databases/com.plexapp.plugins.library.db
|
||||
-- (or wherever PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR points)
|
||||
|
||||
SELECT mi.added_at, datetime(mi.added_at, 'unixepoch'), mp.file
|
||||
FROM metadata_items mi
|
||||
JOIN media_items me ON me.metadata_item_id = mi.id
|
||||
JOIN media_parts mp ON mp.media_item_id = me.id
|
||||
WHERE mp.file LIKE '%your-file%';
|
||||
```
|
||||
|
||||
> **Note:** If the default path returns 0 rows, check your actual data directory:
|
||||
> ```bash
|
||||
> systemctl cat plexmediaserver | grep APPLICATION_SUPPORT
|
||||
> ```
|
||||
|
||||
The `added_at` field is keyed to the **file path** in `media_parts`. As long as the file path doesn't change, the database record — including `added_at` — is untouched even after the file's content is replaced.
|
||||
|
||||
---
|
||||
|
||||
## Why VAAPI Instead of libx265
|
||||
|
||||
On a host with an AMD RX 480/580 (or similar Polaris GPU), hardware HEVC encoding via VAAPI is roughly **9× faster** than software libx265 at comparable quality:
|
||||
|
||||
| Encoder | Speed (1080p) | Notes |
|
||||
|---|---|---|
|
||||
| libx265 -preset medium | ~21 fps / 0.35× | Best quality/size ratio |
|
||||
| hevc_vaapi QP 28 | ~186 fps / 3.1× | Sufficient for streaming content |
|
||||
|
||||
For 1080p streaming content (game streams, podcasts, YouTube archival), the quality difference is imperceptible. libx265 is preferable only for archival encodes where absolute quality matters.
|
||||
|
||||
### Verify VAAPI is working
|
||||
|
||||
```bash
|
||||
vainfo 2>&1 | grep -E "vaapi|HEVC|hevc|Driver"
|
||||
ls /dev/dri/renderD128
|
||||
```
|
||||
|
||||
You need `VAProfileHEVCMain : VAEntrypointEncSlice` in the output. If missing, install `mesa-va-drivers-freeworld` (RPM Fusion) for AMD hardware.
|
||||
|
||||
---
|
||||
|
||||
## The Atomic Swap Strategy
|
||||
|
||||
The key insight: `mv file.tmp file` on the **same filesystem** is an atomic inode rename at the kernel level. Plex sees the same path still present — it never fires a "file removed" event, so the `metadata_items` record (including `added_at`) is preserved.
|
||||
|
||||
**Safe sequence:**
|
||||
1. Encode source → `.hevc.tmp.mp4` alongside the original
|
||||
2. Verify the output with `ffprobe`
|
||||
3. `touch -r original.mp4 temp.mp4` — copy mtime (cosmetic, not required)
|
||||
4. `mv temp.mp4 original.mp4` — atomic replace
|
||||
|
||||
**The one pitfall:** if the original file is deleted *before* the `mv`, Plex orphans the DB record (removes `metadata_items` entry on next scan) and re-indexes the new file with a fresh `added_at`. The original must still exist at swap time.
|
||||
|
||||
---
|
||||
|
||||
## The Batch Script
|
||||
|
||||
Script lives at `~/hevc_batch.sh` on majorhome.
|
||||
|
||||
```bash
|
||||
# Dry run — scan and report what would be encoded, no changes
|
||||
bash ~/hevc_batch.sh --dry-run
|
||||
|
||||
# Full run (default: files >1GB, QP 28)
|
||||
tmux new-session -d -s hevc_batch 'bash ~/hevc_batch.sh'
|
||||
|
||||
# Custom options
|
||||
bash ~/hevc_batch.sh --min-size-gb 2 --qp 26
|
||||
```
|
||||
|
||||
### Queue and resume
|
||||
|
||||
The script writes a queue file at `~/hevc_queue.txt` on first run (scanning all files with ffprobe — takes ~10 min for a large library). On subsequent runs it resumes from where it left off. Completed files are logged to `~/hevc_done.txt`. Failed files go to `~/hevc_failed.txt`.
|
||||
|
||||
To restart from scratch: `rm ~/hevc_queue.txt ~/hevc_done.txt`
|
||||
|
||||
### Log output
|
||||
|
||||
```bash
|
||||
# Structured log lines only (skip ffmpeg progress noise)
|
||||
grep '^\[20' ~/hevc_batch.log
|
||||
|
||||
# Watch live progress
|
||||
tail -f ~/hevc_batch.log | grep '^\[20'
|
||||
```
|
||||
|
||||
Each file logs:
|
||||
- Source size and codec
|
||||
- `Plex added_at before: <unix timestamp>`
|
||||
- ffmpeg exit code and elapsed time
|
||||
- Output size and savings
|
||||
- `DB check: added_at PRESERVED ✓` (or WARN if changed)
|
||||
|
||||
### Space guard
|
||||
|
||||
The script aborts if free space on the Plex volume drops below 20GB (`MIN_FREE_GB`). Worst-case headroom needed is `source_size + tmp_size` simultaneously — on a 4GB source file that's ~8GB peak.
|
||||
|
||||
---
|
||||
|
||||
## ffmpeg Command
|
||||
|
||||
```bash
|
||||
ffmpeg \
|
||||
-vaapi_device /dev/dri/renderD128 \
|
||||
-i "input.mp4" \
|
||||
-vf 'format=nv12,hwupload' \
|
||||
-c:v hevc_vaapi -rc_mode CQP -qp 28 \
|
||||
-c:a copy \
|
||||
-movflags +faststart \
|
||||
-y "output.tmp.mp4"
|
||||
```
|
||||
|
||||
- `-rc_mode CQP -qp 28` — constant quantizer; higher value = smaller file / lower quality. QP 24 is high quality, QP 28 is good for streaming content.
|
||||
- `-vf 'format=nv12,hwupload'` — required to move frames to GPU memory for VAAPI encoding.
|
||||
- `-c:a copy` — passes audio through untouched.
|
||||
- `hevc_vaapi` does not support 10-bit output on Polaris (RX 480/580). For 10-bit HDR sources, fall back to `libx265` with color signaling flags.
|
||||
|
||||
---
|
||||
|
||||
## Plex Data Directory Override
|
||||
|
||||
On majorhome, the Plex data directory is overridden in the systemd unit — the default path `/var/lib/plexmediaserver/` is empty:
|
||||
|
||||
```bash
|
||||
systemctl cat plexmediaserver | grep APPLICATION_SUPPORT
|
||||
# Environment=PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/plex/plexdata/Library/Application Support
|
||||
```
|
||||
|
||||
The actual DB path is therefore:
|
||||
```
|
||||
/plex/plexdata/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [[plex-4k-codec-compatibility]] — Apple TV Direct Play compatibility, HEVC HDR notes
|
||||
- [[snapraid-mergerfs-setup]] — MajorRAID storage pool setup
|
||||
- [[SnapRAID-Majorhome]] — majorhome SnapRAID project
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
created: 2026-04-02T16:03
|
||||
updated: 2026-05-11T07:35
|
||||
updated: 2026-05-15T09:00
|
||||
---
|
||||
* [Home](index.md)
|
||||
* [Linux & Sysadmin](01-linux/index.md)
|
||||
|
|
@ -70,6 +70,7 @@ updated: 2026-05-11T07:35
|
|||
* [Streaming & Podcasting](04-streaming/index.md)
|
||||
* [OBS Studio Setup & Encoding](04-streaming/obs/obs-studio-setup-encoding.md)
|
||||
* [Plex 4K Codec Compatibility (Apple TV)](04-streaming/plex/plex-4k-codec-compatibility.md)
|
||||
* [HEVC Batch Re-Encode for Plex Using VAAPI (AMD GPU)](04-streaming/plex/hevc-vaapi-batch-encode.md)
|
||||
* [Troubleshooting](05-troubleshooting/index.md)
|
||||
* [Apache Outage: Fail2ban Self-Ban + Missing iptables Rules](05-troubleshooting/networking/fail2ban-self-ban-apache-outage.md)
|
||||
* [Mail Client Stops Receiving: Fail2ban IMAP Self-Ban](05-troubleshooting/networking/fail2ban-imap-self-ban-mail-client.md)
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue