wiki: add fail2ban UFW rule bloat and Apache dirscan jail articles (56 articles)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

05-troubleshooting/security/apache-dirscan-fail2ban-jail.md

@@ -0,0 +1,113 @@
+# Custom Fail2ban Jail: Apache Directory Scanning & Junk Methods
+
+## 🛑 Problem
+
+Bots and vulnerability scanners enumerate WordPress directories (`/wp-admin/`, `/wp-includes/`, `/wp-content/`), probe for access-denied paths, or send junk HTTP methods (e.g., `YQEILVHZ`, `DUTEDCEM`). These generate Apache error log entries but are not caught by any default Fail2ban jail:
+
+- `AH01276` — directory index forbidden (autoindex:error)
+- `AH01630` — client denied by server configuration (authz_core:error)
+- `AH00135` — invalid method in request (core:error)
+
+The result is a low success ratio on Netdata's `web_log_1m_successful` metric and wasted server resources processing scanner requests.
+
+---
+
+## ✅ Solution
+
+### Step 1 — Create the filter
+
+Create `/etc/fail2ban/filter.d/apache-dirscan.conf`:
+
+```ini
+# Fail2ban filter for Apache scanning/probing
+# Catches: directory enumeration (AH01276), access denied (AH01630), invalid methods (AH00135)
+
+[Definition]
+failregex = ^\[.*\] \[autoindex:error\] \[pid \d+\] \[client <HOST>:\d+\] AH01276:
+            ^\[.*\] \[authz_core:error\] \[pid \d+\] \[client <HOST>:\d+\] AH01630:
+            ^\[.*\] \[core:error\] \[pid \d+\] \[client <HOST>:\d+\] AH00135:
+
+ignoreregex =
+```
+
+### Step 2 — Add the jail
+
+Add to `/etc/fail2ban/jail.local`:
+
+```ini
+[apache-dirscan]
+enabled = true
+port = http,https
+filter = apache-dirscan
+logpath = /var/log/apache2/error.log
+maxretry = 3
+findtime = 60
+```
+
+Three hits in 60 seconds is aggressive enough to catch active scanners while avoiding false positives from legitimate 403s.
+
+### Step 3 — Test the regex
+
+```bash
+fail2ban-regex /var/log/apache2/error.log /etc/fail2ban/filter.d/apache-dirscan.conf
+```
+
+This shows match counts per regex line and any missed lines.
+
+### Step 4 — Reload Fail2ban
+
+```bash
+fail2ban-client reload
+fail2ban-client status apache-dirscan
+```
+
+---
+
+## 🔍 What Each Pattern Catches
+
+| Error Code | Apache Module | Trigger |
+|---|---|---|
+| `AH01276` | `autoindex:error` | Bot requests a directory with no index file and `Options -Indexes` is set. Classic WordPress/CMS directory enumeration. |
+| `AH01630` | `authz_core:error` | Request denied by `<Directory>` or `<Location>` rules (e.g., probing `/wp-content/plugins/`). |
+| `AH00135` | `core:error` | Request uses a garbage HTTP method that Apache can't parse. Scanners use these to fingerprint servers. |
+
+---
+
+## 🔁 Why Default Jails Miss This
+
+| Default Jail | What It Catches | Gap |
+|---|---|---|
+| `apache-badbots` | Bad User-Agent strings in access log | Doesn't look at error log; many scanners use normal UAs |
+| `apache-botsearch` | 404s for common exploit paths | Only matches access log 404s, not error log entries |
+| `apache-noscript` | Requests for non-existent scripts | Narrow regex, doesn't cover directory probes |
+| `apache-overflows` | Long request URIs | Only catches buffer overflow attempts |
+| `apache-invaliduri` | `AH10244` invalid URI encoding | Different error code — catches URL-encoded traversal, not directory scanning |
+
+The `apache-dirscan` filter fills the gap by monitoring the error log for the three most common scanner signatures that slip through all default jails.
+
+---
+
+## ⚠️ Key Notes
+
+- **`logpath` must point to the error log**, not the access log. All three patterns are logged to `error.log`.
+- **Adjust `logpath`** for your distribution: Debian/Ubuntu uses `/var/log/apache2/error.log`, RHEL/Fedora uses `/var/log/httpd/error_log`.
+- **The `allowipv6` warning** on reload is cosmetic (Fail2ban 1.0+) and can be ignored.
+- **Pair with `recidive`** to escalate repeat offenders to longer bans.
+
+---
+
+## 🔎 Quick Diagnostic Commands
+
+```bash
+# Test filter against current error log
+fail2ban-regex /var/log/apache2/error.log /etc/fail2ban/filter.d/apache-dirscan.conf
+
+# Check jail status
+fail2ban-client status apache-dirscan
+
+# Watch bans in real time
+tail -f /var/log/fail2ban.log | grep apache-dirscan
+
+# Count current error types
+grep -c "AH01276\|AH01630\|AH00135" /var/log/apache2/error.log
+```