wiki: add fail2ban jail for Apache PHP webshell probes

Documents the 2026-04-09 scanner incident where 301-redirected PHP probes
bypassed the existing apache-404scan jail, leaving the scanner unbanned
and firing Netdata web_log_1m_redirects alerts. New jail catches 301/302/
403/404 PHP responses while excluding legitimate WordPress endpoints.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

02-selfhosting/index.md

@@ -1,3 +1,7 @@
+---
+created: 2026-04-13T10:15
+updated: 2026-04-13T10:15
+---
 # 🏠 Self-Hosting & Homelab
 
 Guides for running your own services at home, including Docker, reverse proxies, DNS, storage, monitoring, and security.
@@ -31,6 +35,7 @@ Guides for running your own services at home, including Docker, reverse proxies,
 - [Linux Server Hardening Checklist](security/linux-server-hardening-checklist.md)
 - [Standardizing unattended-upgrades with Ansible](security/ansible-unattended-upgrades-fleet.md)
 - [Fail2ban Custom Jail: Apache 404 Scanner Detection](security/fail2ban-apache-404-scanner-jail.md)
+- [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](security/fail2ban-apache-php-probe-jail.md)
 - [Fail2ban Custom Jail: WordPress Login Brute Force](security/fail2ban-wordpress-login-jail.md)
 - [SELinux: Fixing Fail2ban grep execmem Denial](security/selinux-fail2ban-execmem-fix.md)
 - [UFW Firewall Management](security/ufw-firewall-management.md)

02-selfhosting/security/fail2ban-apache-php-probe-jail.md

@@ -0,0 +1,146 @@
+---
+title: "Fail2ban Custom Jail: Apache PHP Webshell Probe Detection"
+domain: selfhosting
+category: security
+tags:
+  - fail2ban
+  - apache
+  - security
+  - php
+  - webshell
+  - scanner
+status: published
+created: 2026-04-09
+updated: 2026-04-13T10:15
+---
+# Fail2ban Custom Jail: Apache PHP Webshell Probe Detection
+
+## The Problem
+
+Automated scanners flood web servers with rapid-fire requests for non-existent `.php` files — `bless.php`, `alfa.php`, `lock360.php`, `about.php`, `cgi-bin/bypass.php`, and hundreds of others. These are classic **webshell/backdoor probes** looking for compromised PHP files left behind by prior attackers.
+
+On servers that force HTTPS (or have HTTP→HTTPS redirects in place), these probes often return **301 Moved Permanently** instead of 404. That causes three problems:
+
+1. **The `apache-404scan` jail misses them** — it only matches 404 responses
+2. **Netdata fires false `web_log_1m_redirects` alerts** — the redirect ratio spikes to 96%+ during scans
+3. **The scanner is never banned**, and will return repeatedly
+
+This was the exact trigger for the 2026-04-09 `[MajorLinux] Web Log Alert` incident where `45.86.202.224` sent 202 PHP probe requests in a few minutes, all returning 301.
+
+## The Solution
+
+Create a custom Fail2ban filter that matches **any `.php` request returning a redirect, forbidden, or not-found response** — while excluding legitimate WordPress PHP endpoints.
+
+### Step 1 — Create the filter
+
+Create `/etc/fail2ban/filter.d/apache-php-probe.conf`:
+
+```ini
+# Fail2Ban filter to catch PHP file probing (webshell/backdoor scanners)
+# These requests hit non-existent .php files and get 301/302/403/404 responses
+
+[Definition]
+
+failregex = ^<HOST> -.*"(GET|POST|HEAD) /[^ ]*\.php[^ ]* HTTP/[0-9.]+" (301|302|403|404) \d+
+
+ignoreregex = ^<HOST> -.*(wp-cron\.php|xmlrpc\.php|wp-login\.php|wp-admin|index\.php|wp-comments-post\.php)
+
+datepattern = %%d/%%b/%%Y:%%H:%%M:%%S %%z
+```
+
+**Why the ignoreregex matters:** Legitimate WordPress traffic hits `wp-cron.php`, `xmlrpc.php` (often 403-blocked on hardened sites), `wp-login.php`, and `index.php` constantly. Without exclusions the jail would ban your own WordPress admins. Note that `wp-login.php` brute force is caught separately by the `wordpress` jail.
+
+### Step 2 — Add the jail
+
+Add to `/etc/fail2ban/jail.local`:
+
+```ini
+[apache-php-probe]
+enabled = true
+port = http,https
+filter = apache-php-probe
+logpath = /var/log/apache2/access.log
+maxretry = 5
+findtime = 1m
+bantime = 48h
+```
+
+**5 hits in 1 minute** is tight — scanners fire 20–200 PHP probes in seconds, while a real user hitting one broken PHP link won't trip the threshold. The 48-hour bantime is longer than `apache-404scan`'s 24h because PHP webshell scanning is a stronger signal of malicious intent.
+
+### Step 3 — Test the regex
+
+```bash
+fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-php-probe.conf
+```
+
+Verify it matches the scanner requests and does **not** match legitimate WordPress traffic.
+
+### Step 4 — Reload Fail2ban
+
+```bash
+systemctl restart fail2ban
+fail2ban-client status apache-php-probe
+```
+
+## Why This Complements `apache-404scan`
+
+| Jail | Catches | Misses |
+|---|---|---|
+| `apache-404scan` | Any 404 (config file probes, `.env`, random paths) | PHP probes redirected to HTTPS (301) |
+| **`apache-php-probe`** | **PHP webshell probes (301/302/403/404)** | Non-`.php` probes |
+
+Running both jails together covers:
+- **HTTP→HTTPS redirected PHP probes** (301 responses)
+- **Directly-served PHP probes** (404 responses)
+- **Blocked PHP paths** like `xmlrpc.php` in non-WP contexts (403 responses)
+
+## Pair With Recidive
+
+The `recidive` jail catches repeat offenders across all jails:
+
+```ini
+[recidive]
+enabled = true
+bantime = -1
+findtime = 86400
+maxretry = 3
+```
+
+A scanner that trips `apache-php-probe` three times in 24 hours gets a **permanent** firewall-level ban.
+
+## Manual IP Blocking via UFW
+
+For known scanners you want to block immediately without waiting for the jail to trip, use UFW:
+
+```bash
+# Insert at top of rule list (priority over Apache ALLOW rules)
+ufw insert 1 deny from <IP> to any comment "PHP webshell scanner YYYY-MM-DD"
+```
+
+This bypasses fail2ban entirely and is useful for:
+- Scanners you spot in logs after the fact
+- Known-malicious subnets from threat intel
+- Entire CIDR blocks (`ufw insert 1 deny from 45.86.202.0/24`)
+
+## Quick Diagnostic Commands
+
+```bash
+# Count recent PHP probes returning 301/403/404
+awk '/09\/Apr\/2026:18:/ && /\.php/ && ($9==301 || $9==403 || $9==404)' /var/log/apache2/access.log | wc -l
+
+# Top probed PHP filenames (useful for writing additional ignoreregex)
+grep '\.php' /var/log/apache2/access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20
+
+# Top scanner IPs by PHP probe count
+grep '\.php' /var/log/apache2/access.log | awk '$9 ~ /^(301|403|404)$/ {print $1}' | sort | uniq -c | sort -rn | head -10
+
+# Watch bans in real time
+tail -f /var/log/fail2ban.log | grep apache-php-probe
+```
+
+## Key Notes
+
+- **This jail only makes sense on servers that redirect HTTP→HTTPS.** On plain-HTTPS-only servers, PHP probes return 404 and `apache-404scan` already catches them.
+- **Add your own WordPress plugin paths to `ignoreregex`** if you use non-standard endpoints (e.g., custom admin URLs, REST API `.php` handlers).
+- **This filter pairs naturally with Netdata `web_log_1m_redirects` alerts** — during a scan, Netdata fires first (threshold crossed), then fail2ban bans the IP within seconds.
+- Also see: [Fail2ban Custom Jail: Apache 404 Scanner Detection](fail2ban-apache-404-scanner-jail.md) for the sibling 404-based filter.

README.md

@@ -1,6 +1,6 @@
 ---
 created: 2026-04-06T09:52
-updated: 2026-04-13T10:10
+updated: 2026-04-13T10:16
 ---
 # MajorLinux Tech Wiki — Index
 
@@ -80,6 +80,7 @@ updated: 2026-04-13T10:10
 - [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md) — non-root user, SSH key auth, sshd_config, firewall, fail2ban, SpamAssassin
 - [Standardizing unattended-upgrades with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md) — fleet-wide automatic security updates across Ubuntu servers
 - [Fail2ban Custom Jail: Apache 404 Scanner Detection](02-selfhosting/security/fail2ban-apache-404-scanner-jail.md) — custom filter and jail for blocking 404 scanners
+- [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md) — catching PHP webshell/backdoor probes that return 301 on HTTPS-redirecting servers
 - [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md) — access-log-based wp-login.php brute force detection without plugins
 - [SELinux: Fixing Fail2ban grep execmem Denial](02-selfhosting/security/selinux-fail2ban-execmem-fix.md) — resolving execmem AVC denials from Fail2ban's grep on Fedora
 - [UFW Firewall Management](02-selfhosting/security/ufw-firewall-management.md) — managing UFW rules, common patterns, troubleshooting
@@ -162,6 +163,7 @@ updated: 2026-04-13T10:10
 | Date | Article | Domain |
 |---|---|---|
 | 2026-04-13 | [Cron Heartbeat False Alarm: /var/run Cleared by Reboot](05-troubleshooting/cron-heartbeat-tmpfs-reboot-false-alarm.md) | Troubleshooting |
+| 2026-04-09 | [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md) | Self-Hosting |
 | 2026-04-08 | [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) | Troubleshooting |
 | 2026-04-07 | [SSH Config & Key Management](01-linux/networking/ssh-config-key-management.md) | Linux |
 | 2026-04-07 | [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) | Troubleshooting |

SUMMARY.md

@@ -1,6 +1,6 @@
 ---
 created: 2026-04-02T16:03
-updated: 2026-04-13T10:10
+updated: 2026-04-13T10:16
 ---
 * [Home](index.md)
 * [Linux & Sysadmin](01-linux/index.md)
@@ -35,6 +35,7 @@ updated: 2026-04-13T10:10
     * [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md)
     * [Standardizing unattended-upgrades with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md)
     * [Fail2ban Custom Jail: Apache 404 Scanner Detection](02-selfhosting/security/fail2ban-apache-404-scanner-jail.md)
+    * [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md)
     * [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md)
     * [SELinux: Fixing Fail2ban grep execmem Denial](02-selfhosting/security/selinux-fail2ban-execmem-fix.md)
     * [UFW Firewall Management](02-selfhosting/security/ufw-firewall-management.md)

index.md

@@ -1,6 +1,6 @@
 ---
 created: 2026-04-06T09:52
-updated: 2026-04-13T10:11
+updated: 2026-04-13T10:16
 ---
 # MajorLinux Tech Wiki — Index
 
@@ -81,6 +81,7 @@ updated: 2026-04-13T10:11
 - [Linux Server Hardening Checklist](02-selfhosting/security/linux-server-hardening-checklist.md) — non-root user, SSH key auth, sshd_config, firewall, fail2ban, SpamAssassin
 - [Standardizing unattended-upgrades with Ansible](02-selfhosting/security/ansible-unattended-upgrades-fleet.md) — fleet-wide automatic security updates across Ubuntu servers
 - [Fail2ban Custom Jail: Apache 404 Scanner Detection](02-selfhosting/security/fail2ban-apache-404-scanner-jail.md) — custom filter and jail for blocking 404 scanners
+- [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md) — catching PHP webshell/backdoor probes that return 301 on HTTPS-redirecting servers
 - [Fail2ban Custom Jail: WordPress Login Brute Force](02-selfhosting/security/fail2ban-wordpress-login-jail.md) — access-log-based wp-login.php brute force detection without plugins
 - [SELinux: Fixing Fail2ban grep execmem Denial](02-selfhosting/security/selinux-fail2ban-execmem-fix.md) — resolving execmem AVC denials from Fail2ban's grep on Fedora
 - [UFW Firewall Management](02-selfhosting/security/ufw-firewall-management.md) — managing UFW rules, common patterns, troubleshooting
@@ -166,6 +167,7 @@ updated: 2026-04-13T10:11
 | Date | Article | Domain |
 |---|---|---|
 | 2026-04-13 | [Cron Heartbeat False Alarm: /var/run Cleared by Reboot](05-troubleshooting/cron-heartbeat-tmpfs-reboot-false-alarm.md) | Troubleshooting |
+| 2026-04-09 | [Fail2ban Custom Jail: Apache PHP Webshell Probe Detection](02-selfhosting/security/fail2ban-apache-php-probe-jail.md) | Self-Hosting |
 | 2026-04-08 | [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) | Troubleshooting |
 | 2026-04-07 | [SSH Config & Key Management](01-linux/networking/ssh-config-key-management.md) | Linux |
 | 2026-04-07 | [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) | Troubleshooting |