MajorLinux
6592eb4fea
- Fixed 4 broken markdown links (bad relative paths in See Also sections) - Corrected n8n port binding to 127.0.0.1:5678 (matches actual deployment) - Updated SnapRAID article with actual majorhome paths (/majorRAID, disk1-3) - Converted 67 Obsidian wikilinks to relative markdown links or plain text - Added YAML frontmatter to 35 articles missing it entirely - Completed frontmatter on 8 articles with missing fields Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
89 lines
2.7 KiB
Markdown
89 lines
2.7 KiB
Markdown
---
|
|
title: Tuning Netdata Web Log Alerts
|
|
domain: selfhosting
|
|
category: monitoring
|
|
tags:
|
|
- netdata
|
|
- apache
|
|
- monitoring
|
|
- alerts
|
|
- ubuntu
|
|
status: published
|
|
created: '2026-03-06'
|
|
updated: '2026-03-08'
|
|
---
|
|
|
|
# Tuning Netdata Web Log Alerts
|
|
|
|
To stop Netdata's `web_log_1m_redirects` alert from firing on normal HTTP-to-HTTPS redirect traffic, edit `/etc/netdata/health.d/web_log.conf` and raise the redirect threshold to 80%, then reload with `netdatacli reload-health`. The default threshold is too sensitive for any server that forces HTTPS — automated traffic hits port 80, gets a 301, and Netdata flags it as a WARNING even though nothing is wrong.
|
|
|
|
## The Short Answer
|
|
|
|
```bash
|
|
sudo /etc/netdata/edit-config health.d/web_log.conf
|
|
```
|
|
|
|
Change the `warn` line in the `web_log_1m_redirects` template to:
|
|
|
|
```bash
|
|
warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? ( 1 ) : ( 80 )) ) : ( 0 )
|
|
```
|
|
|
|
Then reload:
|
|
|
|
```bash
|
|
netdatacli reload-health
|
|
```
|
|
|
|
## Background
|
|
|
|
Production nodes forcing HTTPS see a lot of 301s. The default Netdata threshold is too sensitive for sites with a high bot-to-human ratio — it was designed for environments where redirects are unexpected, not standard operating procedure.
|
|
|
|
The tuned logic warns only when there are more than 120 requests/minute AND redirects exceed 80% of traffic (dropping to 1% once already in WARNING state to prevent flapping).
|
|
|
|
## Steps
|
|
|
|
1. Identify what's actually generating the redirects
|
|
|
|
```bash
|
|
# Check status code distribution
|
|
awk '{print $9}' /var/log/apache2/access.log | sort | uniq -c | sort -nr
|
|
|
|
# Top redirected URLs
|
|
awk '$9 == "301" {print $7}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head -n 10
|
|
```
|
|
|
|
2. Open the Netdata health config
|
|
|
|
```bash
|
|
sudo /etc/netdata/edit-config health.d/web_log.conf
|
|
```
|
|
|
|
3. Find the `web_log_1m_redirects` template and update the `warn` line
|
|
|
|
```bash
|
|
warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? ( 1 ) : ( 80 )) ) : ( 0 )
|
|
```
|
|
|
|
4. Reload without restarting the service
|
|
|
|
```bash
|
|
netdatacli reload-health
|
|
```
|
|
|
|
5. Verify the alert cleared
|
|
|
|
```bash
|
|
curl -s http://localhost:19999/api/v1/alarms?all | grep -A 15 "web_log_1m_redirec"
|
|
```
|
|
|
|
## Gotchas & Notes
|
|
|
|
- **Ubuntu/Debian only:** The `edit-config` path and `apache2` log location are Debian-specific. On Fedora/RHEL the log is at `/var/log/httpd/access_log`.
|
|
- **The 1% recovery threshold is intentional:** Without it, the alert will flap between WARNING and CLEAR constantly on busy sites. The hysteresis keeps it stable once triggered.
|
|
- **Adjust the 120 req/min floor to your traffic:** Low-traffic sites may need a lower threshold; high-traffic sites may need higher.
|
|
|
|
## See Also
|
|
|
|
- Netdata service monitoring
|