162 lines
5.5 KiB
Markdown
162 lines
5.5 KiB
Markdown
---
|
|
title: "Deploying Netdata to a New Server"
|
|
domain: selfhosting
|
|
category: monitoring
|
|
tags: [netdata, monitoring, email, notifications, netdata-cloud, ubuntu, debian, n8n]
|
|
status: published
|
|
created: 2026-03-18
|
|
updated: 2026-03-22
|
|
---
|
|
|
|
# Deploying Netdata to a New Server
|
|
|
|
This covers the full Netdata setup for a new server in the fleet: install, email notification config, n8n webhook integration, and Netdata Cloud claim. Applies to Ubuntu/Debian servers.
|
|
|
|
## 1. Install Prerequisites
|
|
|
|
Install `jq` before anything else. It is required by the `custom_sender()` function in `health_alarm_notify.conf` to build the JSON payload sent to the n8n webhook. **If `jq` is missing, the webhook will fire with an empty body and n8n alert emails will have no information in them.**
|
|
|
|
```bash
|
|
apt install -y jq
|
|
```
|
|
|
|
Verify:
|
|
|
|
```bash
|
|
jq --version
|
|
```
|
|
|
|
## 2. Install Netdata
|
|
|
|
Use the official kickstart script:
|
|
|
|
```bash
|
|
wget -O /tmp/netdata-install.sh https://get.netdata.cloud/kickstart.sh
|
|
sh /tmp/netdata-install.sh --non-interactive --stable-channel --disable-telemetry
|
|
```
|
|
|
|
Verify it's running:
|
|
|
|
```bash
|
|
systemctl is-active netdata
|
|
curl -s http://localhost:19999/api/v1/info | python3 -c "import sys,json; d=json.load(sys.stdin); print('Netdata', d['version'])"
|
|
```
|
|
|
|
## 3. Configure Email Notifications
|
|
|
|
Copy the default config and set the three required values:
|
|
|
|
```bash
|
|
cp /usr/lib/netdata/conf.d/health_alarm_notify.conf /etc/netdata/health_alarm_notify.conf
|
|
```
|
|
|
|
Edit `/etc/netdata/health_alarm_notify.conf`:
|
|
|
|
```ini
|
|
EMAIL_SENDER="netdata@majorshouse.com"
|
|
SEND_EMAIL="YES"
|
|
DEFAULT_RECIPIENT_EMAIL="marcus@majorshouse.com"
|
|
```
|
|
|
|
Or apply with `sed` in one shot:
|
|
|
|
```bash
|
|
sed -i 's/^#\?EMAIL_SENDER=.*/EMAIL_SENDER="netdata@majorshouse.com"/' /etc/netdata/health_alarm_notify.conf
|
|
sed -i 's/^#\?SEND_EMAIL=.*/SEND_EMAIL="YES"/' /etc/netdata/health_alarm_notify.conf
|
|
sed -i 's/^#\?DEFAULT_RECIPIENT_EMAIL=.*/DEFAULT_RECIPIENT_EMAIL="marcus@majorshouse.com"/' /etc/netdata/health_alarm_notify.conf
|
|
```
|
|
|
|
Restart and test:
|
|
|
|
```bash
|
|
systemctl restart netdata
|
|
/usr/libexec/netdata/plugins.d/alarm-notify.sh test 2>&1 | grep -E '(OK|FAILED|email)'
|
|
```
|
|
|
|
You should see three `# OK` lines (WARNING → CRITICAL → CLEAR test cycle) and confirmation that email was sent to `marcus@majorshouse.com`.
|
|
|
|
> [!note] Delivery via local Postfix
|
|
> Email is relayed through the server's local Postfix instance. Ensure Postfix is installed and `/usr/sbin/sendmail` resolves.
|
|
|
|
## 4. Configure n8n Webhook Notifications
|
|
|
|
Copy the `health_alarm_notify.conf` from an existing server (e.g. majormail) which contains the `custom_sender()` function. This sends enriched JSON payloads to the n8n webhook at `https://n8n.majorshouse.com/webhook/netdata-alert`.
|
|
|
|
> [!warning] jq required
|
|
> The `custom_sender()` function uses `jq` to build the JSON payload. If `jq` is not installed, `payload` will be empty, curl will send `Content-Length: 0`, and n8n will produce alert emails with `Host: unknown`, blank alert/value fields, and `Status: UNKNOWN`. Always install `jq` first (Step 1).
|
|
|
|
After deploying the config, run a test to confirm the webhook fires correctly:
|
|
|
|
```bash
|
|
systemctl restart netdata
|
|
/usr/libexec/netdata/plugins.d/alarm-notify.sh test 2>&1 | grep -E '(custom|n8n|OK|FAILED)'
|
|
```
|
|
|
|
Verify in n8n that the latest execution shows a non-empty body with `hostname`, `alarm`, and `status` fields populated.
|
|
|
|
## 5. Claim to Netdata Cloud
|
|
|
|
Get the claim command from **Netdata Cloud → Space Settings → Nodes → Add Nodes**. It will look like:
|
|
|
|
```bash
|
|
wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh
|
|
sh /tmp/netdata-kickstart.sh --stable-channel \
|
|
--claim-token <token> \
|
|
--claim-rooms <room-id> \
|
|
--claim-url https://app.netdata.cloud
|
|
```
|
|
|
|
Verify the claim was accepted:
|
|
|
|
```bash
|
|
cat /var/lib/netdata/cloud.d/claimed_id
|
|
```
|
|
|
|
A UUID will be present if claimed successfully. The node should appear in Netdata Cloud within ~60 seconds.
|
|
|
|
## 6. Verify Alerts
|
|
|
|
Check that no unexpected alerts are active after setup:
|
|
|
|
```bash
|
|
curl -s 'http://localhost:19999/api/v1/alarms?active' | python3 -c "
|
|
import sys, json
|
|
d = json.load(sys.stdin)
|
|
active = [v for v in d.get('alarms', {}).values() if v.get('status') not in ('CLEAR', 'UNINITIALIZED', 'UNDEFINED')]
|
|
print(f'{len(active)} active alert(s)')
|
|
for v in active:
|
|
print(f' [{v[\"status\"]}] {v[\"name\"]} on {v[\"chart\"]}')
|
|
"
|
|
```
|
|
|
|
## Fleet-wide Alert Check
|
|
|
|
To audit all servers at once (requires Tailscale SSH access):
|
|
|
|
```bash
|
|
for host in majorlab majorhome majormail majordiscord majortoot majorlinux tttpod dca teelia; do
|
|
echo "=== $host ==="
|
|
ssh root@$host "curl -s 'http://localhost:19999/api/v1/alarms?active' | python3 -c \
|
|
\"import sys,json; d=json.load(sys.stdin); active=[v for v in d.get('alarms',{}).values() if v.get('status') not in ('CLEAR','UNINITIALIZED','UNDEFINED')]; print(str(len(active))+' active')\""
|
|
done
|
|
```
|
|
|
|
## Fleet-wide jq Audit
|
|
|
|
To check that all servers with `custom_sender` have `jq` installed:
|
|
|
|
```bash
|
|
for host in majorlab majorhome majormail majordiscord majortoot majorlinux tttpod dca teelia; do
|
|
echo -n "=== $host: "
|
|
ssh -o ConnectTimeout=5 root@$host \
|
|
'has_cs=$(grep -l "custom_sender\|n8n.majorshouse.com" /etc/netdata/health_alarm_notify.conf 2>/dev/null | wc -l); has_jq=$(which jq 2>/dev/null && echo yes || echo NO); echo "custom_sender=$has_cs jq=$has_jq"'
|
|
done
|
|
```
|
|
|
|
Any server showing `custom_sender=1 jq=NO` needs `apt install -y jq` immediately.
|
|
|
|
## Related
|
|
|
|
- [Tuning Netdata Web Log Alerts](tuning-netdata-web-log-alerts.md)
|
|
- [Tuning Netdata Docker Health Alarms](netdata-docker-health-alarm-tuning.md)
|