Deploying Netdata to a New Server

This covers the full Netdata setup for a new server in the fleet: install, email notification config, and Netdata Cloud claim. Applies to Ubuntu/Debian servers.

1. Install

Use the official kickstart script:

wget -O /tmp/netdata-install.sh https://get.netdata.cloud/kickstart.sh
sh /tmp/netdata-install.sh --non-interactive --stable-channel --disable-telemetry

Verify it's running:

systemctl is-active netdata
curl -s http://localhost:19999/api/v1/info | python3 -c "import sys,json; d=json.load(sys.stdin); print('Netdata', d['version'])"

2. Configure Email Notifications

Copy the default config and set the three required values:

cp /usr/lib/netdata/conf.d/health_alarm_notify.conf /etc/netdata/health_alarm_notify.conf

Edit /etc/netdata/health_alarm_notify.conf:

EMAIL_SENDER="netdata@majorshouse.com"
SEND_EMAIL="YES"
DEFAULT_RECIPIENT_EMAIL="marcus@majorshouse.com"

Or apply with sed in one shot:

sed -i 's/^#\?EMAIL_SENDER=.*/EMAIL_SENDER="netdata@majorshouse.com"/' /etc/netdata/health_alarm_notify.conf
sed -i 's/^#\?SEND_EMAIL=.*/SEND_EMAIL="YES"/' /etc/netdata/health_alarm_notify.conf
sed -i 's/^#\?DEFAULT_RECIPIENT_EMAIL=.*/DEFAULT_RECIPIENT_EMAIL="marcus@majorshouse.com"/' /etc/netdata/health_alarm_notify.conf

Restart and test:

systemctl restart netdata
/usr/libexec/netdata/plugins.d/alarm-notify.sh test 2>&1 | grep -E '(OK|FAILED|email)'

You should see three # OK lines (WARNING → CRITICAL → CLEAR test cycle) and confirmation that email was sent to marcus@majorshouse.com.

[!note] Delivery via local Postfix Email is relayed through the server's local Postfix instance. Ensure Postfix is installed and /usr/sbin/sendmail resolves.

3. Claim to Netdata Cloud

Get the claim command from Netdata Cloud → Space Settings → Nodes → Add Nodes. It will look like:

wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh
sh /tmp/netdata-kickstart.sh --stable-channel \
  --claim-token <token> \
  --claim-rooms <room-id> \
  --claim-url https://app.netdata.cloud

Verify the claim was accepted:

cat /var/lib/netdata/cloud.d/claimed_id

A UUID will be present if claimed successfully. The node should appear in Netdata Cloud within ~60 seconds.

4. Verify Alerts

Check that no unexpected alerts are active after setup:

curl -s 'http://localhost:19999/api/v1/alarms?active' | python3 -c "
import sys, json
d = json.load(sys.stdin)
active = [v for v in d.get('alarms', {}).values() if v.get('status') not in ('CLEAR', 'UNINITIALIZED', 'UNDEFINED')]
print(f'{len(active)} active alert(s)')
for v in active:
    print(f'  [{v[\"status\"]}] {v[\"name\"]} on {v[\"chart\"]}')
"

Fleet-wide Alert Check

To audit all servers at once (requires Tailscale SSH access):

for host in majorlab majorhome majormail majordiscord majortoot majorlinux tttpod dca teelia; do
  echo "=== $host ==="
  ssh root@$host "curl -s 'http://localhost:19999/api/v1/alarms?active' | python3 -c \
    \"import sys,json; d=json.load(sys.stdin); active=[v for v in d.get('alarms',{}).values() if v.get('status') not in ('CLEAR','UNINITIALIZED','UNDEFINED')]; print(str(len(active))+' active')\""
done

3.6 KiB Raw Blame History