Add 5 wiki articles from 2026-04-17/18 work
- ghost-smtp-mailgun-setup: two-system email config (newsletter API + transactional SMTP) - firewalld-fleet-hardening: Fedora fleet firewall audit-and-harden pattern with Ansible - clamav-fleet-deployment: fleet deployment with nice/ionice throttling + quarantine - ansible-check-mode-false-positives: when: not ansible_check_mode guard for verify/assert tasks - ghost-emailanalytics-lag-warning: submitted status, lag counter, fetchMissing skip explained
This commit is contained in:
135
05-troubleshooting/ansible-check-mode-false-positives.md
Normal file
135
05-troubleshooting/ansible-check-mode-false-positives.md
Normal file
@@ -0,0 +1,135 @@
|
||||
---
|
||||
title: Ansible Check Mode False Positives in Verify/Assert Tasks
|
||||
domain: selfhosting
|
||||
category: troubleshooting
|
||||
tags:
|
||||
- ansible
|
||||
- check-mode
|
||||
- dry-run
|
||||
- assert
|
||||
- handlers
|
||||
- troubleshooting
|
||||
status: published
|
||||
created: 2026-04-18
|
||||
updated: 2026-04-18T11:13
|
||||
---
|
||||
# Ansible Check Mode False Positives in Verify/Assert Tasks
|
||||
|
||||
## The Problem
|
||||
|
||||
`ansible-playbook --check` (dry-run mode) reports failures on verify and assert tasks that depend on handler-triggered side effects — even when the playbook is correct and would succeed on a real run.
|
||||
|
||||
**Symptom:** Running `--check` produces errors like:
|
||||
|
||||
```
|
||||
TASK [Assert hardened settings are active] ***
|
||||
fatal: [host]: FAILED! => {
|
||||
"assertion": "'permitrootlogin without-password' in sshd_effective.stdout",
|
||||
"msg": "One or more SSH hardening settings not effective"
|
||||
}
|
||||
```
|
||||
|
||||
But a real run (`ansible-playbook` without `--check`) succeeds cleanly.
|
||||
|
||||
## Why It Happens
|
||||
|
||||
In check mode, Ansible simulates tasks but **does not execute handlers**. This means:
|
||||
|
||||
1. A config file task reports `changed` (it would deploy the file)
|
||||
2. The handler (`reload sshd`, `reload firewalld`, etc.) is **never fired**
|
||||
3. A subsequent verify task runs `sshd -T` or `ufw status verbose` against the **current live state** (pre-change)
|
||||
4. The assert compares the current state against the expected post-change state and fails
|
||||
|
||||
The verify task is reading reality accurately — the change hasn't happened yet — but the failure is misleading. It suggests the playbook is broken when it's actually correct.
|
||||
|
||||
## The Fix
|
||||
|
||||
Guard verify and assert tasks that depend on handler side effects with `when: not ansible_check_mode`:
|
||||
|
||||
```yaml
|
||||
- name: Verify effective SSH settings post-reload
|
||||
ansible.builtin.command:
|
||||
cmd: sshd -T
|
||||
register: sshd_effective
|
||||
changed_when: false
|
||||
when: not ansible_check_mode # sshd hasn't reloaded in check mode
|
||||
|
||||
- name: Assert hardened settings are active
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- "'permitrootlogin without-password' in sshd_effective.stdout"
|
||||
- "'x11forwarding no' in sshd_effective.stdout"
|
||||
fail_msg: "SSH hardening settings not effective — check for conflicting config"
|
||||
when: not ansible_check_mode # result would be pre-change state
|
||||
```
|
||||
|
||||
This skips the verify/assert during check mode (where they'd produce false failures) while keeping them active on real runs (where they catch actual misconfigurations).
|
||||
|
||||
## When to Apply This Guard
|
||||
|
||||
Apply `when: not ansible_check_mode` to any task that:
|
||||
|
||||
- Reads the **active/effective state** of a service after a config change (`sshd -T`, `ufw status verbose`, `firewall-cmd --list-all`, `nginx -T`)
|
||||
- **Asserts** that the post-change state matches expectations
|
||||
- Depends on a **handler** having fired first (service reload, daemon restart)
|
||||
|
||||
Don't apply it to tasks that check pre-existing state (e.g., verifying a file exists before modifying it) — those are valid in check mode.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### SSH daemon verify
|
||||
|
||||
```yaml
|
||||
- name: Verify effective sshd settings
|
||||
ansible.builtin.command: sshd -T
|
||||
register: sshd_out
|
||||
changed_when: false
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Assert sshd hardening active
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- "'maxauthtries 3' in sshd_out.stdout"
|
||||
when: not ansible_check_mode
|
||||
```
|
||||
|
||||
### UFW status verify
|
||||
|
||||
```yaml
|
||||
- name: Show UFW status
|
||||
ansible.builtin.command: ufw status verbose
|
||||
register: ufw_status
|
||||
changed_when: false
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Confirm default deny incoming
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- "'Default: deny (incoming)' in ufw_status.stdout"
|
||||
when: not ansible_check_mode
|
||||
```
|
||||
|
||||
### nginx config verify
|
||||
|
||||
```yaml
|
||||
- name: Test nginx config
|
||||
ansible.builtin.command: nginx -t
|
||||
changed_when: false
|
||||
when: not ansible_check_mode
|
||||
```
|
||||
|
||||
## Trade-off
|
||||
|
||||
Guarding with `when: not ansible_check_mode` means check mode won't validate these assertions. The benefit — no false failures — outweighs the gap because:
|
||||
|
||||
- Check mode is showing you what *would* change, not whether the result is valid
|
||||
- Real runs still assert and will catch actual misconfigurations
|
||||
- The alternative (failing check runs) erodes trust in `--check` output
|
||||
|
||||
If you need to verify the effective post-change state in check mode, consider splitting the playbook into a deploy pass and a separate verify-only playbook run without `--check`.
|
||||
|
||||
## See Also
|
||||
|
||||
- [ssh-hardening-ansible-fleet](../02-selfhosting/security/ssh-hardening-ansible-fleet.md)
|
||||
- [ufw-firewall-management](../02-selfhosting/security/ufw-firewall-management.md)
|
||||
- [ansible-getting-started](../01-linux/shell-scripting/ansible-getting-started.md)
|
||||
106
05-troubleshooting/ghost-emailanalytics-lag-warning.md
Normal file
106
05-troubleshooting/ghost-emailanalytics-lag-warning.md
Normal file
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: Ghost EmailAnalytics Lag Warning — What It Means and When to Worry
|
||||
domain: selfhosting
|
||||
category: troubleshooting
|
||||
tags:
|
||||
- ghost
|
||||
- email
|
||||
- mailgun
|
||||
- emailanalytics
|
||||
- docker
|
||||
- troubleshooting
|
||||
status: published
|
||||
created: 2026-04-18
|
||||
updated: 2026-04-18T11:13
|
||||
---
|
||||
# Ghost EmailAnalytics Lag Warning — What It Means and When to Worry
|
||||
|
||||
## The Warning
|
||||
|
||||
Ghost logs a recurring warning every 5 minutes when its EmailAnalytics job falls behind:
|
||||
|
||||
```
|
||||
WARN [EmailAnalytics] Opened events processing is 738.0 minutes behind (threshold: 30)
|
||||
```
|
||||
|
||||
This is followed by:
|
||||
|
||||
```
|
||||
INFO [EmailAnalytics] Job complete - No events
|
||||
INFO [EmailAnalytics] Skipping fetchMissing because end (...) is before begin (...)
|
||||
```
|
||||
|
||||
The counter increments by 5 with every cycle. On a small newsletter, it will grow indefinitely and never reset on its own — until a subscriber opens an email or a new newsletter is sent.
|
||||
|
||||
## Why It Happens
|
||||
|
||||
Ghost's EmailAnalytics polls Mailgun every 5 minutes for new "opened" events. The cursor is anchored to the timestamp of the last email delivery. If no new opened events arrive from Mailgun, the cursor doesn't advance and the lag counter grows.
|
||||
|
||||
This is **expected behavior** when:
|
||||
- All subscribers have already opened (their open was recorded)
|
||||
- One or more subscribers have not opened the email and haven't opened any subsequent emails
|
||||
- There are no new emails to send
|
||||
|
||||
The lag counter = time since the last opened event was recorded, not time since the last email was sent.
|
||||
|
||||
## The `fetchMissing end == begin` Skip
|
||||
|
||||
```
|
||||
INFO [EmailAnalytics] Skipping fetchMissing because end (Fri Apr 17 2026 15:44:57 ...) is before begin (Fri Apr 17 2026 15:44:57 ...)
|
||||
```
|
||||
|
||||
This fires when the cursor window collapses to zero width — the start and end of the query window are identical. Ghost's guard clause skips a nonsensical zero-width Mailgun API call. This is not a bug or data loss — it's a safety check.
|
||||
|
||||
## What `status: submitted` Means
|
||||
|
||||
In Ghost's `emails` database table, all successfully sent newsletters show `status: submitted`. This is the normal terminal state after Ghost hands the email batch off to Mailgun. There is no `status: sent` — `submitted` = success.
|
||||
|
||||
You can verify delivery success by checking the counts:
|
||||
|
||||
```bash
|
||||
docker exec <db-container> mysql -u root -p<password> ghost \
|
||||
-e "SELECT subject, status, email_count, delivered_count, opened_count, failed_count FROM emails ORDER BY created_at DESC LIMIT 5;"
|
||||
```
|
||||
|
||||
A healthy result: `email_count == delivered_count`, `failed_count == 0`, regardless of `opened_count`.
|
||||
|
||||
## When to Actually Worry
|
||||
|
||||
The lag warning is **benign** in these cases:
|
||||
- `delivered_count == email_count` (all emails delivered)
|
||||
- `failed_count == 0`
|
||||
- Mailgun domain state is active
|
||||
- The warning appeared after a successful send and has been growing since
|
||||
|
||||
Investigate further if:
|
||||
- `delivered_count < email_count` — some emails never left Mailgun
|
||||
- `failed_count > 0` — delivery failures
|
||||
- The warning appeared immediately after a Ghost upgrade or Mailgun credential change
|
||||
- Mailgun Events API shows 0 delivered events (not just 0 opened events) for the send window
|
||||
|
||||
## Checking Mailgun Directly
|
||||
|
||||
If you suspect the lag reflects a real delivery problem, query Mailgun's Events API:
|
||||
|
||||
```bash
|
||||
# Check for delivered events in the send window
|
||||
curl -s --user "api:<your-mailgun-api-key>" \
|
||||
"https://api.mailgun.net/v3/<your-domain>/events?event=delivered&begin=<RFC2822-timestamp>&limit=10" \
|
||||
| python3 -m json.tool | grep -E "event|recipient|timestamp" | head -30
|
||||
```
|
||||
|
||||
If delivered events appear for your subscribers, Mailgun is working and the lag warning is purely cosmetic.
|
||||
|
||||
## How It Resolves
|
||||
|
||||
The lag warning self-resolves when:
|
||||
1. **A subscriber opens an email** — Mailgun returns an "opened" event, the cursor advances, lag resets
|
||||
2. **A new newsletter is sent** — the send triggers a fresh analytics cycle, cursor jumps forward
|
||||
3. **Manually resetting the cursor** — possible via direct DB update, but not recommended unless you understand the implications for analytics continuity
|
||||
|
||||
For small newsletters (2–10 subscribers) where one subscriber consistently doesn't open emails, the warning is permanent background noise between sends. It does not indicate data loss or misconfiguration.
|
||||
|
||||
## See Also
|
||||
|
||||
- [ghost-smtp-mailgun-setup](../02-selfhosting/services/ghost-smtp-mailgun-setup.md)
|
||||
- [debugging-broken-docker-containers](../02-selfhosting/docker/debugging-broken-docker-containers.md)
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
created: 2026-03-15T06:37
|
||||
updated: 2026-04-17T09:57
|
||||
updated: 2026-04-17T10:21
|
||||
---
|
||||
# 🔧 General Troubleshooting
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ tags:
|
||||
- powershell
|
||||
status: published
|
||||
created: 2026-04-03
|
||||
updated: 2026-04-07T21:55
|
||||
updated: 2026-04-14T14:27
|
||||
---
|
||||
|
||||
# Windows OpenSSH: WSL as Default Shell Breaks Remote Commands
|
||||
|
||||
@@ -10,7 +10,7 @@ tags:
|
||||
- majorrig
|
||||
status: published
|
||||
created: 2026-04-02
|
||||
updated: 2026-04-07T21:58
|
||||
updated: 2026-04-14T14:27
|
||||
---
|
||||
# Windows OpenSSH Server (sshd) Stops After Reboot
|
||||
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
---
|
||||
title: "yt-dlp YouTube JS Challenge Fix (Fedora)"
|
||||
title: yt-dlp YouTube JS Challenge Fix (Fedora)
|
||||
domain: troubleshooting
|
||||
category: general
|
||||
tags: [yt-dlp, fedora, youtube, javascript, deno]
|
||||
tags:
|
||||
- yt-dlp
|
||||
- fedora
|
||||
- youtube
|
||||
- javascript
|
||||
- deno
|
||||
status: published
|
||||
created: 2026-04-02
|
||||
updated: 2026-04-02
|
||||
updated: 2026-04-14T14:27
|
||||
---
|
||||
# yt-dlp YouTube JS Challenge Fix (Fedora)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user