Add troubleshooting article: wget/curl URLs with special characters

Covers shell quoting for URLs containing &, ?, #, and other characters that Bash interprets as operators. Common gotcha when downloading from CDNs with token-based URLs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:18:34 -04:00
parent 6999b4ae6d
commit 4f0d18575a
5 changed files with 117 additions and 7 deletions
--- a/05-troubleshooting/index.md
+++ b/05-troubleshooting/index.md
@@ -1,6 +1,6 @@
 ---
 created: 2026-03-15T06:37
-updated: 2026-04-07T10:48
+updated: 2026-04-08
 ---
 # 🔧 General Troubleshooting

@@ -17,6 +17,7 @@ Practical fixes for common Linux, networking, and application problems.
 - [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md)
 - [ISP SNI Filtering & Caddy](isp-sni-filtering-caddy.md)
 - [yt-dlp YouTube JS Challenge Fix](yt-dlp-fedora-js-challenge.md)
+- [wget/curl: URLs with Special Characters Fail in Bash](wget-url-special-characters.md)

 ## ⚙️ Ansible & Fleet Management
 - [SSH Timeout During dnf upgrade on Fedora Hosts](ansible-ssh-timeout-dnf-upgrade.md)
--- a/05-troubleshooting/wget-url-special-characters.md
+++ b/05-troubleshooting/wget-url-special-characters.md
@@ -0,0 +1,100 @@
+---
+title: "wget/curl: URLs with Special Characters Fail in Bash"
+domain: troubleshooting
+category: general
+tags: [wget, curl, bash, shell, quoting, url]
+status: published
+created: 2026-04-08
+updated: 2026-04-08
+---
+# wget/curl: URLs with Special Characters Fail in Bash
+
+## Problem
+
+Downloading a URL that contains `&`, `=`, `#`, `?`, or other shell-meaningful characters fails with cryptic errors when the URL is not properly quoted:
+
+```bash
+wget -O output.mp4 https://cdn.example.com/video%20file.mp4?secure=abc123&token=xyz
+```
+
+Bash interprets `&` as a background operator, splitting the command:
+
+```
+bash: token=xyz: command not found
+```
+
+The download either fails outright or downloads only a partial/error page (e.g., 868 bytes instead of 2 GB).
+
+---
+
+## Root Cause
+
+Bash treats several URL-common characters as shell operators:
+
+| Character | Shell Meaning | URL Meaning |
+|---|---|---|
+| `&` | Run previous command in background | Query parameter separator |
+| `?` | Single-character glob wildcard | Start of query string |
+| `#` | Comment (rest of line ignored) | Fragment identifier |
+| `=` | Variable assignment (in some contexts) | Key-value separator |
+| `%` | Job control (`%1`, `%2`) | URL encoding prefix |
+| `!` | History expansion (in interactive shells) | Rarely used in URLs |
+
+If the URL is unquoted, Bash processes these characters before `wget` or `curl` ever sees them.
+
+---
+
+## Fix
+
+**Always single-quote URLs** passed to `wget` or `curl`:
+
+```bash
+wget -O '/path/to/output file.mp4' 'https://cdn.example.com/path/video%20file.mp4?secure=abc123&token=xyz'
+```
+
+Single quotes prevent **all** shell interpretation — no variable expansion, no globbing, no operator parsing. The URL reaches `wget` exactly as written.
+
+### When to use double quotes instead
+
+If the URL contains a shell variable (e.g., a token stored in `$TOKEN`), use double quotes:
+
+```bash
+wget -O output.mp4 "https://cdn.example.com/file.mp4?secure=${TOKEN}&expires=9999"
+```
+
+Double quotes allow variable expansion but still protect `&`, `?`, and `#` from shell interpretation.
+
+### Output filename quoting
+
+The `-O` filename also needs quoting if it contains spaces or special characters:
+
+```bash
+wget -O '/plex/plex/DF Direct Q+A #258.mp4' 'https://example.com/video.mp4'
+```
+
+---
+
+## Quick Reference
+
+| Scenario | Quoting |
+|---|---|
+| Static URL, no variables | Single quotes: `'https://...'` |
+| URL with shell variable | Double quotes: `"https://...${VAR}"` |
+| Output path with spaces | Single or double quotes around `-O` path |
+| URL in a script variable | Assign with double quotes: `URL="https://..."`, then `wget "$URL"` |
+
+---
+
+## Common Symptoms of Unquoted URLs
+
+- `bash: <partial-url>: command not found` — `&` split the command
+- Download completes instantly with a tiny file (error page, not the real content)
+- `wget` reports success but the file is corrupt or truncated
+- `No such file or directory` errors on URL fragments
+- History expansion errors (`!` in URL triggers `bash: !...: event not found`)
+
+---
+
+## See Also
+
+- [Bash Scripting Patterns](../01-linux/shell-scripting/bash-scripting-patterns.md) — general shell quoting and safety patterns
--- a/README.md
+++ b/README.md
@@ -6,8 +6,8 @@ updated: 2026-04-07T21:59

 > A growing reference of Linux, self-hosting, open source, streaming, and troubleshooting guides. Written by MajorLinux. Used by MajorTwin.
 >
-**Last updated:** 2026-04-03
-**Article count:** 76
+**Last updated:** 2026-04-08
+**Article count:** 74

 ## Domains

@@ -17,7 +17,7 @@ updated: 2026-04-07T21:59
 | 🏠 Self-Hosting & Homelab | `02-selfhosting/` | 21 |
 | 🔓 Open Source Tools | `03-opensource/` | 10 |
 | 🎙️ Streaming & Podcasting | `04-streaming/` | 2 |
-| 🔧 General Troubleshooting | `05-troubleshooting/` | 26 |
+| 🔧 General Troubleshooting | `05-troubleshooting/` | 29 |

 ---

@@ -152,6 +152,7 @@ updated: 2026-04-07T21:59
 - [Custom Fail2ban Jail: Apache Directory Scanning](05-troubleshooting/security/apache-dirscan-fail2ban-jail.md) — blocking directory scanners and junk HTTP methods
 - [ClamAV Safe Scheduling on Live Servers](05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md) — preventing clamscan CPU spikes with nice and ionice
 - [Systemd Session Scope Fails at Login](05-troubleshooting/systemd/session-scope-failure-at-login.md) — fixing session-cN.scope failures during login
+- [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) — fixing broken downloads caused by unquoted URLs with &, ?, # characters

 ---

@@ -159,6 +160,7 @@ updated: 2026-04-07T21:59

 | Date | Article | Domain |
 |---|---|---|
+| 2026-04-08 | [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) | Troubleshooting |
 | 2026-04-07 | [SSH Config & Key Management](01-linux/networking/ssh-config-key-management.md) | Linux |
 | 2026-04-07 | [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) | Troubleshooting |
 | 2026-04-07 | [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md) | Troubleshooting |
--- a/SUMMARY.md
+++ b/SUMMARY.md
@@ -1,6 +1,10 @@
 ---
 created: 2026-04-02T16:03
+<<<<<<< HEAD
 updated: 2026-04-07T10:48
+=======
+updated: 2026-04-08
+>>>>>>> 4dc77d4 (Add troubleshooting article: wget/curl URLs with special characters)
 ---
 * [Home](index.md)
 * [Linux & Sysadmin](01-linux/index.md)
@@ -81,3 +85,4 @@ updated: 2026-04-07T10:48
    * [Ansible: SSH Timeout During dnf upgrade on Fedora Hosts](05-troubleshooting/ansible-ssh-timeout-dnf-upgrade.md)
    * [Fedora Networking & Kernel Troubleshooting](05-troubleshooting/fedora-networking-kernel-recovery.md)
    * [Systemd Session Scope Fails at Login](05-troubleshooting/systemd/session-scope-failure-at-login.md)
+    * [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md)
--- a/index.md
+++ b/index.md
@@ -6,8 +6,8 @@ updated: 2026-04-07T21:59

 > A growing reference of Linux, self-hosting, open source, streaming, and troubleshooting guides. Written by MajorLinux. Used by MajorTwin.
 >
-> **Last updated:** 2026-04-07
-> **Article count:** 73
+> **Last updated:** 2026-04-08
+> **Article count:** 74

 ## Domains

@@ -17,7 +17,7 @@ updated: 2026-04-07T21:59
 | 🏠 Self-Hosting & Homelab | `02-selfhosting/` | 21 |
 | 🔓 Open Source Tools | `03-opensource/` | 10 |
 | 🎙️ Streaming & Podcasting | `04-streaming/` | 2 |
-| 🔧 General Troubleshooting | `05-troubleshooting/` | 28 |
+| 🔧 General Troubleshooting | `05-troubleshooting/` | 29 |


 ---
@@ -155,6 +155,7 @@ updated: 2026-04-07T21:59
 - [Custom Fail2ban Jail: Apache Directory Scanning](05-troubleshooting/security/apache-dirscan-fail2ban-jail.md) — blocking directory scanners and junk HTTP methods
 - [ClamAV Safe Scheduling on Live Servers](05-troubleshooting/security/clamscan-cpu-spike-nice-ionice.md) — preventing clamscan CPU spikes with nice and ionice
 - [Systemd Session Scope Fails at Login](05-troubleshooting/systemd/session-scope-failure-at-login.md) — fixing session-cN.scope failures during login
+- [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) — fixing broken downloads caused by unquoted URLs with &, ?, # characters


 ---
@@ -163,6 +164,7 @@ updated: 2026-04-07T21:59

 | Date | Article | Domain |
 |---|---|---|
+| 2026-04-08 | [wget/curl: URLs with Special Characters Fail in Bash](05-troubleshooting/wget-url-special-characters.md) | Troubleshooting |
 | 2026-04-07 | [SSH Config & Key Management](01-linux/networking/ssh-config-key-management.md) | Linux |
 | 2026-04-07 | [Windows OpenSSH: WSL Default Shell Breaks Remote Commands](05-troubleshooting/networking/windows-openssh-wsl-default-shell-breaks-remote-commands.md) | Troubleshooting |
 | 2026-04-07 | [Windows OpenSSH Server (sshd) Stops After Reboot](05-troubleshooting/networking/windows-sshd-stops-after-reboot.md) | Troubleshooting |