Originally published at: Throwing Random Arguments at System Binaries: Real Segfaults vs. Ticket Noise – Fedora Community Blog
Why Would Anyone Do This?
Two and half reasons:
- Smoke testing – You want to know if your system commands actually work, not just when you run them the way the docs say, but when users (or their scripts) feed them garbage.
- AI is excellent at generating potential edge cases, and tracking systems are already all too eager to collect new tickets. I’m being careful not to dump every AI finding into Bugzilla; I don’t want to clutter the backlog and mainly waste developer time on theoretical bugs. Or Should I?
Plus, segfaults don’t lie – either the system crashed or it didn’t, and those are the issues that actually deserve the ticket.
Throwing Random Arguments at System Binaries Until They Crash
Script to do the work:
A pretty straightforward bash script, vibing with AI-generated chaos.
- Grab all binaries from
/usr/binand/usr/sbin - Parse
--helpfor flags (--whatever,-x, you know the drill) - Pick random combos of those flags (1-4 per run)
- Feed them garbage: broken JSON/XML, binary junk, path traversal attempts, format strings, absurdly long lines
Only logs actual crashes – SIGSEGV, SIGABRT, SIGILL, SIGBUS. Exit code 1 from bad args gets ignored.
Core logic looks like this:
# Extract flags from --help flags=$(timeout 3s "$bin" --help 2>&1 | grep -aoE -e '--[a-zA-Z0-9_-]+' -e '-[a-zA-Z]' | grep -avE 'help|version|usage')Pick random flags (1-4 of them)
chosen=$(echo “$flags” | shuf -n $((1 + RANDOM % 4)))
Add a random test file
fuzz_file=“$WORKSPACE/$(random_pick: bad.json, random.bin, longline.txt, …)”
Run it
timeout 5s “$bin” $chosen $fuzz_file
Script skips the obvious no go zones – package managers, rm, network tools, editors. I’m glad to see the script finish with the machine still answering.
Try It Yourself!
Source: run-them-all
Run via Testing Farm (how-to):
testing-farm-public request --test-type fmf \
--git-url https://forge.fedoraproject.org/quality/fmf-tests.git \
--git-ref main \
--compose Fedora-Cloud-Base-AmazonEC2.x86_64-44-1.3 \
--arch x86_64 \
--test run-them-all \
--context "force=yes" \
--plan /plans/all
Run locally: Just use the try-all.sh binary from the test, no requirement needed, run:
curl -sSLO https://forge.fedoraproject.org/quality/fmf-tests/raw/branch/main/system-in-use/run-them-all/try-all-binaries-help-options.sh && bash try-all-binaries-help-options.sh
My results: Testing Farm Artifacts (Fedora 44 RC compose 1.3, ~950 binaries tested)
What I Found
grub2-mkrescue (bootloader utilities), perl (half the system depends on it), eqn from groff (man pages break without it).
What coredumpctl caught:
TIME PID UID GID SIG COREFILE EXE SIZE
Mon 2026-04-20 12:08:09 UTC 52378 0 0 SIGABRT present /usr/bin/edgepaint 80.7K
Mon 2026-04-20 12:08:22 UTC 58926 0 0 SIGSEGV present /usr/bin/eqn 57.8K
Mon 2026-04-20 12:08:41 UTC 77687 0 0 SIGSEGV present /usr/bin/gdbm_dump 23.9K
Mon 2026-04-20 12:09:01 UTC 97901 0 0 SIGSEGV present /usr/bin/grub2-mkrescue 64.7K
Mon 2026-04-20 12:09:08 UTC 106904 0 0 SIGSEGV present /usr/bin/gtshapprox 48.3K
Mon 2026-04-20 12:10:15 UTC 161697 0 0 SIGABRT present /usr/bin/perl5.42.1 94K
Breakdown:
- edgepaint (graphviz) – 4× SIGABRT
- gtshapprox (graphviz) – 3× SIGSEGV
- perl – 4× SIGABRT
- eqn (groff) – 1× SIGSEGV
- gdbm_dump (gdbm) – 1× SIGSEGV
- grub2-mkrescue (grub2) – 1× SIGSEGV
Reproducers
Run it on your side to see the reproducers in action. I’m curious to hear your thoughts on these major findings.
MY CRASHES:
COMMAND: /usr/bin/edgepaint -s -o --random_seed --angle --lightness -v --accuracy --share_endpoint
COMMAND: /usr/bin/edgepaint -v -s --accuracy --color_scheme -o /root/fuzz_lab/empty.dat
COMMAND: /usr/bin/edgepaint --share_endpoint -o --angle -v -s --random_seed --accuracy --color_scheme -
COMMAND: /usr/bin/edgepaint -s --lightness --share_endpoint -o -v --angle --random_seed
COMMAND: /usr/bin/efibootdump --guid -g -f /root/fuzz_lab/large.dat
COMMAND: /usr/bin/eqn -C -f -M -d -v -m -T -s /root/fuzz_lab/fake.png
COMMAND: /usr/bin/gdbm_dump --format -
COMMAND: /usr/bin/grub2-file --is-x86-knetbsd /root/fuzz_lab/longline.txt
COMMAND: /usr/bin/grub2-file --is-x86-knetbsd /root/fuzz_lab/bad.json
COMMAND: /usr/bin/gtshapprox --flat -c /root/fuzz_lab/fake.jpg
COMMAND: /usr/bin/gtshapprox -v -f --verbose -c /root/fuzz_lab/cmd.txt
COMMAND: /usr/bin/gtshapprox -n -h --flat -l --closed -v /root/fuzz_lab/gzip.dat
COMMAND: /usr/bin/gtshapprox --flat --number --log --cost --closed -
COMMAND: /usr/bin/gtshapprox --log -l -f --keep -
COMMAND: /usr/bin/mkfs.xfs -q /root/fuzz_lab/bad_utf8.txt
COMMAND: /usr/bin/mkfs.xfs -K -L -m /root/fuzz_lab/paths.txt
COMMAND: /usr/bin/perl -V -F -T -S -u -p -f -E -W -I /root/fuzz_lab/bad.json
COMMAND: /usr/bin/perl -F -e -n -a -l -u -c -
COMMAND: /usr/bin/perl -S -u -s -E -I -l
COMMAND: /usr/bin/perl -s -p -n -u -D -E -d
COMMAND: /usr/bin/tree -P --filelimit --matchdirs --hyperlink -i /root/fuzz_lab/paths.txtLook, these are edge cases. Nobody’s actually running edgepaint --wtf malformed.json in prod. But segfaults are segfaults – the binary should bail with “invalid option” or “bad input”, not dump core.
Now What?
So I’ve got a pile of crashes. Some in critical components. All reproducible.
File bugs for all of them? That’s a lot of BZ tickets for “yes hm this crashes if you feed it random garbage with weird flags”. Developers have better things to do.
Ignore them? They’re real bugs. And some of these are in grub2 and perl – not exactly throwaway packages.
Still figuring that out.
My Dilemma During Testing on Fedora 44 RC compose 1.X