Show HN: Forkrun – NUMA-aware shell parallelizer (50×–400× faster than parallel)

Summary

forkrun — NUMA-Aware Contention-Free Streaming Parallelization forkrun is a self-tuning, drop-in replacement for GNU Parallel and xargs -P that accelerates shell-based data preparation by 50×–400× on modern CPUs and scales linearly on NUMA architectures. forkrun achieves: 200,000+ batch dispatches/sec (vs ~500 for GNU Parallel) ~95–99% CPU utilization across all cores (vs ~6% for GNU Parallel) Near-zero cross-socket memory traffic (NUMA-aware “born-local” design) forkrun is built for high-frequency, low-latency workloads on deep NUMA hardware — a regime where existing tools leave most cores idle due to IPC overhead and cross-socket data migration. 🚀 Quick Start (Installation & Usage) forkrun is distributed as a single bash file with an embedded, self-extracting compiled C extension. There are no external dependencies (no Perl, no Python). Download and source it directly: source <(curl -sL https://raw.githubusercontent.com/jkool702/forkrun/main/frun.bash) (Note: Sourcing the script sets up the required C loadable builtins in your shell environment). Once sourced, frun acts as a drop-in parallelizer: frun my_bash_func < inputs.txt # parallelize custom bash functions natively! cat file_list | frun -k sed 's/old/new/' # pipe-based input, ordered output frun -k -s sort < records.tsv # stdin-passthrough, ordered output frun -s -I 'gzip -c >{ID}.gz' < raw_logs # stdin-passthrough, unique output names Verifiable Builds: The embedded C-extension is compiled and injected transparently via GitHub Actions. You can trace the git blame of the Base64 blob directly to the public CI workflow run that compiled forkrun_ring.c, guaranteeing the binary contains no hidden malicious code. ⚡ Benchmarks (14-core/28-thread i9-7940x, 100M lines) Workload forkrun GNU Parallel Speedup Notes Default (array + fully-quoted args, no-op) 24 M lines/s 58 k lines/s ~415× forkrun default mode Ordered output (-k, no-op) 24.5 M lines/s 57 k lines/s ~430× ordering is free in forkrun echo (line args) 22.6 ...