Files
osint/deploy/systemd
cedry2k1992andClaude Sonnet 5 e4fa57c3d6 Add bulk-run actions and per-source rate limiting
Run-all buttons (per target and per investigation) queue every applicable,
not-already-in-flight source via a new FindingDispatcher, re-attempting
prior failures automatically. RunFinding now applies a per-source
RateLimited queue middleware so a bulk run can't hammer the free public
APIs (ip-api, crt.sh, whois, sherlock, theharvester) it depends on;
$tries raised to 30 to survive rate-limiter releases, with a failed()
handler so exhausted retries still resolve the Finding instead of leaving
it stuck pending.

Also fixes CrtShRunner treating crt.sh's intermittent 404/502 (a known
flaky free service) as fatal on the first try — added a bounded retry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 11:29:26 +03:00
..

Queue worker (systemd --user)

QUEUE_CONNECTION=database, so OSINT lookups only actually run while a queue:work process is alive. This unit supervises one as a systemd --user service — no root/sudo needed, survives crashes (Restart=always), and restarts hourly (--max-time=3600) to pick up code/config changes and avoid long-lived-process memory growth.

--timeout=180 is intentionally above config('osint.run_timeout') (120s, in config/osint.php) — the worker's timeout must exceed a runner's own timeout, or the worker kills the job before the runner gets a chance to.

Install

cp deploy/systemd/osint-queue-worker.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now osint-queue-worker.service

Requires lingering enabled so the service keeps running without an active login session: loginctl enable-linger $USER (needs sudo once, or ask your sysadmin — already enabled on this box).

Operate

systemctl --user status osint-queue-worker.service
journalctl --user -u osint-queue-worker.service -f
systemctl --user restart osint-queue-worker.service   # after deploying code/.env changes