root/osint
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add bulk-run actions and per-source rate limiting

Browse Source 
Run-all buttons (per target and per investigation) queue every applicable,
not-already-in-flight source via a new FindingDispatcher, re-attempting
prior failures automatically. RunFinding now applies a per-source
RateLimited queue middleware so a bulk run can't hammer the free public
APIs (ip-api, crt.sh, whois, sherlock, theharvester) it depends on;
$tries raised to 30 to survive rate-limiter releases, with a failed()
handler so exhausted retries still resolve the Finding instead of leaving
it stuck pending.

Also fixes CrtShRunner treating crt.sh's intermittent 404/502 (a known
flaky free service) as fatal on the first try — added a bounded retry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
cedry2k1992
2026-07-20 11:29:26 +03:00
co-authored by Claude Sonnet 5
parent 853686f3f3
commit e4fa57c3d6
13 changed files with 387 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deploy/systemd/osint-queue-worker.service
+1 -1
View File
@@ -6,7 +6,7 @@ StartLimitIntervalSec=0
[Service]
Type=simple
ExecStart=/usr/bin/php /home/m105/projects/osint/artisan queue:work --queue=default --sleep=3 --tries=1 --timeout=180 --max-time=3600
ExecStart=/usr/bin/php /home/m105/projects/osint/artisan queue:work --queue=default --sleep=3 --tries=3 --timeout=180 --max-time=3600
WorkingDirectory=/home/m105/projects/osint
Environment="PATH=/home/m105/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Restart=always