Add bulk-run actions and per-source rate limiting

Run-all buttons (per target and per investigation) queue every applicable,
not-already-in-flight source via a new FindingDispatcher, re-attempting
prior failures automatically. RunFinding now applies a per-source
RateLimited queue middleware so a bulk run can't hammer the free public
APIs (ip-api, crt.sh, whois, sherlock, theharvester) it depends on;
$tries raised to 30 to survive rate-limiter releases, with a failed()
handler so exhausted retries still resolve the Finding instead of leaving
it stuck pending.

Also fixes CrtShRunner treating crt.sh's intermittent 404/502 (a known
flaky free service) as fatal on the first try — added a bounded retry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
cedry2k1992
2026-07-20 11:29:26 +03:00
co-authored by Claude Sonnet 5
parent 853686f3f3
commit e4fa57c3d6
13 changed files with 387 additions and 5 deletions
+7 -1
View File
@@ -16,14 +16,20 @@ class CrtShRunner implements Runner
throw new RunnerException('crt.sh only supports "domain" targets.');
}
// crt.sh is a free, frequently overloaded public service that
// intermittently returns 404/502/503 for queries that succeed
// moments later (confirmed by hand: same query, 404 then 200 with
// no code changes) — worth a few retries before treating it as a
// real failure.
$response = Http::timeout((int) config('osint.run_timeout'))
->retry(3, 2000, throw: false)
->get('https://crt.sh/', [
'q' => $target->value,
'output' => 'json',
]);
if (! $response->successful()) {
throw new RunnerException('crt.sh request failed with status '.$response->status().'.');
throw new RunnerException('crt.sh request failed with status '.$response->status().' after retries.');
}
// crt.sh returns an empty body (not valid JSON) when there are no matches.