Add bulk-run actions and per-source rate limiting

Run-all buttons (per target and per investigation) queue every applicable,
not-already-in-flight source via a new FindingDispatcher, re-attempting
prior failures automatically. RunFinding now applies a per-source
RateLimited queue middleware so a bulk run can't hammer the free public
APIs (ip-api, crt.sh, whois, sherlock, theharvester) it depends on;
$tries raised to 30 to survive rate-limiter releases, with a failed()
handler so exhausted retries still resolve the Finding instead of leaving
it stuck pending.

Also fixes CrtShRunner treating crt.sh's intermittent 404/502 (a known
flaky free service) as fatal on the first try — added a bounded retry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
cedry2k1992
2026-07-20 11:29:26 +03:00
co-authored by Claude Sonnet 5
parent 853686f3f3
commit e4fa57c3d6
13 changed files with 387 additions and 5 deletions
+10 -1
View File
@@ -2,6 +2,8 @@
namespace App\Providers;
use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\RateLimiter;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
@@ -19,6 +21,13 @@ class AppServiceProvider extends ServiceProvider
*/
public function boot(): void
{
//
// Self-imposed limits so bulk-running lookups doesn't hammer the
// free public infrastructure these runners depend on. ip-api.com's
// limit (45/min) is theirs; the rest are courtesy limits of ours.
RateLimiter::for('osint-runner:ip-api', fn () => Limit::perMinute(45));
RateLimiter::for('osint-runner:crtsh', fn () => Limit::perMinute(30));
RateLimiter::for('osint-runner:whois', fn () => Limit::perMinute(20));
RateLimiter::for('osint-runner:sherlock', fn () => Limit::perMinute(10));
RateLimiter::for('osint-runner:theharvester', fn () => Limit::perMinute(5));
}
}