Add Gravatar and DNS Records collectors

Two pure-PHP collectors, no Python/venv/subprocess needed:
- GravatarRunner (email): checks gravatar.com/avatar/{md5}?d=404 for a
  200 vs 404. Scoped to that binary signal since Gravatar's profile
  JSON endpoint now requires API auth even for real, public profiles
  (confirmed against Gravatar's own creator's email).
- DnsRunner (domain): dns_get_record() for A/AAAA/MX/NS/TXT/CNAME. Raw
  payload is naturally keyed by record type, which the existing
  generic list-renderer on the investigation page turns into one
  collapsible section per type with no UI changes needed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
cedry2k1992
2026-07-20 11:59:20 +03:00
co-authored by Claude Sonnet 5
parent bb15f13bac
commit 86f73c46b2
8 changed files with 296 additions and 3 deletions
+32 -3
View File
@@ -51,9 +51,9 @@ commands).
- **Targets** — the entity being investigated (person, domain, email,
username, IP, org), scoped to one investigation
- **Sources** — catalog of OSINT tools/techniques (seeded: Sherlock,
theHarvester, crt.sh, WHOIS, ip-api.com, Holehe — all 6 now have a
working runner). `sources.runner` is a display-only label; the real
mapping is `App\Osint\RunnerRegistry`
theHarvester, crt.sh, WHOIS, ip-api.com, Holehe, Gravatar, DNS
Records — all 8 now have a working runner). `sources.runner` is a
display-only label; the real mapping is `App\Osint\RunnerRegistry`
- **Findings** — result of running a source against a target; raw JSON
payload preserved plus a parsed summary
@@ -228,6 +228,35 @@ commands).
live systemd worker, watched it fail with the exit-code-1 quirk
before the fix and complete cleanly after, confirmed the summary
rendered correctly in the UI. Test data cleaned up afterward.
11. **Gravatar + DNS Records collectors** (done) — 7th and 8th
sources, both pure PHP (no Python/venv, no external process at
all for DNS).
- `GravatarRunner` (email) — MD5-hashes the email, checks
`gravatar.com/avatar/{hash}?d=404` for a 200 vs 404. Gravatar's
public profile JSON endpoint (`{hash}.json`) turned out to now
require API auth — confirmed by hand against a real profile
(Matt Mullenweg's, Gravatar's own creator: avatar check returned
200, `.json` returned 404 regardless) — so this stays scoped to
the binary "does an avatar exist" signal rather than pulling
profile details. Unit-tested with `Http::fake()` (avatar
found/not-found/error, invalid email, wrong target type).
- `DnsRunner` (domain) — pure PHP `dns_get_record()`, no HTTP
client or subprocess. Looks up A/AAAA/MX/NS/TXT/CNAME. Its raw
payload is naturally keyed by record type (`A`, `MX`, ...),
which the existing generic list-renderer in the investigation
show page turns into one collapsible section per record type
automatically — no UI changes needed. Unit-tested against a real
DNS lookup (`example.com`, IANA-controlled, stable for decades)
plus a nonexistent-domain case, since `dns_get_record()` isn't
mockable through `Http::fake()`.
- Courtesy rate limits: Gravatar 45/min (cheap single GET), DNS
60/min (local resolver, cheapest operation in the app).
- 9 new tests (84/84 passing). Manually verified end-to-end
through the live systemd worker: Gravatar against a real email
with a known public avatar (found it), DNS against `google.com`
(22 records across all 5 present types, each rendering as its
own labeled section in the UI with correct counts and values).
Test data cleaned up afterward.
## Open questions
- Any legal/ToS constraints on the sources to be queried?