Add OpenCorporates collector for org targets, gated on an API token
Closes the last coverage gap for org (person remains intentionally unaddressed — no good free/non-scraping option exists). Unlike the other 8 sources, OpenCorporates now requires an API token even on its free tier (confirmed: keyless request gets 401). RunnerRegistry::has() checks services.opencorporates.token is configured before exposing the source as runnable, so it degrades to "not yet integrated" in the UI rather than a broken Run button, until OPENCORPORATES_API_TOKEN is set. Also fixes a real bug hit while building this: a RunnerRegistry::REQUIRES class constant mapping source key to a closure crashed on file load (PHP class constants can't hold closures) — inlined the one check instead of building an abstraction for a single case. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
86f73c46b2
commit
dc4388eabf
@@ -52,8 +52,9 @@ commands).
|
||||
username, IP, org), scoped to one investigation
|
||||
- **Sources** — catalog of OSINT tools/techniques (seeded: Sherlock,
|
||||
theHarvester, crt.sh, WHOIS, ip-api.com, Holehe, Gravatar, DNS
|
||||
Records — all 8 now have a working runner). `sources.runner` is a
|
||||
display-only label; the real mapping is `App\Osint\RunnerRegistry`
|
||||
Records, OpenCorporates — 9 total, 8 runnable out of the box;
|
||||
OpenCorporates needs a free API token, see phase 12). `sources.runner`
|
||||
is a display-only label; the real mapping is `App\Osint\RunnerRegistry`
|
||||
- **Findings** — result of running a source against a target; raw JSON
|
||||
payload preserved plus a parsed summary
|
||||
|
||||
@@ -257,6 +258,42 @@ commands).
|
||||
(22 records across all 5 present types, each rendering as its
|
||||
own labeled section in the UI with correct counts and values).
|
||||
Test data cleaned up afterward.
|
||||
12. **OpenCorporates collector** (done) — 9th source, first attempt at
|
||||
closing the `org` gap. Correction to what was assumed when this
|
||||
was first suggested: OpenCorporates now requires an API token even
|
||||
on the free tier — confirmed by hand (keyless request → `401
|
||||
Invalid Api Token`). Free tier still exists (50 req/day, 200/month,
|
||||
register at opencorporates.com/api_accounts/new) but isn't fully
|
||||
keyless like the others.
|
||||
- `services.opencorporates.token` (`OPENCORPORATES_API_TOKEN` in
|
||||
`.env`) holds the token; left blank by default.
|
||||
- `RunnerRegistry::has('opencorporates')` additionally checks the
|
||||
token is configured — without one, the source is hidden from
|
||||
the "runnable" list (shows "not yet integrated", same as any
|
||||
other unbuilt runner) instead of exposing a "Run" button that
|
||||
always fails. Verified both layers by hand: the UI correctly
|
||||
grays it out, and a direct POST bypassing the UI still gets
|
||||
rejected server-side with a clear error and creates no `Finding`
|
||||
row.
|
||||
- Rate limit is their real quota (50/day) via `Limit::perDay(50)`,
|
||||
not a courtesy limit like the other sources — protects the
|
||||
configured token from getting exhausted or suspended.
|
||||
- Hit one real bug while wiring the token-gating: PHP class
|
||||
constants can't hold closures (`Constant expression contains
|
||||
invalid operations`) — an earlier `REQUIRES` map of source key →
|
||||
closure crashed every test process on file load. Fixed by
|
||||
inlining the one special case directly in `has()` instead of
|
||||
building a small closure-map abstraction for what's currently a
|
||||
single entry.
|
||||
- 8 new tests (93/93 passing) — `OpenCorporatesRunner` via
|
||||
`Http::fake()` (token missing, results found/not-found, wrong
|
||||
target type, HTTP failure), `RunnerRegistry` gating logic
|
||||
directly. Not runnable end-to-end without a real token (none
|
||||
configured in this environment) — verified what *can* be
|
||||
verified without one: the gating behavior in both the UI and
|
||||
the controller. **Whoever wants this source live needs to
|
||||
register for a free OpenCorporates account and set
|
||||
`OPENCORPORATES_API_TOKEN` in `.env`.**
|
||||
|
||||
## Open questions
|
||||
- Any legal/ToS constraints on the sources to be queried?
|
||||
|
||||
Reference in New Issue
Block a user