Initial commit: Laravel OSINT platform
Multi-user OSINT investigation tool with owner-controlled per-user and per-group access grants. Includes Sherlock, theHarvester, crt.sh, WHOIS, and ip-api.com collectors wired through a pluggable Runner/queue pipeline, investigation search, and printable/CSV/JSON reporting. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# Python OSINT scripts
|
||||
|
||||
## Setup
|
||||
|
||||
```
|
||||
scripts/python/setup.sh
|
||||
```
|
||||
|
||||
Sets up:
|
||||
- `scripts/python/.venv` — Sherlock (`sherlock-project`) and `python-whois`,
|
||||
both lightweight and dependency-compatible.
|
||||
- `scripts/python/.venv-theharvester` — theHarvester, isolated because it
|
||||
pulls in a much heavier dependency set (playwright, censys, shodan,
|
||||
fastapi). Vendored from `theharvester-src/` at tag `4.6.0` (the last
|
||||
release compatible with Python 3.9+; 4.7+ requires 3.12).
|
||||
|
||||
Neither venv nor `theharvester-src/` is committed — re-run `setup.sh` on
|
||||
any new machine.
|
||||
|
||||
## How runners find these
|
||||
|
||||
`config/osint.php` has the paths; each `App\Osint\Runners\*` class reads
|
||||
from there rather than hardcoding a path.
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Recreates the Python environments for OSINT collectors. Safe to re-run.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "${BASH_SOURCE[0]}")"
|
||||
|
||||
# Sherlock + python-whois share one lightweight venv.
|
||||
python3 -m venv .venv
|
||||
.venv/bin/pip install --upgrade pip -q
|
||||
.venv/bin/pip install sherlock-project python-whois -q
|
||||
|
||||
# theHarvester (>=4.7 needs Python 3.12; pinned to 4.6.0 for 3.9+ compat)
|
||||
# pulls in a much heavier dependency set (playwright, censys, shodan, fastapi),
|
||||
# so it gets its own venv.
|
||||
python3 -m venv .venv-theharvester
|
||||
.venv-theharvester/bin/pip install --upgrade pip -q
|
||||
|
||||
if [ ! -d theharvester-src ]; then
|
||||
git clone --branch 4.6.0 --depth 1 https://github.com/laramies/theHarvester.git theharvester-src
|
||||
rm -rf theharvester-src/.git
|
||||
fi
|
||||
.venv-theharvester/bin/pip install -r theharvester-src/requirements/base.txt -q
|
||||
|
||||
echo "Done."
|
||||
@@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Runs a domain WHOIS lookup and prints a JSON-serializable summary to stdout.
|
||||
|
||||
Used by App\\Osint\\Runners\\WhoisRunner. Takes exactly one argument: the domain.
|
||||
"""
|
||||
import datetime
|
||||
import json
|
||||
import sys
|
||||
|
||||
import whois
|
||||
|
||||
|
||||
def to_jsonable(value):
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [to_jsonable(v) for v in value]
|
||||
if isinstance(value, (datetime.datetime, datetime.date)):
|
||||
return value.isoformat()
|
||||
if isinstance(value, (str, int, float, bool)) or value is None:
|
||||
return value
|
||||
return str(value)
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 2:
|
||||
print(json.dumps({"error": "usage: whois_lookup.py <domain>"}), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
domain = sys.argv[1]
|
||||
|
||||
try:
|
||||
record = whois.whois(domain)
|
||||
except Exception as exc:
|
||||
print(json.dumps({"error": str(exc)}), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
data = {key: to_jsonable(value) for key, value in dict(record).items()}
|
||||
print(json.dumps(data))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user