Replace default Laravel README with project documentation
Describes what this actually is (multi-user OSINT investigation platform), the 9 collectors, and step-by-step setup (DB, Python venvs, queue worker, optional OpenCorporates token). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
cedry2k1992
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
dc4388eabf
commit
5e8b1c8a15
@@ -1,58 +1,136 @@
|
||||
<p align="center"><a href="https://laravel.com" target="_blank"><img src="https://raw.githubusercontent.com/laravel/art/master/logo-lockup/5%20SVG/2%20CMYK/1%20Full%20Color/laravel-logolockup-cmyk-red.svg" width="400" alt="Laravel Logo"></a></p>
|
||||
# OSINT
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/laravel/framework/actions"><img src="https://github.com/laravel/framework/workflows/tests/badge.svg" alt="Build Status"></a>
|
||||
<a href="https://packagist.org/packages/laravel/framework"><img src="https://img.shields.io/packagist/dt/laravel/framework" alt="Total Downloads"></a>
|
||||
<a href="https://packagist.org/packages/laravel/framework"><img src="https://img.shields.io/packagist/v/laravel/framework" alt="Latest Stable Version"></a>
|
||||
<a href="https://packagist.org/packages/laravel/framework"><img src="https://img.shields.io/packagist/l/laravel/framework" alt="License"></a>
|
||||
</p>
|
||||
A multi-user OSINT (Open Source Intelligence) investigation platform built on
|
||||
Laravel. Investigators create **investigations**, add **targets** (a
|
||||
username, domain, email, IP, or org name), and run free/open-source OSINT
|
||||
tools against them. Results are collected as **findings**, searchable across
|
||||
everything you have access to, and exportable as a printable report or
|
||||
CSV/JSON.
|
||||
|
||||
## About Laravel
|
||||
Multi-user by design: each investigation has an owner who controls access,
|
||||
granting `viewer` or `editor` roles to individual users or whole groups.
|
||||
|
||||
Laravel is a web application framework with expressive, elegant syntax. We believe development must be an enjoyable and creative experience to be truly fulfilling. Laravel takes the pain out of development by easing common tasks used in many web projects, such as:
|
||||
## Features
|
||||
|
||||
- [Simple, fast routing engine](https://laravel.com/docs/routing).
|
||||
- [Powerful dependency injection container](https://laravel.com/docs/container).
|
||||
- Multiple back-ends for [session](https://laravel.com/docs/session) and [cache](https://laravel.com/docs/cache) storage.
|
||||
- Expressive, intuitive [database ORM](https://laravel.com/docs/eloquent).
|
||||
- Database agnostic [schema migrations](https://laravel.com/docs/migrations).
|
||||
- [Robust background job processing](https://laravel.com/docs/queues).
|
||||
- [Real-time event broadcasting](https://laravel.com/docs/broadcasting).
|
||||
- **Investigations** — cases that group targets and findings, with
|
||||
owner-controlled access grants (per-user or per-group, `viewer`/`editor`)
|
||||
- **Groups** — simple teams that can be granted access to an investigation
|
||||
as a unit
|
||||
- **Targets** — `username`, `domain`, `email`, `ip`, `org`, or `person`,
|
||||
each with type-specific input validation
|
||||
- **9 OSINT collectors**, all free/open-source, run as background jobs
|
||||
through a shared queue worker:
|
||||
|
||||
Laravel is accessible, powerful, and provides tools required for large, robust applications.
|
||||
| Source | Target type | What it does |
|
||||
|---|---|---|
|
||||
| [Sherlock](https://github.com/sherlock-project/sherlock) | `username` | Checks presence across 400+ social/community sites |
|
||||
| [theHarvester](https://github.com/laramies/theHarvester) | `domain` | Emails/subdomains/hosts from crt.sh, OTX, HackerTarget, and more |
|
||||
| [crt.sh](https://crt.sh) | `domain` | Subdomain enumeration via certificate transparency logs |
|
||||
| WHOIS | `domain` | Registrar, dates, name servers |
|
||||
| DNS Records | `domain` | A/AAAA/MX/NS/TXT/CNAME lookup |
|
||||
| [Holehe](https://github.com/megadose/holehe) | `email` | Checks if an email is registered on 120+ sites |
|
||||
| Gravatar | `email` | Checks for a registered Gravatar image |
|
||||
| [ip-api.com](https://ip-api.com) | `ip` | IP geolocation |
|
||||
| [OpenCorporates](https://opencorporates.com) | `org` | Company registration search — needs a free API token, see below |
|
||||
|
||||
## Learning Laravel
|
||||
- **Bulk actions** — run every applicable source against a target, or every
|
||||
target in an investigation, in one click; skips sources already in
|
||||
flight and automatically retries prior failures
|
||||
- **Search** — find investigations by name or targets by value, scoped to
|
||||
what you have access to
|
||||
- **Reporting** — printable HTML report (browser print-to-PDF), CSV export,
|
||||
JSON export
|
||||
|
||||
Laravel has the most extensive and thorough [documentation](https://laravel.com/docs) and video tutorial library of all modern web application frameworks, making it a breeze to get started with the framework.
|
||||
## Stack
|
||||
|
||||
In addition, [Laracasts](https://laracasts.com) contains thousands of video tutorials on a range of topics including Laravel, modern PHP, unit testing, and JavaScript. Boost your skills by digging into our comprehensive video library.
|
||||
- Laravel 13 (PHP 8.4), Breeze (Blade + Vite) for auth
|
||||
- MySQL for storage, `database` queue driver
|
||||
- Python 3.10 venvs for the Python-based collectors (Sherlock, Holehe,
|
||||
WHOIS, theHarvester); everything else is pure PHP (HTTP calls or
|
||||
`dns_get_record()`), no external process needed
|
||||
|
||||
You can also watch bite-sized lessons with real-world projects on [Laravel Learn](https://laravel.com/learn), where you will be guided through building a Laravel application from scratch while learning PHP fundamentals.
|
||||
See [`PLAN.md`](PLAN.md) for the full build history, design decisions, and
|
||||
known gotchas — it's the more detailed companion to this file.
|
||||
|
||||
## Agentic Development
|
||||
## Setup
|
||||
|
||||
Laravel's predictable structure and conventions make it ideal for AI coding agents like Claude Code, Cursor, and GitHub Copilot. Install [Laravel Boost](https://laravel.com/docs/ai) to supercharge your AI workflow:
|
||||
### 1. PHP dependencies
|
||||
|
||||
```bash
|
||||
composer require laravel/boost --dev
|
||||
|
||||
php artisan boost:install
|
||||
composer install
|
||||
cp .env.example .env
|
||||
php artisan key:generate
|
||||
```
|
||||
|
||||
Boost provides your agent 15+ tools and skills that help agents build Laravel applications while following best practices.
|
||||
### 2. Database
|
||||
|
||||
## Contributing
|
||||
Point `.env` at a MySQL database (`DB_*` variables) and run:
|
||||
|
||||
Thank you for considering contributing to the Laravel framework! The contribution guide can be found in the [Laravel documentation](https://laravel.com/docs/contributions).
|
||||
```bash
|
||||
php artisan migrate
|
||||
php artisan db:seed --class=SourceSeeder
|
||||
```
|
||||
|
||||
## Code of Conduct
|
||||
### 3. Frontend assets
|
||||
|
||||
In order to ensure that the Laravel community is welcoming to all, please review and abide by the [Code of Conduct](https://laravel.com/docs/contributions#code-of-conduct).
|
||||
```bash
|
||||
npm install
|
||||
npm run build # or `npm run dev` while developing
|
||||
```
|
||||
|
||||
## Security Vulnerabilities
|
||||
### 4. Python collectors
|
||||
|
||||
If you discover a security vulnerability within Laravel, please send an e-mail to Taylor Otwell via [taylor@laravel.com](mailto:taylor@laravel.com). All security vulnerabilities will be promptly addressed.
|
||||
```bash
|
||||
scripts/python/setup.sh
|
||||
```
|
||||
|
||||
## License
|
||||
Sets up two venvs (`scripts/python/.venv` for Sherlock/whois/Holehe,
|
||||
`scripts/python/.venv-theharvester` for theHarvester, which has much
|
||||
heavier dependencies) and vendors theHarvester's source. See
|
||||
[`scripts/python/README.md`](scripts/python/README.md) for details.
|
||||
|
||||
The Laravel framework is open-sourced software licensed under the [MIT license](https://opensource.org/licenses/MIT).
|
||||
### 5. Queue worker
|
||||
|
||||
Lookups run as background jobs — nothing executes until a worker is
|
||||
running. For local development:
|
||||
|
||||
```bash
|
||||
php artisan queue:work
|
||||
```
|
||||
|
||||
For anything longer-lived, install the supervised `systemd --user` service
|
||||
in [`deploy/systemd/`](deploy/systemd/README.md) instead.
|
||||
|
||||
### 6. Optional: OpenCorporates
|
||||
|
||||
The `org` collector needs a free API token. Register at
|
||||
[opencorporates.com/api_accounts/new](https://opencorporates.com/api_accounts/new)
|
||||
and set `OPENCORPORATES_API_TOKEN` in `.env`. Without one, that source is
|
||||
hidden rather than shown broken. Restart the queue worker after setting it.
|
||||
|
||||
### 7. Run it
|
||||
|
||||
```bash
|
||||
php artisan serve
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
php artisan test
|
||||
```
|
||||
|
||||
Tests run against a separate `osint_testing` database, configured in
|
||||
`phpunit.xml` — never against your real data.
|
||||
|
||||
## Project layout
|
||||
|
||||
- `app/Osint/` — the collector framework: `Runner` interface,
|
||||
`RunnerRegistry` (maps a source's `key` to its implementation),
|
||||
`Runners/` (one class per collector), `FindingDispatcher` (bulk-run logic)
|
||||
- `app/Jobs/RunFinding.php` — queued job that executes a runner and updates
|
||||
the `Finding` row, with per-source rate limiting
|
||||
- `app/Models/` — `Investigation`, `Target`, `Source`, `Finding`, `Group`,
|
||||
`User`
|
||||
- `scripts/python/` — Python collectors and their setup script
|
||||
- `deploy/systemd/` — the queue worker's supervised-process definition
|
||||
|
||||
Reference in New Issue
Block a user