341 lines
12 KiB
Markdown
341 lines
12 KiB
Markdown
# Error Tracker — Build Plan
|
|
|
|
A self-hosted error tracking application (inspired by [Bugsink](https://www.bugsink.com/) / Sentry) built on this Laravel 13 app.
|
|
|
|
## Stack Summary
|
|
|
|
| Layer | Technology |
|
|
|-------|-----------|
|
|
| Backend | Laravel 13, PHP 8.3+ |
|
|
| Database | SQLite (dev), easily swappable to Postgres/MySQL |
|
|
| Frontend | Blade + Tailwind CSS 4 + Vite 8 |
|
|
| Queue | Database driver (upgrade to Redis for production) |
|
|
| Auth | Laravel Breeze (installed) |
|
|
|
|
---
|
|
|
|
## Phase 1 — Foundation & Auth
|
|
|
|
### 1.1 Install Auth Scaffolding
|
|
- [x] Install Laravel Breeze (Blade + Tailwind stack) for login, registration, password reset, email verification.
|
|
- [x] Authenticated layout, middleware, and user management.
|
|
|
|
### 1.2 Core Data Models & Migrations
|
|
|
|
- [x] `projects` migration & `Project` model
|
|
- [x] `project_user` pivot migration
|
|
- [x] `issues` migration & `Issue` model
|
|
- [x] `events` migration & `Event` model
|
|
- [x] `User` model updated with `projects()` and `ownedProjects()` relationships
|
|
|
|
#### `projects` table
|
|
| Column | Type | Notes |
|
|
|--------|------|-------|
|
|
| id | ulid (pk) | |
|
|
| name | string | |
|
|
| platform | string (nullable) | e.g. "php", "javascript", "python" |
|
|
| dsn_public_key | string (unique) | Auto-generated UUID used in the DSN |
|
|
| dsn_secret_key | string (unique) | Auto-generated secret for server-side SDKs |
|
|
| created_at / updated_at | timestamps | |
|
|
|
|
#### `project_user` pivot table
|
|
| Column | Type | Notes |
|
|
|--------|------|-------|
|
|
| project_id | foreign | |
|
|
| user_id | foreign | |
|
|
| role | string | "owner", "member" |
|
|
|
|
#### `issues` table (groups of similar errors)
|
|
| Column | Type | Notes |
|
|
|--------|------|-------|
|
|
| id | ulid (pk) | |
|
|
| project_id | foreign | |
|
|
| fingerprint | string (index) | Hash used to group events |
|
|
| title | string | Short description, e.g. exception class + message |
|
|
| culprit | string (nullable) | File/function where it occurred |
|
|
| level | string | "fatal", "error", "warning", "info", "debug" |
|
|
| status | string | "unresolved", "resolved", "ignored", "muted" |
|
|
| first_seen_at | datetime | |
|
|
| last_seen_at | datetime | |
|
|
| times_seen | integer | Counter cache |
|
|
| is_regression | boolean | Re-opened after being resolved |
|
|
| metadata | json (nullable) | Platform, runtime, SDK info |
|
|
| created_at / updated_at | timestamps | |
|
|
|
|
#### `events` table (individual error occurrences)
|
|
| Column | Type | Notes |
|
|
|--------|------|-------|
|
|
| id | ulid (pk) | |
|
|
| issue_id | foreign | |
|
|
| project_id | foreign | |
|
|
| event_id | string (unique) | UUID sent by the SDK |
|
|
| level | string | |
|
|
| message | text (nullable) | |
|
|
| platform | string (nullable) | |
|
|
| timestamp | datetime | Event timestamp from SDK |
|
|
| server_name | string (nullable) | |
|
|
| release | string (nullable) | App version/release tag |
|
|
| environment | string (nullable) | "production", "staging", etc. |
|
|
| exception | json | Structured exception data (type, value, stacktrace) |
|
|
| breadcrumbs | json (nullable) | Trail of events leading up to the error |
|
|
| contexts | json (nullable) | OS, browser, device, runtime, etc. |
|
|
| tags | json (nullable) | Key-value pairs for filtering |
|
|
| extra | json (nullable) | Arbitrary extra data |
|
|
| user_context | json (nullable) | Info about the affected user |
|
|
| request_context | json (nullable) | HTTP method, URL, headers, body |
|
|
| sdk | json (nullable) | SDK name + version |
|
|
| raw_payload | json | Full original envelope for debugging |
|
|
| created_at | timestamp | |
|
|
|
|
---
|
|
|
|
## Phase 2 — Event Ingestion API
|
|
|
|
### 2.1 Sentry-Compatible Ingest Endpoint
|
|
|
|
Implement a `/api/{project_id}/store/` endpoint that accepts the Sentry event payload format. This allows using existing Sentry SDKs (for PHP, JS, Python, etc.) as clients without any custom SDK work.
|
|
|
|
- [x] `POST /api/{project_id}/store/` route
|
|
- [x] Validate DSN public key via `sentry_key` query param or `X-Sentry-Auth` header
|
|
- [x] `AuthenticateDsn` middleware
|
|
- [x] `EventIngestController`
|
|
|
|
### 2.2 Event Processing Pipeline
|
|
|
|
- [x] `ProcessEventJob` (queued)
|
|
- [x] `PayloadNormalizer` service
|
|
- [x] `EventProcessor` service (find or create Issue, store Event, update counters)
|
|
- [x] Regression detection (re-open resolved issues)
|
|
|
|
```
|
|
Ingest Request
|
|
→ Authenticate DSN key
|
|
→ Validate & normalize payload
|
|
→ Dispatch job to queue
|
|
→ Compute fingerprint (grouping)
|
|
→ Find or create Issue
|
|
→ Store Event
|
|
→ Update Issue counters (last_seen_at, times_seen)
|
|
→ Check if regression (status was "resolved")
|
|
→ Trigger notifications if needed
|
|
```
|
|
|
|
### 2.3 Fingerprinting / Grouping Strategy
|
|
|
|
- [x] `Fingerprinter` service
|
|
|
|
Group events into issues using a fingerprint hash built from:
|
|
1. Exception type (class name)
|
|
2. Exception message (with variable parts stripped, e.g. IDs, numbers)
|
|
3. Top N frames of the stacktrace (file + function)
|
|
|
|
Users can override with a custom `fingerprint` array in the event payload (Sentry SDK standard).
|
|
|
|
### 2.4 DSN Format
|
|
|
|
```
|
|
https://<public_key>@<host>/api/<project_id>/store/
|
|
```
|
|
|
|
Clients configure this DSN in their Sentry SDK — no changes needed on the SDK side.
|
|
|
|
---
|
|
|
|
## Phase 3 — Web Dashboard
|
|
|
|
### 3.1 Layouts & Navigation
|
|
|
|
- [x] Authenticated layout with sidebar navigation
|
|
- [x] Global project switcher in the sidebar/topbar
|
|
- [x] Sections: **Issues**, **Project Settings**, **Team Members**
|
|
|
|
### 3.2 Issues List Page (`/projects/{project}/issues`)
|
|
|
|
- [x] Table/list of issues sorted by last seen (most recent first)
|
|
- [x] Columns: status badge, title, culprit, event count sparkline, times seen, last seen
|
|
- [x] Filters: status (unresolved/resolved/ignored), level, environment, date range
|
|
- [x] Search: full-text search on title and culprit
|
|
- [x] Bulk actions: resolve, ignore, mute, delete
|
|
- [x] Pagination (cursor-based for performance)
|
|
|
|
### 3.3 Issue Detail Page (`/projects/{project}/issues/{issue}`)
|
|
|
|
- [x] Header: title, status controls (resolve / ignore / unresolve), level badge, first/last seen, times seen
|
|
- [x] **Stacktrace viewer:** collapsible frames, source context (if available), highlighted app frames vs vendor frames
|
|
- [x] **Breadcrumbs timeline:** chronological list of breadcrumb entries
|
|
- [x] **Tags panel:** key-value tag list with links to filter by tag
|
|
- [x] **Context panels:** device, OS, browser, runtime, user info, request details
|
|
- [x] **Events sub-list:** paginated list of individual occurrences with ability to view each one
|
|
- [ ] **Activity:** timeline of status changes (resolved, regressed, etc.)
|
|
|
|
### 3.4 Event Detail Page (`/projects/{project}/issues/{issue}/events/{event}`)
|
|
|
|
- [x] Full event data: raw JSON viewer + structured panels
|
|
- [x] Navigation: previous/next event within the issue
|
|
|
|
### 3.5 Project Settings Page
|
|
|
|
- [x] Rename project, change platform
|
|
- [x] DSN display with copy button
|
|
- [x] Regenerate keys
|
|
- [x] Delete project (with confirmation)
|
|
|
|
### 3.6 Dashboard / Overview Page
|
|
|
|
- [x] Summary cards: total unresolved issues, events in last 24h, most active projects
|
|
- [x] Line chart: events over time (last 7/14/30 days)
|
|
|
|
---
|
|
|
|
## Phase 4 — Notifications & Alerting
|
|
|
|
### 4.1 Notification Channels
|
|
|
|
- [x] **Email:** on first occurrence of a new issue (not on every event)
|
|
- [x] **Database (in-app):** notification bell with unread count
|
|
- [ ] Extensible to Slack/webhooks later
|
|
|
|
### 4.2 Alert Rules (per project)
|
|
|
|
- [x] Notify on: new issue, regression, spike (N events in M minutes)
|
|
- [x] Per-user mute controls
|
|
|
|
---
|
|
|
|
## Phase 5 — Polish & Advanced Features
|
|
|
|
### 5.1 Release Tracking
|
|
|
|
- [ ] Parse `release` field from events
|
|
- [ ] Show which release introduced an issue
|
|
- [ ] Mark issues as resolved-in-release; auto-regress if they appear in a newer release
|
|
|
|
### 5.2 Environment Filtering
|
|
|
|
- [ ] Global environment selector (production / staging / etc.)
|
|
- [ ] Filter issues and events by environment
|
|
|
|
### 5.3 Retention & Cleanup
|
|
|
|
- [ ] Artisan command / scheduled job to delete events older than N days (configurable)
|
|
- [ ] Keep issue summaries even after events are pruned
|
|
|
|
### 5.4 Source Maps / Source Context (optional, stretch)
|
|
|
|
- [ ] Upload source maps for JS projects
|
|
- [ ] Store and display source context lines around each stacktrace frame
|
|
|
|
### 5.5 API Tokens & External API
|
|
|
|
- [ ] Personal API tokens for programmatic access
|
|
- [ ] REST API for listing/resolving issues (useful for CI/CD integrations)
|
|
|
|
---
|
|
|
|
## Implementation Order & Milestones
|
|
|
|
### Milestone 1 — MVP (walking skeleton)
|
|
- [x] 1. Install Breeze, set up auth
|
|
- [x] 2. Create migrations for `projects`, `issues`, `events`, `project_user`
|
|
- [x] 3. Build models with relationships
|
|
- [x] 4. Build the ingest endpoint (`POST /api/{project_id}/store/`)
|
|
- [x] 5. Implement fingerprinting and the `ProcessEventJob`
|
|
- [x] 6. Build a minimal issues list page and issue detail page
|
|
- [ ] 7. Test with a real Sentry SDK (e.g. `sentry/sentry-laravel` on another app pointing at this tracker)
|
|
|
|
### Milestone 2 — Usable product
|
|
- [x] 8. Project CRUD and settings pages
|
|
- [ ] 9. Team/member management (invite users to projects)
|
|
- [x] 10. Stacktrace viewer with collapsible frames
|
|
- [x] 11. Breadcrumbs, tags, context panels on issue detail
|
|
- [x] 12. Filters, search, bulk actions on issues list
|
|
- [x] 13. Status management (resolve, ignore, mute, auto-regression detection)
|
|
|
|
### Milestone 3 — Notifications & dashboard
|
|
- [x] 14. Email notifications on new issue / regression
|
|
- [x] 15. In-app notification system
|
|
- [x] 16. Overview dashboard with charts (events over time)
|
|
|
|
### Milestone 4 — Production-readiness
|
|
- [ ] 17. Rate limiting on ingest
|
|
- [ ] 18. Event retention / cleanup scheduler
|
|
- [ ] 19. Release tracking
|
|
- [ ] 20. Environment filtering
|
|
- [ ] 21. Performance optimization (indexes, query tuning, caching)
|
|
|
|
### Milestone 5 — Advanced (stretch)
|
|
- [ ] 22. Source map upload & processing
|
|
- [ ] 23. API tokens & external REST API
|
|
- [ ] 24. Webhook / Slack notifications
|
|
- [ ] 25. CQRS-style read models for heavy dashboards (if needed)
|
|
|
|
---
|
|
|
|
## File / Directory Structure (planned)
|
|
|
|
```
|
|
app/
|
|
├── Http/
|
|
│ ├── Controllers/
|
|
│ │ ├── DashboardController.php
|
|
│ │ ├── ProjectController.php
|
|
│ │ ├── IssueController.php
|
|
│ │ ├── EventController.php
|
|
│ │ └── Api/
|
|
│ │ └── EventIngestController.php
|
|
│ ├── Middleware/
|
|
│ │ └── AuthenticateDsn.php
|
|
│ └── Requests/
|
|
│ ├── StoreProjectRequest.php
|
|
│ └── UpdateProjectRequest.php
|
|
├── Jobs/
|
|
│ └── ProcessEventJob.php
|
|
├── Models/
|
|
│ ├── Project.php
|
|
│ ├── Issue.php
|
|
│ └── Event.php
|
|
├── Notifications/
|
|
│ ├── NewIssueNotification.php
|
|
│ └── IssueRegressionNotification.php
|
|
├── Services/
|
|
│ ├── EventProcessor.php
|
|
│ ├── Fingerprinter.php
|
|
│ └── PayloadNormalizer.php
|
|
├── Policies/
|
|
│ ├── ProjectPolicy.php
|
|
│ └── IssuePolicy.php
|
|
database/
|
|
├── migrations/
|
|
│ ├── xxxx_create_projects_table.php
|
|
│ ├── xxxx_create_project_user_table.php
|
|
│ ├── xxxx_create_issues_table.php
|
|
│ └── xxxx_create_events_table.php
|
|
resources/
|
|
├── views/
|
|
│ ├── dashboard.blade.php
|
|
│ ├── projects/
|
|
│ │ ├── index.blade.php
|
|
│ │ ├── show.blade.php
|
|
│ │ ├── create.blade.php
|
|
│ │ └── edit.blade.php
|
|
│ ├── issues/
|
|
│ │ ├── index.blade.php
|
|
│ │ └── show.blade.php
|
|
│ └── events/
|
|
│ └── show.blade.php
|
|
routes/
|
|
├── web.php (dashboard, projects, issues, events)
|
|
└── api.php (event ingest endpoint)
|
|
```
|
|
|
|
---
|
|
|
|
## Key Design Decisions
|
|
|
|
1. **Sentry-compatible protocol** — Reuse the massive ecosystem of existing Sentry SDKs instead of building custom clients.
|
|
2. **Queue-based processing** — The ingest endpoint returns `202 Accepted` immediately and processes asynchronously, so SDK clients never block.
|
|
3. **ULID primary keys** — Sortable, time-ordered, no sequential ID leakage.
|
|
4. **Fingerprint-based grouping** — Same approach as Sentry/Bugsink; keeps the issues list manageable.
|
|
5. **Blade + Tailwind (no SPA)** — Keep complexity low; server-rendered pages with sprinkles of Alpine.js where needed (e.g. collapsible stacktrace frames, modals).
|
|
6. **SQLite for dev, Postgres for prod** — SQLite is fine for development and small-scale use. The JSON columns and indexing strategy work on both.
|