trackbugs/PLAN.md

# Error Tracker — Build Plan

A self-hosted error tracking application (inspired by [Bugsink](https://www.bugsink.com/) / Sentry) built on this Laravel 13 app.

## Stack Summary

| Layer | Technology |
|-------|-----------|
| Backend | Laravel 13, PHP 8.3+ |
| Database | SQLite (dev), easily swappable to Postgres/MySQL |
| Frontend | Blade + Tailwind CSS 4 + Vite 8 |
| Queue | Database driver (upgrade to Redis for production) |
| Auth | Laravel Breeze (to be installed) |

---

## Phase 1 — Foundation & Auth

### 1.1 Install Auth Scaffolding
- Install Laravel Breeze (Blade + Tailwind stack) for login, registration, password reset, email verification.
- This gives us the authenticated layout, middleware, and user management out of the box.

### 1.2 Core Data Models & Migrations

#### `projects` table
| Column | Type | Notes |
|--------|------|-------|
| id | ulid (pk) | |
| name | string | |
| platform | string (nullable) | e.g. "php", "javascript", "python" |
| dsn_public_key | string (unique) | Auto-generated UUID used in the DSN |
| dsn_secret_key | string (unique) | Auto-generated secret for server-side SDKs |
| created_at / updated_at | timestamps | |

#### `project_user` pivot table
| Column | Type | Notes |
|--------|------|-------|
| project_id | foreign | |
| user_id | foreign | |
| role | string | "owner", "member" |

#### `issues` table (groups of similar errors)
| Column | Type | Notes |
|--------|------|-------|
| id | ulid (pk) | |
| project_id | foreign | |
| fingerprint | string (index) | Hash used to group events |
| title | string | Short description, e.g. exception class + message |
| culprit | string (nullable) | File/function where it occurred |
| level | string | "fatal", "error", "warning", "info", "debug" |
| status | string | "unresolved", "resolved", "ignored", "muted" |
| first_seen_at | datetime | |
| last_seen_at | datetime | |
| times_seen | integer | Counter cache |
| is_regression | boolean | Re-opened after being resolved |
| metadata | json (nullable) | Platform, runtime, SDK info |
| created_at / updated_at | timestamps | |

#### `events` table (individual error occurrences)
| Column | Type | Notes |
|--------|------|-------|
| id | ulid (pk) | |
| issue_id | foreign | |
| project_id | foreign | |
| event_id | string (unique) | UUID sent by the SDK |
| level | string | |
| message | text (nullable) | |
| platform | string (nullable) | |
| timestamp | datetime | Event timestamp from SDK |
| server_name | string (nullable) | |
| release | string (nullable) | App version/release tag |
| environment | string (nullable) | "production", "staging", etc. |
| exception | json | Structured exception data (type, value, stacktrace) |
| breadcrumbs | json (nullable) | Trail of events leading up to the error |
| contexts | json (nullable) | OS, browser, device, runtime, etc. |
| tags | json (nullable) | Key-value pairs for filtering |
| extra | json (nullable) | Arbitrary extra data |
| user_context | json (nullable) | Info about the affected user |
| request_context | json (nullable) | HTTP method, URL, headers, body |
| sdk | json (nullable) | SDK name + version |
| raw_payload | json | Full original envelope for debugging |
| created_at | timestamp | |

---

## Phase 2 — Event Ingestion API

### 2.1 Sentry-Compatible Ingest Endpoint

Implement a `/api/{project_id}/store/` endpoint that accepts the Sentry event payload format. This allows using existing Sentry SDKs (for PHP, JS, Python, etc.) as clients without any custom SDK work.

- **Route:** `POST /api/{project_id}/store/`
- **Auth:** Validate DSN public key via `sentry_key` query param or `X-Sentry-Auth` header.
- **Rate limiting:** Per-project throttle (e.g. 100 events/minute).
- **No user auth required** — this is an SDK-to-server API.

### 2.2 Event Processing Pipeline

```
Ingest Request
  → Authenticate DSN key
  → Validate & normalize payload
  → Dispatch job to queue
      → Compute fingerprint (grouping)
      → Find or create Issue
      → Store Event
      → Update Issue counters (last_seen_at, times_seen)
      → Check if regression (status was "resolved")
      → Trigger notifications if needed
```

### 2.3 Fingerprinting / Grouping Strategy

Group events into issues using a fingerprint hash built from:
1. Exception type (class name)
2. Exception message (with variable parts stripped, e.g. IDs, numbers)
3. Top N frames of the stacktrace (file + function)

Users can override with a custom `fingerprint` array in the event payload (Sentry SDK standard).

### 2.4 DSN Format

```
https://<public_key>@<host>/api/<project_id>/store/
```

Clients configure this DSN in their Sentry SDK — no changes needed on the SDK side.

---

## Phase 3 — Web Dashboard

### 3.1 Layouts & Navigation

- Authenticated layout with sidebar navigation.
- Global project switcher in the sidebar/topbar.
- Sections: **Issues**, **Project Settings**, **Team Members**.

### 3.2 Issues List Page (`/projects/{project}/issues`)

- Table/list of issues sorted by last seen (most recent first).
- Columns: status badge, title, culprit, event count sparkline, times seen, last seen.
- Filters: status (unresolved/resolved/ignored), level, environment, date range.
- Search: full-text search on title and culprit.
- Bulk actions: resolve, ignore, mute, delete.
- Pagination (cursor-based for performance).

### 3.3 Issue Detail Page (`/projects/{project}/issues/{issue}`)

- Header: title, status controls (resolve / ignore / unresolve), level badge, first/last seen, times seen.
- **Stacktrace viewer:** collapsible frames, source context (if available), highlighted app frames vs vendor frames.
- **Breadcrumbs timeline:** chronological list of breadcrumb entries.
- **Tags panel:** key-value tag list with links to filter by tag.
- **Context panels:** device, OS, browser, runtime, user info, request details.
- **Events sub-list:** paginated list of individual occurrences with ability to view each one.
- **Activity:** timeline of status changes (resolved, regressed, etc.).

### 3.4 Event Detail Page (`/projects/{project}/issues/{issue}/events/{event}`)

- Full event data: raw JSON viewer + structured panels.
- Navigation: previous/next event within the issue.

### 3.5 Project Settings Page

- Rename project, change platform.
- DSN display with copy button.
- Regenerate keys.
- Delete project (with confirmation).

### 3.6 Dashboard / Overview Page

- Summary cards: total unresolved issues, events in last 24h, most active projects.
- Line chart: events over time (last 7/14/30 days).

---

## Phase 4 — Notifications & Alerting

### 4.1 Notification Channels

- **Email:** on first occurrence of a new issue (not on every event).
- **Database (in-app):** notification bell with unread count.
- Extensible to Slack/webhooks later.

### 4.2 Alert Rules (per project)

- Notify on: new issue, regression, spike (N events in M minutes).
- Per-user mute controls.

---

## Phase 5 — Polish & Advanced Features

### 5.1 Release Tracking

- Parse `release` field from events.
- Show which release introduced an issue.
- Mark issues as resolved-in-release; auto-regress if they appear in a newer release.

### 5.2 Environment Filtering

- Global environment selector (production / staging / etc.).
- Filter issues and events by environment.

### 5.3 Retention & Cleanup

- Artisan command / scheduled job to delete events older than N days (configurable).
- Keep issue summaries even after events are pruned.

### 5.4 Source Maps / Source Context (optional, stretch)

- Upload source maps for JS projects.
- Store and display source context lines around each stacktrace frame.

### 5.5 API Tokens & External API

- Personal API tokens for programmatic access.
- REST API for listing/resolving issues (useful for CI/CD integrations).

---

## Implementation Order & Milestones

### Milestone 1 — MVP (walking skeleton)
1. Install Breeze, set up auth.
2. Create migrations for `projects`, `issues`, `events`, `project_user`.
3. Build models with relationships.
4. Build the ingest endpoint (`POST /api/{project_id}/store/`).
5. Implement fingerprinting and the `ProcessEventJob`.
6. Build a minimal issues list page and issue detail page.
7. Test with a real Sentry SDK (e.g. `sentry/sentry-laravel` on another app pointing at this tracker).

### Milestone 2 — Usable product
8. Project CRUD and settings pages.
9. Team/member management (invite users to projects).
10. Stacktrace viewer with collapsible frames.
11. Breadcrumbs, tags, context panels on issue detail.
12. Filters, search, bulk actions on issues list.
13. Status management (resolve, ignore, mute, auto-regression detection).

### Milestone 3 — Notifications & dashboard
14. Email notifications on new issue / regression.
15. In-app notification system.
16. Overview dashboard with charts (events over time).

### Milestone 4 — Production-readiness
17. Rate limiting on ingest.
18. Event retention / cleanup scheduler.
19. Release tracking.
20. Environment filtering.
21. Performance optimization (indexes, query tuning, caching).

### Milestone 5 — Advanced (stretch)
22. Source map upload & processing.
23. API tokens & external REST API.
24. Webhook / Slack notifications.
25. CQRS-style read models for heavy dashboards (if needed).

---

## File / Directory Structure (planned)

```
app/
├── Http/
│   ├── Controllers/
│   │   ├── DashboardController.php
│   │   ├── ProjectController.php
│   │   ├── IssueController.php
│   │   ├── EventController.php
│   │   └── Api/
│   │       └── EventIngestController.php
│   ├── Middleware/
│   │   └── AuthenticateDsn.php
│   └── Requests/
│       ├── StoreProjectRequest.php
│       └── UpdateProjectRequest.php
├── Jobs/
│   └── ProcessEventJob.php
├── Models/
│   ├── Project.php
│   ├── Issue.php
│   └── Event.php
├── Notifications/
│   ├── NewIssueNotification.php
│   └── IssueRegressionNotification.php
├── Services/
│   ├── EventProcessor.php
│   ├── Fingerprinter.php
│   └── PayloadNormalizer.php
├── Policies/
│   ├── ProjectPolicy.php
│   └── IssuePolicy.php
database/
├── migrations/
│   ├── xxxx_create_projects_table.php
│   ├── xxxx_create_project_user_table.php
│   ├── xxxx_create_issues_table.php
│   └── xxxx_create_events_table.php
resources/
├── views/
│   ├── dashboard.blade.php
│   ├── projects/
│   │   ├── index.blade.php
│   │   ├── show.blade.php
│   │   ├── create.blade.php
│   │   └── edit.blade.php
│   ├── issues/
│   │   ├── index.blade.php
│   │   └── show.blade.php
│   └── events/
│       └── show.blade.php
routes/
├── web.php        (dashboard, projects, issues, events)
└── api.php        (event ingest endpoint)
```

---

## Key Design Decisions

1. **Sentry-compatible protocol** — Reuse the massive ecosystem of existing Sentry SDKs instead of building custom clients.
2. **Queue-based processing** — The ingest endpoint returns `202 Accepted` immediately and processes asynchronously, so SDK clients never block.
3. **ULID primary keys** — Sortable, time-ordered, no sequential ID leakage.
4. **Fingerprint-based grouping** — Same approach as Sentry/Bugsink; keeps the issues list manageable.
5. **Blade + Tailwind (no SPA)** — Keep complexity low; server-rendered pages with sprinkles of Alpine.js where needed (e.g. collapsible stacktrace frames, modals).
6. **SQLite for dev, Postgres for prod** — SQLite is fine for development and small-scale use. The JSON columns and indexing strategy work on both.