root/trackbugs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
trackbugs/PLAN.md
T

15 KiB
Raw Permalink Blame History

Error Tracker — Build Plan

A self-hosted error tracking application (inspired by Bugsink / Sentry) built on this Laravel 13 app.

Overview

This is an error tracking application similar to Sentry, designed to receive and aggregate errors from applications running in production. Users integrate SDKs into their applications (PHP, JavaScript, Python, etc.), and those SDKs send error events directly to our tracking server in real-time when errors occur.

How It Works

  1. SDK Integration: Developers install a Sentry-compatible SDK in their application (e.g., sentry/sentry-laravel for Laravel, @sentry/browser for JavaScript, sentry-python for Python).
  2. Configuration: They configure the SDK with a DSN (Data Source Name) pointing to our tracking server.
  3. Automatic Error Capture: When an error occurs in their application, the SDK automatically captures the error context (stacktrace, breadcrumbs, user info, request details) and sends it to our server.
  4. Real-Time Processing: Our server receives the error payload, groups similar errors into issues, and displays them in a web dashboard.
  5. Notifications: Teams are notified when new issues occur or when resolved issues regress.

This architecture means we don't need to build custom SDKs — we leverage the entire ecosystem of existing Sentry SDKs by implementing a Sentry-compatible ingestion API.

Stack Summary

Layer Technology
Backend Laravel 13, PHP 8.3+
Database SQLite (dev), easily swappable to Postgres/MySQL
Frontend Blade + Tailwind CSS 4 + Vite 8
Queue Database driver (upgrade to Redis for production)
Auth Laravel Breeze (installed)

SDK Support & Multi-Language Integration

Supported Languages & Frameworks

Our tracking server accepts error events from any Sentry-compatible SDK, enabling broad language and framework support without custom development:

Language/Platform Official Sentry SDK Notes
PHP sentry/sentry-laravel, sentry/sentry-symfony, sentry/sentry Full Laravel, Symfony, and native PHP support
JavaScript/Node.js @sentry/browser, @sentry/node, @sentry/react, @sentry/vue, @sentry/angular Frontend and backend JS, plus framework-specific integrations
Python sentry-sdk Django, Flask, FastAPI, etc.
Ruby sentry-ruby, sentry-rails Rails and native Ruby support
Java io.sentry:sentry Spring, Android, and JVM languages
Go github.com/getsentry/sentry-go Native Go support
.NET/C# Sentry.AspNetCore, Sentry ASP.NET Core, .NET Framework
Mobile @sentry/react-native, sentry-android, sentry-cocoa React Native, Android (Java/Kotlin), iOS (Swift/Obj-C)
Rust sentry Native Rust support
Elixir sentry Phoenix and Elixir support

SDK Configuration Example

Users simply point the official Sentry SDK to our server:

PHP (Laravel):

// config/sentry.php
'dsn' => 'https://<public_key>@trackbugs.example.com/api/<project_id>/store/',

JavaScript:

Sentry.init({
  dsn: 'https://<public_key>@trackbugs.example.com/api/<project_id>/store/',
});

Python:

import sentry_sdk
sentry_sdk.init(
    dsn="https://<public_key>@trackbugs.example.com/api/<project_id>/store/"
)

No custom SDK development is required — our server implements the Sentry protocol, so all existing SDKs work out of the box.

Phase 1 — Foundation & Auth

1.1 Install Auth Scaffolding

  • Install Laravel Breeze (Blade + Tailwind stack) for login, registration, password reset, email verification.
  • Authenticated layout, middleware, and user management.

1.2 Core Data Models & Migrations

  • projects migration & Project model
  • project_user pivot migration
  • issues migration & Issue model
  • events migration & Event model
  • User model updated with projects() and ownedProjects() relationships

projects table

Column Type Notes
id ulid (pk)
name string
platform string (nullable) e.g. "php", "javascript", "python"
dsn_public_key string (unique) Auto-generated UUID used in the DSN
dsn_secret_key string (unique) Auto-generated secret for server-side SDKs
created_at / updated_at timestamps

project_user pivot table

Column Type Notes
project_id foreign
user_id foreign
role string "owner", "member"

issues table (groups of similar errors)

Column Type Notes
id ulid (pk)
project_id foreign
fingerprint string (index) Hash used to group events
title string Short description, e.g. exception class + message
culprit string (nullable) File/function where it occurred
level string "fatal", "error", "warning", "info", "debug"
status string "unresolved", "resolved", "ignored", "muted"
first_seen_at datetime
last_seen_at datetime
times_seen integer Counter cache
is_regression boolean Re-opened after being resolved
metadata json (nullable) Platform, runtime, SDK info
created_at / updated_at timestamps

events table (individual error occurrences)

Column Type Notes
id ulid (pk)
issue_id foreign
project_id foreign
event_id string (unique) UUID sent by the SDK
level string
message text (nullable)
platform string (nullable)
timestamp datetime Event timestamp from SDK
server_name string (nullable)
release string (nullable) App version/release tag
environment string (nullable) "production", "staging", etc.
exception json Structured exception data (type, value, stacktrace)
breadcrumbs json (nullable) Trail of events leading up to the error
contexts json (nullable) OS, browser, device, runtime, etc.
tags json (nullable) Key-value pairs for filtering
extra json (nullable) Arbitrary extra data
user_context json (nullable) Info about the affected user
request_context json (nullable) HTTP method, URL, headers, body
sdk json (nullable) SDK name + version
raw_payload json Full original envelope for debugging
created_at timestamp

Phase 2 — Event Ingestion API

2.1 Sentry-Compatible Ingest Endpoint

Implement a /api/{project_id}/store/ endpoint that accepts the Sentry event payload format. This allows using existing Sentry SDKs (for PHP, JS, Python, etc.) as clients without any custom SDK work.

  • POST /api/{project_id}/store/ route
  • Validate DSN public key via sentry_key query param or X-Sentry-Auth header
  • AuthenticateDsn middleware
  • EventIngestController

2.2 Event Processing Pipeline

  • ProcessEventJob (queued)
  • PayloadNormalizer service
  • EventProcessor service (find or create Issue, store Event, update counters)
  • Regression detection (re-open resolved issues)
Ingest Request
  → Authenticate DSN key
  → Validate & normalize payload
  → Dispatch job to queue
      → Compute fingerprint (grouping)
      → Find or create Issue
      → Store Event
      → Update Issue counters (last_seen_at, times_seen)
      → Check if regression (status was "resolved")
      → Trigger notifications if needed

2.3 Fingerprinting / Grouping Strategy

  • Fingerprinter service

Group events into issues using a fingerprint hash built from:

  1. Exception type (class name)
  2. Exception message (with variable parts stripped, e.g. IDs, numbers)
  3. Top N frames of the stacktrace (file + function)

Users can override with a custom fingerprint array in the event payload (Sentry SDK standard).

2.4 DSN Format

https://<public_key>@<host>/api/<project_id>/store/

Clients configure this DSN in their Sentry SDK — no changes needed on the SDK side.

Phase 3 — Web Dashboard

3.1 Layouts & Navigation

  • Authenticated layout with sidebar navigation
  • Global project switcher in the sidebar/topbar
  • Sections: Issues, Project Settings, Team Members

3.2 Issues List Page (/projects/{project}/issues)

  • Table/list of issues sorted by last seen (most recent first)
  • Columns: status badge, title, culprit, event count sparkline, times seen, last seen
  • Filters: status (unresolved/resolved/ignored), level, environment, date range
  • Search: full-text search on title and culprit
  • Bulk actions: resolve, ignore, mute, delete
  • Pagination (cursor-based for performance)

3.3 Issue Detail Page (/projects/{project}/issues/{issue})

  • Header: title, status controls (resolve / ignore / unresolve), level badge, first/last seen, times seen
  • Stacktrace viewer: collapsible frames, source context (if available), highlighted app frames vs vendor frames
  • Breadcrumbs timeline: chronological list of breadcrumb entries
  • Tags panel: key-value tag list with links to filter by tag
  • Context panels: device, OS, browser, runtime, user info, request details
  • Events sub-list: paginated list of individual occurrences with ability to view each one
  • Activity: timeline of status changes (resolved, regressed, etc.)

3.4 Event Detail Page (/projects/{project}/issues/{issue}/events/{event})

  • Full event data: raw JSON viewer + structured panels
  • Navigation: previous/next event within the issue

3.5 Project Settings Page

  • Rename project, change platform
  • DSN display with copy button
  • Regenerate keys
  • Delete project (with confirmation)

3.6 Dashboard / Overview Page

  • Summary cards: total unresolved issues, events in last 24h, most active projects
  • Line chart: events over time (last 7/14/30 days)

Phase 4 — Notifications & Alerting

4.1 Notification Channels

  • Email: on first occurrence of a new issue (not on every event)
  • Database (in-app): notification bell with unread count
  • Extensible to Slack/webhooks later

4.2 Alert Rules (per project)

  • Notify on: new issue, regression, spike (N events in M minutes)
  • Per-user mute controls

Phase 5 — Polish & Advanced Features

5.1 Release Tracking

  • Parse release field from events
  • Show which release introduced an issue
  • Mark issues as resolved-in-release; auto-regress if they appear in a newer release

5.2 Environment Filtering

  • Global environment selector (production / staging / etc.)
  • Filter issues and events by environment

5.3 Retention & Cleanup

  • Artisan command / scheduled job to delete events older than N days (configurable)
  • Keep issue summaries even after events are pruned

5.4 Source Maps / Source Context (optional, stretch)

  • Upload source maps for JS projects
  • Store and display source context lines around each stacktrace frame

5.5 API Tokens & External API

  • Personal API tokens for programmatic access
  • REST API for listing/resolving issues (useful for CI/CD integrations)

Implementation Order & Milestones

Milestone 1 — MVP (walking skeleton)

  • 1. Install Breeze, set up auth
  • 2. Create migrations for projects, issues, events, project_user
  • 3. Build models with relationships
  • 4. Build the ingest endpoint (POST /api/{project_id}/store/)
  • 5. Implement fingerprinting and the ProcessEventJob
  • 6. Build a minimal issues list page and issue detail page
  • 7. Test with a real Sentry SDK (e.g. sentry/sentry-laravel on another app pointing at this tracker)

Milestone 2 — Usable product

  • 8. Project CRUD and settings pages
  • 9. Team/member management (invite users to projects)
  • 10. Stacktrace viewer with collapsible frames
  • 11. Breadcrumbs, tags, context panels on issue detail
  • 12. Filters, search, bulk actions on issues list
  • 13. Status management (resolve, ignore, mute, auto-regression detection)

Milestone 3 — Notifications & dashboard

  • 14. Email notifications on new issue / regression
  • 15. In-app notification system
  • 16. Overview dashboard with charts (events over time)

Milestone 4 — Production-readiness

  • 17. Rate limiting on ingest
  • 18. Event retention / cleanup scheduler
  • 19. Release tracking
  • 20. Environment filtering
  • 21. Performance optimization (indexes, query tuning, caching)

Milestone 5 — Advanced (stretch)

  • 22. Source map upload & processing
  • 23. API tokens & external REST API
  • 24. Webhook / Slack notifications
  • 25. CQRS-style read models for heavy dashboards (if needed)

File / Directory Structure (planned)

app/
├── Http/
│   ├── Controllers/
│   │   ├── DashboardController.php
│   │   ├── ProjectController.php
│   │   ├── IssueController.php
│   │   ├── EventController.php
│   │   └── Api/
│   │       └── EventIngestController.php
│   ├── Middleware/
│   │   └── AuthenticateDsn.php
│   └── Requests/
│       ├── StoreProjectRequest.php
│       └── UpdateProjectRequest.php
├── Jobs/
│   └── ProcessEventJob.php
├── Models/
│   ├── Project.php
│   ├── Issue.php
│   └── Event.php
├── Notifications/
│   ├── NewIssueNotification.php
│   └── IssueRegressionNotification.php
├── Services/
│   ├── EventProcessor.php
│   ├── Fingerprinter.php
│   └── PayloadNormalizer.php
├── Policies/
│   ├── ProjectPolicy.php
│   └── IssuePolicy.php
database/
├── migrations/
│   ├── xxxx_create_projects_table.php
│   ├── xxxx_create_project_user_table.php
│   ├── xxxx_create_issues_table.php
│   └── xxxx_create_events_table.php
resources/
├── views/
│   ├── dashboard.blade.php
│   ├── projects/
│   │   ├── index.blade.php
│   │   ├── show.blade.php
│   │   ├── create.blade.php
│   │   └── edit.blade.php
│   ├── issues/
│   │   ├── index.blade.php
│   │   └── show.blade.php
│   └── events/
│       └── show.blade.php
routes/
├── web.php        (dashboard, projects, issues, events)
└── api.php        (event ingest endpoint)

Key Design Decisions

  1. Sentry-compatible protocol — Reuse the massive ecosystem of existing Sentry SDKs instead of building custom clients.
  2. Queue-based processing — The ingest endpoint returns 202 Accepted immediately and processes asynchronously, so SDK clients never block.
  3. ULID primary keys — Sortable, time-ordered, no sequential ID leakage.
  4. Fingerprint-based grouping — Same approach as Sentry/Bugsink; keeps the issues list manageable.
  5. Blade + Tailwind (no SPA) — Keep complexity low; server-rendered pages with sprinkles of Alpine.js where needed (e.g. collapsible stacktrace frames, modals).
  6. SQLite for dev, Postgres for prod — SQLite is fine for development and small-scale use. The JSON columns and indexing strategy work on both.
Reference in New Issue View Git Blame Copy Permalink