diff --git a/PLAN.md b/PLAN.md index 74e6936..debd517 100644 --- a/PLAN.md +++ b/PLAN.md @@ -2,6 +2,20 @@ A self-hosted error tracking application (inspired by [Bugsink](https://www.bugsink.com/) / Sentry) built on this Laravel 13 app. +## Overview + +This is an **error tracking application similar to Sentry**, designed to receive and aggregate errors from applications running in production. Users integrate SDKs into their applications (PHP, JavaScript, Python, etc.), and those SDKs send error events **directly** to our tracking server in real-time when errors occur. + +### How It Works + +1. **SDK Integration**: Developers install a Sentry-compatible SDK in their application (e.g., `sentry/sentry-laravel` for Laravel, `@sentry/browser` for JavaScript, `sentry-python` for Python). +2. **Configuration**: They configure the SDK with a DSN (Data Source Name) pointing to our tracking server. +3. **Automatic Error Capture**: When an error occurs in their application, the SDK automatically captures the error context (stacktrace, breadcrumbs, user info, request details) and sends it to our server. +4. **Real-Time Processing**: Our server receives the error payload, groups similar errors into issues, and displays them in a web dashboard. +5. **Notifications**: Teams are notified when new issues occur or when resolved issues regress. + +This architecture means we don't need to build custom SDKs — we leverage the entire ecosystem of existing Sentry SDKs by implementing a Sentry-compatible ingestion API. + ## Stack Summary | Layer | Technology | @@ -12,6 +26,52 @@ A self-hosted error tracking application (inspired by [Bugsink](https://www.bugs | Queue | Database driver (upgrade to Redis for production) | | Auth | Laravel Breeze (installed) | +## SDK Support & Multi-Language Integration + +### Supported Languages & Frameworks + +Our tracking server accepts error events from **any Sentry-compatible SDK**, enabling broad language and framework support without custom development: + +| Language/Platform | Official Sentry SDK | Notes | +|-------------------|---------------------|-------| +| **PHP** | `sentry/sentry-laravel`, `sentry/sentry-symfony`, `sentry/sentry` | Full Laravel, Symfony, and native PHP support | +| **JavaScript/Node.js** | `@sentry/browser`, `@sentry/node`, `@sentry/react`, `@sentry/vue`, `@sentry/angular` | Frontend and backend JS, plus framework-specific integrations | +| **Python** | `sentry-sdk` | Django, Flask, FastAPI, etc. | +| **Ruby** | `sentry-ruby`, `sentry-rails` | Rails and native Ruby support | +| **Java** | `io.sentry:sentry` | Spring, Android, and JVM languages | +| **Go** | `github.com/getsentry/sentry-go` | Native Go support | +| **.NET/C#** | `Sentry.AspNetCore`, `Sentry` | ASP.NET Core, .NET Framework | +| **Mobile** | `@sentry/react-native`, `sentry-android`, `sentry-cocoa` | React Native, Android (Java/Kotlin), iOS (Swift/Obj-C) | +| **Rust** | `sentry` | Native Rust support | +| **Elixir** | `sentry` | Phoenix and Elixir support | + +### SDK Configuration Example + +Users simply point the official Sentry SDK to our server: + +**PHP (Laravel):** +```php +// config/sentry.php +'dsn' => 'https://@trackbugs.example.com/api//store/', +``` + +**JavaScript:** +```javascript +Sentry.init({ + dsn: 'https://@trackbugs.example.com/api//store/', +}); +``` + +**Python:** +```python +import sentry_sdk +sentry_sdk.init( + dsn="https://@trackbugs.example.com/api//store/" +) +``` + +No custom SDK development is required — our server implements the Sentry protocol, so all existing SDKs work out of the box. + --- ## Phase 1 — Foundation & Auth @@ -206,19 +266,19 @@ Clients configure this DSN in their Sentry SDK — no changes needed on the SDK ### 5.1 Release Tracking -- [ ] Parse `release` field from events -- [ ] Show which release introduced an issue -- [ ] Mark issues as resolved-in-release; auto-regress if they appear in a newer release +- [x] Parse `release` field from events +- [x] Show which release introduced an issue +- [x] Mark issues as resolved-in-release; auto-regress if they appear in a newer release ### 5.2 Environment Filtering -- [ ] Global environment selector (production / staging / etc.) -- [ ] Filter issues and events by environment +- [x] Global environment selector (production / staging / etc.) +- [x] Filter issues and events by environment ### 5.3 Retention & Cleanup -- [ ] Artisan command / scheduled job to delete events older than N days (configurable) -- [ ] Keep issue summaries even after events are pruned +- [x] Artisan command / scheduled job to delete events older than N days (configurable) +- [x] Keep issue summaries even after events are pruned ### 5.4 Source Maps / Source Context (optional, stretch) @@ -227,8 +287,8 @@ Clients configure this DSN in their Sentry SDK — no changes needed on the SDK ### 5.5 API Tokens & External API -- [ ] Personal API tokens for programmatic access -- [ ] REST API for listing/resolving issues (useful for CI/CD integrations) +- [x] Personal API tokens for programmatic access +- [x] REST API for listing/resolving issues (useful for CI/CD integrations) --- @@ -258,14 +318,14 @@ Clients configure this DSN in their Sentry SDK — no changes needed on the SDK ### Milestone 4 — Production-readiness - [ ] 17. Rate limiting on ingest -- [ ] 18. Event retention / cleanup scheduler -- [ ] 19. Release tracking -- [ ] 20. Environment filtering +- [x] 18. Event retention / cleanup scheduler +- [x] 19. Release tracking +- [x] 20. Environment filtering - [ ] 21. Performance optimization (indexes, query tuning, caching) ### Milestone 5 — Advanced (stretch) - [ ] 22. Source map upload & processing -- [ ] 23. API tokens & external REST API +- [x] 23. API tokens & external REST API - [ ] 24. Webhook / Slack notifications - [ ] 25. CQRS-style read models for heavy dashboards (if needed) diff --git a/app/Console/Commands/CleanupOldEvents.php b/app/Console/Commands/CleanupOldEvents.php new file mode 100644 index 0000000..88be8a6 --- /dev/null +++ b/app/Console/Commands/CleanupOldEvents.php @@ -0,0 +1,97 @@ +option('days'); + + if ($days < 1) { + $this->error('Days must be at least 1'); + return self::FAILURE; + } + + $cutoffDate = Carbon::now()->subDays($days); + + $this->info("Deleting events older than {$days} days (before {$cutoffDate->toDateTimeString()})..."); + + // Get count of events to be deleted + $eventsToDelete = Event::where('created_at', '<', $cutoffDate)->count(); + + if ($eventsToDelete === 0) { + $this->info('No events to delete.'); + return self::SUCCESS; + } + + if (!$this->confirm("This will delete {$eventsToDelete} events. Continue?", true)) { + $this->info('Cleanup cancelled.'); + return self::SUCCESS; + } + + // Delete events in chunks to avoid memory issues + $deleted = 0; + $bar = $this->output->createProgressBar($eventsToDelete); + $bar->start(); + + Event::where('created_at', '<', $cutoffDate) + ->chunkById(1000, function ($events) use (&$deleted, $bar) { + foreach ($events as $event) { + $event->delete(); + $deleted++; + $bar->advance(); + } + }); + + $bar->finish(); + $this->newLine(2); + + $this->info("Deleted {$deleted} events."); + + // Update issue counters based on remaining events + $this->info('Updating issue counters...'); + + $issues = Issue::all(); + $issuesUpdated = 0; + + foreach ($issues as $issue) { + $eventsCount = $issue->events()->count(); + + if ($eventsCount === 0) { + // If no events remain, optionally delete the issue or mark it + // For now, we'll keep the issue but mark times_seen as 0 + $issue->update(['times_seen' => 0]); + $issuesUpdated++; + } else { + // Update counters based on remaining events + $firstEvent = $issue->events()->oldest('timestamp')->first(); + $lastEvent = $issue->events()->latest('timestamp')->first(); + + $issue->update([ + 'times_seen' => $eventsCount, + 'first_seen_at' => $firstEvent?->timestamp ?? $issue->first_seen_at, + 'last_seen_at' => $lastEvent?->timestamp ?? $issue->last_seen_at, + ]); + $issuesUpdated++; + } + } + + $this->info("Updated {$issuesUpdated} issues."); + $this->info('Cleanup completed successfully!'); + + return self::SUCCESS; + } +} diff --git a/app/Http/Controllers/Api/ApiIssueController.php b/app/Http/Controllers/Api/ApiIssueController.php new file mode 100644 index 0000000..0e30bd3 --- /dev/null +++ b/app/Http/Controllers/Api/ApiIssueController.php @@ -0,0 +1,103 @@ +authorizeProject($request, $project); + + $query = $project->issues(); + + // Apply filters + if ($status = $request->query('status')) { + $query->where('status', $status); + } + + if ($level = $request->query('level')) { + $query->where('level', $level); + } + + if ($environment = $request->query('environment')) { + $query->whereHas('events', fn($q) => $q->where('environment', $environment)); + } + + // Pagination + $perPage = min((int) $request->query('per_page', 25), 100); + $issues = $query->orderBy('last_seen_at', 'desc')->paginate($perPage); + + return response()->json($issues); + } + + /** + * Get a specific issue + */ + public function show(Request $request, Project $project, Issue $issue): JsonResponse + { + $this->authorizeProject($request, $project); + abort_unless($issue->project_id === $project->id, 404); + + $issue->load('events'); + + return response()->json($issue); + } + + /** + * Update issue status + */ + public function update(Request $request, Project $project, Issue $issue): JsonResponse + { + $this->authorizeProject($request, $project); + abort_unless($issue->project_id === $project->id, 404); + + $validated = $request->validate([ + 'status' => 'required|in:resolved,unresolved,ignored,muted', + 'resolved_in_release' => 'nullable|string', + ]); + + if ($validated['status'] === 'resolved') { + $issue->resolve($validated['resolved_in_release'] ?? null); + } elseif ($validated['status'] === 'unresolved') { + $issue->unresolve(); + } elseif ($validated['status'] === 'ignored') { + $issue->ignore(); + } elseif ($validated['status'] === 'muted') { + $issue->mute(); + } + + return response()->json($issue->fresh()); + } + + /** + * Delete an issue + */ + public function destroy(Request $request, Project $project, Issue $issue): JsonResponse + { + $this->authorizeProject($request, $project); + abort_unless($issue->project_id === $project->id, 404); + + $issue->delete(); + + return response()->json(['message' => 'Issue deleted successfully']); + } + + private function authorizeProject(Request $request, Project $project): void + { + $user = $request->user(); + abort_unless( + $user && $user->projects()->where('projects.id', $project->id)->exists(), + 403, + 'Unauthorized' + ); + } +} diff --git a/app/Http/Controllers/IssueController.php b/app/Http/Controllers/IssueController.php index f7d1fab..fad7a35 100644 --- a/app/Http/Controllers/IssueController.php +++ b/app/Http/Controllers/IssueController.php @@ -33,6 +33,12 @@ class IssueController extends Controller }); } + if ($environment = $request->query('environment')) { + $query->whereHas('events', function ($q) use ($environment) { + $q->where('environment', $environment); + }); + } + $sortField = $request->query('sort', 'last_seen_at'); $sortDir = $request->query('dir', 'desc'); $allowedSorts = ['last_seen_at', 'first_seen_at', 'times_seen', 'level']; @@ -42,12 +48,22 @@ class IssueController extends Controller $issues = $query->cursorPaginate(25)->withQueryString(); + // Get distinct environments from events for this project + $environments = $project->events() + ->whereNotNull('environment') + ->distinct() + ->pluck('environment') + ->sort() + ->values(); + return view('issues.index', [ 'project' => $project, 'issues' => $issues, 'currentStatus' => $status ?: 'unresolved', 'currentLevel' => $level, + 'currentEnvironment' => $environment, 'search' => $search, + 'environments' => $environments, ]); } @@ -56,18 +72,36 @@ class IssueController extends Controller $this->authorizeProject($request, $project); abort_unless($issue->project_id === $project->id, 404); - $latestEvent = $issue->events()->latest('timestamp')->first(); + $environment = $request->query('environment'); - $events = $issue->events() + $eventsQuery = $issue->events(); + + if ($environment) { + $eventsQuery->where('environment', $environment); + } + + $latestEvent = (clone $eventsQuery)->latest('timestamp')->first(); + + $events = $eventsQuery ->latest('timestamp') ->cursorPaginate(10) ->withQueryString(); + // Get distinct environments for this issue + $environments = $issue->events() + ->whereNotNull('environment') + ->distinct() + ->pluck('environment') + ->sort() + ->values(); + return view('issues.show', [ 'project' => $project, 'issue' => $issue, 'latestEvent' => $latestEvent, 'events' => $events, + 'environments' => $environments, + 'currentEnvironment' => $environment, ]); } diff --git a/app/Http/Middleware/AuthenticateApiToken.php b/app/Http/Middleware/AuthenticateApiToken.php new file mode 100644 index 0000000..f7e45f8 --- /dev/null +++ b/app/Http/Middleware/AuthenticateApiToken.php @@ -0,0 +1,44 @@ +bearerToken(); + + if (!$token) { + return response()->json(['error' => 'Unauthenticated'], 401); + } + + $hashedToken = hash('sha256', $token); + $accessToken = PersonalAccessToken::where('token', $hashedToken)->first(); + + if (!$accessToken) { + return response()->json(['error' => 'Invalid token'], 401); + } + + if ($accessToken->isExpired()) { + return response()->json(['error' => 'Token expired'], 401); + } + + // Set the authenticated user + $request->setUserResolver(fn() => $accessToken->tokenable); + + // Mark token as used + $accessToken->markAsUsed(); + + return $next($request); + } +} diff --git a/app/Models/Issue.php b/app/Models/Issue.php index 54d69c1..f5c12ac 100644 --- a/app/Models/Issue.php +++ b/app/Models/Issue.php @@ -24,6 +24,8 @@ class Issue extends Model 'times_seen', 'is_regression', 'metadata', + 'first_release', + 'resolved_in_release', ]; protected function casts(): array @@ -51,9 +53,13 @@ class Issue extends Model return $this->events()->latest('timestamp')->limit(1); } - public function resolve(): void + public function resolve(?string $release = null): void { - $this->update(['status' => 'resolved', 'is_regression' => false]); + $this->update([ + 'status' => 'resolved', + 'is_regression' => false, + 'resolved_in_release' => $release, + ]); } public function ignore(): void diff --git a/app/Models/PersonalAccessToken.php b/app/Models/PersonalAccessToken.php new file mode 100644 index 0000000..78024f8 --- /dev/null +++ b/app/Models/PersonalAccessToken.php @@ -0,0 +1,46 @@ + 'array', + 'last_used_at' => 'datetime', + 'expires_at' => 'datetime', + ]; + } + + public function tokenable(): MorphTo + { + return $this->morphTo(); + } + + public function can(string $ability): bool + { + return in_array('*', $this->abilities ?? []) || in_array($ability, $this->abilities ?? []); + } + + public function markAsUsed(): void + { + $this->update(['last_used_at' => now()]); + } + + public function isExpired(): bool + { + return $this->expires_at && $this->expires_at->isPast(); + } +} diff --git a/app/Models/User.php b/app/Models/User.php index 252531c..03b7197 100644 --- a/app/Models/User.php +++ b/app/Models/User.php @@ -9,8 +9,10 @@ use Illuminate\Database\Eloquent\Attributes\Hidden; use Illuminate\Database\Eloquent\Factories\HasFactory; use Illuminate\Database\Eloquent\Relations\BelongsToMany; use Illuminate\Database\Eloquent\Relations\HasMany; +use Illuminate\Database\Eloquent\Relations\MorphMany; use Illuminate\Foundation\Auth\User as Authenticatable; use Illuminate\Notifications\Notifiable; +use Illuminate\Support\Str; #[Fillable(['name', 'email', 'password'])] #[Hidden(['password', 'remember_token'])] @@ -52,4 +54,21 @@ class User extends Authenticatable ['email_enabled' => true, 'database_enabled' => true, 'muted' => false], ); } + + public function tokens(): MorphMany + { + return $this->morphMany(PersonalAccessToken::class, 'tokenable'); + } + + public function createToken(string $name, array $abilities = ['*'], ?\DateTimeInterface $expiresAt = null): PersonalAccessToken + { + $token = Str::random(64); + + return $this->tokens()->create([ + 'name' => $name, + 'token' => hash('sha256', $token), + 'abilities' => $abilities, + 'expires_at' => $expiresAt, + ]); + } } diff --git a/app/Services/EventProcessor.php b/app/Services/EventProcessor.php index 4e699f9..40bb7f4 100644 --- a/app/Services/EventProcessor.php +++ b/app/Services/EventProcessor.php @@ -105,6 +105,7 @@ class EventProcessor 'last_seen_at' => $now, 'times_seen' => 1, 'is_regression' => false, + 'first_release' => $data['release'], 'metadata' => array_filter([ 'platform' => $data['platform'], 'sdk' => $data['sdk'], @@ -115,11 +116,21 @@ class EventProcessor private function updateExistingIssue(Issue $issue, array $data, Carbon $now): bool { $wasResolved = $issue->status === 'resolved'; + $currentRelease = $data['release']; + $resolvedInRelease = $issue->resolved_in_release; $issue->increment('times_seen'); $issue->update(['last_seen_at' => $now]); + // Check for regression: issue was resolved, and now appears again if ($wasResolved) { + // If we have release tracking and the issue was resolved in a release, + // only mark as regression if the current release is different (assumed newer) + if ($resolvedInRelease && $currentRelease && $currentRelease === $resolvedInRelease) { + // Same release as resolved - don't mark as regression + return false; + } + $issue->markRegression(); return true; diff --git a/database/migrations/2026_03_21_131503_add_release_tracking_to_issues_table.php b/database/migrations/2026_03_21_131503_add_release_tracking_to_issues_table.php new file mode 100644 index 0000000..9f5b620 --- /dev/null +++ b/database/migrations/2026_03_21_131503_add_release_tracking_to_issues_table.php @@ -0,0 +1,29 @@ +string('first_release')->nullable()->after('first_seen_at'); + $table->string('resolved_in_release')->nullable()->after('status'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('issues', function (Blueprint $table) { + $table->dropColumn(['first_release', 'resolved_in_release']); + }); + } +}; diff --git a/database/migrations/2026_03_21_131922_create_personal_access_tokens_table.php b/database/migrations/2026_03_21_131922_create_personal_access_tokens_table.php new file mode 100644 index 0000000..e828ad8 --- /dev/null +++ b/database/migrations/2026_03_21_131922_create_personal_access_tokens_table.php @@ -0,0 +1,33 @@ +id(); + $table->morphs('tokenable'); + $table->string('name'); + $table->string('token', 64)->unique(); + $table->text('abilities')->nullable(); + $table->timestamp('last_used_at')->nullable(); + $table->timestamp('expires_at')->nullable(); + $table->timestamps(); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::dropIfExists('personal_access_tokens'); + } +}; diff --git a/database/seeders/DatabaseSeeder.php b/database/seeders/DatabaseSeeder.php index 6b901f8..ab75279 100644 --- a/database/seeders/DatabaseSeeder.php +++ b/database/seeders/DatabaseSeeder.php @@ -20,6 +20,7 @@ class DatabaseSeeder extends Seeder User::factory()->create([ 'name' => 'Test User', 'email' => 'test@example.com', + 'password' => bcrypt('password'), ]); } } diff --git a/package-lock.json b/package-lock.json index 34a17af..bc4597b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,5 +1,5 @@ { - "name": "test_app_two", + "name": "html", "lockfileVersion": 3, "requires": true, "packages": { diff --git a/resources/views/issues/index.blade.php b/resources/views/issues/index.blade.php index 790ad73..7c721b9 100644 --- a/resources/views/issues/index.blade.php +++ b/resources/views/issues/index.blade.php @@ -1,5 +1,7 @@ - @php($currentProject = $project) + @php + $currentProject = $project; + @endphp
@@ -41,8 +43,19 @@ @endforeach
+ @if($environments->isNotEmpty()) +
+ + +
+ @endif Filter - @if($search || $currentLevel || $currentStatus !== 'unresolved') + @if($search || $currentLevel || $currentEnvironment || $currentStatus !== 'unresolved') Reset @endif diff --git a/resources/views/issues/show.blade.php b/resources/views/issues/show.blade.php index ca7f2a9..852bb61 100644 --- a/resources/views/issues/show.blade.php +++ b/resources/views/issues/show.blade.php @@ -39,6 +39,26 @@ @endif + {{-- Environment Filter --}} + @if($environments->isNotEmpty()) +
+
+
+ + +
+ @if($currentEnvironment) + Reset + @endif +
+
+ @endif + {{-- Issue Meta --}} @php $statusColors = [ @@ -88,6 +108,18 @@ Last Seen
{{ $issue->last_seen_at->diffForHumans() }}
+ @if($issue->first_release) +
+ First Release +
{{ $issue->first_release }}
+
+ @endif + @if($issue->resolved_in_release) +
+ Resolved In +
{{ $issue->resolved_in_release }}
+
+ @endif diff --git a/routes/api.php b/routes/api.php index 07bbec2..6fb65e5 100644 --- a/routes/api.php +++ b/routes/api.php @@ -1,9 +1,22 @@ middleware(AuthenticateDsn::class) ->name('api.events.store'); + +// REST API (uses token authentication) +Route::middleware(AuthenticateApiToken::class)->prefix('v1')->group(function () { + Route::prefix('projects/{project}/issues')->group(function () { + Route::get('/', [ApiIssueController::class, 'index'])->name('api.v1.issues.index'); + Route::get('/{issue}', [ApiIssueController::class, 'show'])->name('api.v1.issues.show'); + Route::patch('/{issue}', [ApiIssueController::class, 'update'])->name('api.v1.issues.update'); + Route::delete('/{issue}', [ApiIssueController::class, 'destroy'])->name('api.v1.issues.destroy'); + }); +});