analytics/PLAN.md

Analytics Platform — Implementation Plan

Current State

The foundation is in place: 7 models with migrations (Website, Visitor, AnalyticsSession, PageView, Event, Goal, GoalConversion), a JavaScript tracker served dynamically per tracking ID, and a POST /api/collect endpoint that ingests page views and events. No authentication, no dashboard, no frontend yet.

---

Phase 1: Authentication & Website Management

The platform needs user accounts and CRUD for tracked websites before anything else is usable.

• Install Laravel Breeze (or Fortify) for registration, login, password reset, email verification
• Build a WebsiteController with full CRUD (index, create, store, show, edit, update, destroy)
• Create views/pages: list of websites, add/edit website form, website settings page
• Build the "embed snippet" page — show the user the <script> tag to copy for each website
• Add form request validation (StoreWebsiteRequest, UpdateWebsiteRequest)
• Scope all website queries to auth()->user() (policy or middleware)
• Add a WebsitePolicy to authorize show/edit/delete actions

---

Phase 2: Dashboard — Real-Time Overview

The main dashboard a user sees after selecting a website, showing key metrics at a glance.

• Create DashboardController scoped to a single website
• Build a date-range picker component (today, last 7 days, last 30 days, custom range)
• Metric cards — display for the selected period:
  • Total page views
  • Unique visitors
  • Total sessions
  • Bounce rate (% of sessions with 1 page view)
  • Average session duration
  • Average pages per session
• Visitors chart — line/area chart of visitors and page views over time (daily or hourly granularity)
• Real-time widget — show active visitors in the last 5 minutes (poll or WebSocket)
• Create an AnalyticsQueryService (or similar) to encapsulate all aggregation queries, keeping controllers thin

---

Phase 3: Detailed Reports

Individual report pages accessible from the dashboard sidebar.

3a — Audience Reports

• Geographic report — visitors by country, region, city (table + world map visualization)
• Technology report — breakdown by browser, OS, device type, screen resolution
• Language report — visitors by browser language
• New vs. returning visitors — segment visitors who have more than one session

3b — Acquisition Reports

• Traffic sources — breakdown by referrer domain (direct, organic, referral, social)
• Campaigns (UTM) — table of sessions grouped by utm_source, utm_medium, utm_campaign
• Referrers — top referring URLs and domains

3c — Behavior Reports

• Pages report — top pages by views, unique views, avg. time on page, bounce rate
• Landing pages — top entry pages with session count and bounce rate
• Exit pages — top exit pages
• User flow — simplified Sankey or funnel showing common page sequences (can be deferred)

3d — Events Report

• Events overview — table of events grouped by name, category, action
• Event detail — drill-down into a single event name with label/value breakdown
• Event trending — chart of event counts over time

---

Phase 4: Goals & Conversions

Allow users to define goals and track conversion rates.

• Build GoalController CRUD (create, edit, delete goals per website)
• Support goal types:
  • URL destination — session visited a specific path
  • Event match — a specific event name/category was fired
  • Duration — session lasted longer than N seconds
  • Pages per session — session hit N+ page views
• Create a GoalEvaluationService that checks incoming data against active goals and records GoalConversion entries
• Hook goal evaluation into CollectController after recording page view / event
• Build a Conversions report — goal completion count, conversion rate (completions / sessions), goal value totals
• Add goal completion sparklines to the main dashboard

---

Phase 5: Collection Pipeline Hardening

The CollectController currently does synchronous DB writes. This won't scale under real traffic.

• Move ingestion to a queued job (ProcessTrackingPayload) dispatched from the controller so the endpoint returns instantly
• Add rate limiting middleware to POST /api/collect (e.g., 120 req/min per IP)
• Add CORS headers to the collect endpoint so browsers allow cross-origin beacons
• Validate the incoming referrer / url against the website's domain to reject spoofed traffic
• Add IP-based geolocation (use MaxMind GeoLite2 via stevebauman/location) to populate country, region, city on the Visitor
• Handle bot filtering — skip known bot user agents (using jaybizzle/crawler-detect)
• Add a tracking_pixel endpoint (1x1 GIF) as a no-JS fallback

---

Phase 6: Data Aggregation & Performance

Raw event tables grow fast. Pre-aggregate data for fast dashboard loads.

• Create daily_aggregates table (migration + model):
  • website_id, date, visitors, sessions, page_views, bounce_count, total_duration
• Create page_daily_aggregates table:
  • website_id, date, path, views, unique_views, total_time_on_page, bounces
• Create a scheduled Artisan command (analytics:aggregate) that rolls up yesterday's raw data into aggregate tables — run daily via schedule:run
• Refactor dashboard and report queries to read from aggregates for past days, and query raw tables only for "today"
• Add database indexes review — ensure all date-range + website_id queries are covered
• Add data_retention_days setting per website (stored in settings JSON) and a cleanup command (analytics:purge) to purge old raw rows

---

Phase 7: Frontend & UI

Choose a frontend approach and build a polished interface.

• Pick a stack:
  • Option A: Blade + Alpine.js (server-rendered, simpler) — selected
• Install Tailwind CSS (already in the Laravel scaffold via Vite) + Inter font
• Build a layout with sidebar navigation (websites list, reports sections) and top bar (date picker, user menu)
• Integrate a charting library (Chart.js + chartjs-chart-geo) for:
  • Line charts (visitors over time)
  • Bar charts (top pages, browsers, sources)
  • Pie/donut charts (device type, traffic source split, new vs returning)
  • World map (geographic report — choropleth)
• Build comparison mode — compare current period vs. previous period with delta percentages on metric cards
• Add loading states, empty states, and error handling throughout
• Make the dashboard responsive (mobile-friendly) — collapsible sidebar with overlay on mobile

---

Phase 8: API for Programmatic Access ✅

Let users pull their analytics data via an API.

• Create API token management (Laravel Sanctum) — UI page to generate/revoke API tokens
• Build versioned API routes (/api/v1/...)
• Endpoints:
  • GET /api/v1/websites — list websites
  • GET /api/v1/websites/{id}/stats — summary metrics with date-range params
  • GET /api/v1/websites/{id}/pages — top pages
  • GET /api/v1/websites/{id}/referrers — traffic sources
  • GET /api/v1/websites/{id}/events — events report
  • GET /api/v1/websites/{id}/visitors — audience breakdown
• Add API resource classes for consistent JSON formatting
• Rate limit API endpoints per token

---

Phase 9: Advanced Features ✅

Features that differentiate from a basic analytics tool.

• Funnels — let users define a sequence of pages/events and visualize drop-off at each step
• Segments — filter all reports by visitor attributes (e.g., mobile users from Germany)
• Custom dashboards — let users pin specific widgets/reports to a custom overview
• Alerts / Notifications — notify users when traffic spikes, drops, or a goal hits a threshold (email and/or in-app)
• Data export — CSV/PDF export for any report
• UTM builder — in-app tool to generate campaign URLs with UTM parameters
• Annotations — let users add notes on specific dates (e.g., "launched ad campaign") visible on charts

---

Phase 10: Testing & Quality ✅

• Write Feature tests for CollectController (valid payloads, invalid tracking IDs, rate limiting)
• Write Feature tests for WebsiteController CRUD and authorization
• Write Unit tests for AnalyticsQueryService aggregation logic
• Write Unit tests for GoalEvaluationService
• Write browser tests (Dusk) for dashboard interactions and date picker (deferred — requires Dusk package + browser driver)
• Add CI pipeline (GitHub Actions): run tests, Pint linting, Larastan static analysis
• Create model factories for all analytics models
• Fix base Controller AuthorizesRequests trait
• Fix DailyAggregate/PageDailyAggregate date cast for SQLite compatibility

---

Phase 11: Deployment & Operations

• Set up production database (PostgreSQL recommended for analytical queries, or MySQL)
• Configure queue worker (Redis + Horizon) for async ingestion jobs
• Set up the scheduler (php artisan schedule:run) for daily aggregation
• Configure caching (Redis) for dashboard queries and script delivery
• Set up HTTPS and CDN for the tracker JS file
• Add application monitoring (Sentry or similar for error tracking)
• Configure backups for the database
• Write a deployment script or use Laravel Forge / Vapor

---

Suggested Order of Execution

Priority	Phase	Reason
1	Phase 1	Can't use anything without auth and website management
2	Phase 7 (partial)	Pick frontend stack and build layout early — everything else renders into it
3	Phase 2	The dashboard is the core product
4	Phase 5	Harden collection before driving real traffic
5	Phase 3	Detailed reports give the platform actual value
6	Phase 4	Goals & conversions are a key differentiator
7	Phase 6	Aggregation becomes necessary as data grows
8	Phase 10	Tests before shipping to production
9	Phase 8	API access for power users
10	Phase 9	Advanced features for product maturity
11	Phase 11	Production deployment