docs: add STATUS.md and MARKETING.md
STATUS.md — full project state, what's done, what's left, deployment checklist, key commands, architecture notes MARKETING.md — go-to-market plan with positioning, target audience, competitive analysis, 90-day content calendar, launch checklist, pricing strategy, metrics
This commit is contained in:
@@ -0,0 +1,242 @@
|
||||
# Analytics Platform — Project Status
|
||||
|
||||
> Last updated: 2026-05-25
|
||||
|
||||
---
|
||||
|
||||
## What Is This
|
||||
|
||||
A self-hosted web analytics platform built with Laravel 11 — a privacy-focused alternative to Google Analytics or Plausible. Users add a lightweight JavaScript snippet to their website and get real-time dashboards, AI-ready data pipelines, goal/funnel tracking, and a full REST API.
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Layer | Choice |
|
||||
|---|---|
|
||||
| Backend | Laravel 11, PHP 8.2+ |
|
||||
| Auth | Laravel Fortify (2FA ready) |
|
||||
| Frontend | Blade + Alpine.js + Tailwind CSS |
|
||||
| Charts | Chart.js |
|
||||
| Database | SQLite (dev) / MySQL or PostgreSQL (prod) |
|
||||
| Queue | Database driver (dev) / Redis + Horizon (prod) |
|
||||
| API | Laravel Sanctum (token auth) |
|
||||
| Geolocation | stevebauman/location |
|
||||
| Bot detection | jaybizzle/crawler-detect |
|
||||
| Build | Vite |
|
||||
|
||||
---
|
||||
|
||||
## Feature Status
|
||||
|
||||
### ✅ Done — Core Platform
|
||||
|
||||
**Authentication & Users**
|
||||
- Registration, login, password reset, email verification via Fortify
|
||||
- Two-factor authentication support
|
||||
- Profile management
|
||||
- API token management (Sanctum)
|
||||
|
||||
**Website Management**
|
||||
- Full CRUD for tracked websites
|
||||
- Embed snippet page with copy-ready `<script>` tag
|
||||
- WebsitePolicy — all queries scoped to authenticated owner
|
||||
|
||||
**Data Collection Pipeline**
|
||||
- `POST /api/e` and `POST /api/collect` endpoints
|
||||
- Async ingestion via `ProcessTrackingPayload` queued job
|
||||
- Bot filtering (CrawlerDetect)
|
||||
- Domain validation (rejects spoofed tracking IDs)
|
||||
- IP geolocation → country, region, city on Visitor
|
||||
- CORS middleware for cross-origin beacons
|
||||
- Tracking pixel (1×1 GIF) as no-JS fallback
|
||||
- Rate limiting (120 req/min per IP)
|
||||
- Ad-blocker resistant alternate endpoints (`/cdn/feedback`, `/cdn/resources/`)
|
||||
|
||||
**Dashboard**
|
||||
- Date-range picker (today / 7d / 30d / custom)
|
||||
- Metric cards: page views, unique visitors, sessions, bounce rate, avg session duration, pages/session
|
||||
- Visitors & page views chart (daily/hourly granularity)
|
||||
- Real-time active visitors widget (last 5 minutes)
|
||||
- Comparison mode vs. previous period with delta %
|
||||
- Goal conversion sparklines
|
||||
|
||||
**Reports — Audience**
|
||||
- Geographic: visitors by country, region, city (table + world map)
|
||||
- Technology: browser, OS, device type, screen resolution
|
||||
- Language: browser language breakdown
|
||||
- New vs. returning visitors
|
||||
|
||||
**Reports — Acquisition**
|
||||
- Traffic sources (direct, organic, referral, social)
|
||||
- UTM campaigns (source, medium, campaign)
|
||||
- Top referrers
|
||||
|
||||
**Reports — Behavior**
|
||||
- Top pages (views, unique views, avg time, bounce rate)
|
||||
- Landing pages (entry pages with session count and bounce rate)
|
||||
- Exit pages
|
||||
|
||||
**Reports — Events**
|
||||
- Events overview (grouped by name/category/action)
|
||||
- Event detail drill-down (label/value breakdown)
|
||||
- Event trending chart
|
||||
|
||||
**Reports — Conversions**
|
||||
- Goal completions, conversion rate, goal value totals
|
||||
|
||||
**Goals & Conversions**
|
||||
- 4 goal types: URL destination, event match, duration, pages per session
|
||||
- GoalEvaluationService — evaluates goals on every incoming payload
|
||||
- Conversion rate reporting
|
||||
|
||||
**Advanced Features**
|
||||
- Funnels — multi-step conversion tracking with drop-off visualization
|
||||
- Segments — filter all reports by visitor attributes
|
||||
- Custom dashboards — pin widgets to a personal overview
|
||||
- Alerts — notify on traffic spikes/drops/goal thresholds (email + in-app)
|
||||
- Annotations — add notes to specific dates on charts
|
||||
- Data export — CSV export for any report
|
||||
- UTM builder — in-app campaign URL generator
|
||||
|
||||
**API (v1)**
|
||||
- `GET /api/v1/websites` — list websites
|
||||
- `GET /api/v1/websites/{id}/stats` — summary metrics with date-range params
|
||||
- `GET /api/v1/websites/{id}/pages` — top pages
|
||||
- `GET /api/v1/websites/{id}/referrers` — traffic sources
|
||||
- `GET /api/v1/websites/{id}/events` — events
|
||||
- `GET /api/v1/websites/{id}/visitors` — audience breakdown
|
||||
- JSON:API resource classes for consistent formatting
|
||||
- Rate limiting per token
|
||||
|
||||
**Data Aggregation**
|
||||
- `daily_aggregates` table — pre-computed per-website daily metrics
|
||||
- `page_daily_aggregates` table — per-page daily metrics
|
||||
- `analytics:aggregate` Artisan command (runs daily at 02:00)
|
||||
- Hybrid query strategy: aggregates for past days, raw tables for today
|
||||
- `analytics:purge` command — purge raw data older than retention window
|
||||
- Configurable `data_retention_days` per website (default: 365)
|
||||
|
||||
**Performance & Indexes**
|
||||
- Composite indexes on all date-range + website_id query paths
|
||||
- SQLite-compatible index migrations (also works with MySQL/PostgreSQL)
|
||||
|
||||
**Testing**
|
||||
- 80 tests, 0 failures
|
||||
- Feature tests: `CollectController`, `WebsiteController`, Auth flows
|
||||
- Unit tests: `AnalyticsQueryService`, `GoalEvaluationService`
|
||||
- Model factories for all analytics models
|
||||
- `withoutVite()` in base TestCase — tests run without a built frontend
|
||||
|
||||
---
|
||||
|
||||
### 🔲 Not Yet Done
|
||||
|
||||
| Item | Notes |
|
||||
|---|---|
|
||||
| **User Flow / Sankey diagram** | Intentionally deferred — complex to visualize, low priority for v1 |
|
||||
| **Production database** | SQLite is dev-only; set up MySQL or PostgreSQL before launch |
|
||||
| **Redis + Horizon** | Replace database queue driver for real traffic; Horizon gives a monitoring UI |
|
||||
| **Cron / scheduler** | Configure `schedule:run` on the server — needed for aggregation, purge, and alerts |
|
||||
| **HTTPS + CDN for tracker JS** | Tracker script should load from a CDN subdomain to avoid ad-blockers |
|
||||
| **Error monitoring** | Wire up Sentry or Flare for production error tracking |
|
||||
| **Database backups** | Automate daily dumps with offsite storage |
|
||||
| **Dusk browser tests** | UI interaction tests deferred — requires Dusk + ChromeDriver setup |
|
||||
|
||||
---
|
||||
|
||||
## Running Locally
|
||||
|
||||
```bash
|
||||
cd ~/Projects/analytics
|
||||
composer install
|
||||
cp .env.example .env
|
||||
php artisan key:generate
|
||||
php artisan migrate
|
||||
npm install && npm run build
|
||||
composer run dev # starts web server + queue worker + log viewer + Vite
|
||||
```
|
||||
|
||||
App runs at `http://localhost:8000`.
|
||||
|
||||
### Background processes required
|
||||
|
||||
```bash
|
||||
# Queue worker — processes tracking events (critical)
|
||||
php artisan queue:work database --tries=3
|
||||
|
||||
# Scheduler — aggregation, purge, alerts (critical)
|
||||
php artisan schedule:work
|
||||
|
||||
# OR add to crontab for production:
|
||||
# * * * * * cd /path/to/analytics && php artisan schedule:run >> /dev/null 2>&1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment Checklist
|
||||
|
||||
- [ ] Provision server (Ubuntu 22+ recommended)
|
||||
- [ ] Install PHP 8.2+, Nginx, MySQL/PostgreSQL, Redis, Node
|
||||
- [ ] Clone repo, `composer install --no-dev`, `npm ci && npm run build`
|
||||
- [ ] Set `APP_ENV=production`, `APP_DEBUG=false`, configure `.env`
|
||||
- [ ] Run `php artisan migrate --force`
|
||||
- [ ] Run `php artisan config:cache route:cache view:cache`
|
||||
- [ ] Configure Supervisor for queue worker (see `HOWTO.md`)
|
||||
- [ ] Add crontab entry for `schedule:run`
|
||||
- [ ] Point Nginx document root to `public/`
|
||||
- [ ] Set up SSL (Let's Encrypt)
|
||||
- [ ] Set up CNAME for tracker script (e.g. `track.yourdomain.com`)
|
||||
- [ ] Configure Sentry DSN in `.env`
|
||||
- [ ] Set up automated database backups
|
||||
|
||||
---
|
||||
|
||||
## Key Artisan Commands
|
||||
|
||||
| Command | Description | Schedule |
|
||||
|---|---|---|
|
||||
| `analytics:aggregate` | Roll up yesterday's raw data into aggregates | Daily 02:00 |
|
||||
| `analytics:purge --force` | Delete raw data older than retention window | Daily 03:00 |
|
||||
| `analytics:check-alerts` | Evaluate alert rules and send notifications | Every hour |
|
||||
| `queue:work database --tries=3` | Process tracking payloads | Always-on process |
|
||||
|
||||
---
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
### Data Flow
|
||||
```
|
||||
Client website (JS snippet)
|
||||
→ POST /api/e (CollectController)
|
||||
→ ProcessTrackingPayload job (queued)
|
||||
→ Creates/updates: Visitor, AnalyticsSession, PageView or Event
|
||||
→ GoalEvaluationService checks active goals
|
||||
→ Daily: analytics:aggregate rolls raw → daily_aggregates
|
||||
→ Dashboard queries: aggregates for history + raw tables for today
|
||||
```
|
||||
|
||||
### Collector Extraction (future)
|
||||
The `collector_app_plan.md` documents a plan to split the data collection endpoints into a separate Laravel app for independent scaling. This is not needed yet but the code is structured to make it straightforward — `CollectController`, `TrackingScriptController`, `ProcessTrackingPayload`, and `GoalEvaluationService` are all self-contained.
|
||||
|
||||
---
|
||||
|
||||
## Database Schema (Core Tables)
|
||||
|
||||
| Table | Purpose |
|
||||
|---|---|
|
||||
| `websites` | Tracked websites, settings, tracking ID |
|
||||
| `visitors` | Unique visitors with device, browser, location |
|
||||
| `analytics_sessions` | Sessions with UTM params, entry/exit page, duration |
|
||||
| `page_views` | Individual page visits |
|
||||
| `events` | Custom events with category, label, value |
|
||||
| `goals` | Conversion goal definitions (4 types) |
|
||||
| `goal_conversions` | Goal completion records |
|
||||
| `funnels` | Multi-step funnel definitions |
|
||||
| `segments` | Custom visitor segment definitions |
|
||||
| `alerts` | Alert rule definitions |
|
||||
| `alert_notifications` | Triggered alert history |
|
||||
| `annotations` | Date annotations visible on charts |
|
||||
| `dashboard_widgets` | Custom dashboard layout per user |
|
||||
| `daily_aggregates` | Pre-computed daily metrics per website |
|
||||
| `page_daily_aggregates` | Pre-computed daily metrics per page |
|
||||
Reference in New Issue
Block a user