analytics/README.md

<p align="center">
  <img src="public/assets/img/auth_logo_icon_main.png" alt="LibreMetrics" width="80">
</p>

<h1 align="center">LibreMetrics</h1>

<p align="center">
  <strong>Open-source, self-hosted web analytics. Your data. Your server. No Google.</strong>
</p>

<p align="center">
  <a href="https://github.com/yourusername/libremetrics/actions"><img src="https://img.shields.io/github/actions/workflow/status/yourusername/libremetrics/tests.yml?label=tests&style=flat-square" alt="Tests"></a>
  <a href="https://github.com/yourusername/libremetrics/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue?style=flat-square" alt="License"></a>
  <img src="https://img.shields.io/badge/PHP-8.2+-purple?style=flat-square" alt="PHP 8.2+">
  <img src="https://img.shields.io/badge/Laravel-11-red?style=flat-square" alt="Laravel 11">
  <img src="https://img.shields.io/badge/tests-80%20passing-brightgreen?style=flat-square" alt="80 tests passing">
</p>

<p align="center">
  <a href="#-features">Features</a> •
  <a href="#-quick-start">Quick Start</a> •
  <a href="#-installation">Installation</a> •
  <a href="#-tracking-setup">Tracking Setup</a> •
  <a href="#-api">API</a> •
  <a href="#-contributing">Contributing</a>
</p>

---

## What is LibreMetrics?

LibreMetrics is a full-featured, self-hosted web analytics platform built with Laravel 11. It gives you the depth of Google Analytics without handing your visitors' data to a third party — and without the per-pageview pricing of SaaS alternatives like Plausible or Fathom.

Install it on a $5 VPS, point your tracking snippet at it, and get real-time dashboards, goal tracking, funnels, segments, custom alerts, and a REST API — all on infrastructure you own and control.

**Why self-host?**
- 🔒 **Privacy by design** — visitor data never leaves your server, GDPR-compliant out of the box
- 💰 **One price: $0** — unlimited websites, unlimited pageviews, no per-seat fees
- 🔧 **Full control** — extend it, customize it, integrate it with anything
- 🚫 **No cookie banners needed** — no cross-site tracking, no third-party scripts

---

## ✨ Features

### Core Analytics
- **Real-time dashboard** — see active visitors right now with a live counter
- **Key metrics** — page views, unique visitors, sessions, bounce rate, avg. session duration, pages/session
- **Comparison mode** — compare any period vs the previous one, with delta percentages on every card
- **Visitors & page views chart** — daily or hourly granularity, with sessions overlay

### Audience Reports
- 🌍 **Geographic** — visitors by country, region, and city with a world map visualization
- 💻 **Technology** — browser, OS, device type, screen resolution breakdown
- 🌐 **Language** — browser language distribution
- 🔄 **New vs. Returning** — segment first-time vs. repeat visitors

### Acquisition Reports
- **Traffic sources** — direct, organic, referral, and social split
- **Campaigns (UTM)** — group sessions by `utm_source`, `utm_medium`, `utm_campaign`
- **Referrers** — top referring domains and full URLs

### Behavior Reports
- **Top pages** — views, unique views, avg. time on page, bounce rate per path
- **Landing pages** — entry pages with session count and bounce rate
- **Exit pages** — where visitors are leaving

### Events & Goals
- **Custom event tracking** — fire events from your frontend with category, label, and value
- **Events report** — grouped by name/category, with drill-down and trending chart
- **4 goal types:**
  - URL destination (exact, starts-with, contains, regex)
  - Event match (name + optional category)
  - Session duration (> N seconds)
  - Pages per session (≥ N pages)
- **Conversions report** — completion count, conversion rate, goal value totals, sparklines on dashboard

### Advanced Features
- **Funnels** — define multi-step sequences and visualize drop-off at each step
- **Segments** — filter any report by visitor attributes (device, country, browser, etc.)
- **Custom dashboards** — pin widgets to a personal overview page
- **Alerts** — get notified by email when traffic spikes, drops, or a goal hits a threshold
- **Annotations** — add notes to specific dates that appear on charts
- **Data export** — export any report to CSV
- **UTM builder** — generate campaign URLs with UTM parameters right in the app

### Developer Features
- **REST API** — pull any metric programmatically with token authentication
- **API token management** — create and revoke tokens from the dashboard
- **Async ingestion pipeline** — collection endpoint returns instantly; processing is queued
- **Bot filtering** — known crawlers are silently rejected
- **1×1 tracking pixel** — no-JavaScript fallback for email or restricted environments
- **Ad-blocker resistant endpoints** — alternate `/cdn/` paths for the tracker script and collection

### Performance & Scale
- **Daily aggregation** — raw data is pre-aggregated nightly for fast dashboard queries
- **Hybrid query strategy** — aggregates for history, raw tables for today's live data
- **Configurable data retention** — set per-website retention from 30 days to forever
- **Purge command** — automatically clean up raw data beyond the retention window
- **Performance indexes** — all date-range + website_id query paths are indexed

---

## 🚀 Quick Start

The fastest path to a running instance:

```bash
git clone https://github.com/yourusername/libremetrics.git
cd libremetrics
composer install
cp .env.example .env
php artisan key:generate
php artisan migrate
npm install && npm run build
composer run dev
```

Open `http://localhost:8000`, register an account, add your first website, and copy the tracking snippet. Done.

> **`composer run dev`** starts four processes in parallel: web server, queue worker, log viewer, and Vite dev server.

---

## 📦 Installation

### Requirements

| Requirement | Version |
|---|---|
| PHP | 8.2+ |
| Composer | 2.x |
| Node.js | 18+ |
| Database | SQLite (dev) / MySQL 8+ / PostgreSQL 14+ |
| Queue | Database (dev) / Redis (production) |

### Step-by-step

**1. Clone and install dependencies**

```bash
git clone https://github.com/yourusername/libremetrics.git libremetrics
cd libremetrics
composer install
npm install
```

**2. Configure environment**

```bash
cp .env.example .env
php artisan key:generate
```

Edit `.env` for your environment. Minimum required changes:

```env
APP_URL=https://analytics.yourdomain.com

# Database (SQLite works great for small installs)
DB_CONNECTION=sqlite

# Or for MySQL/PostgreSQL:
# DB_CONNECTION=mysql
# DB_HOST=127.0.0.1
# DB_DATABASE=libremetrics
# DB_USERNAME=your_user
# DB_PASSWORD=your_password

# Mail (for alerts and email verification)
MAIL_MAILER=smtp
MAIL_HOST=your-smtp-host
MAIL_PORT=587
MAIL_USERNAME=your@email.com
MAIL_PASSWORD=your-password
MAIL_FROM_ADDRESS=analytics@yourdomain.com
```

**3. Run migrations and build assets**

```bash
php artisan migrate
npm run build
```

**4. Start the application**

```bash
# Development (all-in-one):
composer run dev

# Production — use a process supervisor for these:
php artisan serve                             # or configure Nginx + PHP-FPM
php artisan queue:work database --tries=3    # REQUIRED: processes tracking events
php artisan schedule:work                    # REQUIRED: aggregation, purge, alerts
```

> ⚠️ The queue worker and scheduler are **critical**. Without the queue worker, tracking events are dropped. Without the scheduler, reports slow down over time and alerts never fire.

---

### Production Setup (Nginx + PHP-FPM)

Point your Nginx document root to the `public/` directory:

```nginx
server {
    listen 443 ssl http2;
    server_name analytics.yourdomain.com;
    root /var/www/libremetrics/public;

    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }
}
```

**Supervisor config for the queue worker:**

```ini
[program:libremetrics-queue]
command=php /var/www/libremetrics/artisan queue:work database --tries=3 --timeout=60
autostart=true
autorestart=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/var/www/libremetrics/storage/logs/queue.log
```

**Cron entry for the scheduler:**

```cron
* * * * * cd /var/www/libremetrics && php artisan schedule:run >> /dev/null 2>&1
```

---

## 🎯 Tracking Setup

### 1. Add a website

Register, go to **Websites → Add Website**, enter your site's name and domain. You'll get a tracking ID like `TA-XXXXXXXXXX`.

### 2. Install the snippet

Paste this into your HTML before `</head>`:

```html
<script async src="https://analytics.yourdomain.com/api/t/YOUR_TRACKING_ID.js"></script>
```

That's it. Page views, sessions, and visitor data start flowing immediately.

### 3. Track custom events

```javascript
// Track a button click
ta('event', 'signup_click', {
    category: 'engagement',
    label: 'hero_cta',
    value: 1
});

// Track a purchase
ta('event', 'purchase', {
    category: 'ecommerce',
    label: 'checkout',
    value: 49.99
});
```

### 4. Single-page application support

The tracker automatically detects route changes via `history.pushState` and `popstate`. For manual control:

```javascript
ta('pageview'); // manually fire a page view
```

### 5. No-JS fallback (tracking pixel)

```html
<noscript>
  <img src="https://analytics.yourdomain.com/api/pixel?tid=YOUR_TRACKING_ID"
       style="position:absolute;left:-9999px;" alt="">
</noscript>
```

---

## 🔌 API

LibreMetrics includes a versioned REST API for pulling analytics data programmatically.

**Generate a token:** Dashboard → Profile → API Tokens → Create Token

**Authentication:**
```bash
Authorization: Bearer YOUR_API_TOKEN
```

### Endpoints

```
GET /api/v1/websites                           List your websites
GET /api/v1/websites/{id}/stats                Summary metrics (with date range)
GET /api/v1/websites/{id}/pages                Top pages
GET /api/v1/websites/{id}/referrers            Traffic sources
GET /api/v1/websites/{id}/events               Events report
GET /api/v1/websites/{id}/visitors             Audience breakdown
```

**Example — get stats for the last 30 days:**

```bash
curl -H "Authorization: Bearer YOUR_TOKEN" \
  "https://analytics.yourdomain.com/api/v1/websites/1/stats?from=2026-04-25&to=2026-05-25"
```

```json
{
  "data": {
    "page_views": 14823,
    "visitors": 4201,
    "sessions": 5318,
    "bounce_rate": 42.3,
    "avg_duration": 187,
    "avg_pages": 2.8
  }
}
```

---

## ⚙️ Artisan Commands

| Command | Description | Runs |
|---|---|---|
| `analytics:aggregate` | Roll up raw data into daily aggregates | Daily at 02:00 |
| `analytics:purge --force` | Delete raw data beyond retention window | Daily at 03:00 |
| `analytics:check-alerts` | Evaluate alert rules, send notifications | Every hour |

**Run manually:**

```bash
# Aggregate a specific date
php artisan analytics:aggregate --date=2026-05-24

# Aggregate for one website only
php artisan analytics:aggregate --website=1

# Purge old data (prompts for confirmation without --force)
php artisan analytics:purge --force
```

---

## 🧪 Tests

```bash
php artisan test
```

80 tests, all passing. The suite covers collection validation, website authorization, analytics query aggregation, goal evaluation, and auth flows.

```
Tests: 80 passed (180 assertions)
```

---

## 🗺️ Roadmap

- [ ] User flow / Sankey diagram
- [ ] Docker Compose one-command setup
- [ ] One-click deploy (Railway, Render, DigitalOcean)
- [ ] Multi-user teams with role-based access
- [ ] Managed hosting option
- [ ] Mobile app SDKs (iOS, Android, React Native)
- [ ] Webhook notifications for alerts
- [ ] Scheduled email reports (weekly digest)

Have an idea? [Open a discussion](https://github.com/yourusername/libremetrics/discussions).

---

## 🤝 Contributing

Contributions are welcome and appreciated. Here's how to get started:

**1. Fork and clone**

```bash
git clone https://github.com/YOUR_USERNAME/libremetrics.git
cd libremetrics
composer install && npm install
cp .env.example .env && php artisan key:generate
php artisan migrate
```

**2. Create a branch**

```bash
git checkout -b feature/your-feature-name
# or
git checkout -b fix/your-bug-fix
```

**3. Make your changes**

- Write tests for new features
- Follow [PSR-12](https://www.php-fig.org/psr/psr-12/) coding style (enforced by Laravel Pint)
- Keep controllers thin — business logic belongs in services

**4. Run the test suite**

```bash
php artisan test
vendor/bin/pint --test   # check code style
```

**5. Open a pull request**

Describe what you changed and why. Reference any related issues.

### Areas where help is especially welcome

- Dashboard UI polish and new chart types
- User flow / Sankey diagram (see roadmap)
- Docker / deployment tooling
- Integration guides for popular frameworks
- Translations / i18n

---

## 🏗️ Architecture

```
Client website (JS snippet)
  → POST /api/e  (CollectController — validates, queues, returns instantly)
  → ProcessTrackingPayload job (async)
      → Creates/updates: Visitor, AnalyticsSession, PageView or Event
      → GoalEvaluationService checks active goals → records GoalConversion
  → Daily at 02:00: analytics:aggregate
      → Rolls raw tables → daily_aggregates + page_daily_aggregates
  → Dashboard queries:
      → Aggregates for historical dates (fast)
      → Raw tables for today (live)
```

**Key services:**

| Service | Responsibility |
|---|---|
| `AnalyticsQueryService` | All aggregation and report queries — keeps controllers thin |
| `GoalEvaluationService` | Checks every incoming payload against active goals |
| `FunnelAnalysisService` | Computes drop-off rates across funnel steps |
| `ProcessTrackingPayload` | Queued job: visitor resolution, session management, geolocation |

---

## 📄 License

LibreMetrics is open-source software licensed under the [MIT license](LICENSE).

---

## 🙏 Built With

- [Laravel 11](https://laravel.com) — PHP framework
- [Laravel Fortify](https://github.com/laravel/fortify) — Authentication (with 2FA)
- [Laravel Sanctum](https://github.com/laravel/sanctum) — API token authentication
- [Alpine.js](https://alpinejs.dev) — Lightweight frontend reactivity
- [Tailwind CSS](https://tailwindcss.com) — Utility-first CSS
- [Chart.js](https://www.chartjs.org) — Charts and visualizations
- [stevebauman/location](https://github.com/stevebauman/location) — IP geolocation
- [jaybizzle/crawler-detect](https://github.com/JayBizzle/Crawler-Detect) — Bot filtering

---

<p align="center">
  <sub>Made with ❤️ — if LibreMetrics saves you money or headaches, consider giving it a ⭐</sub>
</p>