fixes

README.md

@@ -2,6 +2,34 @@
 
 A comprehensive job aggregator platform built with Laravel 13 that fetches, stores, and displays job listings from multiple free job APIs.
 
+## Quick Start
+
+```bash
+# Clone repository
+git clone <repository-url> && cd jobifygarden
+
+# Install dependencies
+composer install
+
+# Start Docker environment (includes MySQL, Redis)
+./vendor/bin/sail up -d
+
+# Setup application
+./vendor/bin/sail artisan key:generate
+./vendor/bin/sail artisan migrate
+./vendor/bin/sail artisan db:seed
+
+# Install & build frontend assets
+./vendor/bin/sail npm install && ./vendor/bin/sail npm run dev
+
+# Start queue worker (in a separate terminal)
+./vendor/bin/sail artisan queue:work
+
+# Visit http://localhost
+```
+
+**Important:** After setup, configure the cron job (see [Cron Jobs](#cron-jobs-required) section) to enable automatic job fetching.
+
 ## Requirements
 
 - **PHP**: 8.3.x or higher
@@ -40,7 +68,81 @@ A comprehensive job aggregator platform built with Laravel 13 that fetches, stor
 
 ## Installation
 
-### Development Setup
+### Development Setup (Laravel Sail - Recommended)
+
+1. **Clone the repository**
+```bash
+git clone <repository-url>
+cd jobifygarden
+```
+
+2. **Install PHP dependencies**
+```bash
+composer install
+```
+
+3. **Copy environment file**
+```bash
+cp .env.example .env
+```
+
+4. **Configure environment variables**
+Edit `.env` and update the following:
+```env
+APP_NAME=JobifyGarden
+APP_ENV=local
+APP_DEBUG=true
+APP_URL=http://localhost
+
+# Database (Sail uses these defaults)
+DB_CONNECTION=mysql
+DB_HOST=mysql
+DB_PORT=3306
+DB_DATABASE=jobifygarden
+DB_USERNAME=sail
+DB_PASSWORD=password
+
+# Cache & Sessions (Redis)
+CACHE_STORE=redis
+SESSION_DRIVER=redis
+QUEUE_CONNECTION=redis
+
+# Redis (Sail default)
+REDIS_HOST=redis
+REDIS_PASSWORD=null
+REDIS_PORT=6379
+```
+
+5. **Start Docker containers**
+```bash
+./vendor/bin/sail up -d
+```
+
+6. **Generate application key**
+```bash
+./vendor/bin/sail artisan key:generate
+```
+
+7. **Run database migrations**
+```bash
+./vendor/bin/sail artisan migrate
+```
+
+8. **Seed initial data (optional)**
+```bash
+./vendor/bin/sail artisan db:seed
+```
+
+9. **Install frontend dependencies and build assets**
+```bash
+./vendor/bin/sail npm install
+./vendor/bin/sail npm run dev
+```
+
+10. **Visit the application**
+Open `http://localhost` in your browser.
+
+### Production Setup (Traditional Server)
 
 1. **Clone the repository**
 ```bash
@@ -50,8 +152,9 @@ cd jobifygarden
 
 2. **Install dependencies**
 ```bash
-composer install
+composer install --optimize-autoloader --no-dev
 npm install
+npm run build
 ```
 
 3. **Configure environment**
@@ -60,72 +163,141 @@ cp .env.example .env
 php artisan key:generate
 ```
 
-4. **Update `.env` with your database credentials**
+4. **Update `.env` for production**
 ```env
+APP_NAME=JobifyGarden
+APP_ENV=production
+APP_DEBUG=false
+APP_URL=https://yourdomain.com
+
 DB_CONNECTION=mysql
 DB_HOST=127.0.0.1
 DB_PORT=3306
 DB_DATABASE=jobifygarden
 DB_USERNAME=your_username
-DB_PASSWORD=your_password
+DB_PASSWORD=your_secure_password
+
+CACHE_STORE=redis
+SESSION_DRIVER=redis
+QUEUE_CONNECTION=redis
+
+REDIS_HOST=127.0.0.1
+REDIS_PASSWORD=your_redis_password
+REDIS_PORT=6379
 ```
 
 5. **Run migrations**
 ```bash
-php artisan migrate
+php artisan migrate --force
 ```
 
-6. **Seed the database (optional)**
+6. **Optimize for production**
 ```bash
-php artisan db:seed
+php artisan config:cache
+php artisan route:cache
+php artisan view:cache
+php artisan optimize
 ```
 
-7. **Build frontend assets**
+7. **Set proper permissions**
 ```bash
-npm run dev
-```
-
-8. **Start the development server**
-```bash
-php artisan serve
-```
-
-Visit `http://localhost:8000` to view the application.
-
-### Docker Setup (Laravel Sail)
-
-```bash
-# Install dependencies
-composer install
-
-# Start Docker containers
-./vendor/bin/sail up -d
-
-# Run migrations
-./vendor/bin/sail artisan migrate
-
-# Build assets
-./vendor/bin/sail npm install
-./vendor/bin/sail npm run dev
+chown -R www-data:www-data storage bootstrap/cache
+chmod -R 775 storage bootstrap/cache
 ```
 
 ## Production Deployment
 
 See [DEPLOYMENT.md](DEPLOYMENT.md) for detailed production deployment instructions.
 
+## Required Services
+
+### Queue Worker (Required)
+
+The application uses queues for background job processing. You must run a queue worker:
+
+**Using Laravel Sail (Development):**
+```bash
+./vendor/bin/sail artisan queue:work --sleep=3 --tries=3 --timeout=90
+```
+
+**Using Supervisor (Production):**
+Create `/etc/supervisor/conf.d/jobifygarden-worker.conf`:
+```ini
+[program:jobifygarden-worker]
+process_name=%(program_name)s_%(process_num)02d
+command=php /path/to/jobifygarden/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600
+autostart=true
+autorestart=true
+stopasgroup=true
+killasgroup=true
+user=www-data
+numprocs=2
+redirect_stderr=true
+stdout_logfile=/path/to/jobifygarden/storage/logs/worker.log
+stopwaitsecs=3600
+```
+
+Then reload supervisor:
+```bash
+sudo supervisorctl reread
+sudo supervisorctl update
+sudo supervisorctl start jobifygarden-worker:*
+```
+
+### Cron Jobs (Required)
+
+Laravel's task scheduler must run every minute. Add this to your crontab:
+
+**Edit crontab:**
+```bash
+crontab -e
+```
+
+**Add this line:**
+```bash
+* * * * * cd /path/to/jobifygarden && php artisan schedule:run >> /dev/null 2>&1
+```
+
+**For Laravel Sail (Development):**
+```bash
+* * * * * cd /path/to/jobifygarden && ./vendor/bin/sail artisan schedule:run >> /dev/null 2>&1
+```
+
+### Scheduled Tasks
+
+The following tasks are automatically scheduled and will run via the cron job above:
+
+| Task | Schedule | Description |
+|------|----------|-------------|
+| `jobs:fetch` | Every 2 hours | Fetch new jobs from all API sources |
+| `jobs:clean` | Daily at 3:00 AM | Remove expired and old jobs |
+| `jobs:deduplicate` | Daily at 4:00 AM | Remove duplicate job listings |
+| `cache:prune-stale-tags` | Hourly | Clean up stale cache tags |
+
+**Note:** Job fetching intervals can be customized in `app/Console/Kernel.php`
+
 ## Available Commands
 
 ### Job Management
 ```bash
-# Fetch jobs from all sources
+# Fetch jobs from all sources (automatically scheduled)
 php artisan jobs:fetch
 
-# Clean expired jobs
+# Fetch from a specific source
+php artisan jobs:fetch --source=indeed
+
+# Clean expired jobs (automatically scheduled)
 php artisan jobs:clean
 
-# Remove duplicate jobs
+# Clean jobs older than specific days
+php artisan jobs:clean --days=60
+
+# Remove duplicate jobs (automatically scheduled)
 php artisan jobs:deduplicate
 
+# Force dedupe even with different sources
+php artisan jobs:deduplicate --force
+
 # Enrich job data (categories, experience levels)
 php artisan jobs:enrich
 
@@ -133,24 +305,55 @@ php artisan jobs:enrich
 php artisan jobs:test-apis
 ```
 
-### Database Optimization
+### Database Management
 ```bash
-# Optimize database tables and rebuild indexes
+# Run migrations
+php artisan migrate
+
+# Rollback last migration
+php artisan migrate:rollback
+
+# Fresh migration with seeding
+php artisan migrate:fresh --seed
+
+# Optimize database tables
 php artisan db:optimize
 ```
 
 ### Cache Management
 ```bash
 # Clear all caches
+php artisan optimize:clear
+
+# Clear specific caches
 php artisan cache:clear
 php artisan config:clear
 php artisan route:clear
 php artisan view:clear
 
 # Build caches for production
-php artisan config:cache
-php artisan route:cache
-php artisan view:cache
+php artisan optimize
+```
+
+### Queue Management
+```bash
+# Start queue worker
+php artisan queue:work
+
+# Process jobs with options
+php artisan queue:work --queue=high,default --sleep=3 --tries=3
+
+# List failed jobs
+php artisan queue:failed
+
+# Retry all failed jobs
+php artisan queue:retry all
+
+# Retry specific failed job
+php artisan queue:retry {id}
+
+# Clear all failed jobs
+php artisan queue:flush
 ```
 
 ## Project Structure
@@ -194,8 +397,9 @@ GET  /api/jobs/autocomplete/companies   # Company autocomplete
 
 ### Backend
 - **Laravel 13**: PHP framework
+- **PHP 8.3+**: Programming language
 - **MySQL 8**: Primary database
-- **Redis**: Caching and queues
+- **Redis**: Caching, sessions, and queues
 - **Guzzle**: HTTP client for API requests
 
 ### Frontend
@@ -204,6 +408,114 @@ GET  /api/jobs/autocomplete/companies   # Company autocomplete
 - **TailwindCSS**: Utility-first CSS framework
 - **Vite**: Frontend build tool
 
+### DevOps
+- **Laravel Sail**: Docker development environment
+- **Supervisor**: Process manager for queue workers
+- **Nginx/Apache**: Web server
+- **Certbot**: SSL certificate management
+
+## Troubleshooting
+
+### Common Issues
+
+**1. Queue jobs not processing**
+```bash
+# Check if queue worker is running
+ps aux | grep "queue:work"
+
+# Restart queue worker (Supervisor)
+sudo supervisorctl restart jobifygarden-worker:*
+
+# Or start manually for debugging
+php artisan queue:work --verbose
+```
+
+**2. Jobs not being fetched automatically**
+```bash
+# Verify cron is set up
+crontab -l
+
+# Test schedule manually
+php artisan schedule:run
+
+# Check schedule list
+php artisan schedule:list
+```
+
+**3. Redis connection errors**
+```bash
+# Check Redis is running
+redis-cli ping
+
+# For Sail
+./vendor/bin/sail redis redis-cli ping
+
+# Restart Redis (Sail)
+./vendor/bin/sail down
+./vendor/bin/sail up -d
+```
+
+**4. Permission errors**
+```bash
+# Fix storage permissions
+sudo chown -R www-data:www-data storage bootstrap/cache
+sudo chmod -R 775 storage bootstrap/cache
+
+# For development (use your user)
+sudo chown -R $USER:$USER storage bootstrap/cache
+chmod -R 775 storage bootstrap/cache
+```
+
+**5. Slugs not generating for new jobs**
+```bash
+# Regenerate slugs for existing jobs
+php artisan tinker
+# Then run:
+# \App\Models\Job::whereNull('slug')->chunk(100, function($jobs) {
+#     foreach($jobs as $job) {
+#         $job->slug = \App\Models\Job::generateUniqueSlug($job->title, $job->company);
+#         $job->save();
+#     }
+# });
+```
+
+**6. Cache issues**
+```bash
+# Clear all caches
+php artisan optimize:clear
+
+# Rebuild for production
+php artisan optimize
+```
+
+### Performance Tips
+
+1. **Enable OPcache in production** (add to `php.ini`):
+```ini
+opcache.enable=1
+opcache.memory_consumption=256
+opcache.interned_strings_buffer=16
+opcache.max_accelerated_files=20000
+opcache.validate_timestamps=0
+```
+
+2. **Optimize database queries:**
+```bash
+php artisan db:optimize
+```
+
+3. **Use queue workers with multiple processes:**
+```bash
+# Update supervisor config to use more workers
+numprocs=4
+```
+
+4. **Monitor slow queries:**
+```bash
+# Enable query logging in .env
+DB_LOG_QUERIES=true
+```
+
 ## License
 
 This project is open-sourced software licensed under the [MIT license](https://opensource.org/licenses/MIT).