phase 1 and 2
This commit is contained in:
Doar Mihai
@@ -0,0 +1,520 @@
|
||||
# Job Aggregator Platform - Implementation Plan
|
||||
|
||||
## Project Overview
|
||||
Build a comprehensive job aggregator platform using Laravel 13 that fetches, stores, and displays job listings from multiple free job APIs.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Database Design & Setup
|
||||
|
||||
### 1.1 Core Database Tables
|
||||
- **jobs** - Main job listings table
|
||||
- id, external_id, title, company, description, location, salary_min, salary_max, job_type (full-time, part-time, contract, etc.)
|
||||
- source (API provider name), source_url, apply_url, posted_at, expires_at
|
||||
- remote_allowed, experience_level, category
|
||||
- timestamps, soft deletes
|
||||
|
||||
- **job_sources** - API providers configuration
|
||||
- id, name, api_url, api_key (encrypted), is_active, rate_limit, last_fetched_at
|
||||
- configuration (JSON for API-specific settings)
|
||||
|
||||
- **job_categories** - Job categories/industries
|
||||
- id, name, slug, description, icon
|
||||
|
||||
- **job_locations** - Normalized locations
|
||||
- id, city, state, country, country_code, latitude, longitude
|
||||
|
||||
- **user_saved_jobs** - User bookmarks (if implementing user features)
|
||||
- user_id, job_id, timestamps
|
||||
|
||||
- **user_job_alerts** - Job alerts/notifications
|
||||
- user_id, keywords, location, category, frequency
|
||||
|
||||
### 1.2 Migrations
|
||||
Create migrations for all tables with proper indexes and foreign keys
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: API Integration Layer
|
||||
|
||||
### 2.1 Job API Sources to Integrate
|
||||
Research and integrate free job APIs:
|
||||
- **Adzuna API** - Free tier available with registration
|
||||
- **The Muse API** - Free, no key required
|
||||
- **GitHub Jobs API** (deprecated but alternatives exist)
|
||||
- **Remotive API** - Remote jobs, free access
|
||||
- **Indeed API** - May require partnership
|
||||
- **USAJobs API** - US government jobs
|
||||
- **Reed.co.uk API** - UK jobs (requires registration)
|
||||
|
||||
### 2.2 API Client Architecture
|
||||
- Create abstract `JobApiClient` base class
|
||||
- Implement concrete client for each API source:
|
||||
- `AdzunaClient`
|
||||
- `MuseClient`
|
||||
- `RemotiveClient`
|
||||
- etc.
|
||||
|
||||
### 2.3 Data Transformation Layer
|
||||
- Create `JobTransformer` interface
|
||||
- Implement transformers for each API to normalize data into common format
|
||||
- Handle different date formats, salary formats, location formats
|
||||
- Sanitize and validate incoming data
|
||||
|
||||
### 2.4 HTTP Client Setup
|
||||
- Use Laravel HTTP client with retry logic
|
||||
- Implement rate limiting per API
|
||||
- Add request/response logging
|
||||
- Handle API errors gracefully
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Job Fetching System
|
||||
|
||||
### 3.1 Console Commands
|
||||
- `php artisan jobs:fetch {source?}` - Fetch jobs from all or specific source
|
||||
- `php artisan jobs:fetch-all` - Fetch from all active sources
|
||||
- `php artisan jobs:clean` - Remove expired/old jobs
|
||||
- `php artisan jobs:deduplicate` - Remove duplicate jobs across sources
|
||||
|
||||
### 3.2 Queue Jobs
|
||||
- `FetchJobsFromSource` - Queue job to fetch from specific API
|
||||
- `ProcessJobListing` - Process and store individual job
|
||||
- `DeduplicateJobs` - Background job deduplication
|
||||
- `NotifyUserJobAlerts` - Send alerts to users
|
||||
|
||||
### 3.3 Scheduling
|
||||
Set up Laravel scheduler for automated fetching:
|
||||
- Hourly fetches from fast-updating sources
|
||||
- Daily fetches from slower sources
|
||||
- Daily cleanup of expired jobs
|
||||
- Weekly deduplication runs
|
||||
|
||||
### 3.4 Deduplication Logic
|
||||
- Create job fingerprinting algorithm (hash of title + company + location)
|
||||
- Implement similarity matching for near-duplicates
|
||||
- Prefer jobs from higher-quality sources
|
||||
- Keep audit trail of merged/deduplicated jobs
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Business Logic Layer
|
||||
|
||||
### 4.1 Repositories
|
||||
- `JobRepository` - CRUD and query operations for jobs
|
||||
- `JobSourceRepository` - Manage API sources
|
||||
- `CategoryRepository` - Category management
|
||||
|
||||
### 4.2 Services
|
||||
- `JobAggregatorService` - Orchestrate fetching from multiple sources
|
||||
- `JobSearchService` - Advanced search with filters
|
||||
- `JobDeduplicationService` - Handle duplicate detection
|
||||
- `JobEnrichmentService` - Enhance job data (geocoding, categorization)
|
||||
|
||||
### 4.3 Search & Filter Features
|
||||
- Full-text search (Laravel Scout with Meilisearch/Algolia or database)
|
||||
- Filter by:
|
||||
- Location (city, state, country, remote)
|
||||
- Job type (full-time, part-time, contract)
|
||||
- Experience level
|
||||
- Salary range
|
||||
- Date posted
|
||||
- Company
|
||||
- Category/industry
|
||||
- Sort by: relevance, date, salary
|
||||
|
||||
### 4.4 Job Enrichment
|
||||
- Geocoding service for location standardization
|
||||
- Auto-categorization using keywords
|
||||
- Salary normalization across different formats
|
||||
- Company logo fetching (Clearbit API, Brandfetch)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: API & Controllers
|
||||
|
||||
### 5.1 API Endpoints
|
||||
RESTful API for job listings:
|
||||
```
|
||||
GET /api/jobs - List jobs with pagination & filters
|
||||
GET /api/jobs/{id} - Single job details
|
||||
GET /api/jobs/search - Advanced search
|
||||
GET /api/categories - List categories
|
||||
GET /api/locations - List locations (autocomplete)
|
||||
GET /api/stats - Platform statistics
|
||||
POST /api/jobs/{id}/apply - Track apply clicks
|
||||
POST /api/jobs/{id}/save - Save job (authenticated)
|
||||
DELETE /api/jobs/{id}/save - Unsave job
|
||||
```
|
||||
|
||||
### 5.2 Web Controllers
|
||||
- `JobController` - Display jobs, search, details
|
||||
- `HomeController` - Homepage with featured/recent jobs
|
||||
- `CategoryController` - Jobs by category
|
||||
- `LocationController` - Jobs by location
|
||||
- `DashboardController` - User dashboard (saved jobs, alerts)
|
||||
|
||||
### 5.3 Admin Panel
|
||||
- Dashboard with statistics
|
||||
- Manage job sources (enable/disable, configure)
|
||||
- Manual job moderation/removal
|
||||
- View fetch logs and errors
|
||||
- System health monitoring
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Frontend Development
|
||||
|
||||
### 6.1 Public Pages
|
||||
- **Homepage**
|
||||
- Hero search bar
|
||||
- Featured categories
|
||||
- Latest jobs
|
||||
- Statistics counter
|
||||
|
||||
- **Job Listings Page**
|
||||
- Advanced filters sidebar
|
||||
- Search bar
|
||||
- Job cards with key info
|
||||
- Infinite scroll or pagination
|
||||
|
||||
- **Job Details Page**
|
||||
- Full job description
|
||||
- Apply button (external link)
|
||||
- Similar jobs
|
||||
- Share buttons
|
||||
|
||||
- **Search Results Page**
|
||||
- Results with highlighting
|
||||
- Saved search functionality
|
||||
|
||||
- **Category/Location Pages**
|
||||
- Browse by category
|
||||
- Browse by location
|
||||
- Related categories/locations
|
||||
|
||||
### 6.2 User Features (Optional)
|
||||
- Authentication (Laravel Breeze/Jetstream)
|
||||
- User dashboard
|
||||
- Save/bookmark jobs
|
||||
- Job alerts setup
|
||||
- Application tracking
|
||||
|
||||
### 6.3 UI/UX Components
|
||||
- Responsive design (mobile-first)
|
||||
- Job card component
|
||||
- Filter components
|
||||
- Search autocomplete
|
||||
- Loading states
|
||||
- Empty states
|
||||
- Error handling
|
||||
|
||||
### 6.4 Frontend Stack
|
||||
- Blade templates with Alpine.js (lightweight)
|
||||
- OR Inertia.js + Vue/React (SPA experience)
|
||||
- TailwindCSS for styling
|
||||
- Vite for asset bundling
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Monitoring & Analytics
|
||||
|
||||
### 7.1 Logging
|
||||
- API fetch success/failure logs
|
||||
- Job processing errors
|
||||
- User activity logs
|
||||
- Performance metrics
|
||||
|
||||
### 7.2 Metrics & Monitoring
|
||||
- Track jobs fetched per source
|
||||
- API response times
|
||||
- Queue processing times
|
||||
- Database query performance
|
||||
- User engagement metrics
|
||||
|
||||
### 7.3 Error Handling
|
||||
- Graceful API failure handling
|
||||
- Retry mechanisms with exponential backoff
|
||||
- Admin notifications for critical failures
|
||||
- Error reporting (Sentry/Bugsnag)
|
||||
|
||||
### 7.4 Analytics
|
||||
- Track popular searches
|
||||
- Most viewed jobs
|
||||
- Click-through rates
|
||||
- Geographic distribution
|
||||
- Source effectiveness
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Performance Optimization
|
||||
|
||||
### 8.1 Caching Strategy
|
||||
- Cache job listings (Redis/Memcached)
|
||||
- Cache search results (5-15 minutes)
|
||||
- Cache categories and locations
|
||||
- Cache statistics/counters
|
||||
- HTTP cache headers for static content
|
||||
|
||||
### 8.2 Database Optimization
|
||||
- Proper indexing (title, location, posted_at, category)
|
||||
- Full-text indexes for search
|
||||
- Query optimization
|
||||
- Connection pooling
|
||||
- Read replicas for heavy reads
|
||||
|
||||
### 8.3 Queue Optimization
|
||||
- Separate queues for priority tasks
|
||||
- Queue workers tuning
|
||||
- Failed job handling
|
||||
- Queue monitoring
|
||||
|
||||
### 8.4 Asset Optimization
|
||||
- Image optimization and lazy loading
|
||||
- CSS/JS minification
|
||||
- CDN for static assets
|
||||
- Gzip compression
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: SEO & Marketing
|
||||
|
||||
### 9.1 SEO Optimization
|
||||
- Dynamic meta tags for job pages
|
||||
- Structured data (JSON-LD) for jobs
|
||||
- XML sitemap generation
|
||||
- Robots.txt optimization
|
||||
- Canonical URLs
|
||||
- Open Graph tags
|
||||
- Fast page loads (Core Web Vitals)
|
||||
|
||||
### 9.2 Content Strategy
|
||||
- Job landing pages per category
|
||||
- Location-specific landing pages
|
||||
- Career advice blog (optional)
|
||||
- Company profiles
|
||||
|
||||
### 9.3 Social Sharing
|
||||
- Share job buttons (LinkedIn, Twitter, Facebook)
|
||||
- Auto-posting to social media (optional)
|
||||
- Email sharing functionality
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Testing & Quality Assurance
|
||||
|
||||
### 10.1 Testing Strategy
|
||||
- **Unit Tests**
|
||||
- API client tests
|
||||
- Transformer tests
|
||||
- Service layer tests
|
||||
- Repository tests
|
||||
|
||||
- **Feature Tests**
|
||||
- Job fetching workflow
|
||||
- Search and filter functionality
|
||||
- User authentication flow
|
||||
- Job saving/bookmarking
|
||||
|
||||
- **Integration Tests**
|
||||
- API endpoint tests
|
||||
- Database operations
|
||||
- Queue processing
|
||||
|
||||
- **Browser Tests (Dusk)**
|
||||
- Critical user flows
|
||||
- Search and apply workflow
|
||||
- Responsive design testing
|
||||
|
||||
### 10.2 Quality Checks
|
||||
- Code style (Laravel Pint)
|
||||
- Static analysis (PHPStan/Larastan)
|
||||
- Security scanning
|
||||
- Performance profiling
|
||||
|
||||
---
|
||||
|
||||
## Phase 11: Deployment & DevOps
|
||||
|
||||
### 11.1 Infrastructure
|
||||
- Production server setup (VPS/Cloud)
|
||||
- Database server (MySQL/PostgreSQL)
|
||||
- Redis server for cache and queues
|
||||
- SSL certificate (Let's Encrypt)
|
||||
- CDN setup (Cloudflare/AWS CloudFront)
|
||||
|
||||
### 11.2 CI/CD Pipeline
|
||||
- GitHub Actions or GitLab CI
|
||||
- Automated testing on push
|
||||
- Automated deployment to staging
|
||||
- Manual deployment to production
|
||||
- Database migration automation
|
||||
|
||||
### 11.3 Environment Configuration
|
||||
- Environment-specific .env files
|
||||
- Secrets management
|
||||
- API key rotation strategy
|
||||
- Backup strategy
|
||||
|
||||
### 11.4 Monitoring & Maintenance
|
||||
- Server monitoring (uptime, CPU, memory)
|
||||
- Application monitoring (Laravel Telescope/Horizon)
|
||||
- Log aggregation (ELK stack or cloud service)
|
||||
- Automated backups
|
||||
- Incident response plan
|
||||
|
||||
---
|
||||
|
||||
## Phase 12: Launch & Post-Launch
|
||||
|
||||
### 12.1 Pre-Launch Checklist
|
||||
- [ ] All APIs configured and tested
|
||||
- [ ] Database migrations completed
|
||||
- [ ] Initial job data seeded
|
||||
- [ ] Search functionality tested
|
||||
- [ ] Mobile responsiveness verified
|
||||
- [ ] SEO optimizations in place
|
||||
- [ ] Analytics configured
|
||||
- [ ] Error monitoring active
|
||||
- [ ] Backup system tested
|
||||
- [ ] SSL certificate installed
|
||||
|
||||
### 12.2 Launch Tasks
|
||||
- Initial data population (run fetchers)
|
||||
- Submit sitemap to search engines
|
||||
- Social media announcement
|
||||
- Monitor for errors
|
||||
- Performance monitoring
|
||||
|
||||
### 12.3 Post-Launch Iterations
|
||||
- Monitor user feedback
|
||||
- Analyze usage patterns
|
||||
- Add more job sources
|
||||
- Improve search relevance
|
||||
- Add requested features
|
||||
- Performance optimization based on real data
|
||||
|
||||
---
|
||||
|
||||
## Technical Stack Summary
|
||||
|
||||
**Backend:**
|
||||
- Laravel 13 (PHP 8.3)
|
||||
- MySQL/PostgreSQL for primary database
|
||||
- Redis for cache and queues
|
||||
- Laravel Horizon for queue monitoring
|
||||
- Laravel Scout for search (with Meilisearch/Algolia)
|
||||
|
||||
**Frontend:**
|
||||
- Blade templates + Alpine.js OR Inertia.js + Vue/React
|
||||
- TailwindCSS
|
||||
- Vite
|
||||
|
||||
**DevOps:**
|
||||
- Docker for development (Laravel Sail)
|
||||
- GitHub Actions for CI/CD
|
||||
- Server: Ubuntu 24.04 LTS
|
||||
- Web Server: Nginx
|
||||
- Process Manager: Supervisor (for queues)
|
||||
|
||||
**Third-party Services:**
|
||||
- Job APIs (multiple sources)
|
||||
- Geocoding API (Google Maps/OpenStreetMap)
|
||||
- Email service (SendGrid/Mailgun/SES)
|
||||
- Error tracking (Sentry)
|
||||
- Analytics (Google Analytics/Plausible)
|
||||
|
||||
---
|
||||
|
||||
## Development Timeline (Estimated)
|
||||
|
||||
- **Week 1-2:** Database design, migrations, seeders
|
||||
- **Week 2-3:** API integration layer (3-5 sources)
|
||||
- **Week 3-4:** Job fetching system, queue jobs, commands
|
||||
- **Week 4-5:** Business logic, services, repositories
|
||||
- **Week 5-6:** API endpoints and web controllers
|
||||
- **Week 6-8:** Frontend development (all pages)
|
||||
- **Week 8-9:** Search optimization, caching, performance
|
||||
- **Week 9-10:** Testing, bug fixes, SEO
|
||||
- **Week 10-11:** Deployment, monitoring, documentation
|
||||
- **Week 11-12:** Beta testing, refinements, launch
|
||||
|
||||
**Total: ~12 weeks for MVP**
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
- **Technical Metrics:**
|
||||
- Jobs fetched per day: 1,000+
|
||||
- API success rate: >95%
|
||||
- Search response time: <200ms
|
||||
- Page load time: <2s
|
||||
- Uptime: >99.5%
|
||||
|
||||
- **Business Metrics:**
|
||||
- Daily active users
|
||||
- Job views per day
|
||||
- Apply click-through rate
|
||||
- User retention rate
|
||||
- SEO rankings for target keywords
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- **Phase 2 Features:**
|
||||
- Employer job posting (paid feature)
|
||||
- Premium job listings
|
||||
- Resume builder
|
||||
- Application tracking system
|
||||
- Company reviews
|
||||
- Salary insights
|
||||
- Job recommendations (ML-based)
|
||||
- Mobile app (iOS/Android)
|
||||
- API for third-party access
|
||||
- Browser extension
|
||||
- Slack/Discord bot integration
|
||||
|
||||
- **Monetization Options:**
|
||||
- Featured job listings
|
||||
- Employer accounts
|
||||
- Affiliate commissions
|
||||
- Premium user features
|
||||
- Job board white-labeling
|
||||
- Sponsored content
|
||||
|
||||
---
|
||||
|
||||
## Resources & Documentation
|
||||
|
||||
- **Laravel Documentation:** https://laravel.com/docs/13.x
|
||||
- **Job API Documentation:**
|
||||
- Adzuna: https://developer.adzuna.com/
|
||||
- The Muse: https://www.themuse.com/developers/api/v2
|
||||
- Remotive: https://remotive.com/api
|
||||
- USAJobs: https://developer.usajobs.gov/
|
||||
|
||||
- **Tools & Libraries:**
|
||||
- Laravel Horizon: Queue monitoring
|
||||
- Laravel Scout: Full-text search
|
||||
- Laravel Telescope: Debugging
|
||||
- Spatie packages: Various Laravel utilities
|
||||
- GuzzleHTTP: HTTP requests
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- Start with 3-5 reliable free APIs, expand later
|
||||
- Focus on data quality over quantity
|
||||
- Implement robust error handling from day one
|
||||
- Keep job data fresh (remove outdated listings)
|
||||
- Consider GDPR/privacy regulations for user data
|
||||
- Plan for scaling from the beginning
|
||||
- Document all API integrations thoroughly
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** April 2, 2026
|
||||
**Version:** 1.0
|
||||
@@ -0,0 +1,110 @@
|
||||
<?php
|
||||
|
||||
namespace App\Console\Commands;
|
||||
|
||||
use App\Services\JobApi\Exceptions\ApiException;
|
||||
use App\Services\JobApi\JobApiClientFactory;
|
||||
use Illuminate\Console\Attributes\Description;
|
||||
use Illuminate\Console\Attributes\Signature;
|
||||
use Illuminate\Console\Command;
|
||||
|
||||
#[Signature('jobs:test-apis {source?} {--limit=5}')]
|
||||
#[Description('Test job API integrations by fetching sample jobs')]
|
||||
class TestJobApis extends Command
|
||||
{
|
||||
/**
|
||||
* Execute the console command.
|
||||
*/
|
||||
public function handle(): int
|
||||
{
|
||||
$sourceName = $this->argument('source');
|
||||
$limit = (int) $this->option('limit');
|
||||
|
||||
$this->info('Testing Job API Integrations...');
|
||||
$this->newLine();
|
||||
|
||||
try {
|
||||
if ($sourceName) {
|
||||
// Test specific source
|
||||
$this->testSingleSource($sourceName, $limit);
|
||||
} else {
|
||||
// Test all active sources
|
||||
$this->testAllSources($limit);
|
||||
}
|
||||
|
||||
return self::SUCCESS;
|
||||
|
||||
} catch (\Exception $e) {
|
||||
$this->error("Failed to test APIs: {$e->getMessage()}");
|
||||
return self::FAILURE;
|
||||
}
|
||||
}
|
||||
|
||||
private function testAllSources(int $limit): void
|
||||
{
|
||||
$clients = JobApiClientFactory::makeAll();
|
||||
|
||||
if (empty($clients)) {
|
||||
$this->warn('No active job sources found. Run: php artisan db:seed --class=JobSourceSeeder');
|
||||
return;
|
||||
}
|
||||
|
||||
$this->info('Testing ' . count($clients) . ' active job sources...');
|
||||
$this->newLine();
|
||||
|
||||
foreach ($clients as $client) {
|
||||
$this->testClient($client, $limit);
|
||||
$this->newLine();
|
||||
}
|
||||
}
|
||||
|
||||
private function testSingleSource(string $sourceName, int $limit): void
|
||||
{
|
||||
try {
|
||||
$client = JobApiClientFactory::makeByName($sourceName);
|
||||
$this->testClient($client, $limit);
|
||||
} catch (ApiException $e) {
|
||||
$this->error("Failed to create client for '{$sourceName}': {$e->getMessage()}");
|
||||
}
|
||||
}
|
||||
|
||||
private function testClient($client, int $limit): void
|
||||
{
|
||||
$sourceName = $client->getSourceName();
|
||||
|
||||
$this->line("<fg=cyan>Testing: {$sourceName}</>");
|
||||
|
||||
try {
|
||||
// Fetch sample jobs
|
||||
$jobs = $client->fetchJobs(['limit' => $limit]);
|
||||
|
||||
if (empty($jobs)) {
|
||||
$this->warn(" No jobs returned from {$sourceName}");
|
||||
return;
|
||||
}
|
||||
|
||||
$this->info(" ✓ Successfully fetched " . count($jobs) . " jobs");
|
||||
|
||||
// Display sample jobs
|
||||
$this->table(
|
||||
['ID', 'Title', 'Company', 'Location', 'Remote'],
|
||||
collect($jobs)->take(3)->map(fn($job) => [
|
||||
substr($job->externalId, 0, 10) . '...',
|
||||
substr($job->title, 0, 40),
|
||||
substr($job->company, 0, 20),
|
||||
substr($job->location ?? 'N/A', 0, 20),
|
||||
$job->remoteAllowed ? 'Yes' : 'No',
|
||||
])->toArray()
|
||||
);
|
||||
|
||||
} catch (ApiException $e) {
|
||||
$this->error(" ✗ API Error: {$e->getMessage()}");
|
||||
if ($context = $e->getContext()) {
|
||||
$this->line(" Context: " . json_encode($context, JSON_PRETTY_PRINT));
|
||||
}
|
||||
} catch (\Exception $e) {
|
||||
$this->error(" ✗ Unexpected Error: {$e->getMessage()}");
|
||||
$this->line(" " . $e->getTraceAsString());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\BelongsTo;
|
||||
use Illuminate\Database\Eloquent\Relations\BelongsToMany;
|
||||
use Illuminate\Database\Eloquent\SoftDeletes;
|
||||
|
||||
class Job extends Model
|
||||
{
|
||||
use SoftDeletes;
|
||||
|
||||
protected $fillable = [
|
||||
'external_id',
|
||||
'title',
|
||||
'company',
|
||||
'description',
|
||||
'location',
|
||||
'salary_min',
|
||||
'salary_max',
|
||||
'salary_currency',
|
||||
'job_type',
|
||||
'job_source_id',
|
||||
'source_url',
|
||||
'apply_url',
|
||||
'posted_at',
|
||||
'expires_at',
|
||||
'remote_allowed',
|
||||
'experience_level',
|
||||
'job_category_id',
|
||||
'job_location_id',
|
||||
'fingerprint',
|
||||
];
|
||||
|
||||
protected $casts = [
|
||||
'posted_at' => 'datetime',
|
||||
'expires_at' => 'datetime',
|
||||
'remote_allowed' => 'boolean',
|
||||
'salary_min' => 'decimal:2',
|
||||
'salary_max' => 'decimal:2',
|
||||
];
|
||||
|
||||
/**
|
||||
* Get the source that provides this job.
|
||||
*/
|
||||
public function source(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(JobSource::class, 'job_source_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the category that this job belongs to.
|
||||
*/
|
||||
public function category(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(JobCategory::class, 'job_category_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the location of this job.
|
||||
*/
|
||||
public function jobLocation(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(JobLocation::class, 'job_location_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the users who saved this job.
|
||||
*/
|
||||
public function savedByUsers(): BelongsToMany
|
||||
{
|
||||
return $this->belongsToMany(User::class, 'user_saved_jobs')
|
||||
->withTimestamps();
|
||||
}
|
||||
|
||||
/**
|
||||
* Scope a query to only include active jobs.
|
||||
*/
|
||||
public function scopeActive($query)
|
||||
{
|
||||
return $query->where(function ($q) {
|
||||
$q->whereNull('expires_at')
|
||||
->orWhere('expires_at', '>', now());
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Scope a query to only include remote jobs.
|
||||
*/
|
||||
public function scopeRemote($query)
|
||||
{
|
||||
return $query->where('remote_allowed', true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Scope a query to filter by job type.
|
||||
*/
|
||||
public function scopeOfType($query, string $type)
|
||||
{
|
||||
return $query->where('job_type', $type);
|
||||
}
|
||||
|
||||
/**
|
||||
* Scope a query to filter by experience level.
|
||||
*/
|
||||
public function scopeExperienceLevel($query, string $level)
|
||||
{
|
||||
return $query->where('experience_level', $level);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\HasMany;
|
||||
|
||||
class JobCategory extends Model
|
||||
{
|
||||
protected $fillable = [
|
||||
'name',
|
||||
'slug',
|
||||
'description',
|
||||
'icon',
|
||||
];
|
||||
|
||||
/**
|
||||
* Get the jobs for this category.
|
||||
*/
|
||||
public function jobs(): HasMany
|
||||
{
|
||||
return $this->hasMany(Job::class, 'job_category_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the job alerts for this category.
|
||||
*/
|
||||
public function jobAlerts(): HasMany
|
||||
{
|
||||
return $this->hasMany(UserJobAlert::class, 'job_category_id');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\HasMany;
|
||||
|
||||
class JobLocation extends Model
|
||||
{
|
||||
protected $fillable = [
|
||||
'city',
|
||||
'state',
|
||||
'country',
|
||||
'country_code',
|
||||
'latitude',
|
||||
'longitude',
|
||||
];
|
||||
|
||||
protected $casts = [
|
||||
'latitude' => 'decimal:7',
|
||||
'longitude' => 'decimal:7',
|
||||
];
|
||||
|
||||
/**
|
||||
* Get the jobs for this location.
|
||||
*/
|
||||
public function jobs(): HasMany
|
||||
{
|
||||
return $this->hasMany(Job::class, 'job_location_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the full location string.
|
||||
*/
|
||||
public function getFullLocationAttribute(): string
|
||||
{
|
||||
$parts = array_filter([
|
||||
$this->city,
|
||||
$this->state,
|
||||
$this->country,
|
||||
]);
|
||||
|
||||
return implode(', ', $parts);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\HasMany;
|
||||
|
||||
class JobSource extends Model
|
||||
{
|
||||
protected $fillable = [
|
||||
'name',
|
||||
'api_url',
|
||||
'api_key',
|
||||
'is_active',
|
||||
'rate_limit',
|
||||
'last_fetched_at',
|
||||
'configuration',
|
||||
];
|
||||
|
||||
protected $casts = [
|
||||
'is_active' => 'boolean',
|
||||
'last_fetched_at' => 'datetime',
|
||||
'configuration' => 'array',
|
||||
];
|
||||
|
||||
/**
|
||||
* Get the jobs from this source.
|
||||
*/
|
||||
public function jobs(): HasMany
|
||||
{
|
||||
return $this->hasMany(Job::class, 'job_source_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Scope a query to only include active sources.
|
||||
*/
|
||||
public function scopeActive($query)
|
||||
{
|
||||
return $query->where('is_active', true);
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,8 @@ use Database\Factories\UserFactory;
|
||||
use Illuminate\Database\Eloquent\Attributes\Fillable;
|
||||
use Illuminate\Database\Eloquent\Attributes\Hidden;
|
||||
use Illuminate\Database\Eloquent\Factories\HasFactory;
|
||||
use Illuminate\Database\Eloquent\Relations\BelongsToMany;
|
||||
use Illuminate\Database\Eloquent\Relations\HasMany;
|
||||
use Illuminate\Foundation\Auth\User as Authenticatable;
|
||||
use Illuminate\Notifications\Notifiable;
|
||||
|
||||
@@ -29,4 +31,21 @@ class User extends Authenticatable
|
||||
'password' => 'hashed',
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the jobs saved by this user.
|
||||
*/
|
||||
public function savedJobs(): BelongsToMany
|
||||
{
|
||||
return $this->belongsToMany(Job::class, 'user_saved_jobs')
|
||||
->withTimestamps();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the job alerts for this user.
|
||||
*/
|
||||
public function jobAlerts(): HasMany
|
||||
{
|
||||
return $this->hasMany(UserJobAlert::class);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\BelongsTo;
|
||||
|
||||
class UserJobAlert extends Model
|
||||
{
|
||||
protected $fillable = [
|
||||
'user_id',
|
||||
'keywords',
|
||||
'location',
|
||||
'job_category_id',
|
||||
'frequency',
|
||||
'is_active',
|
||||
'last_sent_at',
|
||||
];
|
||||
|
||||
protected $casts = [
|
||||
'is_active' => 'boolean',
|
||||
'last_sent_at' => 'datetime',
|
||||
];
|
||||
|
||||
/**
|
||||
* Get the user who created the alert.
|
||||
*/
|
||||
public function user(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(User::class);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the category for this alert.
|
||||
*/
|
||||
public function category(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(JobCategory::class, 'job_category_id');
|
||||
}
|
||||
|
||||
/**
|
||||
* Scope a query to only include active alerts.
|
||||
*/
|
||||
public function scopeActive($query)
|
||||
{
|
||||
return $query->where('is_active', true);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
<?php
|
||||
|
||||
namespace App\Models;
|
||||
|
||||
use Illuminate\Database\Eloquent\Model;
|
||||
use Illuminate\Database\Eloquent\Relations\BelongsTo;
|
||||
|
||||
class UserSavedJob extends Model
|
||||
{
|
||||
protected $fillable = [
|
||||
'user_id',
|
||||
'job_id',
|
||||
];
|
||||
|
||||
/**
|
||||
* Get the user who saved the job.
|
||||
*/
|
||||
public function user(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(User::class);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the saved job.
|
||||
*/
|
||||
public function job(): BelongsTo
|
||||
{
|
||||
return $this->belongsTo(Job::class);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Clients;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\ApiException;
|
||||
use App\Services\JobApi\Transformers\AdzunaTransformer;
|
||||
use App\Services\JobApi\Transformers\JobTransformer;
|
||||
use Illuminate\Http\Client\PendingRequest;
|
||||
|
||||
class AdzunaClient extends JobApiClient
|
||||
{
|
||||
private string $appId;
|
||||
private string $appKey;
|
||||
private string $country = 'us'; // Default to US
|
||||
|
||||
protected function getTransformer(): JobTransformer
|
||||
{
|
||||
return new AdzunaTransformer();
|
||||
}
|
||||
|
||||
protected function attachAuthentication(PendingRequest $client): PendingRequest
|
||||
{
|
||||
// Parse API key as "app_id:app_key"
|
||||
if ($this->source->api_key && str_contains($this->source->api_key, ':')) {
|
||||
[$this->appId, $this->appKey] = explode(':', $this->source->api_key, 2);
|
||||
} else {
|
||||
throw new ApiException(
|
||||
'Adzuna API key must be in format "app_id:app_key"',
|
||||
$this->source->name
|
||||
);
|
||||
}
|
||||
|
||||
// Extract country from configuration if set
|
||||
if (!empty($this->source->configuration['country'])) {
|
||||
$this->country = $this->source->configuration['country'];
|
||||
}
|
||||
|
||||
return $client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch jobs from Adzuna API
|
||||
*
|
||||
* @param array $params [
|
||||
* 'what' => string, // Job title or keywords
|
||||
* 'where' => string, // Location
|
||||
* 'results_per_page' => int,
|
||||
* 'page' => int,
|
||||
* 'category' => string,
|
||||
* 'full_time' => int, // 1 for full-time only
|
||||
* 'part_time' => int, // 1 for part-time only
|
||||
* 'contract' => int, // 1 for contract only
|
||||
* 'max_days_old' => int, // Jobs posted within last N days
|
||||
* ]
|
||||
* @return array<JobData>
|
||||
*/
|
||||
public function fetchJobs(array $params = []): array
|
||||
{
|
||||
try {
|
||||
$queryParams = $this->buildQueryParams($params);
|
||||
|
||||
// Adzuna API endpoint structure: /v1/api/jobs/{country}/search/{page}
|
||||
$page = $queryParams['page'] ?? 1;
|
||||
unset($queryParams['page']);
|
||||
|
||||
$endpoint = "search/{$page}";
|
||||
|
||||
$response = $this->makeRequest('GET', $endpoint, $queryParams);
|
||||
|
||||
$data = $response->json();
|
||||
|
||||
if (!isset($data['results']) || !is_array($data['results'])) {
|
||||
throw new ApiException(
|
||||
'Invalid response structure from Adzuna API',
|
||||
$this->source->name,
|
||||
['response' => $data]
|
||||
);
|
||||
}
|
||||
|
||||
$this->updateLastFetched();
|
||||
|
||||
return $this->transformer->transformMany($data['results']);
|
||||
|
||||
} catch (ApiException $e) {
|
||||
throw $e;
|
||||
} catch (\Throwable $e) {
|
||||
throw new ApiException(
|
||||
'Failed to fetch jobs from Adzuna',
|
||||
$this->source->name,
|
||||
['error' => $e->getMessage()],
|
||||
0,
|
||||
$e
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build query parameters for the API request
|
||||
*/
|
||||
private function buildQueryParams(array $params): array
|
||||
{
|
||||
$queryParams = [
|
||||
'app_id' => $this->appId,
|
||||
'app_key' => $this->appKey,
|
||||
'results_per_page' => min($params['results_per_page'] ?? 20, 50),
|
||||
'page' => $params['page'] ?? 1,
|
||||
];
|
||||
|
||||
if (!empty($params['what'])) {
|
||||
$queryParams['what'] = $params['what'];
|
||||
}
|
||||
|
||||
if (!empty($params['where'])) {
|
||||
$queryParams['where'] = $params['where'];
|
||||
}
|
||||
|
||||
if (isset($params['category'])) {
|
||||
$queryParams['category'] = $params['category'];
|
||||
}
|
||||
|
||||
if (isset($params['full_time'])) {
|
||||
$queryParams['full_time'] = (int) $params['full_time'];
|
||||
}
|
||||
|
||||
if (isset($params['part_time'])) {
|
||||
$queryParams['part_time'] = (int) $params['part_time'];
|
||||
}
|
||||
|
||||
if (isset($params['contract'])) {
|
||||
$queryParams['contract'] = (int) $params['contract'];
|
||||
}
|
||||
|
||||
if (isset($params['max_days_old'])) {
|
||||
$queryParams['max_days_old'] = (int) $params['max_days_old'];
|
||||
}
|
||||
|
||||
return $queryParams;
|
||||
}
|
||||
|
||||
/**
|
||||
* Override buildUrl to include country in path
|
||||
*/
|
||||
protected function buildUrl(string $endpoint): string
|
||||
{
|
||||
$baseUrl = rtrim($this->source->api_url, '/');
|
||||
$endpoint = ltrim($endpoint, '/');
|
||||
return "{$baseUrl}/{$this->country}/{$endpoint}";
|
||||
}
|
||||
|
||||
/**
|
||||
* Get supported countries
|
||||
*/
|
||||
public function getSupportedCountries(): array
|
||||
{
|
||||
return [
|
||||
'au' => 'Australia',
|
||||
'at' => 'Austria',
|
||||
'br' => 'Brazil',
|
||||
'ca' => 'Canada',
|
||||
'fr' => 'France',
|
||||
'de' => 'Germany',
|
||||
'in' => 'India',
|
||||
'it' => 'Italy',
|
||||
'nl' => 'Netherlands',
|
||||
'nz' => 'New Zealand',
|
||||
'pl' => 'Poland',
|
||||
'ru' => 'Russia',
|
||||
'sg' => 'Singapore',
|
||||
'za' => 'South Africa',
|
||||
'ch' => 'Switzerland',
|
||||
'gb' => 'United Kingdom',
|
||||
'us' => 'United States',
|
||||
];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Clients;
|
||||
|
||||
use App\Models\JobSource;
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\ApiException;
|
||||
use App\Services\JobApi\Exceptions\RateLimitException;
|
||||
use App\Services\JobApi\Transformers\JobTransformer;
|
||||
use Illuminate\Http\Client\PendingRequest;
|
||||
use Illuminate\Http\Client\Response;
|
||||
use Illuminate\Support\Facades\Http;
|
||||
use Illuminate\Support\Facades\Log;
|
||||
|
||||
abstract class JobApiClient
|
||||
{
|
||||
protected PendingRequest $client;
|
||||
protected JobTransformer $transformer;
|
||||
protected int $maxRetries = 3;
|
||||
protected int $retryDelay = 1000; // milliseconds
|
||||
|
||||
public function __construct(
|
||||
protected JobSource $source
|
||||
) {
|
||||
$this->client = $this->buildHttpClient();
|
||||
$this->transformer = $this->getTransformer();
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the HTTP client with base configuration
|
||||
*/
|
||||
protected function buildHttpClient(): PendingRequest
|
||||
{
|
||||
$client = Http::timeout(30)
|
||||
->retry($this->maxRetries, $this->retryDelay, function ($exception, $request) {
|
||||
// Retry on server errors (5xx) but not on client errors (4xx)
|
||||
if ($exception instanceof \Illuminate\Http\Client\RequestException) {
|
||||
$statusCode = $exception->response?->status();
|
||||
return $statusCode >= 500 && $statusCode < 600;
|
||||
}
|
||||
return false;
|
||||
})
|
||||
->withHeaders($this->getHeaders());
|
||||
|
||||
if ($this->source->api_key) {
|
||||
$client = $this->attachAuthentication($client);
|
||||
}
|
||||
|
||||
return $client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the transformer instance for this API
|
||||
*/
|
||||
abstract protected function getTransformer(): JobTransformer;
|
||||
|
||||
/**
|
||||
* Get default headers for API requests
|
||||
*/
|
||||
protected function getHeaders(): array
|
||||
{
|
||||
return [
|
||||
'Accept' => 'application/json',
|
||||
'User-Agent' => 'JobifyGarden/1.0',
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Attach authentication to the HTTP client
|
||||
*/
|
||||
abstract protected function attachAuthentication(PendingRequest $client): PendingRequest;
|
||||
|
||||
/**
|
||||
* Fetch jobs from the API
|
||||
*
|
||||
* @param array $params Query parameters (keywords, location, page, etc.)
|
||||
* @return array<JobData> Array of standardized job data
|
||||
*/
|
||||
abstract public function fetchJobs(array $params = []): array;
|
||||
|
||||
/**
|
||||
* Make an API request with error handling and logging
|
||||
*/
|
||||
protected function makeRequest(string $method, string $endpoint, array $params = []): Response
|
||||
{
|
||||
$url = $this->buildUrl($endpoint);
|
||||
|
||||
Log::info("API Request: {$this->source->name}", [
|
||||
'method' => $method,
|
||||
'url' => $url,
|
||||
'params' => $params,
|
||||
]);
|
||||
|
||||
try {
|
||||
$response = match (strtoupper($method)) {
|
||||
'GET' => $this->client->get($url, $params),
|
||||
'POST' => $this->client->post($url, $params),
|
||||
default => throw new ApiException(
|
||||
"Unsupported HTTP method: {$method}",
|
||||
$this->source->name
|
||||
),
|
||||
};
|
||||
|
||||
if ($response->failed()) {
|
||||
$this->handleFailedResponse($response);
|
||||
}
|
||||
|
||||
Log::info("API Response: {$this->source->name}", [
|
||||
'status' => $response->status(),
|
||||
'url' => $url,
|
||||
]);
|
||||
|
||||
return $response;
|
||||
|
||||
} catch (\Illuminate\Http\Client\ConnectionException $e) {
|
||||
throw new ApiException(
|
||||
"Connection failed to {$this->source->name}",
|
||||
$this->source->name,
|
||||
['error' => $e->getMessage()],
|
||||
0,
|
||||
$e
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build full URL from endpoint
|
||||
*/
|
||||
protected function buildUrl(string $endpoint): string
|
||||
{
|
||||
$baseUrl = rtrim($this->source->api_url, '/');
|
||||
$endpoint = ltrim($endpoint, '/');
|
||||
return "{$baseUrl}/{$endpoint}";
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle failed API responses
|
||||
*/
|
||||
protected function handleFailedResponse(Response $response): void
|
||||
{
|
||||
$statusCode = $response->status();
|
||||
|
||||
if ($statusCode === 429) {
|
||||
$retryAfter = $response->header('Retry-After');
|
||||
throw new RateLimitException(
|
||||
$this->source->name,
|
||||
$retryAfter ? (int) $retryAfter : null,
|
||||
['response' => $response->body()]
|
||||
);
|
||||
}
|
||||
|
||||
throw new ApiException(
|
||||
"API request failed with status {$statusCode}",
|
||||
$this->source->name,
|
||||
[
|
||||
'status' => $statusCode,
|
||||
'response' => $response->body(),
|
||||
],
|
||||
$statusCode
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the source name
|
||||
*/
|
||||
public function getSourceName(): string
|
||||
{
|
||||
return $this->source->name;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the API is available
|
||||
*/
|
||||
public function isAvailable(): bool
|
||||
{
|
||||
return $this->source->is_active;
|
||||
}
|
||||
|
||||
/**
|
||||
* Update the last fetched timestamp
|
||||
*/
|
||||
protected function updateLastFetched(): void
|
||||
{
|
||||
$this->source->update(['last_fetched_at' => now()]);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Clients;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\ApiException;
|
||||
use App\Services\JobApi\Transformers\JobTransformer;
|
||||
use App\Services\JobApi\Transformers\RemotiveTransformer;
|
||||
use Illuminate\Http\Client\PendingRequest;
|
||||
|
||||
class RemotiveClient extends JobApiClient
|
||||
{
|
||||
protected function getTransformer(): JobTransformer
|
||||
{
|
||||
return new RemotiveTransformer();
|
||||
}
|
||||
|
||||
protected function attachAuthentication(PendingRequest $client): PendingRequest
|
||||
{
|
||||
// Remotive API doesn't require authentication
|
||||
return $client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch jobs from Remotive API
|
||||
*
|
||||
* @param array $params [
|
||||
* 'search' => string, // Job title keywords
|
||||
* 'company_name' => string,
|
||||
* 'category' => string, // e.g., 'software-dev', 'customer-support'
|
||||
* 'limit' => int, // Number of results (default 50)
|
||||
* ]
|
||||
* @return array<JobData>
|
||||
*/
|
||||
public function fetchJobs(array $params = []): array
|
||||
{
|
||||
try {
|
||||
$queryParams = $this->buildQueryParams($params);
|
||||
|
||||
$response = $this->makeRequest('GET', 'remote-jobs', $queryParams);
|
||||
|
||||
$data = $response->json();
|
||||
|
||||
if (!isset($data['jobs']) || !is_array($data['jobs'])) {
|
||||
throw new ApiException(
|
||||
'Invalid response structure from Remotive API',
|
||||
$this->source->name,
|
||||
['response' => $data]
|
||||
);
|
||||
}
|
||||
|
||||
$this->updateLastFetched();
|
||||
|
||||
return $this->transformer->transformMany($data['jobs']);
|
||||
|
||||
} catch (ApiException $e) {
|
||||
throw $e;
|
||||
} catch (\Throwable $e) {
|
||||
throw new ApiException(
|
||||
'Failed to fetch jobs from Remotive',
|
||||
$this->source->name,
|
||||
['error' => $e->getMessage()],
|
||||
0,
|
||||
$e
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build query parameters for the API request
|
||||
*/
|
||||
private function buildQueryParams(array $params): array
|
||||
{
|
||||
$queryParams = [];
|
||||
|
||||
if (!empty($params['search'])) {
|
||||
$queryParams['search'] = $params['search'];
|
||||
}
|
||||
|
||||
if (!empty($params['company_name'])) {
|
||||
$queryParams['company_name'] = $params['company_name'];
|
||||
}
|
||||
|
||||
if (!empty($params['category'])) {
|
||||
$queryParams['category'] = $params['category'];
|
||||
}
|
||||
|
||||
if (!empty($params['limit'])) {
|
||||
$queryParams['limit'] = min((int) $params['limit'], 100);
|
||||
}
|
||||
|
||||
return $queryParams;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get available job categories from Remotive
|
||||
*/
|
||||
public function getCategories(): array
|
||||
{
|
||||
// Remotive categories (as of API documentation)
|
||||
return [
|
||||
'software-dev',
|
||||
'customer-support',
|
||||
'design',
|
||||
'marketing',
|
||||
'sales',
|
||||
'product',
|
||||
'business',
|
||||
'data',
|
||||
'devops',
|
||||
'finance',
|
||||
'hr',
|
||||
'legal',
|
||||
'operations',
|
||||
'qa',
|
||||
'writing',
|
||||
];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Clients;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\ApiException;
|
||||
use App\Services\JobApi\Transformers\JobTransformer;
|
||||
use App\Services\JobApi\Transformers\TheMuseTransformer;
|
||||
use Illuminate\Http\Client\PendingRequest;
|
||||
|
||||
class TheMuseClient extends JobApiClient
|
||||
{
|
||||
protected function getTransformer(): JobTransformer
|
||||
{
|
||||
return new TheMuseTransformer();
|
||||
}
|
||||
|
||||
protected function attachAuthentication(PendingRequest $client): PendingRequest
|
||||
{
|
||||
// The Muse API v2 doesn't require authentication for basic queries
|
||||
// If API key is provided, attach it as a query parameter
|
||||
if ($this->source->api_key) {
|
||||
$client = $client->withQueryParameters(['api_key' => $this->source->api_key]);
|
||||
}
|
||||
|
||||
return $client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch jobs from The Muse API
|
||||
*
|
||||
* @param array $params [
|
||||
* 'page' => int, // Page number (default 0)
|
||||
* 'descending' => bool, // Sort order (default true)
|
||||
* 'category' => string, // Job category
|
||||
* 'level' => string, // e.g., 'Entry Level', 'Mid Level', 'Senior Level'
|
||||
* 'location' => string, // Location filter
|
||||
* 'company' => string, // Company name or ID
|
||||
* ]
|
||||
* @return array<JobData>
|
||||
*/
|
||||
public function fetchJobs(array $params = []): array
|
||||
{
|
||||
try {
|
||||
$queryParams = $this->buildQueryParams($params);
|
||||
|
||||
$response = $this->makeRequest('GET', 'jobs', $queryParams);
|
||||
|
||||
$data = $response->json();
|
||||
|
||||
if (!isset($data['results']) || !is_array($data['results'])) {
|
||||
throw new ApiException(
|
||||
'Invalid response structure from The Muse API',
|
||||
$this->source->name,
|
||||
['response' => $data]
|
||||
);
|
||||
}
|
||||
|
||||
$this->updateLastFetched();
|
||||
|
||||
return $this->transformer->transformMany($data['results']);
|
||||
|
||||
} catch (ApiException $e) {
|
||||
throw $e;
|
||||
} catch (\Throwable $e) {
|
||||
throw new ApiException(
|
||||
'Failed to fetch jobs from The Muse',
|
||||
$this->source->name,
|
||||
['error' => $e->getMessage()],
|
||||
0,
|
||||
$e
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build query parameters for the API request
|
||||
*/
|
||||
private function buildQueryParams(array $params): array
|
||||
{
|
||||
$queryParams = [
|
||||
'page' => $params['page'] ?? 0,
|
||||
'descending' => $params['descending'] ?? true,
|
||||
];
|
||||
|
||||
if (!empty($params['category'])) {
|
||||
$queryParams['category'] = $params['category'];
|
||||
}
|
||||
|
||||
if (!empty($params['level'])) {
|
||||
$queryParams['level'] = $params['level'];
|
||||
}
|
||||
|
||||
if (!empty($params['location'])) {
|
||||
$queryParams['location'] = $params['location'];
|
||||
}
|
||||
|
||||
if (!empty($params['company'])) {
|
||||
$queryParams['company'] = $params['company'];
|
||||
}
|
||||
|
||||
return $queryParams;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get available job categories from The Muse
|
||||
*/
|
||||
public function getCategories(): array
|
||||
{
|
||||
return [
|
||||
'Account Management',
|
||||
'Business & Strategy',
|
||||
'Creative & Design',
|
||||
'Customer Service',
|
||||
'Data Science',
|
||||
'Editorial',
|
||||
'Education',
|
||||
'Engineering',
|
||||
'Finance',
|
||||
'Healthcare & Medicine',
|
||||
'HR & Recruiting',
|
||||
'Legal',
|
||||
'Marketing & PR',
|
||||
'Operations',
|
||||
'Project & Product Management',
|
||||
'Retail',
|
||||
'Sales',
|
||||
'Social Media & Community',
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get available experience levels
|
||||
*/
|
||||
public function getLevels(): array
|
||||
{
|
||||
return [
|
||||
'Internship',
|
||||
'Entry Level',
|
||||
'Mid Level',
|
||||
'Senior Level',
|
||||
'Executive',
|
||||
];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\DTO;
|
||||
|
||||
use DateTime;
|
||||
|
||||
class JobData
|
||||
{
|
||||
public function __construct(
|
||||
public string $externalId,
|
||||
public string $title,
|
||||
public string $company,
|
||||
public string $description,
|
||||
public ?string $location = null,
|
||||
public ?float $salaryMin = null,
|
||||
public ?float $salaryMax = null,
|
||||
public string $salaryCurrency = 'USD',
|
||||
public ?string $jobType = null,
|
||||
public ?string $sourceUrl = null,
|
||||
public ?string $applyUrl = null,
|
||||
public ?DateTime $postedAt = null,
|
||||
public ?DateTime $expiresAt = null,
|
||||
public bool $remoteAllowed = false,
|
||||
public ?string $experienceLevel = null,
|
||||
public ?string $category = null,
|
||||
public ?array $rawData = null,
|
||||
) {}
|
||||
|
||||
/**
|
||||
* Convert to array for database insertion
|
||||
*/
|
||||
public function toArray(): array
|
||||
{
|
||||
return [
|
||||
'external_id' => $this->externalId,
|
||||
'title' => $this->title,
|
||||
'company' => $this->company,
|
||||
'description' => $this->description,
|
||||
'location' => $this->location,
|
||||
'salary_min' => $this->salaryMin,
|
||||
'salary_max' => $this->salaryMax,
|
||||
'salary_currency' => $this->salaryCurrency,
|
||||
'job_type' => $this->jobType,
|
||||
'source_url' => $this->sourceUrl,
|
||||
'apply_url' => $this->applyUrl,
|
||||
'posted_at' => $this->postedAt,
|
||||
'expires_at' => $this->expiresAt,
|
||||
'remote_allowed' => $this->remoteAllowed,
|
||||
'experience_level' => $this->experienceLevel,
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a fingerprint for deduplication
|
||||
*/
|
||||
public function generateFingerprint(): string
|
||||
{
|
||||
$data = strtolower(trim($this->title . '|' . $this->company . '|' . $this->location));
|
||||
return hash('sha256', $data);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Exceptions;
|
||||
|
||||
use Exception;
|
||||
|
||||
class ApiException extends Exception
|
||||
{
|
||||
public function __construct(
|
||||
string $message,
|
||||
public readonly string $apiSource,
|
||||
public readonly ?array $context = null,
|
||||
int $code = 0,
|
||||
?\Throwable $previous = null
|
||||
) {
|
||||
parent::__construct($message, $code, $previous);
|
||||
}
|
||||
|
||||
public function getContext(): array
|
||||
{
|
||||
return array_merge([
|
||||
'api_source' => $this->apiSource,
|
||||
'message' => $this->getMessage(),
|
||||
'code' => $this->getCode(),
|
||||
], $this->context ?? []);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Exceptions;
|
||||
|
||||
class RateLimitException extends ApiException
|
||||
{
|
||||
public function __construct(
|
||||
string $apiSource,
|
||||
public readonly ?int $retryAfter = null,
|
||||
array $context = []
|
||||
) {
|
||||
$message = "Rate limit exceeded for {$apiSource}";
|
||||
if ($this->retryAfter) {
|
||||
$message .= ". Retry after {$this->retryAfter} seconds";
|
||||
}
|
||||
|
||||
parent::__construct($message, $apiSource, $context, 429);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Exceptions;
|
||||
|
||||
class TransformationException extends ApiException
|
||||
{
|
||||
public function __construct(
|
||||
string $apiSource,
|
||||
string $message = 'Failed to transform API response',
|
||||
array $context = []
|
||||
) {
|
||||
parent::__construct($message, $apiSource, $context);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi;
|
||||
|
||||
use App\Models\JobSource;
|
||||
use App\Services\JobApi\Clients\AdzunaClient;
|
||||
use App\Services\JobApi\Clients\JobApiClient;
|
||||
use App\Services\JobApi\Clients\RemotiveClient;
|
||||
use App\Services\JobApi\Clients\TheMuseClient;
|
||||
use App\Services\JobApi\Exceptions\ApiException;
|
||||
|
||||
class JobApiClientFactory
|
||||
{
|
||||
/**
|
||||
* Client class mappings
|
||||
*/
|
||||
private const CLIENT_MAP = [
|
||||
'remotive' => RemotiveClient::class,
|
||||
'the-muse' => TheMuseClient::class,
|
||||
'adzuna' => AdzunaClient::class,
|
||||
];
|
||||
|
||||
/**
|
||||
* Create an API client for the given source
|
||||
*
|
||||
* @param JobSource $source
|
||||
* @return JobApiClient
|
||||
* @throws ApiException
|
||||
*/
|
||||
public static function make(JobSource $source): JobApiClient
|
||||
{
|
||||
$sourceName = strtolower($source->name);
|
||||
$normalizedName = self::normalizeSourceName($sourceName);
|
||||
|
||||
if (!isset(self::CLIENT_MAP[$normalizedName])) {
|
||||
throw new ApiException(
|
||||
"No client implementation found for source: {$source->name}",
|
||||
$source->name,
|
||||
['available_clients' => array_keys(self::CLIENT_MAP)]
|
||||
);
|
||||
}
|
||||
|
||||
$clientClass = self::CLIENT_MAP[$normalizedName];
|
||||
|
||||
return new $clientClass($source);
|
||||
}
|
||||
|
||||
/**
|
||||
* Create clients for all active sources
|
||||
*
|
||||
* @return array<JobApiClient>
|
||||
*/
|
||||
public static function makeAll(): array
|
||||
{
|
||||
$sources = JobSource::active()->get();
|
||||
|
||||
return $sources->map(function (JobSource $source) {
|
||||
try {
|
||||
return self::make($source);
|
||||
} catch (ApiException $e) {
|
||||
// Log the error but continue with other sources
|
||||
\Log::warning("Failed to create client for source: {$source->name}", [
|
||||
'error' => $e->getMessage(),
|
||||
]);
|
||||
return null;
|
||||
}
|
||||
})->filter()->values()->all();
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a client by source name
|
||||
*
|
||||
* @param string $sourceName
|
||||
* @return JobApiClient
|
||||
* @throws ApiException
|
||||
*/
|
||||
public static function makeByName(string $sourceName): JobApiClient
|
||||
{
|
||||
$source = JobSource::where('name', $sourceName)
|
||||
->orWhere('name', 'LIKE', "%{$sourceName}%")
|
||||
->firstOrFail();
|
||||
|
||||
return self::make($source);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all available client types
|
||||
*
|
||||
* @return array
|
||||
*/
|
||||
public static function getAvailableClients(): array
|
||||
{
|
||||
return array_keys(self::CLIENT_MAP);
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize source name for lookup
|
||||
*/
|
||||
private static function normalizeSourceName(string $name): string
|
||||
{
|
||||
// Convert "Remotive", "remotive.io", "Remotive.io" to "remotive"
|
||||
$name = strtolower($name);
|
||||
$name = str_replace(['.io', '.com', ' '], '', $name);
|
||||
|
||||
// Handle special cases
|
||||
$name = match (true) {
|
||||
str_contains($name, 'muse') => 'the-muse',
|
||||
str_contains($name, 'remotive') => 'remotive',
|
||||
str_contains($name, 'adzuna') => 'adzuna',
|
||||
default => $name,
|
||||
};
|
||||
|
||||
return $name;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Transformers;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\TransformationException;
|
||||
use DateTime;
|
||||
|
||||
class AdzunaTransformer implements JobTransformer
|
||||
{
|
||||
public function transform(array $data): JobData
|
||||
{
|
||||
try {
|
||||
return new JobData(
|
||||
externalId: (string) $data['id'],
|
||||
title: $data['title'] ?? 'Untitled',
|
||||
company: $data['company']['display_name'] ?? 'Unknown Company',
|
||||
description: $this->sanitizeDescription($data['description'] ?? ''),
|
||||
location: $this->parseLocation($data),
|
||||
salaryMin: $data['salary_min'] ?? null,
|
||||
salaryMax: $data['salary_max'] ?? null,
|
||||
salaryCurrency: 'USD',
|
||||
jobType: $this->mapJobType($data['contract_type'] ?? null),
|
||||
sourceUrl: $data['redirect_url'] ?? null,
|
||||
applyUrl: $data['redirect_url'] ?? null,
|
||||
postedAt: $this->parseDate($data['created'] ?? null),
|
||||
expiresAt: null,
|
||||
remoteAllowed: $this->isRemote($data),
|
||||
experienceLevel: null,
|
||||
category: $data['category']['label'] ?? null,
|
||||
rawData: $data,
|
||||
);
|
||||
} catch (\Throwable $e) {
|
||||
throw new TransformationException(
|
||||
'Adzuna',
|
||||
'Failed to transform Adzuna job data: ' . $e->getMessage(),
|
||||
['data' => $data, 'error' => $e->getMessage()]
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
public function transformMany(array $jobs): array
|
||||
{
|
||||
return array_map(fn($job) => $this->transform($job), $jobs);
|
||||
}
|
||||
|
||||
private function sanitizeDescription(string $description): string
|
||||
{
|
||||
$clean = strip_tags($description);
|
||||
$clean = html_entity_decode($clean, ENT_QUOTES | ENT_HTML5, 'UTF-8');
|
||||
$clean = preg_replace('/\s+/', ' ', $clean);
|
||||
return trim($clean);
|
||||
}
|
||||
|
||||
private function parseLocation(array $data): ?string
|
||||
{
|
||||
$location = $data['location']['display_name'] ?? null;
|
||||
|
||||
if (empty($location)) {
|
||||
// Fallback to area
|
||||
$area = $data['location']['area'] ?? [];
|
||||
if (!empty($area)) {
|
||||
return implode(', ', $area);
|
||||
}
|
||||
}
|
||||
|
||||
return $location;
|
||||
}
|
||||
|
||||
private function isRemote(array $data): bool
|
||||
{
|
||||
$title = strtolower($data['title'] ?? '');
|
||||
$description = strtolower($data['description'] ?? '');
|
||||
$location = strtolower($data['location']['display_name'] ?? '');
|
||||
|
||||
return str_contains($title, 'remote') ||
|
||||
str_contains($description, 'remote') ||
|
||||
str_contains($location, 'remote');
|
||||
}
|
||||
|
||||
private function parseDate(?string $date): ?DateTime
|
||||
{
|
||||
if (empty($date)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
return new DateTime($date);
|
||||
} catch (\Exception $e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private function mapJobType(?string $type): ?string
|
||||
{
|
||||
if (empty($type)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return match (strtolower($type)) {
|
||||
'permanent', 'full_time' => 'full-time',
|
||||
'part_time' => 'part-time',
|
||||
'contract' => 'contract',
|
||||
'temporary' => 'temporary',
|
||||
default => 'full-time',
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Transformers;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
|
||||
interface JobTransformer
|
||||
{
|
||||
/**
|
||||
* Transform a single job from API response to JobData DTO
|
||||
*
|
||||
* @param array $data Raw job data from API
|
||||
* @return JobData Standardized job data
|
||||
*/
|
||||
public function transform(array $data): JobData;
|
||||
|
||||
/**
|
||||
* Transform multiple jobs from API response
|
||||
*
|
||||
* @param array $jobs Array of raw job data from API
|
||||
* @return array<JobData> Array of standardized job data
|
||||
*/
|
||||
public function transformMany(array $jobs): array;
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Transformers;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\TransformationException;
|
||||
use DateTime;
|
||||
|
||||
class RemotiveTransformer implements JobTransformer
|
||||
{
|
||||
public function transform(array $data): JobData
|
||||
{
|
||||
try {
|
||||
return new JobData(
|
||||
externalId: (string) $data['id'],
|
||||
title: $data['title'] ?? 'Untitled',
|
||||
company: $data['company_name'] ?? 'Unknown Company',
|
||||
description: $this->sanitizeDescription($data['description'] ?? ''),
|
||||
location: $this->parseLocation($data),
|
||||
salaryMin: $this->parseSalary($data['salary'] ?? null, 'min'),
|
||||
salaryMax: $this->parseSalary($data['salary'] ?? null, 'max'),
|
||||
salaryCurrency: 'USD',
|
||||
jobType: $this->mapJobType($data['job_type'] ?? null),
|
||||
sourceUrl: $data['url'] ?? null,
|
||||
applyUrl: $data['url'] ?? null,
|
||||
postedAt: $this->parseDate($data['publication_date'] ?? null),
|
||||
expiresAt: null,
|
||||
remoteAllowed: true, // Remotive only has remote jobs
|
||||
experienceLevel: $this->mapExperienceLevel($data['candidate_required_location'] ?? null),
|
||||
category: $data['category'] ?? null,
|
||||
rawData: $data,
|
||||
);
|
||||
} catch (\Throwable $e) {
|
||||
throw new TransformationException(
|
||||
'Remotive',
|
||||
'Failed to transform Remotive job data: ' . $e->getMessage(),
|
||||
['data' => $data, 'error' => $e->getMessage()]
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
public function transformMany(array $jobs): array
|
||||
{
|
||||
return array_map(fn($job) => $this->transform($job), $jobs);
|
||||
}
|
||||
|
||||
private function sanitizeDescription(string $description): string
|
||||
{
|
||||
// Remove HTML tags and clean up whitespace
|
||||
$clean = strip_tags($description);
|
||||
$clean = html_entity_decode($clean, ENT_QUOTES | ENT_HTML5, 'UTF-8');
|
||||
$clean = preg_replace('/\s+/', ' ', $clean);
|
||||
return trim($clean);
|
||||
}
|
||||
|
||||
private function parseLocation(array $data): ?string
|
||||
{
|
||||
$location = $data['candidate_required_location'] ?? null;
|
||||
|
||||
if (empty($location)) {
|
||||
return 'Remote';
|
||||
}
|
||||
|
||||
return $location;
|
||||
}
|
||||
|
||||
private function parseSalary(?string $salary, string $type): ?float
|
||||
{
|
||||
if (empty($salary)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Try to extract numbers from salary string
|
||||
if (preg_match_all('/[\d,]+/', $salary, $matches)) {
|
||||
$numbers = array_map(fn($n) => (float) str_replace(',', '', $n), $matches[0]);
|
||||
|
||||
if (count($numbers) >= 2) {
|
||||
return $type === 'min' ? min($numbers) : max($numbers);
|
||||
} elseif (count($numbers) === 1) {
|
||||
return $numbers[0];
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private function parseDate(?string $date): ?DateTime
|
||||
{
|
||||
if (empty($date)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
return new DateTime($date);
|
||||
} catch (\Exception $e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private function mapJobType(?string $type): ?string
|
||||
{
|
||||
if (empty($type)) {
|
||||
return 'full-time';
|
||||
}
|
||||
|
||||
return match (strtolower($type)) {
|
||||
'full_time', 'full-time' => 'full-time',
|
||||
'part_time', 'part-time' => 'part-time',
|
||||
'contract' => 'contract',
|
||||
'freelance' => 'freelance',
|
||||
default => 'full-time',
|
||||
};
|
||||
}
|
||||
|
||||
private function mapExperienceLevel(?string $location): ?string
|
||||
{
|
||||
// Remotive doesn't provide experience level directly
|
||||
// This is a placeholder that could be enhanced with ML/NLP
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
<?php
|
||||
|
||||
namespace App\Services\JobApi\Transformers;
|
||||
|
||||
use App\Services\JobApi\DTO\JobData;
|
||||
use App\Services\JobApi\Exceptions\TransformationException;
|
||||
use DateTime;
|
||||
|
||||
class TheMuseTransformer implements JobTransformer
|
||||
{
|
||||
public function transform(array $data): JobData
|
||||
{
|
||||
try {
|
||||
return new JobData(
|
||||
externalId: (string) $data['id'],
|
||||
title: $data['name'] ?? 'Untitled',
|
||||
company: $data['company']['name'] ?? 'Unknown Company',
|
||||
description: $this->sanitizeDescription($data['contents'] ?? ''),
|
||||
location: $this->parseLocation($data),
|
||||
salaryMin: null, // The Muse doesn't provide salary info
|
||||
salaryMax: null,
|
||||
salaryCurrency: 'USD',
|
||||
jobType: $this->mapJobType($data['type'] ?? null),
|
||||
sourceUrl: $data['refs']['landing_page'] ?? null,
|
||||
applyUrl: $data['refs']['landing_page'] ?? null,
|
||||
postedAt: $this->parseDate($data['publication_date'] ?? null),
|
||||
expiresAt: null,
|
||||
remoteAllowed: $this->isRemote($data),
|
||||
experienceLevel: $this->mapExperienceLevel($data['levels'] ?? []),
|
||||
category: $this->parseCategory($data['categories'] ?? []),
|
||||
rawData: $data,
|
||||
);
|
||||
} catch (\Throwable $e) {
|
||||
throw new TransformationException(
|
||||
'The Muse',
|
||||
'Failed to transform The Muse job data: ' . $e->getMessage(),
|
||||
['data' => $data, 'error' => $e->getMessage()]
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
public function transformMany(array $jobs): array
|
||||
{
|
||||
return array_map(fn($job) => $this->transform($job), $jobs);
|
||||
}
|
||||
|
||||
private function sanitizeDescription(string $description): string
|
||||
{
|
||||
$clean = strip_tags($description);
|
||||
$clean = html_entity_decode($clean, ENT_QUOTES | ENT_HTML5, 'UTF-8');
|
||||
$clean = preg_replace('/\s+/', ' ', $clean);
|
||||
return trim($clean);
|
||||
}
|
||||
|
||||
private function parseLocation(array $data): ?string
|
||||
{
|
||||
if (empty($data['locations'])) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$locations = array_map(fn($loc) => $loc['name'] ?? '', $data['locations']);
|
||||
return implode(', ', array_filter($locations));
|
||||
}
|
||||
|
||||
private function isRemote(array $data): bool
|
||||
{
|
||||
if (empty($data['locations'])) {
|
||||
return false;
|
||||
}
|
||||
|
||||
foreach ($data['locations'] as $location) {
|
||||
$name = strtolower($location['name'] ?? '');
|
||||
if (str_contains($name, 'remote') || str_contains($name, 'flexible')) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
private function parseDate(?string $date): ?DateTime
|
||||
{
|
||||
if (empty($date)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
return new DateTime($date);
|
||||
} catch (\Exception $e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private function mapJobType(?string $type): ?string
|
||||
{
|
||||
if (empty($type)) {
|
||||
return 'full-time';
|
||||
}
|
||||
|
||||
return match (strtolower($type)) {
|
||||
'full-time', 'fulltime' => 'full-time',
|
||||
'part-time', 'parttime' => 'part-time',
|
||||
'contract' => 'contract',
|
||||
'temporary', 'temp' => 'temporary',
|
||||
'internship', 'intern' => 'internship',
|
||||
'freelance' => 'freelance',
|
||||
default => 'full-time',
|
||||
};
|
||||
}
|
||||
|
||||
private function mapExperienceLevel(array $levels): ?string
|
||||
{
|
||||
if (empty($levels)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// The Muse uses: Entry Level, Mid Level, Senior Level
|
||||
$firstLevel = strtolower($levels[0]['name'] ?? '');
|
||||
|
||||
return match (true) {
|
||||
str_contains($firstLevel, 'entry') => 'entry',
|
||||
str_contains($firstLevel, 'mid') => 'mid',
|
||||
str_contains($firstLevel, 'senior') => 'senior',
|
||||
str_contains($firstLevel, 'executive') => 'executive',
|
||||
default => null,
|
||||
};
|
||||
}
|
||||
|
||||
private function parseCategory(array $categories): ?string
|
||||
{
|
||||
if (empty($categories)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Return the first category name
|
||||
return $categories[0]['name'] ?? null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
services:
|
||||
laravel.test:
|
||||
build:
|
||||
context: './vendor/laravel/sail/runtimes/8.5'
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
WWWGROUP: '${WWWGROUP}'
|
||||
image: 'sail-8.5/app'
|
||||
extra_hosts:
|
||||
- 'host.docker.internal:host-gateway'
|
||||
ports:
|
||||
- '${APP_PORT:-80}:80'
|
||||
- '${VITE_PORT:-5173}:${VITE_PORT:-5173}'
|
||||
environment:
|
||||
WWWUSER: '${WWWUSER}'
|
||||
LARAVEL_SAIL: 1
|
||||
XDEBUG_MODE: '${SAIL_XDEBUG_MODE:-off}'
|
||||
XDEBUG_CONFIG: '${SAIL_XDEBUG_CONFIG:-client_host=host.docker.internal}'
|
||||
IGNITION_LOCAL_SITES_PATH: '${PWD}'
|
||||
volumes:
|
||||
- '.:/var/www/html'
|
||||
networks:
|
||||
- sail
|
||||
depends_on:
|
||||
- mysql
|
||||
mysql:
|
||||
image: 'mysql:8.4'
|
||||
ports:
|
||||
- '${FORWARD_DB_PORT:-3306}:3306'
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: '${DB_PASSWORD}'
|
||||
MYSQL_ROOT_HOST: '%'
|
||||
MYSQL_DATABASE: '${DB_DATABASE}'
|
||||
MYSQL_USER: '${DB_USERNAME}'
|
||||
MYSQL_PASSWORD: '${DB_PASSWORD}'
|
||||
MYSQL_ALLOW_EMPTY_PASSWORD: 1
|
||||
MYSQL_EXTRA_OPTIONS: '${MYSQL_EXTRA_OPTIONS:-}'
|
||||
volumes:
|
||||
- 'sail-mysql:/var/lib/mysql'
|
||||
- './vendor/laravel/sail/database/mysql/create-testing-database.sh:/docker-entrypoint-initdb.d/10-create-testing-database.sh'
|
||||
networks:
|
||||
- sail
|
||||
healthcheck:
|
||||
test:
|
||||
- CMD
|
||||
- mysqladmin
|
||||
- ping
|
||||
- '-p${DB_PASSWORD}'
|
||||
retries: 3
|
||||
timeout: 5s
|
||||
networks:
|
||||
sail:
|
||||
driver: bridge
|
||||
volumes:
|
||||
sail-mysql:
|
||||
driver: local
|
||||
+1
-1
@@ -38,7 +38,7 @@ return [
|
||||
'database' => [
|
||||
'driver' => 'database',
|
||||
'connection' => env('DB_QUEUE_CONNECTION'),
|
||||
'table' => env('DB_QUEUE_TABLE', 'jobs'),
|
||||
'table' => env('DB_QUEUE_TABLE', 'queue_jobs'),
|
||||
'queue' => env('DB_QUEUE', 'default'),
|
||||
'retry_after' => (int) env('DB_QUEUE_RETRY_AFTER', 90),
|
||||
'after_commit' => false,
|
||||
|
||||
@@ -11,7 +11,7 @@ return new class extends Migration
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('jobs', function (Blueprint $table) {
|
||||
Schema::create('queue_jobs', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->string('queue')->index();
|
||||
$table->longText('payload');
|
||||
@@ -50,7 +50,7 @@ return new class extends Migration
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('jobs');
|
||||
Schema::dropIfExists('queue_jobs');
|
||||
Schema::dropIfExists('job_batches');
|
||||
Schema::dropIfExists('failed_jobs');
|
||||
}
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
/**
|
||||
* Run the migrations.
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('job_categories', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->string('name');
|
||||
$table->string('slug')->unique();
|
||||
$table->text('description')->nullable();
|
||||
$table->string('icon')->nullable();
|
||||
$table->timestamps();
|
||||
|
||||
$table->index('slug');
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Reverse the migrations.
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('job_categories');
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,36 @@
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
/**
|
||||
* Run the migrations.
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('job_locations', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->string('city')->nullable();
|
||||
$table->string('state')->nullable();
|
||||
$table->string('country');
|
||||
$table->string('country_code', 2);
|
||||
$table->decimal('latitude', 10, 7)->nullable();
|
||||
$table->decimal('longitude', 10, 7)->nullable();
|
||||
$table->timestamps();
|
||||
|
||||
$table->index(['country', 'state', 'city']);
|
||||
$table->index('country_code');
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Reverse the migrations.
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('job_locations');
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,37 @@
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
/**
|
||||
* Run the migrations.
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('job_sources', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->string('name');
|
||||
$table->string('api_url');
|
||||
$table->text('api_key')->nullable();
|
||||
$table->boolean('is_active')->default(true);
|
||||
$table->integer('rate_limit')->default(100);
|
||||
$table->timestamp('last_fetched_at')->nullable();
|
||||
$table->json('configuration')->nullable();
|
||||
$table->timestamps();
|
||||
|
||||
$table->index('is_active');
|
||||
$table->index('last_fetched_at');
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Reverse the migrations.
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('job_sources');
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,57 @@
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
/**
|
||||
* Run the migrations.
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('jobs', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->string('external_id')->nullable();
|
||||
$table->string('title');
|
||||
$table->string('company');
|
||||
$table->text('description');
|
||||
$table->string('location')->nullable();
|
||||
$table->decimal('salary_min', 10, 2)->nullable();
|
||||
$table->decimal('salary_max', 10, 2)->nullable();
|
||||
$table->string('salary_currency', 3)->nullable()->default('USD');
|
||||
$table->enum('job_type', ['full-time', 'part-time', 'contract', 'temporary', 'internship', 'freelance'])->nullable();
|
||||
$table->foreignId('job_source_id')->constrained('job_sources')->onDelete('cascade');
|
||||
$table->string('source_url')->nullable();
|
||||
$table->string('apply_url')->nullable();
|
||||
$table->timestamp('posted_at')->nullable();
|
||||
$table->timestamp('expires_at')->nullable();
|
||||
$table->boolean('remote_allowed')->default(false);
|
||||
$table->enum('experience_level', ['entry', 'mid', 'senior', 'lead', 'executive'])->nullable();
|
||||
$table->foreignId('job_category_id')->nullable()->constrained('job_categories')->onDelete('set null');
|
||||
$table->foreignId('job_location_id')->nullable()->constrained('job_locations')->onDelete('set null');
|
||||
$table->string('fingerprint')->nullable()->unique();
|
||||
$table->timestamps();
|
||||
$table->softDeletes();
|
||||
|
||||
$table->index(['job_source_id', 'external_id']);
|
||||
$table->index('posted_at');
|
||||
$table->index('expires_at');
|
||||
$table->index('job_type');
|
||||
$table->index('experience_level');
|
||||
$table->index('remote_allowed');
|
||||
$table->index('job_category_id');
|
||||
$table->index('job_location_id');
|
||||
$table->fullText(['title', 'company', 'description']);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Reverse the migrations.
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('jobs');
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,32 @@
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
/**
|
||||
* Run the migrations.
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('user_saved_jobs', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->foreignId('user_id')->constrained('users')->onDelete('cascade');
|
||||
$table->foreignId('job_id')->constrained('jobs')->onDelete('cascade');
|
||||
$table->timestamps();
|
||||
|
||||
$table->unique(['user_id', 'job_id']);
|
||||
$table->index('user_id');
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Reverse the migrations.
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('user_saved_jobs');
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,37 @@
|
||||
<?php
|
||||
|
||||
use Illuminate\Database\Migrations\Migration;
|
||||
use Illuminate\Database\Schema\Blueprint;
|
||||
use Illuminate\Support\Facades\Schema;
|
||||
|
||||
return new class extends Migration
|
||||
{
|
||||
/**
|
||||
* Run the migrations.
|
||||
*/
|
||||
public function up(): void
|
||||
{
|
||||
Schema::create('user_job_alerts', function (Blueprint $table) {
|
||||
$table->id();
|
||||
$table->foreignId('user_id')->constrained('users')->onDelete('cascade');
|
||||
$table->string('keywords')->nullable();
|
||||
$table->string('location')->nullable();
|
||||
$table->foreignId('job_category_id')->nullable()->constrained('job_categories')->onDelete('set null');
|
||||
$table->enum('frequency', ['instant', 'daily', 'weekly'])->default('daily');
|
||||
$table->boolean('is_active')->default(true);
|
||||
$table->timestamp('last_sent_at')->nullable();
|
||||
$table->timestamps();
|
||||
|
||||
$table->index('user_id');
|
||||
$table->index('is_active');
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Reverse the migrations.
|
||||
*/
|
||||
public function down(): void
|
||||
{
|
||||
Schema::dropIfExists('user_job_alerts');
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,62 @@
|
||||
<?php
|
||||
|
||||
namespace Database\Seeders;
|
||||
|
||||
use App\Models\JobSource;
|
||||
use Illuminate\Database\Seeder;
|
||||
|
||||
class JobSourceSeeder extends Seeder
|
||||
{
|
||||
/**
|
||||
* Run the database seeds.
|
||||
*/
|
||||
public function run(): void
|
||||
{
|
||||
$sources = [
|
||||
[
|
||||
'name' => 'Remotive',
|
||||
'api_url' => 'https://remotive.com/api',
|
||||
'api_key' => null, // No API key required
|
||||
'is_active' => true,
|
||||
'rate_limit' => 60, // requests per hour
|
||||
'configuration' => [
|
||||
'description' => 'Remote jobs API - Free, no authentication required',
|
||||
'docs' => 'https://remotive.com/api-documentation',
|
||||
],
|
||||
],
|
||||
[
|
||||
'name' => 'The Muse',
|
||||
'api_url' => 'https://www.themuse.com/api/public',
|
||||
'api_key' => null, // Optional API key for higher limits
|
||||
'is_active' => true,
|
||||
'rate_limit' => 100, // requests per hour
|
||||
'configuration' => [
|
||||
'description' => 'The Muse job board API - Free tier available',
|
||||
'docs' => 'https://www.themuse.com/developers/api/v2',
|
||||
],
|
||||
],
|
||||
[
|
||||
'name' => 'Adzuna',
|
||||
'api_url' => 'https://api.adzuna.com/v1/api/jobs',
|
||||
'api_key' => env('ADZUNA_API_KEY'), // Format: app_id:app_key
|
||||
'is_active' => env('ADZUNA_API_KEY') ? true : false,
|
||||
'rate_limit' => 100, // requests per hour
|
||||
'configuration' => [
|
||||
'country' => 'us',
|
||||
'description' => 'Adzuna job search API - Requires free API key',
|
||||
'docs' => 'https://developer.adzuna.com/overview',
|
||||
'signup' => 'https://developer.adzuna.com/signup',
|
||||
],
|
||||
],
|
||||
];
|
||||
|
||||
foreach ($sources as $source) {
|
||||
JobSource::updateOrCreate(
|
||||
['name' => $source['name']],
|
||||
$source
|
||||
);
|
||||
}
|
||||
|
||||
$this->command->info('Job sources seeded successfully!');
|
||||
}
|
||||
}
|
||||
+1
-2
@@ -23,8 +23,7 @@
|
||||
<env name="BCRYPT_ROUNDS" value="4"/>
|
||||
<env name="BROADCAST_CONNECTION" value="null"/>
|
||||
<env name="CACHE_STORE" value="array"/>
|
||||
<env name="DB_CONNECTION" value="sqlite"/>
|
||||
<env name="DB_DATABASE" value=":memory:"/>
|
||||
<env name="DB_DATABASE" value="testing"/>
|
||||
<env name="DB_URL" value=""/>
|
||||
<env name="MAIL_MAILER" value="array"/>
|
||||
<env name="QUEUE_CONNECTION" value="sync"/>
|
||||
|
||||
Reference in New Issue
Block a user