Initial commit: Laravel scaffold + Common Crawl site indexing
Adds Site/SitePage models and the crawlindex:fetch command, which pulls CDX index records from Common Crawl and optionally enriches them with page title/description/keywords/body text extracted from the WARC archive. CDX/WARC access and the site/page upsert logic live in CommonCrawlIndexService and WarcRecordFetcher.
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
@import 'tailwindcss';
|
||||
|
||||
@source '../../vendor/laravel/framework/src/Illuminate/Pagination/resources/views/*.blade.php';
|
||||
@source '../../storage/framework/views/*.php';
|
||||
|
||||
@theme {
|
||||
--font-sans: 'Instrument Sans', ui-sans-serif, system-ui, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji',
|
||||
'Segoe UI Symbol', 'Noto Color Emoji';
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
//
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user