Add tech-stack fingerprinting (server, powered-by, generator)
WarcRecordFetcher was splitting out and discarding the HTTP response
headers to get at the HTML body — now it keeps and parses them, so
Server/X-Powered-By headers get captured alongside the <meta
name="generator"> tag already sitting in the parsed DOM. No extra
requests: same WARC segment already fetched for --with-metadata.
Site copies these from its homepage capture, same as
title/description/keywords. New badges on the site detail page show
the detected platform (e.g. "Wix.com Website Builder") and server
signature at a glance.
Verified against real .es data: correctly detected Wix
("00padel.es"), nginx/Apache/Cloudflare server headers, and PHP
X-Powered-By headers across the batch.
This commit is contained in:
@@ -14,6 +14,9 @@ class Site extends Model
|
||||
'title',
|
||||
'description',
|
||||
'keywords',
|
||||
'server',
|
||||
'powered_by',
|
||||
'generator',
|
||||
];
|
||||
|
||||
public function pages(): HasMany
|
||||
|
||||
@@ -27,6 +27,9 @@ class SitePage extends Model
|
||||
'title',
|
||||
'description',
|
||||
'keywords',
|
||||
'server',
|
||||
'powered_by',
|
||||
'generator',
|
||||
'content',
|
||||
'raw',
|
||||
'record_hash',
|
||||
|
||||
Reference in New Issue
Block a user