Track and display subdomains under their root domain
Adds sites.root_domain (indexed string, backfilled for existing rows)
computed via a naive "last two labels" heuristic (App\Support\RootDomain)
— good enough for example.com/somesite.es, but wrong for two-part
public suffixes like .co.uk. Doing that correctly needs the Mozilla
Public Suffix List; flagged as a known simplification rather than
built now.
Deliberately not a hard FK: like Backlink.target_domain, a subdomain
can get indexed before its root domain does (or vice versa), so
Site::subdomains()/rootSite() match on the domain string rather than
requiring the relationship to exist at write time.
Site detail page: a root domain shows a paginated "Subdomains"
section (same guest-preview cap as Pages/Backlinks); a subdomain
shows "Subdomain of {root}" linking back, when the root is itself
indexed.
Verified against real data: 21k.ro correctly lists its two known
subdomains (buzau.21k.ro, timisoara.21k.ro), each of which correctly
links back to 21k.ro. Added SubdomainsTest (root lists subdomains,
subdomain links to root, empty state) — full suite (31 tests) passes.
This commit is contained in:
@@ -6,6 +6,7 @@ use App\Models\Backlink;
|
||||
use App\Models\Site;
|
||||
use App\Models\SitePage;
|
||||
use App\Support\CommonCrawl\WarcRecordFetcher;
|
||||
use App\Support\RootDomain;
|
||||
use Illuminate\Support\Collection;
|
||||
use Illuminate\Support\Facades\Http;
|
||||
|
||||
@@ -116,6 +117,7 @@ class CommonCrawlIndexService
|
||||
|
||||
$homepage = $pages->sort(fn ($a, $b) => $this->compareCaptures($a, $b))->first();
|
||||
$site->url = $homepage['url'];
|
||||
$site->root_domain = RootDomain::of($group['domain']);
|
||||
|
||||
if ($withContent) {
|
||||
$content = $this->fetchContent($homepage, $contentCache);
|
||||
|
||||
Reference in New Issue
Block a user