Track and display subdomains under their root domain

Adds sites.root_domain (indexed string, backfilled for existing rows)
computed via a naive "last two labels" heuristic (App\Support\RootDomain)
— good enough for example.com/somesite.es, but wrong for two-part
public suffixes like .co.uk. Doing that correctly needs the Mozilla
Public Suffix List; flagged as a known simplification rather than
built now.

Deliberately not a hard FK: like Backlink.target_domain, a subdomain
can get indexed before its root domain does (or vice versa), so
Site::subdomains()/rootSite() match on the domain string rather than
requiring the relationship to exist at write time.

Site detail page: a root domain shows a paginated "Subdomains"
section (same guest-preview cap as Pages/Backlinks); a subdomain
shows "Subdomain of {root}" linking back, when the root is itself
indexed.

Verified against real data: 21k.ro correctly lists its two known
subdomains (buzau.21k.ro, timisoara.21k.ro), each of which correctly
links back to 21k.ro. Added SubdomainsTest (root lists subdomains,
subdomain links to root, empty state) — full suite (31 tests) passes.
This commit is contained in:
m105
2026-07-25 15:44:12 +03:00
parent 6435bccb20
commit 72b87ab579
8 changed files with 196 additions and 0 deletions
@@ -6,6 +6,7 @@ use App\Models\Backlink;
use App\Models\Site;
use App\Models\SitePage;
use App\Support\CommonCrawl\WarcRecordFetcher;
use App\Support\RootDomain;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Http;
@@ -116,6 +117,7 @@ class CommonCrawlIndexService
$homepage = $pages->sort(fn ($a, $b) => $this->compareCaptures($a, $b))->first();
$site->url = $homepage['url'];
$site->root_domain = RootDomain::of($group['domain']);
if ($withContent) {
$content = $this->fetchContent($homepage, $contentCache);