How Similarweb got its users

Similarweb's unit is one domain. The template is /website/[domain]/, titled "[domain] Traffic Analytics, Ranking & Audience [Month Year]". The list is not bought: /corp/ourdata/ names its four pillars, contributors network, first party analytics, public data, partners. Pick the single unit your product already measures and can restate monthly. Hand-write that template for three real units before scripting anything. If it reads the same with the unit swapped, there is no page.

done whenTake your three hand-written pages and swap the unit name in each. If every sentence still reads fine unchanged, your template carries no real data and scripting it just multiplies a blank page 5,000 times.

One dataset, several templates. Off the same measurements Similarweb runs /ai-traffic/[domain]/, /website/[a]/vs/[b]/ ("airbnb.com vs. vrbo.com Ranking Comparison"), /website/[domain]/competitors/, and the grid /top-websites/[country]/[category]/[subcategory]/, which holds 12,831 URLs in their top-websites-001 sitemap. Ship one template and get it indexed first. Add the vs/ pairing only after single-unit pages rank: pairs square your page count and your thin pages.

done whenCount the templates you have planned against the templates with at least one page actually ranking. Until the first is indexed and pulling clicks, a second template is duplicated work on an unproven format.

Make every page a crawl hub. Each /website/[domain]/ page carries a "Top 10 [domain] Competitors" module ranked by affinity across keyword traffic, audience targeting and market overlap, and links out to all ten. The /top-websites/[country]/[category]/ grid feeds downward into those same pages. Copy it: every unit page links to five or ten siblings chosen by a computed relation, never alphabetical. Your long tail gets found through those links, not through the nav.

done whenPick a page fifteen levels deep in your own set and count the clicks from your homepage to reach it. Past three, it sits uncrawled no matter how good the underlying data is.

Stamp the period into the page. Similarweb titles read "airbnb.com Traffic Analytics, Ranking & Audience [July 2026]" and the AI page H1 reads "airbnb.com AI Traffic for July 2026". Their index at /sitemaps/sitemap_index.xml.gz carries a fresh lastmod per shard. Only pick this channel if your numbers actually move month to month. Static facts make a directory, and a directory gets scraped and outranked. Recomputed numbers earn the recrawl.

done whenRun your generator twice, a month apart, on the same unit and diff the two outputs. If nothing changed, you have a static directory and no reason for a crawler to return, so pick a different channel.

Give away the answer, gate the next question. The public /website/ page hands over global rank, country rank, bounce rate, pages per visit, visit duration, top countries and ten competitors. Deeper cuts sit behind "Start a trial" and "Register to download PDF" buttons placed inside each section, not at the top. Copy that placement: the visitor gets exactly what they searched for, then meets the wall the moment they want the layer below it.

done whenOpen your own generated page logged out and read only what loads free. If you cannot answer the query you targeted without signing up, you rank and bounce. If you answer everything, nobody signs up.