a single source, nothing corroborates it

Rank a rival's archive by argument, not shares

Crawl a competitor's blog, exclude the tag and pagination URLs so a capped crawl is spent on articles, and extract the comment counter into a ranked table. An article people argue about is the kind that later attracts links, which makes comment volume a better content map than share data. Reddit's per-domain listing does the same job for communities.

The method

1. 01

   Rank a rival blog's archive by comment count rather than by social shares, because an article people argue about is the kind that later attracts links.

2. 02

   Copy the XPath of the comment counter straight out of the browser inspector and feed it to Screaming Frog custom extraction, which turns any blog archive into a ranked table in one pass.

3. 03

   Crawl a competitor with Screaming Frog but blocklist the URL patterns that are not posts first, so a capped crawl is spent on articles instead of tag and pagination pages.

4. 04

   Rent a high-memory cloud instance when a scrape gets big, and raise the crawler's memory ceiling by hand rather than trusting its default.

5. 05

   Query Reddit's per-domain listing page to see everywhere a site has been submitted, which surfaces subreddits and individual users worth knowing.

What it returned

• 392

  Scraping Copyblogger returned 392 posts carrying ten or more comments, 221 carrying fifty or more and 81 carrying a hundred or more, which Glen Allsopp treats as a better content map than BuzzSumo share data.

• 95

  Glen Allsopp says XPath taken this way has worked for him about 95 percent of the time, and a run finishes in minutes under 500 URLs or a few hours past 50,000.

• 500

  Screaming Frog is free up to 500 URLs, and Glen Allsopp fills the Configuration then Exclude box with dot star patterns and unchecks most Spider options to keep the export clean.

• 400,000

  Glen Allsopp uses an EC2 r3.4xlarge with 166GB for crawls of over 400,000 URLs, cutting days to hours at roughly $25 a day, and notes Screaming Frog defaults to only 512MB unless told otherwise.

Sources

• 13 Advanced Link Building Strategies You (Probably) Haven't Used - Detailed.comGlen Allsopp · detailed.com · A Unique Formula for Finding Popular, Linked-to Content · 2019-08-05