DEV Community

Cover image for Check which companies run Shopify, WordPress, Webflow, Wix or Squarespace, in bulk
Joshua Smith
Joshua Smith

Posted on

Check which companies run Shopify, WordPress, Webflow, Wix or Squarespace, in bulk

You have company domains from a trade-show exhibitor list or a CRM export and want to know which run Shopify, WordPress or WooCommerce, Webflow, Wix or Squarespace. A Shopify app vendor sizing its market, a WordPress agency, a migration consultancy: all need the same table. This is research about companies and their public websites, not about people. Below are how platform detection works, where it goes wrong, and a script that writes a CSV with one platform per company and a confidence segment.

Why "runs Shopify" is harder than it looks

A platform is not one fact you can read off a page. A company can run its store on Shopify and its blog on WordPress, or hide Shopify behind a Next.js front end. A WordPress site can embed a Shopify Buy Button. A one-column "Shopify: yes/no" hides all of that. You want the platform, how sure you are, the evidence, and a flag on rows to check.

What each platform leaves behind

Detection matches the served page against open-source webappanalyzer fingerprints. The signals that work over plain HTTP:

  • Shopify: script URLs on cdn.shopify.com, a shopify-digital-wallet meta tag, _shopify_* cookies, an x-shopify-stage header.
  • WordPress: paths under /wp-content/ or /wp-includes/, a generator meta tag, a link: <...>; rel="https://api.w.org/" header. WooCommerce additionally needs its own scripts or meta tag, and the rules require WordPress to be present.
  • Webflow: a generator meta tag and a data-wf-site attribute on the <html> element.
  • Wix: X-Wix-* response headers, scripts from static.parastorage.com, a generator meta tag.
  • Squarespace: a Server: Squarespace header. Its other fingerprints are JavaScript globals, which an HTTP-only check never sees.
  • DNS records (NS, SOA, TXT, MX, CNAME) show who hosts the domain's DNS and email and which SaaS tools verified the domain, not what serves the page. In my test run, Wix's own site showed Cloudflare only through a cloudflare_dashboard_sso TXT record.

Why confidence matters

Each fingerprint pattern has a weight, 100 unless the rule says otherwise. A technology's confidence is the sum of the weights that matched, capped at 100. Most patterns weigh 100, so a single strong hit gives 100. Weaker patterns carry less: for Shopify, a <link> to shopify.com counts 50 and an x-shopid header 50. Implied technologies get at most the confidence of the one that implied them.

So 100 means one strong signal or several weak ones; 50 means one weak signal. The script calls 100 with evidence from the page itself confirmed, 50 to 99 or DNS-only evidence check by hand, and anything lower a weak signal.

The false positives and misses to expect

  • Subdomain splits. The Actor analyses only the page you give it, after redirects. If the store is shop.brand.example and the home page is WordPress, you get WordPress. Add the store or blog host as a second row when it matters.
  • Headless sites. A Next.js or Nuxt front end with a headless CMS can sit on top of Shopify or BigCommerce, leaving a 50-confidence trace or nothing. "No platform found" is not "no platform".
  • Embedded widgets. A Buy Button script from sdks.shopifycdn.com on a WordPress site scores Shopify at 100. The script flags rows with more than one platform.
  • CDNs in front. Behind Cloudflare or Akamai, the Server header names the CDN, not the origin. HTML signals still come through; a header-only detection like Squarespace's can be lost.
  • Redirects to another domain. In my test, magento.com redirected to business.adobe.com, so the result describes Adobe's page. The script flags these.
  • Vendor sites. shopify.com is detected as Shopify. Remove vendors and agencies before counting.

Run the detection

Tech Stack Detector is an open-source Apify Actor that checks each page against 7,600+ fingerprints. Paste the domains into Website URLs (the scheme is optional), switch on Include detection evidence, and click Start. Or send this input (placeholder domains):

{
    "urls": [
        "outdoor-gear.example",
        "www.coffee-roaster.example",
        "brand-a.example",
        "shop.brand-a.example",
        "furniture-studio.example"
    ],
    "detectViaDns": true,
    "includeEvidence": true
}
Enter fullscreen mode Exit fullscreen mode

From the API, with the JSON saved as input.json:

curl -X POST "https://api.apify.com/v2/acts/josh99smith~tech-stack-detector/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" -d @input.json -o results.json
Enter fullscreen mode Exit fullscreen mode

For thousands of domains, start the run in the Console and download the dataset as JSON.

Segment the list with a script

The script picks one platform per site (most specific first, so WooCommerce wins over WordPress), assigns a segment, adds notes for the cases above, writes segments.csv and prints a summary. Save it as segment.py; it needs Python 3.8 or later and no packages.

import csv
import json
import sys
from collections import Counter

# Most specific first: WooCommerce beats WordPress, Squarespace Commerce beats Squarespace.
PLATFORMS = {
    "Shopify": "Shopify",
    "WooCommerce": "WooCommerce",
    "BigCommerce": "BigCommerce",
    "Magento": "Magento",
    "Salesforce Commerce Cloud": "Salesforce Commerce Cloud",
    "Wix eCommerce": "Wix",
    "Squarespace Commerce": "Squarespace",
    "Webflow": "Webflow",
    "Wix": "Wix",
    "Squarespace": "Squarespace",
    "WordPress": "WordPress",
    "Drupal": "Drupal",
}
HEADLESS_HINTS = {"Next.js", "Nuxt.js", "Gatsby", "Hydrogen", "Vercel", "Netlify",
                  "Contentful", "Sanity", "Builder.io", "Storyblok"}
CONFIRMED, CHECK = 100, 50  # confidence thresholds


def site(url):
    host = (url or "").split("//")[-1].split("/")[0].lower()
    return ".".join(host.split(".")[-2:])  # crude: treats shop.co.uk as co.uk


def strong(tech):
    # Evidence other than DNS records or "implied-by". Without includeEvidence, trust confidence.
    ev = tech.get("evidence")
    if ev is None:
        return True
    return any(not e.startswith(("dns:", "implied-by:")) for e in ev)


rows = []
for r in json.load(open(sys.argv[1], encoding="utf-8")):
    row = {"input_url": r["url"], "final_url": r.get("finalUrl", ""), "segment": "failed",
           "platform": "", "confidence": "", "evidence": "", "also_seen": "", "notes": ""}
    rows.append(row)
    if not r["success"]:
        row["notes"] = r.get("errorType", "")
        continue
    techs = {t["name"]: t for t in r["technologies"]}
    found = [techs[n] for n in PLATFORMS if n in techs]
    hints = sorted(HEADLESS_HINTS & techs.keys())
    notes = []
    if site(r["url"]) != site(r["finalUrl"]):
        notes.append("redirected to another domain")
    if not found:
        row["segment"] = "no platform found"
        if hints:
            notes.append("custom or headless: " + ", ".join(hints))
    else:
        best = found[0]
        for t in found[1:]:  # a strong match on a later platform beats a weak one earlier
            if t["confidence"] > best["confidence"] and strong(t):
                best = t
        conf = best["confidence"]
        row.update(platform=PLATFORMS[best["name"]], confidence=conf,
                   evidence=" ".join(sorted({e.split(":")[0] for e in best.get("evidence", [])})))
        others = {PLATFORMS[t["name"]] for t in found} - {row["platform"]}
        if row["platform"] == "WooCommerce":
            others.discard("WordPress")  # WooCommerce always runs on WordPress
        row["also_seen"] = ", ".join(sorted(others))
        if conf >= CONFIRMED and strong(best):
            row["segment"] = "confirmed"
        elif conf >= CHECK:
            row["segment"] = "check by hand"
            if not strong(best):
                notes.append("DNS or implied evidence only")
        else:
            row["segment"] = "weak signal"
        if hints and row["segment"] != "confirmed":
            notes.append("possibly headless: " + ", ".join(hints))
        if others:
            notes.append("more than one platform")
    row["notes"] = "; ".join(notes)

with open("segments.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=list(rows[0]))
    w.writeheader()
    w.writerows(rows)

print(f"{len(rows)} sites -> segments.csv\n")
for (platform, segment), n in sorted(Counter((r["platform"] or "-", r["segment"]) for r in rows).items()):
    print(f"{platform:26} {segment:18} {n}")
print("\nReview by hand:")
for r in rows:
    if r["segment"] not in ("confirmed", "failed") or r["notes"]:
        print(f"  {r['input_url']:28} {r['segment']:18} {r['platform'] or '-':8} {r['confidence']!s:4} {r['notes']}")
Enter fullscreen mode Exit fullscreen mode

Run python segment.py results.json. Change CONFIRMED and CHECK to move the thresholds; add platforms to PLATFORMS by the names the Actor returns.

Read the result

On a test run of 20 well-known public sites, 14 were confirmed (5 Shopify, 3 WordPress, one each on six other platforms), 1 needed checking, 3 showed no platform and 2 failed. The review list is where the work is (names replaced):

  brand-b.example     check by hand      Shopify  50   possibly headless: Contentful, Next.js, Vercel
  brand-c.example     no platform found  -             custom or headless: Vercel
  brand-d.example     no platform found  -             redirected to another domain
Enter fullscreen mode Exit fullscreen mode

The first is the headless case: Next.js on Vercel with Contentful, and Shopify only through one <link>. The second has no platform signal at all. The third now redirects to someone else. For a market-size number, count confirmed and report check by hand as the upper range. Before outreach, a person should open each check by hand row.

Cost and limits

The Actor charges $0.008 per successfully analysed site. Failures (dns, timeout, blocked and so on) cost nothing, and a run stops at the maximum cost you set.

  • A list of 2,000 domains where 10% fail is 1,800 analysed sites: 1,800 x $0.008 = $14.40.
  • Adding a store or blog host for 300 of them: 300 x $0.008 = $2.40.
  • Apify's free plan includes $5 of usage a month, enough for 625 sites.

Limits to plan around:

  • HTTP only. No browser runs, so fingerprints based on JavaScript globals are not checked. Platforms usually show in HTML and headers; client-side widgets often do not.
  • One page per site. The page you supply, after redirects. Sub-pages are not crawled.
  • Bot protection. Some sites block cloud IPs and come back as free blocked records. Apify Proxy can help; proxy traffic is billed by Apify separately.

If this saves you time, a review on the Store page helps other people find it, and a star on GitHub helps too.

Disclosure: I built this Actor. Source: github.com/josh99smith/tech-stack-detector. Shopify, WordPress, WooCommerce, Webflow, Wix and Squarespace are trademarks of their respective owners; the Actor is not affiliated with any of them.

Top comments (0)