I run a small browser tool for making Data Matrix codes. Server-side analytics counted 166 human page views on it in one day, plus 52 from bots. Search Console says the whole site got 205 impressions and 1 click from Google in the last four weeks.
Both can't be right about people. I didn't know which one to believe until this week.
How it was measured
The server logged every page view and labelled it human or bot from the User-Agent: anything containing bot, crawl, curl, python, headless and so on is a bot. There are no cookies and no third-party scripts, by choice.
The front end only sent events once someone reached the paywall: hook shown, limit hit, upgrade opened. When I audited it at the end of October, those events over 30 days came to 0 from humans and 9 from bots. So I could not answer the most basic question: did anyone who arrived actually make a code?
What I added
Two things, both still without cookies or storage:
- Tool events: first input, code ready, code downloaded, batch ready, label PDF.
-
A page-leave beacon. Each page load gets a random id that lives only in memory. While the page is visible, it adds up seconds and records the deepest scroll. Every time the page goes hidden (tab switch, close, navigation) it sends one
page_leavewithnavigator.sendBeacon.
To connect a person's pages within a day, the server hashes salt + UTC date + IP + User-Agent and keeps the first 6 bytes. It changes every day and the IP isn't stored.
The useful side effect is that only a browser that actually ran the page's JavaScript can send page_leave. A scraper with a Chrome User-Agent can't.
What the first hours showed
In the first 3 days after the release:
- 391 page views from 231 visitor ids were labelled
humanby the User-Agent check. - Only 12 of those 231 ever sent a
page_leave. - A sister site on the same code, released at the same moment, had 64 of 324 over the same 3 days, so the beacon works.
The "human" traffic had a pattern once I looked. In about an hour, 11 visits arrived with a Google referrer, roughly one every 5–10 minutes, each from a new visitor id and each on a different page, including the terms, privacy and refund pages. Real searchers don't land on a refund policy from Google one after another.
The real visitors seem to be the handful who type into the box. The 166 a day was mostly crawlers dressed as Chrome.
What I'm still unsure about
-
Is "sent a page-leave beacon" a fair definition of a person? Some browsers kill pages without firing
visibilitychange, and some privacy tools block beacons. I'd undercount, but I don't know by how much. - Hiding the bots in reports vs. blocking them. For now they're labelled, not dropped.
- Is the daily hash too coarse? People behind the same office IP with the same browser look like one person.
If you want something to test a scanner with while you're here, the site has a set of Data Matrix test codes.
What I'd like to know
- On a small site without cookies, how do you separate real people from bots that send a normal browser User-Agent?
- Do you trust "ran JavaScript" as the line, or use something stricter, like an interaction?
- What do you track on a free tool before anyone gets near a paywall?
I'll post a full week's numbers once I have them.
Top comments (1)
Yeah, those look like two different things being called “human visitors.”
A page view with a browser-looking user agent isn’t the same as a browser that actually ran the page and sent a leave beacon. I’d keep the 166 and the 12 separate, with a clear definition for each. Otherwise, a crawler dressed as Chrome quietly becomes a person in the report.