On the evening of 11 November 2026, Apify told account holders that it had rotated all API tokens and Proxy passwords, ended sessions and revoked GitHub sign-in. Old tokens stopped working at once.
We run 33 Actors on one Apify account. This is a plain account of what that meant for us, what we did, and what we would set up in advance next time. It covers only our own account. We are not adding any explanation of why the rotation happened beyond what Apify told us.
What we had
The account held about 30 personal tokens, 33 Actors, 172 saved Tasks, 58 schedules and one webhook. That is not unusual for a small data business, but it is enough that "just replace the token" stops being a one-minute job.
What broke
In roughly the order we noticed it:
- The command line logged out. That meant no builds and no runs from our machines until we signed in again.
- Actors that read private stores returned 401. Several of our Actors read from private key-value stores using a token held in a secret environment variable. Once the old token was dead, those reads failed.
- Customer-facing Actors failed closed. They stopped with a plain message, for example "change log unavailable, nothing was read or charged". Nobody was billed for a run that could not read its data.
- A local publish step was held. It depends on a stored token, so it stopped rather than guessing.
- Scheduled canaries went red. We run small scheduled checks against our public Actors. They told us within hours which ones were affected.
- A daily summary Actor failed with 401. It is triggered by a webhook and uses a token to read run information.
How we recovered
The first surprise was the Console's copy button on an old token. It copies a value, but the value is invalid. A rotated token cannot be recovered that way, so every token we still needed had to be rotated or created again.
The second was environment variables. A secret variable only reaches a run through a build made after you set it. Setting the new value in the Console is not enough; the running build still carries the old one. We rebuilt eleven Actors.
Then we tested. For each new token we made one tiny owner run, capped at $0.05. The total cost of all of them was well under a cent. We also scanned every Task, schedule and the webhook for old tokens. None held one.
One limit we could not get round: the Console's token list shows no created or last-used dates, and there is no audit log. From the Console alone you cannot prove that nothing else used a given token. We can say what we checked, but not what we could not see.
What we learned
Copy cannot recover a rotated token
If you are counting on being able to copy a token out of the Console later, you cannot. Treat a token as something you can only create again. Know which of your jobs would stop if every token vanished tonight, and know how to make each replacement.
Rebuild after every variable change
A secret variable reaches a run only through a build made after it was set. After any change to an Actor's variables, rebuild and then check with a run. We now treat "set the variable" and "rebuild the Actor" as one step with two parts, not two separate tasks.
One scoped token per job
Give each job its own token, named for the job and the Actor that uses it, and keep a written map of where each one lives. When a rotation arrives, finding what to replace becomes a lookup rather than a hunt. A shared token that feeds six things is the one that costs you an afternoon. Keep the map somewhere that does not depend on any of the tokens working.
Fail closed, with a message people can read
When an Actor cannot read its data, it should stop, say so in words a customer understands, and charge nothing. Ours did, and that kept a credential problem on our side from turning into a billing problem on theirs. A bare stack trace or an empty dataset that looks like success would have been worse.
Canaries find the affected Actors quickly
A small scheduled run against each public Actor, with an alert on failure, told us within hours which Actors were hit. Without that, we would have learned from customers. If a public Actor has no canary, it is the first thing we would add.
Write the list before you need it
Make a list of every place a token is used, before you need it:
- Actor environment variables
- local stored credentials
- routines and scheduled jobs
- shell variables
- integrations
- webhooks
The ones that are easy to forget are the quiet ones: a webhook-triggered job that runs once a day, or a local step that only runs when you publish. Ours was the daily summary Actor.
Test each new token with the smallest possible run
Once a token is in place, prove it with one tiny run and put a spending cap on it. We capped ours at $0.05. A new token that has never been used is a guess, and a capped run is a cheap way to find out.
Keep tokens out of chat and logs
We use small scripts that read the token from hidden input, store it, and print only whether it worked. The token never appears in a chat window, a terminal history or a log. That also means we never have to ask anyone, or any tool, to read one back.
A checklist to write down now
- [ ] A list of every token and where it is used (Actor variables, local credentials, routines, shell variables, integrations, webhooks)
- [ ] One scoped token per job, named for the job and the Actor
- [ ] A rebuild step after every variable change
- [ ] A canary and failure alert on every public Actor
- [ ] A customer-readable message when data cannot be read, with no charge
- [ ] One capped test run for each new token
- [ ] Hidden-input scripts so tokens never reach chat or logs
About us
We publish free weekly UK and Australian shelf-price indices under CC BY 4.0 at data.yappman.com, and our Actors are on the Apify Store at apify.com/yappman.
Top comments (0)