Networks are unreliable but our systems cannot be.
What is Idempotency?
Idempotency is a property of API design that ensures that makin...
For further actions, you may consider blocking this person and/or reporting abuse
I have been doing this for twenty years now. Astonishingly, I have won not a single convert in that time.
If there is a concern for collisions, you can easily generate the UUIDs on the server or get them directly from the database. I used to generate fifty at a time with a query to PostgreSQL, then load them in the
<head>of the page as a queue. When a form was submitted, it would shift a UUID off the queue and then do a PUT instead of a post passing the UUID (I generally converted them to Base58).On the back end, the PUT would either replace the record entirely if it existed, or create a new record if it didn't, returning a 200 if updated or a 201 if created, along with the full record.
I used PATCH to update records (rather than replace), returning a 200 and the updated record.
DELETE returned a 204 and GET a 200 with the record(s).
This way GET remained nullipotent and PUT, PATCH, and DELETE were idempotent. No duplicate records, no back button nonsense. Easy peasy.
If the queue ever got down to say, ten or twenty, I'd do a query to bump it back up again. Again, this was probably overly cautious, but I like rock solid reliability.
You can also just generate the UUIDs on the client as needed. You can use version 1 if you're OK with the MAC address being encoded. Twenty years ago the quality of the JS UUID libraries was questionable. These days you can probably get away with
crypto.randomUUID()on most browsers.With PostgreSQL for example, you can simply do
INSERT ... ON CONFLICT (id) DO UPDATE ....I have never understood why there was so much push back against this method.
It's tough to convince others to adopt new methods despite their effectiveness. 😅
Thank you for sharing your experience. Really helpful.
Thanks @chasm for sharing your approach!
I want to understand how do you ensure that these UUIDs are not misused in the system? Is it by authorization? Do UUIDs have expiry then?
Thanks, Mudit. I don't really understand your question. UUIDs are unguessable. They only represent what you choose to make them represent. It's not like they are crypto – you can't "decode" them. They are just big numbers.
So your question sounds to me like "how dow you ensure that 7 and 9 are not misused in the system?" and "does 33 have an expiry?"
Those questions make no sense to me, but maybe I am misunderstanding you or we are talking about two different things. Sorry if I'm missing something.
What I was trying to ask is if there are security concerns of pre-generating UUIDs and in that case, should they have an expiry?
This pattern works well if we ignore network errors that leads to clients to retry. When there is a network problem to receive the uuid on the client, but the server have already commited, the idenpotency is violated. One alternative is expected from the client an UUID on the request, so it can be reused on retries. Other alternative is hash the request payload to use as identifier of what have already commited.
Had heard about idempotency but never looked into it much.
I love all your articles!
Thank youu :D
"Networks are unreliable but our systems cannot be." is powerful haha!
Learnt a lot as usual
Glad it was helpful 🙌
You're the best!)
Thanks a lot 🙌
This was a good read. 🔥
Thank you!
Well, by design (the minimum bare of good design), most Rest API are idempotent. Using POST as an example, if for the same dataset you are creating the 'same' resource twice, is because of your design being wrong and not because you need to enforce idempotency.
There are situations where enforcing idempotency is required even in well-designed APIs. For example - in scenarios where network issues or other errors can cause a request to be retried.
I believe enforcing idempotency is not about compensating for design flaws but rather about adding an extra layer of safety and consistency to API interactions - even if a request is duplicated/retried, the overall system remains in a predictable/consistent state.
Hello, I fully agree on this one. I see this at the company I work for, that without enforcing the idempotency at API level, we are now building other checks elsewhere.
I agree with @karishmashukla totally.
We use idempotency a lot at work (I work in payments).
Idempotence keys are not important but rather crucial for certain use cases like building robust payment systems. In the payment system, there is a risk of double payments due to retries, which can be avoided by idempotence keys.
REST api by design should be stateless whereas in your examples you are making your server stateful.
Imho, this is over-engineering and is not really idempotent.
The example of creation of a new user is not suitable.
If post request containing the same user info is sent two times for user creation, the first request should create the user and send a status of 201 (assuming the user does not exist yet) and the second should return an error indicating why the request has failed, user already exist....therefore making it not idempotent.
Your implementation should not be using "idempotency keys" but rather concentrate on the type of resource being created. If the resource needs to be unique like email/username in the case of user creation, your function should cater for it and not rely on request keys.
What if the same request is sent with another request key...in your implementation, you would have end up with duplicate resource whereas focus should be on verifying where this email/username is available....or in the real world your servers should be able to scale horizontally and the second request would probably be executed by a different server
I would recommend you to read the article once again. The example of creation of a new user IS NOT explaining the concept of idempotence keys, rather it is to very simply explain the statement "Idempotency of an API is determined by the changes it makes to the system, not the response it provides." As you clearly stated if the user already exists we get an error response but the state of the system remains the same. This makes it simple for beginners to get the core idea :)
There is NO WHERE the author is asking you to use idempotence keys for user creation. Instead the author clearly mentions Real World Use Cases where idempotency will be of help
The author also clearly mentions "The code examples are just to illustrate fundamental concept of idempotent keys."
At my work place we do use idempotency and enforcing idempotency in production would require a lot more code and hence the code examples are just "examples" (already clarified by the author)
Thanks for pointing it out to me. I confused idempotency in rest api (for GET method) as opposed to the usage of idempotency keys to circumvent arising issues in case of retries.
No problem. Glad I could help :D
I wonder how you can avoid that the client side requests a new UUID, and so making all of this still not Idempotent.
Can you explain about this.
Assume a client goes back 2 pages, change some of the data and submits again.
Would the latest info still be stored as new or as update, or is it then just skipped?
Well-documented article!
I had a doubt you mentioned that POST is not idempotent right. The example you gave about creating a new user if it doesn't exist what http method will you use for this. Will it be PUT?
As a verb POST is not idempotent. As part of your bussiness logic, however, you will ensure (or want to) that said request perfom as such. For the 'new user' example, email/phone/username are candidate keys. Meaning, you won't have two user with those same values