Conventional wisdom seems to be that changing a DynamoDB design is extremely difficult and you want to avoid it at all costs.
It is repeated without question that you must determine all your access patterns up front because you cannot easily change your DynamoDB design after the fact. For example:
"you cannot change primary indexes so the only option is to create a new (table), migrate the data with transformations and delete the old table. This process is excruciating, especially in production."
However, with single table designs, this thinking is out of date and today, is just plain wrong.
With single table DynamoDB designs, your keys and attributes are uncoupled from the physical implementation, so you can evolve your DynamoDB design much more easily.
This may be the most important benefit of single table designs.
While planning your query access patterns is hugely important, even essential, this does not mean that the only way to change a DynamoDB implementation is a "fork-lift" upgrade where you are forced create a new table.
Over the past year, we have extensively evolved our SenseDeep DynamoDB implementation without any downtime. We have not once had to re-create our DynamoDB table or modify our primary keys. We have added new entities, new access patterns, new indexes, changed inter-entity references and extended our item collections.
We did this by using single table designs that uncoupled our keys and attributes from the physical table keys and attributes. This is an essential tenent of single table designs. We used the DynamoDB OneTable Library which makes defining your entities and attributes and uncoupling your keys straight forward.
We then used sequenced, reversible DynamoDB migrations to evolve our DynamoDB design without downtime. These migrations are small, scriptable atomic database mutations that add new capabilities before removing or modifying existing attributes or entities. Using migrations expressed as code, you can confidently evolve your DynamoDB design knowing that you can easily and quickly reverse a change.
We use the OneTable CLI to sequence and manage (or rollback) database migrations interspersed with code deploys.
More?
You can read more about OneTable at: OneTable and OneTable CLI.
SenseDeep with OneTable
At SenseDeep, we've used OneTable and the OneTable CLI extensively with our SenseDeep serverless troubleshooter. All data is stored in a single DynamoDB table and we extensively use single table design patterns. We could not be more satisfied with DynamoDB implementation. Our storage and database access costs are insanely low and access/response times are excellent.
Please try our Serverless trouble shooter SenseDeep.
Contact
You can contact me (Michael O'Brien) on Twitter at: @SenseDeepCloud, or email and ready my Blog.
To learn more about SenseDeep and how to use our troubleshooter, please visit https://www.sensedeep.com/.
Top comments (0)