Canva has redesigned its session revocation infrastructure to support hundreds of millions of active sessions while avoiding networked database lookups for most authentication requests. The new architecture stores revocation data in Amazon S3 as compact, immutable records and distributes the data to application gateways as in-memory indexes. Canva said the approach improved deployment speed, reduced database infrastructure, and cut the memory footprint of its revocation cache by 87.5%.
Canva stores session information in encrypted browser cookies, allowing gateways to authenticate requests without contacting a networked datastore. However, revoked sessions and changes to permissions need to be reflected in near real time. Canva previously kept 12 hours of revocation data in memory, while session refreshes continued to check MySQL. As the platform grew, hundreds of gateway instances could each request more than a million revocations from MySQL during deployments, creating a coordinated database load.
Canva chose Amazon S3 over Redis to avoid operating another datastore while providing durable storage for revocation data. The 12-hour revocation window is divided into 30-minute S3 objects, which gateways download as needed. Each revocation is represented as a 16-byte binary record containing a principal and timestamp. Sorted arrays enable direct in-memory searches, reducing the cache footprint by 87.5%. Gateways use conditional GETs to download changed chunks and discard data older than 12 hours. Asynchronous workers scan for new revocations, merge them into the latest chunk, and upload the result. Conditional PUTs provide optimistic concurrency control when workers update the same object, while ZooKeeper leader election reduces conflicts but is not required for correctness.
Sharing the article on LinkedIn, software engineer Adam Urban highlighted the use of Amazon S3 as more than an object store, pointing to its use of immutable objects and conditional writes as coordination primitives.
How a worker task copies revocations from database into S3 (Source: Canva Blog Post)
The design also addresses recovery and deployment. A gateway can reconstruct its local revocation state by downloading the relevant S3 chunks rather than requiring a database to rebuild the cache. Canva said the worker can process more than 2,000 revocations per second, exceeding its expected requirements, while even a chunk containing one million revocations represents about 16 megabytes of binary data.
The architecture prompted discussion on Reddit, where one commenter suggested
Just use a refresh token scheme, keep access token lifetimes short and keep refresh tokens in a database instead, no need to check each session against a million cached revocations.
Canva engineer Llew Vallis responded to a Reddit discussion,
Keeping revocation data in memory provided better tradeoffs because frequent token refreshes would increase database load and make availability dependent on the database during refresh operations.
Following the migration, Canva reduced its session revocation database to two read replicas for redundancy and improved deployment speed. Database load also became more predictable, scaling with revocation write throughput and overall site traffic rather than the number of gateway instances loading the cache. Canva said testing multiple implementations on real infrastructure helped validate that the selected design could meet its scalability requirements, including workloads involving hundreds of thousands of revocations in a single array.