Zerodha had put a 1 MB limit on each saved preference on its Kite trading platform because its previous database, SSDB, was getting slower with larger records. Custom workspaces and chart layouts had outgrown a single machine system.
In an engineering post on Nov. 19, Zerodha software engineer Sarat Chandra explained how the team migrated those settings to Blobasaur, an open-source storage system built on top of SQLite.
The team was given a starting point in terminal mode. Its workspaces were transferred to Blobasaur upon its launch. For the older charts, a different methodology had to be adopted, as traders were able to change the layout while Zerodha was duplicating the older settings.
Kite kept each of the changes in both databases for a month. Blobasaur failed to return preference, Kite could still get it from SSDB. SSDB was retired by Zerodha as there were no discrepancies found for that month.
Slowdown in new saves due to removal of old settings.
One of the bottlenecks was due to routine house-keeping. Blobasaur split its records into multiple SQLite files, but each file could only support one writer at a time. Fresh saves waited on cleanup jobs that deleted expired settings in one big batch. Some writes were between 30 and 50 msec.
Kite already ignored expired settings while reading, so deleting all of them at once didn’t make much sense. The engineers break the cleanup into smaller chunks, and have them alternate with the incoming saves. This got rid of the spikes, says Zerodha.
During a full trading session, roughly 99% of storage commands on each of its three machines finished in about 2 milliseconds or less. That number represents the time spent in Blobasaur handling a storage request.
