Rohit Swami · 10 long reads
Writing
Long reads on the kind of work I do, from flaky tests and fast pipelines to retrieval, and on the problems inside the products I have built. Each has figures you can push around.
Engineering notes
- 01A test that always fails isn't flakyFailure rate ranks the wrong tests. What flakestat measures instead, why its verdicts wait for evidence, and the times it fooled itself before it could fool anyone else.
- 02Ten times faster, mostly by doing lessI rebuilt a graph engine from R to Python and Polars, and it got ten times faster on 80% less memory. Very little of that came from changing languages.
- 03Push, don't pollGetting a notification from a background job to a browser in under 100 milliseconds is easy once. The hard parts arrive when it has to happen for everyone at once.
- 04Spot instances without the 3am pageSpare cloud capacity is cheap until it's taken back. Interruptions, checkpoints, retries and the spot/on-demand mix behind a batch pipeline bill that came down by 28%.
- 05Drawing a graph that's too big to drawSpatial indexes, culling, level of detail and a single event listener: the unglamorous techniques that make a huge biological network feel instant.
- 06Retrieval is the hard partWhen a RAG system gives a bad answer, the model usually takes the blame. More often it was handed the wrong context. Embeddings, chunking and approximate search, seen from the retrieval side.
Building my own products
- 07Counting days the way the government doesAustralia, Canada, the UK, the US and the Schengen area all limit how long you can be away, and no two count a day the same way. How CitizenDays reads the rules like code, and why it is a day stricter than one official calculator on purpose.
- 08A ledger its own server can't readClearOwe keeps track of money between people. It works offline, follows you to a new phone and shares a balance by link, while the server in the middle learns almost nothing. The three problems that took more thought than the cryptography.
- 09Marked like the real thingThree exam-prep apps, for the FAA drone licence, the US military's enlistment test and the Australian citizenship test, run on one engine. What it takes for practice to predict the real exam honestly, and why the readiness score is hard to flatter.
- 10A WhatsApp export is not a file formatYAPD turns an exported chat into a Wrapped-style recap, and the chat never leaves the browser. Most of the engineering went into reading a format nobody designed, and into letting a language model quote people without ever misquoting them.
Earlier, on Medium
- Customer churn prediction of a music app using PySpark ↗
- 4 crunching facts of the Stack Overflow Developer Survey '17 ↗
- All posts on Medium, @rowhitswami ↗
And four peer-reviewed papers, listed on the front page.