A quick engineering-related post: we built a cache plugin for Play that uses capped collections in Mongo. It's available on Github if you'd like to use it.
Motivation
Domino's web frontend is built in Scala, on Play — and, for now, somewhat begrudgingly, hosted on Heroku.
We have multiple dynos running, and we want each dyno to access the same cache instance. So a cache that lives in memory of the web process, or on disk of the machine, isn't going to work.
We initially tried using Memcached via Memcachier but we were getting intermittent timeouts that we could never resolve, even with help from the Memcachier folks.
Solution
Capped Collections in Mongo have characteristics that work pretty well for a cache: fast writes and FIFO eviction. Our plugin augments that with an optional expiration time for each cached item.
Instructions for using this yourself are in the readme file in the project.
Caveats
This is not meant for insane throughput. We are not doing anything at "web scale" and I doubt this solution would hold up under massive load. This is a fairly basic solution that's probably good enough for many use cases — not something engineered to be industrial strength.
Nick Elprin is the CEO and co-founder of Domino Data Lab, provider of the open data science platform that powers model-driven enterprises such as Allstate, Bristol Myers Squibb, Dell and Lockheed Martin. Before starting Domino, Nick built tools for quantitative researchers at Bridgewater, one of the world's largest hedge funds. He has over a decade of experience working with data scientists at advanced enterprises. He holds a BA and MS in computer science from Harvard.
Summary
RELATED TAGS
SHARE
Other posts you might be interested in
Subscribe to the Domino Newsletter
Receive data science tips and tutorials from leading Data Science leaders, right to your inbox.
By submitting this form you agree to receive communications from Domino related to products and services in accordance with Domino's privacy policy and may opt-out at anytime.