A quick engineering-related post: we built a cache plugin for Play that uses capped collections in Mongo. It's available on Github if you'd like to use it.
We have multiple dynos running, and we want each dyno to access the same cache instance. So a cache that lives in memory of the web process, or on disk of the machine, isn't going to work.
We initially tried using Memcached via Memcachier but we were getting intermittent timeouts that we could never resolve, even with help from the Memcachier folks.
Capped Collections in Mongo have characteristics that work pretty well for a cache: fast writes and FIFO eviction. Our plugin augments that with an optional expiration time for each cached item.
Instructions for using this yourself are in the readme file in the project.
This is not meant for insane throughput. We are not doing anything at "web scale" and I doubt this solution would hold up under massive load. This is a fairly basic solution that's probably good enough for many use cases — not something engineered to be industrial strength.
New to Domino? Consider a Guided Tour.Watch a Demo of Domino
Recent PostsSnowflake and RAPIDS For On-Demand Computing by a Storm Parallel Computing with Dask: A Step-by-Step Tutorial Lightning fast CPU-based image captioning pipelines with Deep Learning and Ray Everything You Need to Know about Feature Stores 5 MLOps Best Practices for Large Organizations Choosing a Data-Governance Framework for Your Organization Transformers - Self-Attention to the rescue How data science can fail faster to leap ahead N-shot and Zero-shot learning with Python A Hands-on Tutorial for Transfer Learning in Python
Other posts you might be interested in
Subscribe to the Data Science Blog
Receive data science tips and tutorials from leading Data Scientists right to your inbox.