Adopting the Ceph storage platform at Ocado Technology
For many working with data, it has become clear that there’s only one way for it to go: up!
At Ocado, we produce huge amounts of data, from our customer-friendly websites to our highly-automated warehouses and our IoT-ready vans.
My team is responsible for building the high-performance data systems that power Ocado; our architecture needs to be scalable, maintainable and free of downtime. We also believe in adopting a DevOps approach to work, where self-provisioning becomes very important.
When we started looking for a solution to manage our fast-growing storage platform, we realised Ceph provided the optimal answer to our quest.
Here is a description of Ceph from their website:
Ceph is storage platform designed to present object, block, and file storage from a single distributed computer cluster. Ceph’s main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available. The data is replicated, making it fault tolerant. The system is designed to be both self-healing and self-managing and strives to reduce both administrator and budget overhead. All this while running on commodity hardware.
Having a solution that accomplishes your needs is always good, but when that solution is also open source, it makes everything even better. Using Ceph, we’ve found that whenever there were some use cases that didn’t accomplish our needs, we could always add them ourselves, or alternatively ask the community for support – guiding the feature set in the direction we thought the product should go.
It also meant that whenever we found an issue, we could easily work with other Ceph developers and guide the development towards our goals and needs.
The Ocado Technology team also contributed some commits back to the community and we’re also contributing bug reports to help make Ceph a better product.
For example, we have contributed a new feature allowing users to self provision themselves, when Ceph is used in a OpenStack environment with domain based authentication – i.e. Keystone v3. Thanks to this feature, users can become completely independent as long as they have access to the system and there are enough free resources for them to use.
We’ve chosen to add this feature not just for Ocado, but because we saw that other community members had been actively requesting it for some time; this is the base ethos of open source software.
Also as early adopters of Ceph, we have gained the useful expertise that has allowed us to find issues, report the appropriate bugs, and be an active member of the community, contributing to mailing lists and helping other users.
Luis Periquito, Unix Team Lead