Twitter took the wraps off their new Manhattan NoSQL database this morning. The design goals are very similar to Cassandra’s, but tailored for Twitter’s workloads and needs. Unlike the oxymoronic WebScaleSQL, Manhattan incorporates important lessons from the last ten years; like Cassandra, Manhattan focuses on availability first, but allows opting in to consistency as necessary.
When you have a unique workload like Twitter’s, it can make sense to build custom systems to make sure your requirements are met as exactly as possible. Still, I would advise caution; it’s very tough to keep up as the state of the art moves forward when you go it alone. Project Voldemort is a good example: LinkedIn created it to meet internal needs at the beginning of the NoSQL era, but despite being open sourced it never grew a wider community and today is far behind Cassandra features like lightweight transactions and rapid read protection.
Since Twitter started work on Manhattan in 2012, Cassandra has added a new query language and protocol, request tracing, virtual nodes, and many internal enhancements, driven by the needs of hundreds of companies. Cassandra 2.1 is around the corner with even more innovation. It’s going to be tough for any single company to keep up.