Netvibes is the Dashboard of Everything–an online dashboard that allows you to track everything that matters to you or your brand.
For individual users, Netvibes is a personalized home page and RSS reader. You can drag and drop a variety of widgets (there are over 250,000 in the Netvibes catalog) to create your ultimate dashboard. You can have your email, weather, to do list, your favorite blogs and websites, and all your social media feeds. Literally anything that matters to you online can be in your all-in-one dashboard, making it easy to see at a glance everything that is happening.
The Enterprise version of Netvibes takes that same concept but adds on additional functionality for teams and brands. Netvibes Premium delivers data analytics and visualization, so you can create customized graphs and drag-to-compare anything. With Netvibes’ unique MisoData feature, you can also import any data, including data from the web or from external files within your organization. Analyze how your sales numbers compare with your Twitter mentions, or see how your website traffic correlates with your media coverage. You can calculate ROI on anything.
Netvibes is very popular with PR, marketing and digital agencies who use it for their clients. They can monitor all of their clients’ mentions everywhere across social media and traditional news channels and blogs. They also use Netvibes to monitor key trends and do automated reporting, so clients can see exactly how they’re being perceived online in real time.
The NoSQL technology landscape in 2008 was quite limited. Even Cassandra was at the beginning, and it was quite controversial at the time. We initially decided to use Tokyo Cabinet; it’s an old technology, a key-value store with no scaling feature, so we had to do our own. In contrast, Cassandra is able to do that natively, so it was a huge boost for us.
The best candidates during our re-evaluation were Apache Cassandra, MongoDB, and Riak. They are a bit different; Riak was close to Cassandra in terms of philosophy, but they were both good for our use case.
We conducted a number of performance benchmarks primarily looking at the speed of read and write queries. Scalability and performance were the main things for us, as the free part of the product is entirely the aggregation job that we are doing. The aggregation requires a database that can scale with very low latency, and that’s Cassandra.
Next to the benchmarks, we tried to look at the costs. We are a point where we have both back ends, Tokyo Cabinet and Cassandra, because the migration is not finished, and we are able to compare the costs. Today, with Cassandra our costs are divided by 4. It means the cost per article is reduced between 70 to 80%. It’s a big win for us. So, we decided to change and Cassandra was the clear winner.
Cassandra won the benchmarks, and we will go live with it this year. Cassandra is used to store every article in the dashboards. An article can be anything from a RSS article, a tweet, a Facebook post, or whatever. All this kind of information is stored in Cassandra.
The goal of the migration to Cassandra was to be able to scale to give the opportunity for our customers to store and analyze as much data as they need. If you create an account on Netvibes.com, and you have some tweets on your page, they will be stored in Cassandra. Our free users have at least 2 rolling weeks of storage, and our Premium users have 3 months. Premium users can choose to extend this period of storage at will. Our most popular option is the 9 month extension making Netvibes’ Premium Dashboard store absolutely everything, without any limitation of size, for a full rolling year. We call this feature AutoSave.
That was the main goal of our Cassandra migration, to be able to deliver the AutoSave feature to our customers. PR agencies, marketing departments, and the like are dealing with a lot of information. If you add social networks, the amount of articles is pretty huge. You need to store more, and sometimes you need to analyze all the stats. You need to compare data from this month to the previous months, or maybe compare to 6 months ago. It means you need to store a lot of data, and we can achieve that with Cassandra.
The system we have built now, it’s about 20+ machines running on version 2.0.9. With a bit less than 1 tera of hard drive space per node. The first goal of this system is to be able to manage 1 billion articles. We start with 20+ machines, which will fit and we’ll have the option to add extra machines when needed.
Our data model is simple, it’s only 6 tables and every query is done through CQL. We had the advantage of already knowing the data well when we started with Cassandra, how to manage it; we already knew our Java.
We quite fell in love with 2 things with Cassandra. The first is the ease of scale – how it’s easy to scale with Cassandra, how it’s easy to add nodes. The other thing is CQL. In the tech team, we all knew Cassandra before the CQL era and we knew that it was, lets say, more difficult to enter at that time, because of the parting and its difference for new-comers. Thanks to CQL, we are now very confident, even to introduce some new developer to Cassandra. It was a game changer. When we learned about CQL, it was the key to simplify everything.
In addition to both things, the ease of scalability, and the CQL, the performance is pretty damn good. So, at the end, these 3 key things, was why we choose Cassandra.
In Paris our team has attended a number of meetups, or talks. Also, we appreciate Planet Cassandra’s YouTube channel. There are many videos about the Cassandra conferences, so it was sometimes very, very helpful, because some guys are very full of technical feedbacks. What to do, what things to avoid. So, it was very, very interesting.
I would say, forget about the past; the pre-CQL era. It is the roots of Cassandra, but now CQL is very mature and it’s a game changer. Every web developer knows about MySQL, for example, and the gap between MySQL and CQL is very close. Even young developers should not be afraid of Cassandra. The learning curve is very easy, and it’s not the technology that people should be afraid of. Sometimes, especially in big company, when you are choosing a technology that nobody has heard of, it can be complicated. Trust this technology, it is amazing.