Rearchitecting CouchDB Secondary Indexes on Top of FoundationDB - Garren Smith, Red Comet Labs

Rearchitecting CouchDB Secondary Indexes on Top of FoundationDB - Garren Smith, Red Comet Labs

How do you rewrite a database that’s been around since 2005, so your users never notice a change in behavior or have to sacrifice features, but get all the improvements in performance and consistency that FoundationDB brings?

That’s the challenge we’ve taken on in rewriting Apache CouchDB to run on top of FDB. One of the biggest architectural redesigns is CouchDB’s secondary indexes: Map/Reduce, Mango, and Text Search.

This talk covers the architectural solutions we’ve found for challenges such as: finding the most suitable data model for each index; managing global index state and avoiding conflicts when building indexes; ensuring that the results returned are sorted correctly; and using Apache Lucene on top of FDB for Text Search.

I’ll also share the results of the benchmarking we’ve done, along with further plans we have to improve the architecture of each index.