Michael Stonebraker | Big Data is (at least) Four Different Problems

Yes, that slide deck in isolation looks an ad so it's probably better to get some background on Michael Stonebraker's "big picture" beliefs and core premises: ++ He believes SQL is the "right" and most proven access model. (Note that SQL does not mean RDBMS like Oracle/MSSQL.) ++ He believes in ACID ++ He believes that other db engines' (NoSQL, etc) approaches to gain speed and scaling of clusters by sacrificing SQL and ACID is wrong and his proof is that those other products end up eventually bolting on SQL and ACID anyway. I think a better presentation to flesh out these ideas is the following youtube video: [this video] It's 55 minutes long so to save time, HN readers may prefer to browse it by pressing "L" key to step through it quickly. Just read the slides shown behind him and if the bullet points look interesting, rewind and listen to his presentation. One can dwell on those SQL/ACID merits in isolation and consider separately, the question of whether VoltDB-as-a-product fulfills those ideals.

Firstly, disclosure: I work on XTDB, where we are in the process of building out a new columnar SQL engine that also (still) supports edn-flavoured Datalog, see [link] Perhaps the most fundamental aspect here is the inherent tension between (A) typical columnar relational schemas which primarily target efficient OLAP across very wide & sparse tables, and (B) the nature of 'graph' systems (whether OLAP or OLTP) which tend towards handling a huge number of tables and/or recursive self-joins + incremental view maintenance. I believe Michael Stonebraker's conjecture continues to be that a "one size fits all" database engine is essentially unachievable, and therefore you can't expect a single database system to be competitive across both domains simultaneously. However some databases (SQL Server springs to mind) are apparently pretty good at running multiple engines in parallel. With the v2 XTDB engine design we are sacrificing some of our graph capabilities in the short term (WCOJ and self-join optimisations) for what we feel are more pressing advantages of the columnar model (low-cost retention of history and vectorized processing). We are also aiming to retain reasonable OLTP performance as well, in a similar vein to SingleStore etc., since the target audience is still application developers rather than data analysts. If you are looking for a way of using Datalog on existing columnar storage engines today, you should definitely look at [link] Also, this Stonebraker talk on "Big