2 HN users recommended this talk3 months2020–2026score 3.5
Years with an HN recommendation, 2007–2026
HN recommends it as the definitive talk from the implementer on DuckDB's motivation and design.
Mark Raasveldt of CWI presents DuckDB, an embeddable analytical database described as the SQLite for analytics, in a CMU Database Group talk. He explains why it was built, what it is for, and how it works.
“one of the implementers giving a talk to CMU students about what DuckDB is, why they made it, and how it works” (HN, 2020)
If you are a data scientist or do anything with data... duckdb is like a swiss army knife. So many great ways it can help your workflow. The original video from CMU in 2020 [1] is a classic. Minutes 3-8 present a good argument for adding duckdb to your data cleaning/processing workflow. And if you want to add a semantic layer on top of data, Malloy [2] is my favorite so far (it has duckdb built in): [1]: [this video] [2]: [link]
Classic talk making the case for DuckDB in data cleaning.
I was wondering the same--I guess because it's fairly new (2018) and it came out of a database research group at a European university, rather than a SV tech firm. Therefore, limited marketing budget. By the way, here's a YT video of a talk given by one of the DuckDB implementers about why they made it, what it's for, and how it works: [this video]
Implementer explains why DuckDB was made, what it is for, how it works.
Here's a YT video of one of the implementers giving a talk to CMU students about what DuckDB is, why they made it, and how it works: [this video] In a nutshell, it's a file-per-database embedded DB like SQLite, and it's installable as a Python or R package from PyPI or CRAN, but unlike SQLite, it's a column store with vectorized operations so it's more performant for analytical queries.
Implementer's talk on what DuckDB is, why built, how it works.
Enjoyed his roundup in the "Shoving Ducks into Everything" section. DuckDB is a great tool. In April 2020, the creator of DuckDB gave a talk at CMU. In the beginning he makes a convincing argument (in 5 minutes) why data scientists don't use RDBMS and how this was the genesis of DuckDB. Here is a video that starts 3 minutes into the talk (where is argument starts): [this video]
Argues why data scientists avoid RDBMS, DuckDB's origin.
DuckDB looks very interesting and I'm quite excited to examine it more closely in the next few days! I just wanted to add to the discussion that an unchanging file format, or at least a backwards compatible one, is a key feature of sqlite. See for example Richard Hipp's comments here [1] (I think he also mentioned earlier in the talk that the file format has become a limiting factor now in terms of some of the refactoring that they can do). The file format therefore seems likely to be a major factor in the long term success of this project and I am glad to see that you are taking your time before settling on any architecture here. Given that you are targeting the data science and analytics space, what are your plans for integration with arrow and the feather file format? From a purely user/developer perspective, arrow's aim of shared memory data structures across different analytics tools, seems like a great goal. I know Wes McKinney and Ursa Labs have also spent quite some time at the file storage part of this, see for example the Feather V2 announcement [2]. What are your thoughts on the tradeoffs they considered and how do you see the requirements of DuckDB in relation to theirs? From the Carnegie Mellon DuckDB talk [3], I saw that you already have a zero-copy reader to the pandas memory data structures, so the vision I have is that DuckDB could be the universal SQL interface to arrow datasets which can then also be shared with more complex ML models. Is that something tha
If you're using sqlite, but for a static website, you might consider using duckdb. It's still evolving, but it has huge potential. It's also an embedded single file database like sqlite, but stores data in columns. Duckdb to sqlite is like redshift to postgres. Hopefully they will add compression for string columns, so that host your entire static site from a single file in a very efficient way. [link] [this video]
I agree with this 100%. The creator of duckdb argues that people using pandas are missing out of the 50 years of progress in database research, in the first 5 minutes of his talk here [1]. I've been using Malloy [2], which compiles to SQL (like Typescript compiles to Javascript), so instead of editing a 1000 line SQL script, it's only 18 lines of Malloy. I'd love to see a blog post comparing a pandas approach to cleaning to an SQL/Malloy approach. [1] [this video] [2] [link]
DuckDB creator argues pandas users miss decades of database research.
Video list and Hacker News comment labels from HackerTube (prompted by @karpathy), released under ODC-By 1.0, built on the open-index/hacker-news archive. Hacker News data is provided by Y Combinator; comments remain the work of their authors. Video titles, channel names and thumbnails belong to their creators and are shown via YouTube. HackerTube Watch is an unofficial edition. Data export of 2026-10-31.
Plays via YouTube's privacy-enhanced embed
7 Hacker News comments
From the HN archive; up to 20 kept per video, favoring different commenters. Recommendations first.
Citations, criticism and jokes
Read more