Shouting in the Datacenter

+1, in general. To adda bit of content: 1) Power Hungry, the total power envelope of a rack position is the limiting issue. You can fit 16-20 disks per RU no problem. The constraint on total density is your 5kVa or 10kVa power budget. Watts matter. 3) sensitive to vibration, be very very quiet [this video] 4) Density, actually 4TB is hitting the sweet spot for $ per byte last time I looked. If you need absolute density look for 5 & 6TB Real Soon Now. If you can tolerate some loss variable density disks around 5TB look quite a bit more cost effective. 5) Read ahead, you only need to read ahead by a couple of chunks. For video ball park it at 2MB. You don't really need lots of RAM, think 32-64GB per chassis. Additionally each 8GB dimm costs about the same power as a disk. By going with 32GB instead of 64 I can fit another 2-3 disks in the power budget per chassis.

> For example, imagine an evil SSD which had a 1% chance of rolling a sector back to a previous version, a 1% chance of saying a write failed when it didn't, a 1% chance of writing data to the wrong sector number, a 1% chance of flipping some bits in the written data, and a 1% chance of disconnecting and reconnecting a few seconds later. There are stories from the ZFS folks of dealing with these issues and things ran just fine. While not directly involved with ZFS development (IIRC), Bryan Cantrill was very 'ZFS-adjacent' since he used Solaris-based systems for a lot of his career, and he has several rants about firmware that you can find online. A video that went viral many years ago, with Cantrill and Brendan Gregg, is "Shouting in the Datacenter": * [this video] * [link] (making of)