External SSD/NVMe recommendations?

[reichman]

With a Mac Studio on the way, I need to up my storage game. I have several mediocre SSDs, and lots of 3.5" spinners in my storage tower. Does anyone have a recommendation for a 6TB or 8TB SSD that delivers those epic NVMe speeds north of 1000MB/s? Thanks.

[EGS]

This enclosure claims "Transfer Rate: Up to 64Gbps (Depending on SSD Speed)":
https://www.newegg.com/icy-dock-mb72...9SIB36ZHUM0647

[Darryl Ramm]

Originally Posted by EGS

This enclosure claims "Transfer Rate: Up to 64Gbps (Depending on SSD Speed)":
https://www.newegg.com/icy-dock-mb72...9SIB36ZHUM0647

Uh no. That's an OCUlink enclosure... a direct PCIe wired connection as you might find on *some* PCs/servers. It's not Thunderbolt 3/4 as required on these Mac Studios for external storage.

And the physical packaging is for PC type enclosure and they want to plug these big fugly rack modules into those cases.

The 64 Gbps spec is pretty meaningless, it's the raw link speed, all it is saying is it supports 1 x PCIe 4 x 4 lane... kinda irrelevant for Thunderbolt 3/4 with PCIe 3 only speeds.

I'll post more soon.

[EGS]

Originally Posted by Darryl Ramm

...It's not Thunderbolt 3/4...

Oops. How about this one? https://eshop.macsales.com/item/OWC/...RoCW-QQAvD_BwE

[Darryl Ramm]

Originally Posted by EGS

Oops. How about this one? https://eshop.macsales.com/item/OWC/...RoCW-QQAvD_BwE

Getting warmer...

But ugh these !@#$ OWC boxes suck. Cheap and nasty design, they lack a proper PCIe switch internally and split one PCIe lanes internally to each of 4 daughter cards, so you get a fraction of the performance possible in many cases. And say starting off running one internal M.2 drive in this case really sucks. Cheap nasty. Just one of the reasons I think there are many better vendors for SSD storage than OWC... but they have a niche with in-box upgrades of Mac cards, and I *love* much of other OWC stuff.

I'll post more soon

[Darryl Ramm]

As with most Apple products that are not easily expandable my advice here is to purchase as much internal storage as your wallet can handle. The good sign is these Studios have socketed internal SSD drives, or at least socketed FLASH chips, that hopefully third party vendors *might* be able to target--they are not M.2 drives, and the controller setup is Apple unusual so they may well not ever get there but we'll see.

I *really* hope the next Mac Pro has multiple wide PCIe 4 slots that will let us run M.2 cards in slot adapters.

There are maybe not great options out there for large external NVMe SSDs. I'd hope that vendors come out maybe targeting the Studio form factor for some add-on modular storage. I'd love to hear what folks are using in the 8TB+ NVMe size range.

For portable external NVMe storage you have stuff like the Samsung X5, and the Glyph Technologies Atom Pro drives, and products from OWC and others. The Glyph gets my attention now as it goes up to 8TB (but dual 4TB drive may be a better performance/cost/redundancy choice?). But I would prefer active cooling (i.e. a fan) for serious desktop use, which none of these have. I like lots of OWC products, but am not a fan of OWC SSDs. And I've got a strong aversion to anything where some small company is messing with this stuff, and prefer things from companies like Samsung that own the whole flash/controller/firmware stack. The Glyph uses the nice Western Digital Black M.2 NVMe drives internally, but you can't be sure they are not messing with firmware etc. (I hope not). I also would like to either measure myself or see Anandtech benchmark/review any of these SSDs. These are super complex SSDs, and sadly too complex for many reviewers to evaluate properly, Andtech was a rare exception, but seem to be doing less extensive benchmarking.

My preference today at the highest end be to use well regarded NVMe M.2 card in your own portable enclosure. And by portable enclosure I mean a PCIe slot chassis where you can install the right PCIe to M.2 adapter cards, esp. cards with PCIe switches on them to get efficient speed out of the drives, unlike what OWC did with the drive enclosure I just trashed. .. and I they have (fairly quiet) cooling fans/the whole box can be put beneath a desk and does not cause me any noise concern).

I'm running mostly Samsung (what a surprise) 980 Pro M.2 drives in different external Sonnet Thunderbolt 3 expansion chassis-old Sonnet GPU chassis (NVIDIA on Mac did not last so long, thanks Apple) or an Echo Express 3 slot chassis with multiple PCIe to M.2 cards in it. If you have an expansion chassis for Avid HD Native or HDX cards then you might stick storage in that as well. My 980 Pro drives are 2TB each but I do not stripe or concatenate them. I've posted multiple times in the past about my usual preference to just use these as JBOD drives. I would either use multiple single slot M.2 to PCIe adapter cards in these chassis (just about any brand but I use Syba) or if wanting a PCIe adapter card with multiple M.2 cards on it then look at Sonnet or Syba cards that are PCIe switch based.

Now we are talking stupid fast performance compared to legacy mechanical disks or even SATA SSD but one thing to remember is Thunderbolt 3/4 performance @ 395 MB/s per PCIe 3 lane means each a thunderbolt 3/4 link at best can deliver ~ 3.9 GB/s and in reality will see a lot less than this. So a whole external three slot Sonnet PCIe chassis is limit4ed to that performance. If you want maximum performance you might hang multiple drives/PCIe expansion chassis off different Thunderbolt ports.

Current high end M.2 NVMe drives are already PCIe 4, but most yet not making much use of that extra bandwidth, but with all these systems you are limited to at most 4x PCIe 3 lanes, which on paper is half the maximum PCIe 4 performance, but the actual drive performance is nowhere near that yet.

A lot here depends on budget, how much you want flexibility and future upgrades etc., how much you are willing to dink around and assemble stuff. (I'm far out on the tinkering edge, and use these SDs for much more demanding stuff than audio work).

And this is only part of it. Don't forget backup and restore.

[Darryl Ramm]

And let me dump on another product.

https://www.sonnetstore.com/products...underbolt-dock

I'd genuinely hope Sonnet did better, as I look across a desktop littered with Sonnet products, but here is another M2 NVMe chassis/case that just splits the PCIe lanes internally. You can see that by the single drive limit of up to 1500 MB/sec and the dual drive/RAID claim of up to 3000 MB/sec. That made me think they are just derating specs for 2 x PCIe 3 lanes and 4 x PCIe 3 lanes respectively, meaning inside the box they are taking 4 lanes off a Thunderbolt chip and sending two to each of the M.2 slots. And I confirmed with Sonnet support that is indeed what they are doing.

Now is "up to 1600 MB/s" impressive, yes clearly it is... but.... But I'd want to see what actual performance delivers here as you starve an actual M.2 card of half its potential PCIe bandwidth, including newer PCIe 4 M.2 cards that you are then starving down to a quarter of their theoretical maximum bandwidth if they were plugged into PCIe4 (which Thunderbolt 3/4 does not support). I expect you are going to run into the limit if you are really pushing I/O and will become more significant with newer higher performance PCIe 4 and PCIe 5 NVMe cards used in these PCIe3 slots. Many users might not notice, but to me it's just especially a cheap approach to delivering a product. And sure, it may be something to worry about in future upgrades as SSDs get faster, but I wanted folks to be aware of this, and for what you get in this doc it's not exactly low cost.... but likely quality stuff for the rest of what it does and I do like the cooling fan and what looks like efforts to reduce noise in the fan location/design.

If these systems had implemented this "properly" with a PCIe switch there would be no lower single drive limit. whichever M.2 card was being accessed would be able to consume up to the maximum total PCIe bandwidth, if you access both M.2 cards simultaneously then obviously they have to share that bandwidth. This at least with dual PCIe 3 lanes going to each M.2 card is less obnoxious than the OWC storage case I dinged earlier which only has one lane going to each card, and Sonnet may also be being clearer here than OWC was with their product with big clear limit called out per M.2 drive.

Can you stripe or concatenate the M.2 drives and get a higher aggregate performance, sure, but it's highly depends on workload, and a concatenated drives where you are reading/writing out a lot of stuff sequentially likely within the same area of the drive very likely will be writing to only one M.2 drive at a time if they are concatenated. And I sure prefer to avoid striping SSDs for reasons I've discussed on DUC before.

Lots of users may never notice stuff like this, but some might, and it's the counter intuitive thing that if you want fast performance on a system with lots of Thunderbolt connected I/O you may be better off buying small singe M.2 drive cases, which have only a single M.2 slot and connecting them up to different thunderbolt ports, than say using a big "serious looking" chassis/case. Or if you do want to run multiple M.2 drives in an expansion chassis and don't want reduced lane counts to each M.2 card then make sure it's internally using a PCIe switch (like the great Sonnet Express III and other PCIe card expansion chassis do.. and if you put a multi-M2 card adapter in any PCie slot in those chassis make sure it also uses a switch and does not just split lanes) that means it can dynamically provide maximum bandwidth to any M.2 card installed in the chassis, even if the aggregate link to a Thunderbolt 3/4 chassis only equal 4 x PCIe 3 lanes. Again it may be best to have multiple boxes separately connected to multiple separate Thunderbolt busses.

Rant over.

[EGS]

Originally Posted by reichman

... need to up my storage game ... epic NVMe speeds...

Fast work-drives are important i.e. boot, samples, & active sessions. For backups though, I'm more concerned with size & reliability rather than speed. This one goes up to 16TB: https://www.newegg.com/seagate-ironw...-796-_-Product

[reichman]

Thank you all for the info! I'm inclined to get a bus-powered single slot drive so I can travel with it more easily. The Glyph 8TB is a bit expensive, Darryl, I don't always love OWC products, but the 8TB Envoy Express seems appealing. And if I move up to the 4-slot, I could take out the drive and keep using it. What do you think?

And EGS, I have probably (30) or so 6TB spinners, and a few new 16TB spinners from Seagate. The 16TBs are amazing, routinely giving me 240MB/s (much faster than the usual 160MB/s). But even that is not fast enough for a big session with video and an Atmos recorder. My hope is that NVMe would let me run all of that on one drive.

[Darryl Ramm]

Originally Posted by reichman

Thank you all for the info! I'm inclined to get a bus-powered single slot drive so I can travel with it more easily. The Glyph 8TB is a bit expensive, Darryl, I don't always love OWC products, but the 8TB Envoy Express seems appealing. And if I move up to the 4-slot, I could take out the drive and keep using it. What do you think?

My preference really is to have something fan cooled if you think you are going to push performance. Now I don't know if you really are going to push I/O for extended time but if these drives and their onboard powerful controllers can put out a fair amount of heat... and then they start throttling performance, but you were asking about how to get high performance...

A M.2 drive housed in a small well designed aluminum case with thermal bonding to the M.2 card and heat sink fins on the case etc. probably all help with cooling. And some of the more aggressively finned cases seem to be able to handle the heat well (although you never know how much firmware has been hacked to trade off throttling). Unfortunately this that OWC case just doesn't seem well designed for thermals, it's another cheap looking drive thing by OWC. No internal thermal pads, no heat sinks, no fan of course, the thing largely seems to be a novelty about making it easy to clip onto an adhesive mount on the back of a iMac or similar. But if it overheats and throttles I doubt many users would notice. But I've not seen anybody I care about benchmark/stress test these and I'm not going to do it. So...

I'm not sure which 4 slot OWC product you mean. The OWC one I mentioned gives only one PCIe 3 lane to each NVMe card... again not a good idea IMNSHO.

The Mac Studios at least have a lot of Thunderbolt 3/4 busses, not just ports but lots of busses, and I hope somebody is going to come out with external M.2 chassis that has multiple Thunderbolt 3 links to the Mac Studio and supports a small number of M.2 cards with great bandwidth.

But in the meantime if I was building new desktop Thunderbolt storage with M.2 drives this is probably what I'd use (and it's close to what I have now, except with the larger Sonnet chassis, and a mix of M.2 drives, but mostly 980 Pro)

1 x $550 Sonnet Echo Express SE IIIe https://www.amazon.com/Sonnet-Expres.../dp/B07XHP2D12
3 x $13 ea. Crest M.2 adapter card https://www.amazon.com/Crest-NGFF-PC.../dp/B01I5VABFY

And the actual M.2 drives: (pick any one of the following)

3 x ~$300 ea. 2TB Samsung 980 Pro 2TB (might as well get the model with heatsink)
- or -
3 x ~$300 ea. WD Black SN850 https://www.amazon.com/WD_Black-SN85.../dp/B08PMLYV9H
- or - (at a little lower performance)
3 x ~$200 ea. 2TB Samsung 970 Evo Plus https://www.amazon.com/dp/B07MFZXR1B

And Kudos to WD for these M.2 black drives, they are out with serious competition against Samsung. (WD acquired some nice technology and teams when they acquired SanDisk).

Should not be too hard to put all this together, but the most important thing is to make sure every M.2 drive is on the current firmware before using them.

That's 6TB done with three seperate M.2 drives on separate adapter cards plugged into three PCIe slots in the chassis. They are all going to share the total Thunderbolt bandwidth in whatever way makes best sense since the Sonnet Echo Express chassis has a PCIe switch distributing PCIe traffic between the different PCIe slots. If you want to add another M.2 card you can add a switch based PCIe card in one of the PCIe slots. (I am not sure if non-switch based cards in one of the 8x lane slots would work, I've not tried that with the Sonnet expansion chassis). But for now If I could get by with 6TB of SSD I'd just use 3 x 2TB M.2 drives, and expect I might be later on adding additional new cards as things are evolving rapidly. And if you really want more performance beyond this then add a second Echo Express chassis off a separate Thunderbolt 3/4 port.

These Sonnet Echo Express SE III chassis are not huge, but not tiny either. Fans don't seem too annoying to me. They are smaller than the full size Echo Express III I use now (and many HDX folks use, but you don't really need to accommodate full length PCIe cards for this M.2 storage. The only thing I don't like about the smaller Echo Express SE III is they have an external power wart while the fill size Echo Express III have a mains IEC plug to an internal power supply. The full size ones can also be mounted two in a desktop duo dock or in a 19" rack unit... nice packaging, but *lots* of space is taken up for small M.2 drives. (ultimately in my case I'm running other PCIe cards in these chassis anyhow so I do want the large chassis). OTOH for this point in time, until better packaged/focused Thunderbolt M.2 storage products are delivered I quite like the flexibility these PCIe slot expansion chassis provide.