First experience with NVMe M.2 in enclosure - bad.

[Righty27]

I’m in a similar situation to Ben - just acquired a work 16-inch MBP Max that I also want to use for remote. I’m now deciding on my personal rig (replacing a retired Intel iMac) based on a Mac Studio, that will either be a ‘stock’ 512GB Max model, combined with external storage (that could be used with the MacBook also) or a CTO with 2TB.

I’ve been reading-up on external enclosures and, considering the small 512GB SSD option would primarily be used for applications + Library (e.g. Mac Mail archives), resulting in everything else (not just PT but also my personal files, Final Cut libraries, etc.) on the external drive(s), I’m wary about quality/reliability having looked at all the Oricos, Startechs, etc., so could do with a bit of Darryl’s expertise/experience too … and Ben’s total data loss scenario (based on overheating, NVME drive itself or enclosure reliability) is giving me pause for thought …

Darryl - any Thunderbolt 3/4 enclosure that you would recommend, or specific controller chipsets to look out for e.g. from ASMedia or others? A reliable external enclosure that I could switch between Macs (with applications installed on both) would probably be ideal.

Or, if I bite the bullet and go for http://https://www.sonnettech.com/product/echo-dual-nvme-thunderbolt-dock/overview.html with a single drive to start, could I use/benefit from the heatsink version of the Samsung 980 Pro?

Would this more-expensive storage/hub setup also be suitable as a boot/system drive i.e. just using whichever Thunderbolt-connected computer for processing - presumably one could easily switch between Mac Studio and MBP with this setup?

Thanks! :)

[glasswing]

I use and so far love the Sonnet Echo Dual NVMe enclosures. With Mac Studio Ultra I have problems getting them - and the machine - to sleep. But other Mac Studio owners have the same problem with different config's.

[Righty27]

Thanks Richard - very useful feedback!

Any risk of data loss if/when ‘sleep’ works + is the connection otherwise stable/reliable? So do you have to shutdown/reboot every time you leave/return to the computer + does the dock stay constantly warm/hot?

Have Sonnet not been able to resolve this? Any idea if this is their own design or a generic board/chipset e.g. I see similar products from the likes of Sabrent and Orico but would rather pay a premium for quality, reliability and effective support. Have the Samsungs been ruled out as a cause for the sleep issue - I note that Western Digital appear to provide a Mac version of their SSD management software*, and this is used to control sleep behaviour, for example, whereas Samsung don’t offer Mac software (I have some Samsung SSDs myself)?

A few more questions, if I may ….

How do you find the Samsung 970 EVO Plus SSDs - is real-world performance combined with the Sonnet dock similar/close to the internal drive in the Studio?

Is the fan quiet + effective in keeping the unit cool(er) e.g. do you use fan control software and monitor actual temperatures?

Cheers! :)

[* Edit - checked and not compatible with WD SN850 model, which is closest equivalent to Samsung 980 Pro]

[Ben Jenssen]

About the
Echo Dual NVMe Thunderbolt Dock
Price isn't bad, $349, but it's a bit overkill for my purposes.

But I also would like to know; is it noisy?
If it is, then it's definitely out. I love how quiet my room is with the M1.

[Darryl Ramm]

Originally Posted by Ben Jenssen

About the
Echo Dual NVMe Thunderbolt Dock
Price isn't bad, $349, but it's a bit overkill for my purposes.

But I also would like to know; is it noisy?
If it is, then it's definitely out. I love how quiet my room is with the M1.

I have no experience with using those Echo Dual NVMe Thunderbolt docks. Sonnet seems generally concerned about fan noise, has been whacked for that before and now seem to largely use Noctura fans. That dock also has a serpentine cooling path clearly designed to reduce fan noise, but I've never heard one.. and anytime you have tiny diameter fans there is a potential issue with noise at higher RPMs.

As I've warned about in the past with these Echo Dual NVMe Thunderbolt Sonnet Docks, and lower-end OWC Express 4M2 enclosures....

This dock does not have a PCIe switch, it supports two cards by just splitting the 4 x PCIe 3 lanes that come off the back of the Intel Thunderbolt chip inside the chassis and sending 2 lanes to one M.2 card and 2 lanes to the other M.2 card. If you use it with only one M.2 card you are halving the bandwidth to that card.

Which means each M.2 drive can only getting half of the maximum bandwidth it would get had the enclosure used a PCIe switch. If both drives were equally very busy that's OK you start to nottlenect on the overall bandwidth coming int or thunderbolt. But for many uses we tend to put most load on one drive at a time e.g. when backing it up or restoring it.

And if storage performance is not important then you may well never notice any of this. And the OWC 4M2 is worse than this in the sense that it splits the 4 x PCIe 3 lanes into individual lanes so that each M.2 card only gets 1/4 or the maximum bandwidth it otherwise could, even if it's the only M.2 card in the enclosure. And all this is unfortunate given Thunderbolt 3/4 is already a performance bottleneck with current M.2 cards supporting 4 x PCIe 4 lanes.

Now you can partially address this by striping across two M.2 cards with SoftwareRAID. I advise against just casually adding softwareRAID. It can be buggy, and can complicate systems enough that it leads to user errors potentially causing data loss (have backups) and can sure make data recovery more complex (also potentially leading to data loss).

SoftwarRAID has its place, but if you are going to use that, look around on the vendors user forums and look at the kinds of problems folks run into and try to avoid them. And ask yourself questions like what will happen if a M.2 drive fails or starts giving errors. Where will I see those errors, what should I do? And what's the actual procedure to replace a failed/failing drive and recover my data from backups.

It's just all a cheap an nasty way of building an M.2 enclosure as it avoids paying a few $10 to ~$100? additional BOM cost to include a PCIe switch and associated hardware (development and QA costs would also increase).

The PCIe expansion chassis products like the Sonnet Echo Express SE IIIe I use (and I have older larger Sonnet chassis) already have a high-end Broadcom/Plex PCIe switch to provide PCIe expansion between the three PCIe expansion card slots, when you plug a dumb M.2 to PCIe card into those slots it's leveraging the enclosure PCIe switch to do it's stuff. And the (quiet) cooling fan keeps the drives well cooled... but these are larger boxes, and cost $700... but to me since I use them for other uses with network and other PCIe cards and am frequently swapping/messing with NVMe storage and those cards so it's well worth it. Heck I'll probably buy another soon.

There are lots of choices for Thunderbolt expansion from single M.2 enclosures up to the switch based PCIe enclosures. And performance may *not* be important to many folks. I just want users to know what they are getting for their money especially in the boxes without a PCIe switch. BTW I think may of those single card enclosures may be OK without a fan for casual use if the card is well thermally bonded to the aluminium chassis and ideally also have some vents in the chassis. Even better keep/mount those enclosures in the cooling air flow for the Mac studio. They start to get unwieldy if you have many M.2 drives (I have between 3 and 6 M.2 drives in my enclosures). It's unfortunate no vendors has really targeted a system at mac Studios yet, they could do a lot better modular packaging than any system has today, e.g.. supporting multiple (locking!) Thunderbolt 3/4 links into a box with easy pluggable M.2 card access maybe with or without optional internal PCIe switches and with a great cooling fan. I've plead my case to folks like Sonnet, while complaining about their Echo Dual NVMe Thunderbolt dock lacking a switch. I have no idea if they are doing anything.

[Ben Jenssen]

Thank you, Darryl.
I have a
OWC Envoy Express
on order.
I'll just see how that works out.

[Darryl Ramm]

Originally Posted by Ben Jenssen

Thank you, Darryl.
I have a
OWC Envoy Express
on order.
I'll just see how that works out.

Would love to hear how that goes, it would be great to have a low-end single carrier that we all know works well for others, and I expect OWC to do a good job with this.

[Darryl Ramm]

Originally Posted by Righty27

I’ve been reading-up on external enclosures and, considering the small 512GB SSD option would primarily be used for applications + Library (e.g. Mac Mail archives), resulting in everything else (not just PT but also my personal files, Final Cut libraries, etc.) on the external drive(s), I’m wary about quality/reliability having looked at all the Oricos, Startechs, etc., so could do with a bit of Darryl’s expertise/experience too … and Ben’s total data loss scenario (based on overheating, NVME drive itself or enclosure reliability) is giving me pause for thought …

My advice is always add as much internal SSD storage on a modern mac as you can afford. And especially on a Studio, that's some fast SSD.

From an engineering snob viewpoint the gold standard PCIe switch technology is Broadcom/Plex. They dominate the high end of the market and I've run into them multiple times with engineering teams building high performance interconnect technology for supercomputers etc. The downside is the vendor is Broadcom, with Broadcom's attitude and Broadcom's pricing. Historical lack of competition in this space, especially the higher end has not helped. But it's good to see ASMedia challenging them, especially at the lower end. Now I buy Sonnet expansion chassis because I trust Sonnet, with a seemingly solid Engineering culture/team to get stuff right, that they use Plex is nice. If Sonnet turned around and came out with an ASMedia chassis I'd consider that, because well Sonnet.

Now in the Echo Express IIIe chassis if I want just one M.2 card on a PCIe adapter I'll use a cheap dumb card, no real engineering there since there is no switch or smarts on the Card, my preferred card is https://www.amazon.com/gp/product/B01I5VABFY. Only thing I don't' like about it is lack of cooling holes in the PCIe slot exterior panel, but a visit to the drill press can fix that, and the chassis may have enough air venting this is not a problem. I've looked for better vented cards just not found them. If I want more density by using dual M.2 to PCIe slot adapters then I use this IOCrest Syba (ASmedia) switch based card https://www.amazon.com/gp/product/B07HYZY7P2. Now you have two different vendors chips doing the PCIe switching. You could go with the quad Sonnet M.2 cards, which I believe are PLX switch based, but that's expensive and requires a full length expansion chassis not he short one I mainly use. If I was setting up a higher end studio and wanted lots of M.2 in a Chassis that's probably the way I'd go... well except I'm hanging out to see what the new Mac Pro offers and I'd expect to have multiple PCIe 4 (or 5?) M.2 internal expansion slots or PCIe slots that can take a PCIe 4 (or 5?) adapter card.

The global semiconductor shortage is likely affecting this space. The PEX8724 used in my Sonnet expansion chassis is currently unobtanium in retail semiconductor channels and lists for ~$130+ in small quantities. I don't know if there is a later substitute. PLX switch costs and engineering development may add several hundred dollars to a chassis price.

Darryl - any Thunderbolt 3/4 enclosure that you would recommend, or specific controller chipsets to look out for e.g. from ASMedia or others? A reliable external enclosure that I could switch between Macs (with applications installed on both) would probably be ideal.

If you can't get everything internal you may not need a fancy external box if you just want say a 2TB workspace on an external drive... wait and hear what Ben says about the OWC single drive dock and maybe get that.

But if you are looking for a higher-end chassis by the time you get to expansion chassis with switches the one I recommend is the Sonnet Echo Express SE IIIe. More expensive than some other lower cost ones, but frankly I just want Sonnet and trust them to get all the integration correct. Somebody might have lots of experienced with some other chassis and be able to recommend them. You are paying ~$300 for a single slot Thunderbolt to single PCIe slot expansion chassis from cheaper brands, so it's not outrageous to pay $700 for a three slot Sonnet chassis with switch, and from Sonnet for peace of mind, quiet cooling, etc.

Or, if I bite the bullet and go for http://https://www.sonnettech.com/product/echo-dual-nvme-thunderbolt-dock/overview.html with a single drive to start,

See the warning I posted above, with one drive installed you get half the PCIe 3 Bandwidth you otherwise would, and a modern PCIe 4 M.2 card is already throttled by PCIe 3. And the performance is already a fraction of what you would get from Studio internal SSD... so I'd sure be looking at spending money on the Studio internal SSD, especially to get dual internal NAND cards.

could I use/benefit from the heatsink version of the Samsung 980 Pro?

It should only hurt if the heatsink won't fit, in which case you can remove it, not hard to do. You can also add third party heatsinks to existing non heatsink M.2. With the big PCIe card chassis I use... there is lots of space and airflow from a large fan flowing over all heatsinks.

Would this more-expensive storage/hub setup also be suitable as a boot/system drive i.e. just using whichever Thunderbolt-connected computer for processing - presumably one could easily switch between Mac Studio and MBP with this setup?

You can boot off any external NVMe drive with modern Macs, maybe with some restrictions with RAID and/or secure boot. Well I have multiple boot images in my Echo Express chassis, use those for say testing a new macOS version. But I think most of the time you want to rely on the internal boot volume.

And your MBP is Intel and the Studio is Apple Silicon and you should not be wanting to have them share a boot/system drive. And increasingly over time it will become absolutely impossible to do that. So don't.

You also are living in a world of T2 authenticated/encrypted systems you should ready up on that before wanting to change stuff around.

[Righty27]

Ben - I look forward to hearing your experiences with the OWC Envoy Express.

In case it's of interest, I sent out a few support enquiry tickets over the weekend and Acasis have already replied regarding their latest TBU405 enclosure. They confirm that this uses the same Thunderbolt dual chipset as their preceding TBU401 model (various reviews online): JHL7440 + RTL9210B - Intel's Titan Ridge TB3 product that has been around since 2018 (according to Wikipedia). The newer Acasis enclosure apparently delivers a few benefits, including improved heat dissipation.

Darryl -

Originally Posted by Darryl Ramm

And your MBP is Intel and the Studio is Apple Silicon and you should not be wanting to have them share a boot/system drive. And increasingly over time it will become absolutely impossible to do that. So don't.

Quick clarification - the MBP is the latest 16-inch with M1 Max, and this forms part of my best endeavours, coordinated (!) migration to Apple Silicon. I've now ruled out booting off an external drive, and the idea is to do fresh installs on both machines having held out for a critical mass of native applications, albeit apparently relying on Rosetta for a while still ... ahem!

FYI, I've had an option on a base Mac Studio (with 512GB SSD) which has been in-stock at my local official Apple reseller over the weekend, so would probably have to commit today. I was hoping that the Sonnet (or even the OWC Express 4M2) might provide near-equivalent performance to an internal SSD in a high-quality external unit, whilst adding some expansion and flexibility for data sharing across the two Macs. I do not plan on using any version of RAID to compensate for the performance hit, and I actually intend to access each as an individual drive ... and also not considering moving up to the larger Sonnet PCIe expansion boxes (with a PCIE switch) + cards.

Therefore, I (believe I?!) find myself having to do a bit more 'research' on performance levels using the Sonnet/OWC 'unswitched' expansion boxes/docks to determine if sufficient for e.g. editing 4K video in FCPX directly off the external drive(s) e.g. maybe the 2-bay Sonnet with 1/2 throughput is still 'good enough', whereas the 1/4 bandwidth of the OWC is probably a stretch, for video in particular?

However, with Thunderbolt throughput to these devices evidently restricted, the latest/fastest NVME drives might be irrelevant, and real-world performance might actually be matched by DIY enclosures (such as the Acasis) or even off-the-shelf units like the Samsung T7, with each drive connected to a separate port on the Mac Studio (not necessarily Thunderbolt at this point).

Alternatively, as Darryl recommends, it might be better to wait for a CTO Mac and add the first 2TB of storage to a clutter-reduced Studio (although I appreciate that this wouldn't achieve the highest speeds for internal storage like the 4/8TB models) and simply use external drive(s) for a sync'd subset of data when using the MBP (which has 1TB, so enough for the occasional remote project).

Thanks for the detailed explanations, especially into how the 4x lanes of PCIe works over Thunderbolt 3/4 + impact of not having a PCIe switch, which was all new to me! :)

Edit - I’ve been researching some more and found this, albeit with usual risks dealing with possibly less familiar manufacturers. However, the Netstor product reviews that I have found are very positive and this unit, including a PCIe switch, appears to be quite premium and far from low-cost: https://www.netstor.com.tw/product_i...90328202742977

[Righty27]

Originally Posted by glasswing

I use and so far love the Sonnet Echo Dual NVMe enclosures. With Mac Studio Ultra I have problems getting them - and the machine - to sleep. But other Mac Studio owners have the same problem with different config's.

Hi Richard - could you possibly elaborate e.g. is the sleep issue rare/occasional or regular ... do the external drives (in the dock) reconnect after waking ... any issues with data loss ... do the drives themselves sleep when inactive whilst the Mac Studio remains active ... have you ruled out specific models of NVMe drives ... are Sonnet aware or do they expect to resolve?

The Sonnet looks about right for me but this would be a showstopper as I tend to leave my Mac switched on.

FYI, I have found the following regarding another dock, however the explanation appears to relate to a generic MacOS issue - I wonder if this is what you're experiencing?

https://www.caldigit.com/thunderbolt...nder-macos-12/