Backup software conflict

[Magnetic]

Once 2020.12 dropped, I finally pulled the trigger on Catalina. Since that update, my backup software, Arq, has stopped playing nice with ProTools. Anytime Arq is in the process of writing a backup, ProTools throws this error:

Could not save "" because Assertion in "/Volumes/Development/protools/release/2020/r3/ProTools/NewFileLibs/Sys/Sys_Volume.cpp", line 224

Once the backup completes, ProTools saves as usual.

After much troubleshooting, the problem is related to Arq using APFS Snapshots. If I disable the Arq preference to use APFS Snapshots, ProTools saves as usual, even during an active backup session. Arq support, of course, says "We've never seen that before". I've already tried all the optimization/delete prefs fixes. Any thoughts?

iMac 14.3
3.1 GHz Quad-core i7
16 GB Ram
OS 10.15.7
ProTools Ultimate (native) 2020.12.0
Arq 7.2

[musicman691]

Most Mac users here use Carbon Copy Cloner. I suggest you check it out. And never do a backup with PT or any other application running. Ideally you do NOT want your backup drive powered on and connected while working normally. Only when you want to make a backup.

[Magnetic]

Thanks Jack. I'm aware of, and use CCC. Been backing up since 1993, so I'm on board with the process.

[Darryl Ramm]

But to harp on Jacks point about not backing up while running applications like Pro Tools.... you just should not do that. It's maybe good this got caught here and you should just stop doing it. The Snapshot does not help here.. even if you can recover a snapshot of all files on disk for a session that are perfectly synced to a point in time that does not mean that the overall state is not corrupt/is usable by Pro Tools. Pro Tools state is spread across memory and disk and nobody told it that *it* needed to flush all of it's in-memory state to disk so that the snapshot is truly logically consistent and usable by Pro Tools again. The only way to be sure is to not be running Pro Tools when you take the backup.

[boboroshi]

I am hitting this as well, but Arq is only enabled to upload/backup from 11 to 6 overnight. Does it mean that it's in a "holding" state and therefore locking the drive or some equivalent as far as ProTools is concerned?

I just went through reinstalling everything over 2 days to only be taunted by " line 224" again, so going to try to disable this.

[Darryl Ramm]

Originally Posted by boboroshi

I am hitting this as well, but Arq is only enabled to upload/backup from 11 to 6 overnight. Does it mean that it's in a "holding" state and therefore locking the drive or some equivalent as far as ProTools is concerned?

I just went through reinstalling everything over 2 days to only be taunted by " line 224" again, so going to try to disable this.

What are you doing *exactly*? Is Pro Tools left running while the backup is happening? With or without a session open? What are you backing up to? disk? nas? cloud? Is the backup starting when you expect? And very importantly, does it actually finish.... if a nightly backups never actually finishes you may have way more problems than just this error. Are you backing up to slow cloud and the backup does not finish in that time window? What do the Arq logs show?

I do not use Arq, but these are the first questions to ask. For the many more of us using Carbon Copy Cloner... which on APFS also uses snapshots...at least by default for non-boot/system volumes and that seems to works like a charm if used correctly. I suspect the biggest risk is users not understanding snapshots and poking around and messing stuff up, and running out of disk space if old snapshots are on the system.

[Darryl Ramm]

Oh and to see what snapsots might be left on your computer after each backup you use...

$ tmutil listlocalsnapshots <mount point>

Like

$ tmutil listlocalshanpshots /

if looking at the root/boot/system drive.

Assuming the issue here really is related to snapshots. then why Pro Tools fails with an assert just because a snapshot is taken is unclear, it's possible it's checking (calling an assert) for something in a non-standard or very subtle way. In principle Pro Tools should just keep running... a test would be to reproduce this by just taking a manual snapshot with tmutil. In which case I'd hope that is something that Avid would treat like a bug.

[Magnetic]

Hey Darryl,

Thanks to you and Bob for your insight. I have used Arq as a proxy for TimeMachine for years. It does a running backup of my home folder and my Pro Tools work drive to the cloud (Wasabi) every hour. It's never been an issue until I reformatted my SSD system drive to APFS for Catalina. The work drive is a spinner, still MacOS Extended (Journaled).

I'm under no illusion that Arq would catch an in-progress ProTools session without the opportunity for error. That's what Pro Tools automated backups are for. I also use Arq to backup the work drive nightly to a local NAS, and then backup that NAS to a different cloud provider, so I always have three copies (two of which are off-site) in addition to the work drive while the project is in-process.

I also use CCC for a middle of the night clone of my system drive (to a local NAS).

tmutil shows no snapshots on any of my local drives

Also, "/Volumes/Development/protools/release/2020/r3/ProTools/NewFileLibs/Sys/Sys_Volume.cpp" doesn't look like a valid file path. I don't find a hidden "Development" volume on my system.

[musicman691]

Originally Posted by Magnetic

Hey Darryl,

Thanks to you and Bob for your insight. I have used Arq as a proxy for TimeMachine for years. It does a running backup of my home folder and my Pro Tools work drive to the cloud (Wasabi) every hour. It's never been an issue until I reformatted my SSD system drive to APFS for Catalina. The work drive is a spinner, still MacOS Extended (Journaled).

I'm under no illusion that Arq would catch an in-progress ProTools session without the opportunity for error. That's what Pro Tools automated backups are for. I also use Arq to backup the work drive nightly to a local NAS, and then backup that NAS to a different cloud provider, so I always have three copies (two of which are off-site) in addition to the work drive while the project is in-process.

I also use CCC for a middle of the night clone of my system drive (to a local NAS).

tmutil shows no snapshots on any of my local drives

Also, "/Volumes/Development/protools/release/2020/r3/ProTools/NewFileLibs/Sys/Sys_Volume.cpp" doesn't look like a valid file path. I don't find a hidden "Development" volume on my system.

What part of 'do not do backups while PT is running' do you not get? Is there any real reason that you're doing what you do?

[Darryl Ramm]

Originally Posted by Magnetic

"/Volumes/Development/protools/release/2020/r3/ProTools/NewFileLibs/Sys/Sys_Volume.cpp" doesn't look like a valid file path. I don't find a hidden "Development" volume on my system.

No that's the C++ source file on the system where Pro Tools was built that has the assert in it. See https://en.wikipedia.org/wiki/Assert...e_development) That assert is firing on your system, but for what exact reason we have no idea.

"Well behaved" software just should not assert because a checkpoint is taken on the filesystem that the software has files open on. While that might be happening here, it also might be something else entirely is going on. And the most basic stuff like is there a Pro Tools session open during all this, when exactly does the error occur, etc. is still not clear to me. maybe you can try Arq with checkpointing disabled as just another check on this to make sure its not some other APFS related thing.

It would be interesting to see the cause here, if this is somehow happening due to a checkpoint being taken then Avid would hopefully try to fix it...or produce a more useful error "just because"... and no not to allow people to backup using checkpoints when Pro Tools is running. Unfortunately Avid development seems to over rely on Asserts and some asserts that happen frequently linger for years when the developers should be hardening the code and/or producing a better user understandable error message.

Different alerts do get fired from Sys_Volume.cpp... across different of version of Pro Tools, going back a long time, but outside of looking at the source code there is no easy way of knowing what's going on...or even if these are the related or very different assets. The exact line number shown in these assert messages may be different across different Pro Tools versions even for the exact same asset cause as other code changes in the source file change the line number in different versions of Pro Tools.