Audio from past two days of recording missing... ideas?

[formfunction]

Originally Posted by flommer

Type in some search term and if it doesn't have any hits then start using the additional search options to try different methods to search them out. You have to click the little + symbol to get at the other options. In the dropdown list (kind) there is an "options..." option with lots of different things to use to search. .

Yes, those options help narrow it down... especially the date function. ff

[pmklein]

Speaking of backups, have you tried opening any of the Pro Tools session file backups? (If you have that option enabled.)

[Rail Jon Rogut]

The only time I've ever seen this.. after checking the studio security tapes they saw one of their employees had accidently formatted the drive and done a restore... and not told anyone/denied it. Is your system accessible to anyone else?

Normally if you were to lose sound files, the regions would still exist in the session as offline.

Rail

[formfunction]

Originally Posted by formfunction

either that or one of your skilled buds is "having a go" with you. ff

Thats what I came up with Rail!!

[Dark Pines]

I experienced pretty much what you have... except it happened without a glitch, shutdown, crash, reboot or anything else... all during the same session.

They were random regions that were missing.

Some of the files were recoverable, some were not.

(Most of the text with Digi support on the case follows)
I'm running pretty much a standard HD3 rig here...

Power is 15kva Isolated power, through a Furman 1220 isolated power conditioner. (Power IS stable)

Intel Mac Quad Core 2.66 w/9Gb RAM
3 HDD's - 250Gb main drive, 2 - 160Gb internal drives for audio
OS-X 10.6

PT 8.0.3cs2
D-command
2-Aurora 16's
1-192
Sync HD

The situation;
A very light tracking session - 6 track acoustic artist @ 24/48KHz

During playback after one of the takes, the display on the edit window "travel" freezes, the system is unresponsive on the D-command and from the keyboard and I have to force quit, shut down and restart. (Which is under my other case number... and in which Digi Support told me to upgrade to 8.03cs2... which I promptly did under that case number.)

I reopened the session and the tracks are all there... whew!

We finished tracking two more takes and even added overdubs, to this "session 1".

After tracking the first song, saved the session, and then closed it.

We then created a new song/session and imported the session data from session 1.

Lather, rinse, repeat for song 3.

We only deleted data in the regions list on the new import.

The rest of the session is running without a hitch... WHEW! We bust ass on getting 3 songs put in the can.

It's getting late, so we decide to mix song 3... ok... all is well... did the bounce... done.

Open song/session #2... a glitch is flashing the trim button on the Icon for one track that is unresponsive to automation. It WILL not clear... ok... we rode it manually for the final mix... a PITA, but doable for rough bounce.

We go to open session #1 and of the 4 takes, there should be 56 wav's... there are only 12! The rest of the wav's are GONE!

Through the finder, the files are not on the drive.

I go to the regions list and check to see if PT had done something stoopid like enabled round robin... nope. (Which I also have a problem with, but have not addressed with Avid.)

I searched the entire computer and the files are not present.

Of course this happens at 10pm and there's no support available...

After searching DUC/AVID and the web, the overall recommended utility is Data Rescue 3.

Yesterday morning, I contacted Sweetwater support who evidently was on the phone with their Digi folks trying to figure out wtf happened... their only conclusion was that it must be an Apple OS issue, and that Drive Rescue would be the best answer.

I ran Data Rescue and it found several of the files... but not all of them. (Of course it wouldn't find two of the main files for the take I need!)

35 years of IT/Hardware the files should, by reasoning, have ALL been deleted into a single class of folder. What I discovered was that the recovered files were in 4 classes of folder... according to Magic Number matching.

2 files, of differing size, of the same take were able to match. No others were able to match.

-----

Next, Digi Support indicates that I'm too stoopid to understand what Round Robin is, and that that's the issue. Which it isn't, because Round Robin becomes active when I attach my Glyph... (even when Round Robin's disabled!).. and the Glyph wasn't even powered up.

----
As I accurately stated, in what I thought was enough detail, the recovered files came from the SAME drive, but were recovered to four different folders on the 2nd internal audio drive. 2 folders were named for the session::audio files, one was placed in a folder called Recovered::WAV, the other in Recovered::WAV2

Why 2 were duplicate names of the session::audio files folder, I cannot understand, but seems to most likely to have something to do with associated playlists. But not all files from the same playlist were in the same folder... just as not all takes or any other recognizable pattern seems to be associated with the WAV or WAV2 folders.

-----

Digi's solution is to continually deny that there could possibly be a failure of any part of ProTools, except a user error.

[DigiTechSupt]

I go to the regions list and check to see if PT had done something stoopid like enabled round robin... nope.

When you open the session with the missing files and 'Show Full Pathnames' in the Regions bin - where does it say those audio files should be? And, if you go to that location on the drive, are they there?

Digi's solution is to continually deny that there could possibly be a failure of any part of ProTools, except a user error.

I highly doubt that's the case - in fact, from what you're writing, they said that it may be a drive problem, which happens every now and then. If the drive directory is corrupted, it will sometimes lose track of where files are.

I'm not saying there isn't a bug here, but our experience with these kinds of things shows that, if Pro Tools was at fault, we hear about it far and wide. Isolated incidents are typically system specific and not something in Pro Tools itself causing the problem.

Can you give me the case number, so I can look at the history and see what's been covered so far?

[jeremyroberts]

Originally Posted by Dark Pines

We then created a new song/session and imported the session data from session 1.

Lather, rinse, repeat for song 3.

We only deleted data in the regions list on the new import.

Why did you import the audio from song 1 into the song 2 session?

It sounds like you deleted regions. I am sure you didn't intend on doing this, but it sure looks that way.

The better workflow when importing session data the way you are working, is to select "DO NOT IMPORT" playlist, and then ALWAYS "copy audio from source media", just in case. This way, any audio that come in are full copies, never pointing to the original.

I'm guessing the reason you are using import session data is to keep routing, naming, plugins, etc., right? But you don't need the audio. Maybe on a click track -- so make that a separate import, and COPY the audio to the new session.

Storage is so freaking cheap now that is there is little justification for not having 100% of a session's audio completely self contained. It's the sure thing. There may be some intelligent uses for using cloned audio -- but then the workflow should NEVER involve deleting regions. This is just an invitation for disaster.

I am sorry to hear about your data loss - but it really sounds like you deleted these regions... ProTools does not randomly delete audio. it just doesn't do this. Else, they would be out of business.

[Dark Pines]

I would normally agree that it looks like a stoopid user trick in that we had indeed chosen the option to delete regions. However, I learned that lesson the hard way several months ago and thankfully, it was a throw away session... when I was first getting used to the rig.

However...

The full path for the missing regions was the same as the rest of the session. Same drive, same folder... so, Round Robin was NOT enabled.

If I had deleted the regions, instead of removing the regions (when I imported session data), ALL of the regions should have been deleted in that session. I cannot understand why they were randomly missing.

Additionally, if I had indeed deleted the regions, when I went to recover the files, they should have been recovered to either one folder, or to possibly two folders. IF, I had indeed deleted the regions, they should have been deleted at the same time and their recovery structure should have reflected that they were indeed deleted at the same time, and thus in one folder... or more likely similar folders as grouped by time of creation or modification.

I have done quite extensive data recovery over the years, and you typically get a logical data structure on the recovered files by some common factor. The few files I was able to recover really didn't have a logical order in which they were found/recovered.

If I had screwed myself, ok... fine.. I did it to myself. But there were two of us sitting next to each other and neither one of us noticed anything we were doing out of the ordinary. The few times we actually import session data, (usually low track count sessions) we are ALWAYS careful in NOT deleting regions, but only removing the regions list.

I've triple checked the drive integity, and they all check to be good under 4 different utilities. So, I have confidence in the drives.

I've run every system utility I know of, on the system, and everything checks out clean.

I've checked quite a few logs, and I can see various actions that indicate file deletion, but NOTHING in the logs around the time that these files went missing... e.g., no deleted files that day. There may very well be logs/entries that Avid creates and uses that I don't know about or haven't found, but the one's I have found, there just isn't anything that indicated a user oriented delete instruction/command.

Unless PT does something really tweaky with disk I/O operations at the OS level, I'm just plain stumped as to what happened... and I sure can't afford to experience this again.

[jeremyroberts]

Originally Posted by Dark Pines

I have done quite extensive data recovery over the years

A statement like this makes me nervous! I have been actively using hard drives since 1986. And I can count on one hand the number of times I tried to play with data recovery. And it's rarely successful, nor logical. I had to pay drive savers once in my career. Cost me $$$ and a major client.

Data recovery is something I fear, and I look at data recovery like an airbag in my car -- I really don't ever want to find out if it works or not.

I'm just plain stumped as to what happened

Did you delete regions from the region bin after importing session data into song#2 ? That's my guess as to where the audio went...

[Dark Pines]

Originally Posted by jeremyroberts

A statement like this makes me nervous! I have been actively using hard drives since 1986. And I can count on one hand the number of times I tried to play with data recovery. And it's rarely successful, nor logical. I had to pay drive savers once in my career. Cost me $$$ and a major client.

Data recovery is something I fear, and I look at data recovery like an airbag in my car -- I really don't ever want to find out if it works or not.

I've been playing with HDD's since RLL's... like 1980.. spent almost 16 years as an IT/Systems Admin, shoving tons of Tb's of bandwidth around a network... When you're talking about a coupla dozen RAID drives going to Drive Saver's @ $200/Hr/drive... you end up having to learn how to edit sectors when you get a bad crash... just comes with the territory. I've had to do data recovery at least a coupla dozen times on various drive configurations... usually because of stoopid user tricks. NT4 Server was seriously nasty about over-writing files that got deleted, after 15 minutes of emptying the trash. You'ld have to recover the trash cache, then decipher that to know what sector to go look for file headers... ick... Now I remember why I gave it up to be a poor studio owner... <shudder>

Did you delete regions from the region bin after importing session data into song#2 ? That's my guess as to where the audio went...

No... we only removed the regions list. Like I said... I learned that one the hard way a few months ago... I made a HUGE note and put it on the keyboard drawer... "DO NOT DELETE REGIONS"... and again... If I had accidentally selected "delete regions"... there should NOT have been ANY regions left. There should not have been 12 of the 56 remaining... correct?