Re: Lost Files...Who can save the day???
This is from one of the other forums from one of the Digi Test Engineers. Not sure if it will work in this situation.
Quote:
First and most important thing: Do NOT use the hard disk for anything!! Don't even launch Pro Tools with the thing connected. Pro Tools may very well update the .ddb file on that drive which could overwrite important data. Just leave it unconnected until you are ready to attempt recovery, which I will describe next.
You will need three programs (well, two actually, but a text editor makes life simpler):
1. Terminal (found in your Applications:Utilities folder)
2. HexEdit (freeware)
3. TextEdit (in your Apps folder) or, my favorite, TextEdit (by Haxial)
You will also need an extra hard disk or two, and a Pro Tools system capable of playing the same audio files you are trying to recover (i.e.: if you're trying to get back 96kHz files, you need a 002 or HD rig.) For simplicity's sake we're going to refer to your precious hard disk as the Source Disk and the extra hard disks as your Destination Disk(s).
Here's what you're going to do
1. Read raw data off of the hard disk, 1GB at a time, to a second hard disk.
2. Attach audio file wrapper data to each raw data file. Do this THREE times if you are trying to recover 24-bit audio data. Do it TWICE if you're recovering 16-bit data. I'll explain.
3. Import the new audio files into Pro Tools.
4. Manually comb through the audio files in Pro Tools to find the data you want. This part of the process may take quite a while and test your patience, but if the data is really important, you'll do it.
Some caveats (in no particular order)
1. This is going to take time. Quite a bit of time. If you need this done quickly, and you're getting paid, it might be better to send it to a data recovery company and pay big $$. That's up to you.
2. This is going to take up a lot of disk space. The bigger the hard disk you are trying to recover, the more space you'll eventually need. As a rule of thumb, figure on 4x the amount of disk space from the original drive. If you don't have this much extra space, you can do it in chunks, but some of the time-saving techniques won't be as useful.
3. You need to know what type of audio files you're looking for. File format, sample rate and bit depth. If you are trying to recover a whole bunch of different file types or you don't know the types, you're in for a REALLY LONG HAUL. You'll basically have to repeat this entire procedure for each file type. If you're willing to do that you must have some really important audio files to recover. Good luck.
4. Again (worth repeating), do NOT write to your affected disk drive until you have recovered your audio or given up. This is really important. Since you emptied the trash, the computer doesn't know where those audio files are and could write over them without warning if you save something to that disk.
How to do it
1. Connect your Source and Destination Disks to your computer.
2. Boot up.
3. Launch Terminal.
4. In Terminal type su. Enter your password. You're now in superuser mode. Be careful.
Now you need to find out some info via unix:
5. In Terminal type df. This will show the mount point of your drives. On far right is the name of the disk; e.g. /Volumes/MyDiskName. On far left is "unix device mount point", e.g. /dev/disk1s9. 1 is the disk number (e.g. 1st disk found since booting). 9 is the partition number.
6. Find your disk in this list by the name and note down the mount point. This is how you will tell unix where to read raw data from.
Here goes the main recovery effort. I suggest you read ahead before you actually type this stuff.
7. In Terminal type dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/01 count=2m
Okay, what's all this about?
dd This is a unix command that allows raw data reading/writing.
if= Tells dd command that this is the Input File.
/dev/rdisk1s9 This is your mount point that you found in step 5. The numbers will probably be different on your system. The r is added in to tell dd that you want to do a raw disk read. The r is very important!
of= Tells dd command that this is the Output File.
/Volumes/MyDestinationDisk/01 Here's where you need to put in your Destination Disk name. Keep the /Volumes/ part since this is the same on all OS X systems and then type the name of the hard disk volume you want to write to. Add a slash after the name and then type a file name for the new raw data file you're going to create. I use numbers, like 01 since this command will be executed many times. Each time I increase this file number by 1 to keep things organized. (Unix scripters will see opportunity for automation here but doing it manually gives the same results.)
count=2m This tells dd how much data you want to read. 2m=1GB. (Unix deals in 512 Kilobyte chunks.)
8. If this all looks fine and dandy to you, press Enter. You won't see much going on but if you look at your drive access lights you'll see that reads and writes are occuring.
Now you may be wondering how unix knows which 1GB of data to read off of your drive? Simple. It just reads the first 1GB of data. So how do you get it to read the 2nd GB? or the 3rd? Or the 49th? Easy. Just add an additional command at the end of the dd line that says skip=2m. This tells dd to start reading raw data 1GB from the beginning of the disk. You'd use this to create your second raw data file. Your third file would need skip=4m added to it. The fourth will need skip=6m. Etc. A handy equation for this is of=N skip=(N*2-2)m. I.e.: Your Output File number is N and the number in the skip part is N*2 - 2.
So your second raw data recovery will look like:
dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/02 count=2m skip=2m
Your third raw data recovery will look like:
dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/03 count=2m skip=4m
Your fourth raw data recovery will look like:
dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/04 count=2m skip=6m
And so on. Until you've recovered all your data or run out of disk space on Destination Disk. If you have to stop in the middle, make note of where you left off and DISCONNECT your Source Disk.
How to speed up the raw data recovery process.
You may have noticed that each 1GB of data takes a long time to recover. I don't suggest you use your computer for any other tasks during this process so you probably want to do this stuff late at night or whenever the computer is not in use. But you don't want to be there babysitting the thing all night long. There's a solution. You can type commands into unix one after another if you seperate them with a semicolon. That's what I use TextEdit for. It's much easier to copy and paste a whole bunch of those commands into TextEdit, then scroll through and change the Output File names (the numbers) and the skip commands. Then you can copy and paste out of TextEdit back into Terminal. You'll end up with something like this:
dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/01 count=2m; dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/02 count=2m skip=2m; dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/03 count=2m skip=4m; dd if=/dev/rdisk1s9 of=/Volumes/MyDestinationDisk/04 count=2m skip=6m; etc.
I find that resizing the TextEdit window so that one dd command fits perfectly on a line also helps in making the edits correctly.
Be careful! This is a powerful technique for getting things done in a big batch, but if you make a small typing error you can do disastrous things to your hard disks. Or you can accidentally write each new raw data file over the previous one, which means you won't really know what you recovered and you'll have to start over. If you know how to write shell scripts in unix, I'm sure you'll be automating this whole process. I found it more satisfying to be able to see each and every command written out before I let them loose on my system.
Also, don't forget that you need to do the superuser mode change each time you launch Terminal. And you should recheck the df command to make sure the mount points didn't change (they can every time the system boots.)
You're one-third of the way there.
The next step is to add the audio file wrappers so you can import these new raw data files into Pro Tools, but first you need to create the wrapper files. There are two files to make a wrapper, one that comes first, then your raw data, then a second wrapper. I'll call these wrappers header and footer. You'll use the same footer for each raw data file, but you'll need to create three headers for each raw data file. Why? Pro Tools records 24-bit data. 24-bit data is broken up into three 8-bit chunks. Since you recovered raw data, you don't know what the right ordering of the three chunks. The only way to make sure you can get all audio data back is to create three versions of each raw data file, each version being a different ordering of the 8-bit chunks. So you'll need to create three header files, the second one being 1 byte longer than the first, and the third being 2 bytes longer than the first. When you attach the raw data to each of these, the byte ordering will start in each of the three possible places allowing you to find all the audio data that may be in the raw files. Sounds a little complicated but it's really not.
Creating header and footer files
1. Disconnect your precious Source Disk!!!
2. Launch Pro Tools.
3. Create a session in the same file format, sample rate and bit depth as the data you're trying to recover. If you don't know, you'll have to create headers and footers for each file type and do everything from here on over and over until you find your data. If this is your situation, think long and hard about how important those files are. Unless you have original recordings of spacealiens I suggest you forget trying to recover and re-record stuff. If you DID have original recordings of spacealiens and you didn't make a backup copy immediately then I suggest you sell your computer and get a job in a less intellectually demanding field. That said, mistakes happen. I made such a mistake. Luckily, I remembered the file type, sample rate, bit depth and even the number of channels of what I was looking for. So I pressed on.
4. Make a selection on your timeline that will create an audio file 1GB in size.
5. Press Option-Shift-3 to create a blank audio file with your selection.
6. You'll need to do this several times to zero in on the exact time needed to create a 1GB file. I'm not sure how exact you have to be but I got it to the exact file size through trial and error. Come to think of it, it's probably better to have a little bigger file than exactly 1GB because the headers and footers will take a little space. Oh well. My recovery worked well. I got back over 99% of my lost data (about 4 hours worth) and that was good enough for me.
7. Quit Pro Tools once you have your 1GB audio file. Name that file something like 1GBAudioFile.
8. Now launch HexEdit.
9. Open 1GBAudioFile in HexEdit. A sea of numbers will fill the window. Don't flinch.
10. HexEdit shows the raw hex data of the open file on the left side and the "human readable" version on the right. Sometimes you can make out words on the right side, but for audio data it just looks like garbage. Scroll down through the data until you see a large amount of zeroes that goes on forever. This is the audio data. Since you created a blank audio file, it's just zeroes in there, so it's easy to see where it begins and ends.
11. Find the beginning of the audio data.
12. Copy all the data from just before the audio data all the way to the beginning of the file.
13. Create a new file in HexEdit and copy this data into it.
14. Name this file "HeaderA".
15. In HeaderA file, copy and paste a single 00 to the end of the file.
16. Name this file "HeaderB".
17. In HeaderB file, copy and paste a single 00 to the end of the file.
18. Name this file "HeaderC". You now have three header files, each one byte longer than the previous one. Now for the footer.
19. Find the end of the audio data in the original 1GBAudioFile. It's the where the zeros all end and you get a few non-zero numbers showing up. There will be more zeros after this (where the waveform drawing data is stored) so make sure you scroll in far enough to get the end of actual audio data. This could take a while since you're looking at A LOT of data.
20. Copy all the data from the end of the audio data to the end of the file.
21. Create a new file and copy this data into it.
22. Name this file "Footer".
23. Quit HexEdit.
24. Eat something, take a break.
25. Copy the three Header and one Footer files into the same folder with your raw audio data files.
26. Launch Terminal.
27. Type cd and then drag the folder containing your raw audio files into the Terminal window. It will autofill the pathname for you. Hit Enter. Now you are in the same directory (folder) as your raw audio files so commands will look for files in this directory.
28. Type cat HeaderA 01 Footer > 01A; cat HeaderB 01 Footer > 01B; cat HeaderC 01 Footer > 01C This is three commands in a row (note the semicolons.) Each one says concatenate file HeaderA then file 01 then file Footer in that order into a new file called 01A. Then do the same for B, then C. So you now have three copies of your raw data file with the audio file wrappers on them. They are now ready to be imported into Pro Tools.
29. Repeat step 28 for every raw data file you recovered. You can queue up a lot of these commands with semicolons just like you did with the dd command earlier, as long as you have the disk space. As you can now see, you'll have three more 1GB files on your drive for each 1GB raw data file, making 4GB of data. This is where the 4x space requirement comes from. If you're doing an 80GB drive, you'll eventually need 320GB of space to check every bit of space on the disk for audio (no pun intended.) Breaking it up into smaller chunks, like doing 10GB of a time might be necessary if you don't have lots of free disk space. And remember, don't use your original precious hard disk!! Don't even think about writing to it until you are completely done recovering data from it.
30. If you are trying to recover 16-bit data, then you only need to create two copies of each raw data file. There are only two bytes of 8 bits each in this case.
The moment of truth
You have three 1GB files for each 1GB of space on your original disk. Now you need to put them in Pro Tools and start listening to them and looking at them to identify audio data and save it.
1. Launch Pro Tools.
2. Create a new session at the same sample rate, bit depth, format as your recovered data.
3. Import files 01A, 01B and 01C to Audio Tracks.
4. Let the waveforms redraw. If they don't automatically redraw, select them in the Region Bin and force the redraw.
5. You are now looking at the the first 1GB of data from your drive, presented with three different byte orderings. You will probably see large regions of what looks like solid full code noise (looks like a brick when zoomed out). Interspersed with these bricks you'll hopefully see what looks like regular audio.
6. Turn down your speakers!!! You're going to be hearing some unpleasant noises coming from your system. Do not use headphones.
7. Solo one of the tracks and start playing the parts that look like actual audio data. If it sounds right, then go ahead and select and delete the regions on the other two tracks that are in the same place as the good audio data. You're deleting what is the same data as the good stuff, just shifted one or two bits so what should be the bottom 8 bits of your audio is now the top 8 bits or something like that. Go ahead and listen to some of it. Depending on the original material, there may be some interesting stuff in there especially if you like noise and distortion.
8. Repeat step 7 for each of the three tracks until you've gotten rid of all the stuff that you know is junk. You'll be left with mostly real audio and probably some unknown sections that don't appear to have audio on them in any of the tracks. Listen to the unknown stuff and determine if it's of any value. Most likely it's junk.
9. Repeat it all with the each and every file you recovered and created.
10. You may have noticed while going through the audio that some stuff will get cut off at the end of a file and then pick up at the beginning of the next one. This is helpful so you can edit stuff back together. You'll also notice that you've lost all meaningful timing relationships between tracks that were recorded at the same time, or edited together later. You'll have to reassemble your session manually. This may sound nightmarish, but if this data is really that important to recover and you've come this far, you can probably recreate the session even better than you did originally. Good luck.
The End
If you completed this process like I did, you'll now have most or all of your original audio back and ready to work within Pro Tools. You will be elated to have recovered what was seemingly lost forever. You'll learn several things along the way, but most importantly, you'll learn the importance of backing up important data.
There might be a few typos here and there. Please post corrections if you find problems and I'll try to check back in and edit this post.
|
|
05-31-2004, 09:08 AM
Similar Threads
Linear Mode
