Avid Pro Audio Community

Avid Pro Audio Community (https://duc.avid.com/index.php)
-   Storage Subsystems (https://duc.avid.com/forumdisplay.php?f=14)
-   -   If I use a crap HD to copy audio files, would it cause any problems? (https://duc.avid.com/showthread.php?t=394692)

Darryl Ramm 10-27-2017 09:49 PM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
To the original issue/cause here. If you suspect the USB stick put it aside or throw it out. Check that the problem is not something you are doing wrong, e.g. copying files while Pro Tools or another application might still be writing the files. Not properly unmounting a USB stick before removing it from a computer likely causes most corruption with those devices.

NTFS on Windows and HFS+ on Mac (and in future APFS will be even better) are more robust filesystems than FAT32 or EXFAT. Archive backup volumes should be on NFTS or HFS+ (APFS in future) not EXFAT or FAT32. EXFAT can be a handy filesystem to use on a USB transfer stick, but don't rely on that a backup.

Quote:

Originally Posted by Rockman413 (Post 2456603)
I'm not asking how HDD or SSD manages sectors.

My questions are:
1. If an HDD/SSD have bad sectors, I use it to transfer files to my other computer, will my other computer get any corrupted files?

Or

As long as my other computer can read/copy the files without any errors, then I can fully trust all copied files are non-corrupted ?

No. and Yes. As far as anybody here needs to worry about, the file will either succeed or fail to transfer.

There is a much much rarer concern in computer systems storage design about exotic forms of silent data corruption. Not something to worry here.

Quote:

2. If I write files to a good HDD/SSD, and later it got hit the ground/got damaged and may have new or even a lot of bad sectors- meaning my previously written files are now on bad sectors. So I copy them out to my other computer, am I going to get corrupted files? or as long as I can copy them out without any error, all files are exactly the same as the original fils?
Worse case the whole drive might be unreadable and unrecoverable, or whole partitions/filesystems may be unreadable (and others on the drive are), or individual blocks within files data can't be read or it takes multiple retires (transparently to you) for the disk to read them. If a file read fails often applications will crash or a copy program will report an error.

Your primary protection against data loss is having backups.

SSDs might complicate the discussion because they are much more complex to understand internally that a HDD. But to the end-user they operate similarly. Good quality SSDs should have higher-reliability than HDD when used for active storage, for offline/archive storage you want to use a HDD. Don't put backups or archives in SSDs they are not intended for long-term powered off storage, but that is an active area of R&D.

For a HDD or SSD if SMART is reporting errors throw it out. Nobody here should really be worrying about deeper details of how drives work. What you should be worrying about is you backup and recovery plan, what failures you are protecting against (disk drive failure, building failure (e.g. fire, earthquake), theft, etc.) and testing both the backups and recovery actually work. You likely should be using different methods to backup boot/system drives and to make archives of sessions. And think about how far you want to be able to roll back either of those. And don't have just one active backup, the classic time for things to go South is when you are trying to write the backup and the read of the parent disk fails, now you have a failing disk and a possibly half-overwritten useless backup. Ad similar for archives, if something is missing it might also be missing.

And the most common cause of data loss is *not* disk failure, its the idiot user. So you want to think though what happens if you accidentally delete lots of audio content out of some sessions. And don't notice for weeks, has your backup plan overwritten all those useful old backups with the later now useless new session folder by the time you notice? etc.

You need multiple separate backups, ideally some stored off-site and/or in the cloud, and on different types of media. NAS systems can be fantastic as one of the things to archiving to (but don't rely on having all eggs in one basket).

QuikDraw 10-27-2017 09:58 PM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Quote:

Originally Posted by Rockman413 (Post 2456603)
I'm not asking how HDD or SSD manages sectors.

My questions are:
1. If an HDD/SSD have bad sectors, I use it to transfer files to my other computer, will my other computer get any corrupted files?

Any file that spans bad sectors will most likely not transfer at all. The disk/OS may try several times before giving up and reporting that the file is unreadable. The target disk will not "inherit" any bad sectors.

Quote:

Originally Posted by Rockman413 (Post 2456603)
Or
As long as my other computer can read/copy the files without any errors, then I can fully trust all copied files are non-corrupted ?

If your computer can read/copy without errors then you won't have errors.
Can you "fully trust" all copied files are non-corrupted? No. Even with brand new drives files can be corrupted.

Quote:

Originally Posted by Rockman413 (Post 2456603)
2. If I write files to a good HDD/SSD, and later it got hit the ground/got damaged and may have new or even a lot of bad sectors- meaning my previously written files are now on bad sectors. So I copy them out to my other computer, am I going to get corrupted files? or as long as I can copy them out without any error, all files are exactly the same as the original fils?

If the files can be read without error then the copies will not have errors.

Just try it, and see what you get. If you end up with a few corrupted files then you might try a data recovery service if the files are valuable to you.

Rockman413 10-28-2017 05:03 AM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Thank you both for the detailed explain.

So , can I understand it in this way:

As long as I have copied data from a HDD/SSD to my mac without seeing any error messages, or as long as the copy/read is complete, I can trust I have got all the files and nothing is corrupted. Correct?

Of course, if my target computer drive is corrupted, but I have also complete the copying from it to another new target drive and if the copy is complete without any system error messages , then I can trust all the data is non-corrupted, Correct?

JFreak 10-28-2017 05:24 AM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Quote:

Originally Posted by Darryl Ramm (Post 2456605)
the most common cause of data loss is *not* disk failure, its the idiot user

This made my day :D :D :D

Darryl Ramm 10-28-2017 08:32 AM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Quote:

Originally Posted by JFreak (Post 2456632)
This made my day :D :D :D



It is supposed to make you do a backup.


Sent from my iPad using Tapatalk

albee1952 10-28-2017 10:08 AM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Indeed...feel free to use a drive that is questionable, IF(the great big IF) you adhere to the axiom(thanks to Craig Anderton)"if it doesn't exist in at least 3 places, its not backed up);) SSD's are pretty darn solid, but spinners.....here's what happened to me just last week: Copying files from one system to another and it just stopped mid-stream. After a few minutes of restarts and head scratching, I notice a ticking noise. Yep, 1 500GB WD drive that had been sitting in a drawer for several months decided to quit for no reason I could figure. Then the next day, I am moving stuff in my office and snag a wire behind the desk as I am moving the phone and I hear a loud bang. It was an external drive that was sitting on top of a 6' cabinet(all my modem/router/network stuff sits up there). A 6' drop will kill a spinner.........:eek::eek::eek:

Extreme Mixing 10-28-2017 10:30 AM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
A six foot drop would hurt you, too!!!

Steve

Rockman413 10-28-2017 06:59 PM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Quote:

Originally Posted by QuikDraw (Post 2456608)
Any file that spans bad sectors will most likely not transfer at all. The disk/OS may try several times before giving up and reporting that the file is unreadable. The target disk will not "inherit" any bad sectors.


If your computer can read/copy without errors then you won't have errors.
Can you "fully trust" all copied files are non-corrupted? No. Even with brand new drives files can be corrupted.

--->If my target computer/drive completes copying without errors, then can I trust all copied files are same as original and non-corrupted? Of course if the target computer/drive have a lot of bad sectors the files can be corrupted, but I copy them to another new target drive and it completes the copying without any system error message, then I can trust that all the data is the same as original, unless the new target disk have bad sectors to damage the datas... correct?

meaning, is there any possibility that I copy audio files from my computer to a target computer drive(which has no bad sectors) using a questionable drive , though I didn't get any system copy-error messages, but the data/audio files on the target computer drive can be corrupted because of the questionable drive which is used to transfer the audio files?

SigFriedChicken 10-28-2017 07:47 PM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
Quote:

Originally Posted by Rockman413 (Post 2456721)
--->If my target computer/drive completes copying without errors, then can I trust all copied files are same as original and non-corrupted? Of course if the target computer/drive have a lot of bad sectors the files can be corrupted, but I copy them to another new target drive and it completes the copying without any system error message, then I can trust that all the data is the same as original, unless the new target disk have bad sectors to damage the datas... correct?

meaning, is there any possibility that I copy audio files from my computer to a target computer drive(which has no bad sectors) using a questionable drive , though I didn't get any system copy-error messages, but the data/audio files on the target computer drive can be corrupted because of the questionable drive which is used to transfer the audio files?

This is a simple-seeming question, but it really depends on the specifics.

When you copy files, validation is usually handled by the operating system. I don't believe that operating systems generally perform a full "verify" operation when you copy a file, though they probably compare metadata, like file size, to confirm that the source and target are the same size. Generally, if no error is reported, you can be confident that the file is not corrupt. But "not corrupt" is not the same thing as "exactly the same as the source."

There are specialized copying and backup applications that will perform a full "verify" operation, where the source and target are compared, block by block, to ensure that they are exactly the same. Full verify operations are time-consuming and often take just as long as the actual copy operation. An alternative is to manually generate a checksum for a file, before it is copied, and then get a checksum for the file after it is copied and confirm that they match.

Rockman413 10-28-2017 08:56 PM

Re: If I use a crap HD to copy audio files, would it cause any problems?
 
So in realistic daily use:

even though I used a questionable HDD/SSD to copy the audio files, if I didn't get any error message when copying, then I should not worry about I may got corrupted audio files and I can trust all the audio files sounds exactly the same as the original audio files, correct?


All times are GMT -7. The time now is 01:15 AM.

Powered by: vBulletin, Copyright ©2000 - 2008, Jelsoft Enterprises Limited. Forum Hosted By: URLJet.com