Hard drives from the last 20 years are now slowly dying.
Isnt the standard preservation system tape drives? They tested the longevity of different storage solutions ages ago to avoid stuff like this.
It is. Magnetic tape is still king.
Not really. Disk took over when the clouds started offering cheap archiving. Tape is getting more and more rare.
Not for long term archival they didn’t. HDDs and SSDs suffer from bitrot among other issues when they haven’t been powered and/or refreshed in awhile.
Tape is still king for long term archival, just about every major company uses it for the long term archival of critical data.
They may also use cloud archival services, because when it comes to backups if you don’t have multiple across multiple mediums and multiple places, you don’t have a backup.
Tape suffers from bit rot too. Radiation doesn’t target just HDDs and SSDs. Look, I don’t know what to tell you. I deal with a lot of large companies and I lived through tape’s hayday. The cost to archive data on disk is not high and companies don’t have issues doing it. Having it on disk prevents bit rot, because the pools are massive and are auto-healing. Also, the only way that your archive is not going to be long term is if humaity ends. Seriously, what do you think it would take to destroy a multi-AZ glacier archive?
Ultrium media has a lifetime of 30 years after the first write to the cartridge, and that’s assuming you write to it up to the maximum number of full writes (260).
Assuming you write to your cartridges once, store them (and the drive) in an airtight dustproof box, and don’t expose them to extreme temperature or humidity, you’re pretty much guaranteed to have complete data integrity after the 30 year mark.
At that point, you can probably transfer all the data into your petascale holographic storage installed at the base of your skull.
They aren’t talking about availability. This is about data integrity over time.
Which is higher on an active disk pool with auto-healing.
Sure if we’re powering every drive. For long term shelf stable backups, it’s still tape.
Not really. Disk is king now since S3 storage took the crown when cloud services started offering cheap archiving. Anything still on disk from the 90s is some neglected archive that has been deemed by the company to have no value.
I would assume they’re finding this out now because they’re trying to feed their whole archive to the AI beast.
Yeah no, thats not an “archive” you are talking about thats just a bunch of storage. Archives are for things like historical, government, artistic data. That stuff sits in airtight cases on tape storage in a bunker.
Obviously any drive that is constantly in use to deliver data to customers is gonna die, thats never going to change. But these were actually intended to be used for archiving but failed at doing exactly that.
Archive is whatever companies want it to be. I’ve been told anything that’s not microfilm isn’t an archive, so there you go.
Words have meaning, it doesnt matter what some company says.
Sure, in the world of social media you can enforce whatever arbitrary terms you wish.
“Hello Customer CIO, unexposedhazard on Lemmy says you’re using the term Archive wrong, so I’m going to have to ask you to stop.”
You’re currently having a conversation on an article about cold storage. The comment you replied to was about this article, and hence also about cold storage. It makes absolutely no sense to come into this conversation saying that they’re wrong about how cold storage works because your experience with hot storage doesn’t line up.
S3 archive is on glacier which is all tape
How do I get Glacier instant retrieval from a tape?
No one really knows, and Amazon won’t say. There’s speculation it’s tape, low-rpm drives connected to custom logic boards, Blu ray, etc.
It’s almost certainly tape, a single LTO9 cartridge can hold 18TBs of data for cheap compared to the equivalent drive.
Blu-ray is unlikely, only quad layer BR have a decent capacity at 125GB each and quality ones are hard to find these days. Sony has even stopped making their blu ray based Optical Disk Archival system thing.
It’s almost certainly spare space from massive S3 disk pools that’s unused.
Disk is king when you need lots of active storage.
When it comes to archival Tape is king. I would never trust an HDD to be left unpowered for years like you could a tape cartridge.
And a single LTO9 cartridge can hold 18TBs for dirt cheap compared to the equivalent HDD
S3 Glaciers practices are secretive, but it’s almost certainly tape based
I don’t disagree with you that tape has a different value prop, but I sell backup systems and almost nobody I sell to uses tape anymore. The truth with tape is that it’s a cheap media, but you still need to pay someone to vault it for you, it cannot be accessed easily, it makes physical moves which cause damage and tape drive tech is still one of the most complicated things in the Datacenter.
Most companies I deal with want the data to be “online” in at least some form that can be easily accessed for AI, lawsuits, new research, business continuity, etc. Tape allows none of that, and so the value of it is pretty limited. The truth these days is I can stuff a TB of data into cloud archive, with instant retrieval, for really, really cheap, with like 99.99999999999% data durability guarantees.
I have a crate of old hard drives going back to the late nineties. Am I the only person that migrates the data to new drives regularly? At this point it is a yearly tradition for me to pick up larger drives during the Black Friday / Cyber Monday sales. Why rely on old 4tb drives when you can move them all to fresh 14tb drives?
NAS is another option instead of relying on random assortment of drives.
But it’s most cost-effective to use cold storage like Backblaze if you don’t need to access that data and just want to archive it.
What I meant by drives are NAS. I buy the drives on sale spin up a new array, migrate the data, and redirect the mount point.
I use to cold store until I realized that unless I have access to it, it might as well not exist. Now I keep everything live, even backups going back to 1997.
The only data I have “lost” are copies of my old warez CDs from eastern Europe because I have no idea where I have stashed them, and a pack of Zip Disks because I have no functioning Zip Drive.
Phew, I was imagining a closet of drives. NAS is great.
Cold storage is always controversial as you are storing it on someone else’s hardware, but it is by far the most cost-effective option. Just a single month’s electricity cost in some places can match years of cold storage.
Using both of course is recommended, as cold storage acts as another backup vector in case your own storage ever gets catastrophic failure due to fire or flooding. 3-2-1 rule and all. But cost is always a factor in people using the best practices.
I’m the opposite: I migrated 2 4TB drives from my first NAS into the actual one. The drives are going strong and nearing ten years (!) of run time. Two out of eight drives died in this server since 2017. Both were newer. I’m not going to change a single disk before it dies. Most value for money in my opinion.
But I can afford this „risk“: My server has a redundancy of 2 disks. It has a local USB backup, is mirrored to two remote servers in different locations with local backups as well.
One reason why I love btrfs is the ability to add (and remove) arbitrarily sized drives to the disk array while maintaining multiple redundant copies of my files.
unRAID can do this with both xfs and btrfs.
Drive failure in the 00s was really common. I lost 2 or 3 separate drives from different mfg over the course of a couple years. Newer drives are better but even in modern nas setups, I planned on losing at least 1 drive per year on a 4 drive nas even fresh out of the box.
Always keep data your care about in at least 3 places and in at least 2 different mediums with one preferably offsite. I like to have one drive in use, one backup that sync’s daily, and one that I keep in cold storage unplugged. Then swap the sync drve and cold storage drive every so often.
So far over a span of maybe 40 years of computing I’ve only lost two HDDs. A number of 5 1/4 floppies back then but that was typical. Both drives I was able to pull most of the info off to a new drive, so yay for the mechanical drive, where a SSD you’re left with either a miracle, or looking for the experts to retrieve something. I’m no power user, so perhaps that’s part of the reason, but ever since we got into the giga and tera range of storage my first thought is always…wow, that’s a lot to lose at one time.
30 year old hard drives failing?
Color me shocked
