I recently decided to replace the SD card in my Raspberry Pi and reinstall the system. Without any special backups in place, I turned to rsync to duplicate /var/lib/docker with all my containers, including Nextcloud.
Step #1: I mounted an external hard drive to /mnt/temp.
Step #2: I used rsync to copy the data to /mnt/tmp. See the difference?
Step #3: I reformatted the SD card.
Step #4: I realized my mistake.
Moral: no one is immune to their own stupidity 😂
Fuck up #1: no backups
Fuck up #2: using SD cards for data storage. SD cards and USB drives are ephemeral storage devices, not to be relied on. Most of the time they use file systems like FAT32 which are far less safe than NTFS or ext4. Use reliable storage media, like hard drives.
Fuck up #3: no backups.
Would an SSD be any better than a pen drive or should it be stored on spinning rust?
In my experience, flash drives are way more reliable than SD cards and I’d put SSD and HDD above both of those.
I wish they’d just ditch the SD card on the Pi already as it’s always the most likely reason why your stuff stops working. For my Pi running Home Assistant, I’ve swapped to an SDD as the boot drive. For the others, I still use SD cards but they’re just doing basic stuff like running Klipper on my 3d printer or a (WIP) live photo frame that can be easily swapped with a replacement SD later.
It really depends how you define reliability. SD cards are physically nigh indestructible, but can show failure when overwritten often. Hence for one off backups it’s actually a good alternative. It will start showing problems when used as a medium that often writes and overwrites the same data often.
I would recommend backups on SD cards in an A/B fashion when you want to give a backup to someone else to store safely.
Reliability in that I’ve used flash drives and SD cards for years but have only ever had issues with corrupt SD cards (probably at least half a dozen times) while I’ve never had any with flash drives.
Constant writes is an issue with them, which is why I think it’s stupid that the Raspberry Pi Foundation continues to use them as the default storage/OS drive. Then again, they continue to make insane choices with power supplies as well, so it shouldn’t be a big surprise.
I really wish they’d just give us an m.2 slot on the back
The best way to ensure your data lasts a long time is to use a laser to beam it to the darkest part of the sky. Read speed is abysmal though
Much better. SSDs and HDDs do monitor the health of the drives (and you can see many parameters through SMART), while pen drives and SD cards don’t.
Of course, they have their limits which is why raid exists. File systems like ZFS are built on the premise that drives are unreliable. It’s up to you if you want that redundancy. The most important thing to not lose data is to have backups. Ideally at least 3 copies, 1 off site (e.g. on a cloud, or on a disk at some place other than your home).
RAID is not a backup.
Though not every fail state is going to show up. If you start seeing weird intermittent behaviour from a drive, for goodness sake find a way to back it up immediately.
My mum’s new nuc started having some issues, SMART showed perfect drive health. After trying a few things to diagnose, I rebooted to run memtest and check for bad ram, and that was the last time it ever booted into windows. Controller or something on the nvme ssd died. Far too expensive to try and repair for data recovery. Thankfully had a… Somewhat recent backup. Not as recent as we would have liked.
SD cards and pen drives are (usally) made from lower quality, cheaper nand (the little memory chips that store the data) and also lack health monitoring, that being said ssds can and do die so it’s important to have backups