I took over 400 CDs and DVDs to the shredding company this week.
This was a collection of old System disks, software that won’t run anymore, and 828,000 photos in an archive that I once thought to be impregnable. Optical discs were the safest, most reasonable, non-volatile, non-moving storage medium in the history of civilization. What could possibly go wrong?
(I wrote about this in an earlier blog where I described the NAS server that I built, and the process of moving all of the data on my optical discs to that storage device.)
What went wrong was light. Regular white light, and some ultraviolet light that crept in, and perhaps a little bit of infrared light.
My discs were stored in clear plastic boxes on shelves in my office. I have a database of their contents, so I can find almost anything in a matter of minutes. Each disc was given a name, which was then recorded into the database, and all the contents cataloged. This worked great.
Until one day when I couldn’t read one of the discs. Its writable surface was damaged by exposure to light. It was optically erased. There might have been some data left on the disc, but the important part – the index – was erased, and short of giving the disc to the NSA, there was no way to get the information off of the disc. I was distraught.
In the end I found back-ups of most of the files on that disc, and I managed to stumble along with those files. But the finished documents were forever lost to light exposure.
In a moderate panic, I rushed to convert all of that precious data to an archive on a set of spinning hard drives, where the data exist to this day. It’s a RAID 5 array, so it’s reasonably safe. But there is only one copy of that array, and without it, I would be in big trouble. In the end I lost about five percent of my stored files to light exposure. Those irredeemable discs went into the shredder this week along with those I could read. I no longer need to store these slowly decaying plastic discs.
My NAS has worked wonderfully now for several years. Mostly I ignore it, and I use it to make archival copies of important files. It’s not a back-up, it’s an archive.
The issue at hand is whether anything can be stored forever using any technology currently available to humankind. All hard drives fail, or they become technically obsolete and need to be replaced. A friend once told me that there are two kinds of hard drives: those that have failed, and those that will fail. What can we do?
I have thought about building a “disk” array of SSD drives. Samsung makes 4TB SSD drives that are stunningly fast, and they have the physical size and the same connectors as hard drives have. They have no moving parts, they work much faster than hard drives. They are also expensive; to put four of them into a RAID array would cost about $6,000 for the drives alone.
This small package packs a punch! Four terabytes of storage with no moving parts. I would love to get four of these and build an array, but I can’t afford that. I’ll have to stick with moving drives for the short term. Image from Amazon.com.
The problem with SSD drives, as I learned earlier this year, is that when they fail, they fail absolutely. There is no way to retrieve any data from an SSD drive that has failed. They hit bottom and they stop. End of story.
I have a MacBook Air lap-top computer. It came with 256 GB of SSD storage. I outran that storage after about two years, and had the local MacSuperstore install a 400+ GB SSD drive inside as a replacement. The SSD upgrade was manufactured by OWC, a supplier of such things (I have their 1TB SSD upgrade in my Mac Pro). That upgrade lasted over a year, but then it failed. Fortunately, OWC’s warranty covered the cost of replacement, and I was back to work shortly after the problem showed up. The problem was that its contents were lost.
Fortunately I had no data on that MacBook Air, only System and application software. So replacing the SSD was not a crisis. I lost nothing that I couldn’t replace. My personal policy is that I only use that computer for teaching and working while traveling. No documents are ever stored on the machine. Instead, I put them on a portable SSD drive (which is equally vulnerable to failure but more portable).
I sync the external SSD drive almost every day with my desktop computer, so even if it fails, everything on that drive is a copy of something stored elsewhere.
In some future era, historians will probably wonder what happened to the late 20th and early 21st centuries. There will be printed books and printed photographs, but there will be no record of much of anything that occurred in the first 50 years of the computer age.
I saw an ad the other day for a 1TB SSD card for cameras. That’s an excellent example of the extraordinary capabilities of SSD storage, and it is roughly the same technology that is packed into those Samsung SSD drives I want. It seems reasonable that storage companies like Western Digital will, quite soon, move away from spinning hard drives and convert all of us over to solid-state drives.
And as they do that, the technology will improve in integrity and safety, and we will eventually have some sort of storage device that will be indestructible, like CDs and DVDs once were.
