The Digital Cornucopia

In ancient Greek mythology, the cornucopia was the symbol of abundance and nourishment. It was a curved goat's horn, usually pictured overflowing with fruits, vegetables, and other foods. No matter how much food you took out of the cornucopia, there would always be more. The cornucopia represents limitless plenty.

Limitless plenty might seem like a hard concept to find in modern life. After all, just a few years ago in 2008, we had a worldwide economic crisis. As I write this, governments in Europe are debating more and more Draconian austerity measures. Oil prices climb higher every year.

However, limitless plenty already exists all around us, if you know where to look. The computer world is full of it.

The CDROM of Plenty

If you are a writer, how long would you have to write to fill up a single CD ROM disc? I'm talking about a standard 650 megabyte compact disc rewritable, like you can buy at almost any corner pharmacy these days.

Well, first we have to have some sense of how big a novel is. Let's take J.R.R. Tolkien as an example. Each of J.R.R. Tolkien's Lord of the Rings novels weighs in at around 200,000 words each. In the English language, words have 5 leters on average. So we can say that with no compression whatsoever, each epic novel is about a megabyte.

Now, an author might reasonably be expected to produce a few epic novels, but 650? You would have to be cranking those things out at top speed to have any hope of filling that CDROM within your lifetime.

Now, you could quibble with a few of my assumptions here. Any reasonble person will use compression when storing his data, which will cut the amount of storage needed at least in half. And modern word processors can be very inefficient when they store things on-disk, which could increase the amount of storage needed quite significantly. But you can't dispute the basic conclusion here: text is free.

The Cornucopia Point

Each individual has a limited ability to consume over the course of his lifetime. No matter how avid a reader you are, there are only so many books that you'll be able to read in a hundred years. No matter how many movies you watch, the number will be finite. When technology advances to the point where storing all the media an individual can consume in a lifetime becomes essentially free, we've reached the cornucopia point for that medium.

Each medium has a different cornucopia point. Text became free sometime in the late 1980s, when storage devices with hundreds of megabytes became commonplace. Pictures became free during the late 1990s, when you could easily get hundreds of gigabytes of space. Music became free sometime during the 2000s, when 1 terrabyte drives came out.

Movies probably still haven't hit their cornucopia point. If the average DVD weighs in at 8GB, you can still only hold about 380 or so of them on a 3 terrabyte drive. You could conceivably watch that many movies in a year or two.

Conclusions

Will all of this digital plenty result in radical economic or political changes? I don't know. I'm not trying to consider that here-- I'm just pointing out that it exists.

Will consumers stop buying larger storage devices at some point? There is a very real possibility that they will. As of 2012, the exponential growth in storage has continued, but that may change.

Arguably, the current driver for denser consumer storage is video games. The video game market is huge, and games have historically been good at finding new uses for storage. I predict that long after storing movies has become trivial, games will still be finding ways to cram in more graphical detail.

However, not everyone is a gamer. So we may see the market bifurcate into work PCs and game PCs. Or perhaps the PC as a game platform will die out, and consoles will be where the newest storage devices get deployed. We'll just have to see.

There are other markets where more storage is still desirable-- like the Big Data market. I currently work on HDFS, the Hadoop Distributed Filesystem, a very popular filesystem in that market. However, as big as it is, the market for Big Data is small compared to the market for video games. Economic logic dictates that we must frequently re-purpose technology from the consumer market to our own ends.

In the meantime, I'm looking for a bottomless drinking horn to go with that cornucopia.