A Cure for Storage Hardware Dependence

Netizens, like digital drug addicts, keep buying ever-larger fixes of storage for all the high- definition content on the planet. But back at the knowledge ranch, weird science is going to help us kick the storage habit and the hardware economics that keep us addicted.

Consider that corporate data storage is exploding at a 60% annual clip, driven by the need to meet regulatory compliance dictates, keep full-motion video online and support ever-increasing amounts of analysis and correlations for that elusive competitive advantage. Small wonder that the incessant demand for storage means storage revenue grows almost twice as fast as that of the lowly server segment.

These halcyon days of hardware-dominated storage can't last. Current storage costs depend on the economics of hardware. Hardware factors such as magnetic and optical densities determine how much intelligence is needed in controllers, how to optimize redundancies and how to engineer expansion. Optimizing hardware will never remove storage bottlenecks for users pushing the limits of system performance defined by more hardware.

The future of storage lies in kicking the hardware habit. The storage equation must be inverted, with smarter software processing the information before it ever sees hardware. The brave new world of storage economics requires very intelligent software with new algorithms that automate data compression before it's sent to the storage farm.

Entrepreneurs without a stake in the existing world order are moving beyond the rules of data communications and storage laid down by Claude Shannon in 1948. Shannon, dubbed the father of information theory, said that data is a collection of bits and bytes that have no intrinsic value and are mere commodities to be moved and stored.

With the economics of storage, we know now that information has intrinsic value. Patterns and knowledge do in fact exist in Shannon's random collection of bits and bytes. A huge marketplace is waiting to form around the notion that information can be ordered, the similarities removed, and only the differences stored. Imagine the savings.

Focusing on the changes is a proven methodology today, used in display and presentation algorithms like Flash and AJAX. By sending only the parts of the display that change, we can save huge amounts of communications bandwidth and push system bottlenecks back into the storage quagmire.

A promising theoretical fix to the hardware-based economics of storage is autosophy (www.autosophy.com), invented and marketed by a small San Francisco-based research firm of the same name. The word autosophy comes from the Greek words auto (self) and sophia (knowledge or wisdom), and its inventors call it the new science of "self-assembling structures."

Autosophy is based on algorithms that selectively acquire knowledge from information and data in random environments. This acquired knowledge grows complex data constructs without human intervention. Examples of self-assembling structures in nature include trees, crystals and even complex societies like beehives and ant colonies.

Here's how autosophy is going to cure our storage addiction. It starts with the concept of an "engram" - a unit of knowledge. Knowledge can be learned only once. You can't learn what you already know, and repeating what the receiver already knows adds no knowledge. Efficient communications send only engrams, telling receivers what they don't yet know and thus omitting all the background information already known or acquired. Focusing only on engrams obviously yields a huge economic benefit by eliminating all the transmission and storage of redundant information.

A second key to the autosophy concept is an address token called a tip. Unlike the monetary reward for good service, an autosophy tip is the address for an engram. In video communications and storage, each tip could represent that part of a frame that changed - the engram of new information that moves the action in the video. The variability of an engram eliminates any relationship between information and data volume - a tip can represent any amount of data, from a letter to an entire book or video.

The last part of your autosophy primer is the completed whole, or "hyperspace knowledge library." This new type of data construct must be identical in both the transmitter and receiver, just like in parity-controlled traditional data storage and communications. Autosophy communications becomes more efficient as the hyperspace knowledge library grows and assembles. It can be shared prior to transmission or self-assembled from the tip codes during transmission.

Among humans, the hyperspace knowledge library analog is that people without much knowledge communicate less efficiently than experts with a great deal of knowledge, or engrams must be in short supply in Washington.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Mark Willoughby

Show Comments

Brand Post

Most Popular Reviews

Latest Articles


PCW Evaluation Team

Luke Hill


I need power and lots of it. As a Front End Web developer anything less just won’t cut it which is why the MSI GT75 is an outstanding laptop for me. It’s a sleek and futuristic looking, high quality, beast that has a touch of sci-fi flare about it.

Emily Tyson

MSI GE63 Raider

If you’re looking to invest in your next work horse laptop for work or home use, you can’t go wrong with the MSI GE63.

Laura Johnston

MSI GS65 Stealth Thin

If you can afford the price tag, it is well worth the money. It out performs any other laptop I have tried for gaming, and the transportable design and incredible display also make it ideal for work.

Andrew Teoh

Brother MFC-L9570CDW Multifunction Printer

Touch screen visibility and operation was great and easy to navigate. Each menu and sub-menu was in an understandable order and category

Louise Coady

Brother MFC-L9570CDW Multifunction Printer

The printer was convenient, produced clear and vibrant images and was very easy to use

Edwina Hargreaves

WD My Cloud Home

I would recommend this device for families and small businesses who want one safe place to store all their important digital content and a way to easily share it with friends, family, business partners, or customers.

Featured Content

Product Launch Showcase

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?