We are in the midst of a digital revolution that is changing the way we communicate, work and live. This, in turn, has had significant ramifications on how we produce, store and recall information, almost every bit of which is becoming digitised.
Collectively, we are on pace to churn out 35 zettabytes, or 35 trillion gigabytes, of information by 2020. To put this in perspective, if each piece of digital information were a DVD, the stack of DVDs would then reach halfway to Mars.
We are creating so much digital content that we have already exceeded the amount of digital storage space available. An estimate says that we would have a shortfall of around 35 per cent if we tried to store every single piece of digital data created in 2009. This gap will only increase over time (see the figure)
Given this, it is critical that we have a reliable, cost-effective and scalable storage solution in the form of an easily managed unified storage platform.
Why unified storage
Traditional servers have become very expensive as a long-term solution, especially when factoring in the costs of the equipment and software licences and also the need for management by a network administrator. In recent years, a new class of storage solutions which consolidates network-attached storage (NAS) appliances and storage area networks has become readily available. These storage appliances run on a single storage platform instead of multiple servers running on various platforms, providing enterprises with reliable, scalable and easy-to-use tools to effectively manage their storage needs.
High reliability. Given the growing dependence on digital information for business transactions, a single minute of downtime has direct impact on the revenue of a company. For a company like Taobao that processes all of its transactions via its website, losing one minute of uptime can cost the company $36,000, or roughly $2 million, per hour.
Taobao selected a unified storage platform with active-active node configuration that could survive single- or multiple-node failure through workload redistribution across a clustered solution. In addition, it could recover data almost instantaneously through data snapshots. With these safeguards in place, companies can enjoy the assurance of 99.999 per cent availability.
High scalability. Enterprises also need to consider future business and data growth in their selection of storage solutions. For example, Shanghai Ocean University was looking to manage 100 terabytes of e-books and videos within its digital library on campus while also planning ahead for its role as a super-computing centre. With scalability and flexibility in mind, it chose a unified storage platform that was not only a high-performance file serving solution but also scalable up to 15 petabytes in the future. Dynamic storage tiering (DST) within this platform ensured the most efficient use of storage locations for the university’s data.
DST enables data to dynamically move to different storage tiers depending on usage needs. For example, frequently accessed data would be stored on tier-one servers that consist of faster disks, while less volatile data resides on tier-two or tier-three servers. Because most organisations access only 10 per cent of their data frequently, 90 per cent of the university’s library data would be redirected to the less expensive storage using DST technology, which is expected to bring more than $700,000 in savings.
High performance. The constant demand for data is putting a strain on storage system performance. To meet this demand, there is a need for even utilisation across multiple nodes with-in a single storage system. A unified storage system addresses this by enabling every NAS engine node to share the same storage pool simultaneously. In addition, it could make it possible for multiple NAS engines to read and write the same document simultaneously, thus improving response time and performance when many users are trying to access a single document.
[stextbox id=”info” caption=”Drowning in oceans of data”]
The increase in data creation is taking place at personal and enterprise levels. At the personal level, the rise of personal digital devices speaks volumes about the way we create and share personal data. Photos have moved from physical photo albums to things like smartphones and photo-sharing websites. On Facebook alone, more than 25 billion pieces of content are shared each month, requiring over 60,000 servers to handle the storage and transactions.
At the enterprise level, there is unprecedented growth in online transactions and services that need to be stored, secured and managed. For companies like Taobao, an e-commerce site in China comparable to eBay, reliable data storage is critical, considering that in one day in September 2009 the company sold 4.561 million items and processed 64,808 transactions. With each item holding an accompanying description, photos and customer data, there is a burgeoning need for robust and scalable storage solutions.
[/stextbox]
Management simplicity. While technologies are becoming more sophisticated, their management has to be simple and intuitive. Unified storage systems offer centralised management tools that are easy to use and access, whether through a graphical user interface application or a Web interface. System administrators can use these management tools to set notification alerts via multiple communication mechanisms in the case of a system failure. Data backups can be configured and monitored for success and failures.
