Storage virtualization, as a concept, has been around about 60 years now. However, it became more prominent and practical with the advancement of computing technologies in the late 1990s and early 2000s, in particular with the emergence of storage area networks (SANs) and network-attached storage (NAS). As these systems became more prevalent, storage virtualization technologies started to take advantage of these architectures, offering more scalable and flexible solutions for managing your storage resources.
Storage Virtualization Architectures
The idea of storage virtualization abstracts away the physical storage resources from the logical view presented to users or applications. This abstraction allows for more efficient management of storage resources in terms of time and resources. The separation enables pooling of multiple physical storage devices into a single logical storage pool. Furthermore, different types of disks can be used to tier data from faster, but more expensive storage backends, to slower, but cheaper ones, hence optimizing cost and performance. Providing the user with a logical (or virtual) device, all complexities are hidden, providing the client with heterogeneous storage.
On top of the physical resource management, additional features help improve the storage efficiency utilizing deduplication and compression, as well as providing features like thin provisioning, data migration, and replication without the need for direct interaction with underlying hardware.
Storage virtualization architectures provide built-in high-availability, as well as redundancy and fault tolerance by integrating solutions such as disk mirroring, checksums, transaction support, and other durability features. Therefore, those systems easily mitigate the loss of one or more backing storage devices and provide the necessary integrity for enterprise level data storages.
Today, storage virtualization is a fundamental component of modern storage infrastructures, with various technologies such as software-defined storage (SDS), virtual storage appliances (VSAs), and storage hypervisors, enabling organizations to achieve greater agility, scalability, and efficiency in managing their storage environments.
Storage in the Cloud Age
With the emergence of cloud environments, as well as companies redesigning their existing services or building newer architectures, requirements for storage also shifted to the cloud, losing many beloved features along the way.
When consuming storage-as-a-service, such as Amazon EBS, we can take advantage of some of the storage virtualization concepts, while there are also limitations on the existing cloud storage services we have to live with.
As users, we have to choose between the following options:
Local, fast NVMe storage, which has affordable cost of performance and capacity, but does not scale. A fixed ratio of compute to storage almost always leads to either under- or over-utilization, and does not live beyond the lifecycle of the single compute instance. It is therefore neither highly durable nor highly available. Further, it does not come with additional data services which are effective to increase storage efficiency and decrease storage costs or increase security and reliability, such as thin provisioning, deduplication, compression, encryption, attack detection, snapshots, copy-on-write clones, and many more.
Local, fast block storage, which overcomes some of the limitations of local storage, but it is slower, though much more expensive on a per-performance unit. Also, depending on the specific type and costs, is not highly durable, cannot be multi-attached, replicated across availability zones, or doesn’t have special cyber-security protection features.
Distributed file storage, which is comfortable and supports multi-tenancy, but is expensive (per capacity, per performance unit), not as fast as local or networked block storage, not replicated across zones, and does not provide a range of data services.
Object storage, which is the cheapest type of storage. It is highly scalable and works cross-availability zone, but requires a separate programmatic interface. Furthermore, it is immutable and really slow with high access latency. A limited amount of data services is available. It does not allow for features such as instant snapshotting and copy-on-write clones.
Storage Virtualization and the Cloud Future
Moving to the cloud robbed us of the possibility to combine the features we wanted, and have to settle on what is provided by the cloud provider. At simplyblock we strongly believe that this is not the way it should be. People should have the freedom of choice they've known with their SAN and NAS storage solutions. Our industry doesn’t need to take a step back, but move forward.
That said, at simplyblock, we take the fundamental ideas and concepts of storage virtualization and adapt them to achieve the next level of cloud computing. We transparently combine the different advantages of cloud storage types and provide a single logical device. This device is exposed as the most common industry interface for fast storage, NVMe (over TCP). A stack that is readily available on all modern Linux and Windows servers.
With this approach, you can combine the cost efficiency and reliability of object storage, with the convenience and performance of networked storage. You can even utilize the ultra-low latency of your local storage. All storage is transparently tiered and moved between the layers. A common set of data services, such as thin provisioning, instant snapshots, instant cloning, replication, cyber-security features and deduplication, are available across all of them.
This way we make the best of the latest generation of high-end data center storage systems and architectures and make them available to you, the way you need them. Fully configurable.
The future is now
At simplyblock we already build the future. We believe that the freedom to combine storage requirements according to your needs is the primary concern to start with. Unfortunately, cloud providers have a different agenda, trying to build a solution that works for as many as possible, but also helping the cloud provider’s financial business side.
Anyhow, the technology is available and can be used today. No need to wait for tomorrow. The future is now.
If you want to learn more about how simplyblock works, you must read our “why simplyblock” page, and when you’re ready to try it out, let us know.