The hunger for analytics and the popularity of the cloud is profoundly straining traditional storage architectures. The huge costs required to scale linearly with legacy storage appliances are untenable, not to mention the time it would take to scale to the level demanded by today’s digitized enterprise.
What’s more, even adding multiple servers could not accommodate storage demands. Vertical storage architecture contains bottlenecks that slow performance to an unacceptable level.
Because software-defined storage (SDS) decouples the programming that controls storage-related tasks from the physical storage hardware, it dramatically reduces costs associated with hardware. Fewer, less-expensive servers can be used to improve both capacity and performance. Administration is simplified and made more flexible and efficient. SDS enables users to allocate and share storage assets across all workloads.
Understandably, the industry has embraced SDS. Gartner recently reported that by 2020, anywhere from 70 to 80 percent of unstructured data will be stored and managed on lower-cost hardware supported by software-defined storage.
File Systems Are Critical
Storage solutions that offer file systems currently represent 80 percent of the market. That’s because 80 percent of data is unstructured. While it is widely understood that unstructured data is best managed with a file system, for some reason, many SDS offerings focus solely on block or object storage. Few focus on file systems, or do them well. Without a file system overlaying this data, it becomes very difficult to manage that data.
Of the three flavors of storage—block, object and file system—each has its purpose:
- Block is the basic foundation of storage. It is used for storing virtual machines or databases, but you need files as well to deal with all the unstructured data.
- Object storage is the exciting new kid on the block. Object is used for machine-to-machine/IoT transactions and other applications that require extreme scalability, but it isn’t that much better than block when it comes to managing data.
- File systems are the bread and butter of storage. It isn’t as hyped as object, but it is the best at handling unstructured data.
Because software-defined storage providers know that file systems are important, some claim to provide file systems with their offerings. However, these file systems are usually based on Samba, and exclude some features most Windows users are used to.
Many in need of a file system have turned to Samba, the freeware module that enables support for SMB and allows end users to access and use files on the company’s intranet or network. However, providing file services through Samba, which is open source, often means going without needed features.
The importance of features can’t be underestimated. It’s not just the file system that organizations need in order to deal with unstructured data; file-related features are also necessary. These include:
- Snapshot: A snapshot is a read-only copy of the contents of a file system or independent file set taken at a single point in time. When a snapshot of an independent file set is taken, all files and nested dependent file sets will be included in the snapshot.
- Quota: You can use quotas to help monitor the amount of storage being used. You can set a soft limit quota that will warn you when part of a file system is close to reaching its storage limit but still allow data to be saved. If you set up a hard limit quota, after the quota is reached, no new data can be saved.
- Retention: This feature enables you to automatically create a single folder or a hierarchy of folders on file servers, to be deleted according to assigned policies.
- Tiering: A policy enables you to designate where a specific file is to be placed and if and when the file will be migrated between file system pools. You can define both file placement and migration policies. By using a policy you create a filter which designates a specific file type to a particular tier. Tiered storage is more efficient and boosts performance.
It’s an odd position to be in: needing a file system to manage unstructured data, which makes up 80 percent of your pool, but realizing that most SDS providers offer only a token file system. Open source options offer limited feature sets. The perfect solution is a unified approach that includes object, block and file system – a solution that can handle all storage types. As you explore file system approaches, do your due diligence to make sure you are getting one that provides the full-featured storage your organization needs.
For more in-depth information, check out Aberdeen’s comprehensive report on how Best-in-Class organizations are leveraging storage innovations to boost their business.
Stefan Bernbo is the founder and CEO of Compuverde. For 20 years, Stefan has designed and built numerous enterprise-scale data storage solutions designed to be cost effective for storing huge data sets.