IDC analysts are expecting global datasphere to grow to 175 zettabytes by 2025.
Mobile devices, internet, social media platforms, communication channels, and online shopping produce colossal amounts of data each day. The volume of data that’s generated and is being transferred is staggering and is accelerating at an unprecedented rate.
Gartner predicts that more than half of major new business processes and systems will incorporate some element of the internet of things (IoT) in their organization by 2020. And with that, the amount of data generated, transferred to be analyzed by Big Data applications and stored is bound to grow at a mind-boggling rate.
Due to this upsurge in big data and increased cloud adoption across organizations, the demand for scalable storage solutions that are capable of handling and archiving increased digital content is increasing. Hence, the cloud storage solutions’ market is seeing an upward trend.
Cloud storage market growth
Global cloud storage market is expected to grow at a CAGR of 21.9% and reach $207.05 billion by 2026, according to a recent research by Research and Markets.
More and more businesses are moving their critical IT infrastructure and data to the cloud, as storing data on cloud allows them to store files online and access them from any location via the internet. Businesses get three main cloud storage solutions to select from:
Public cloud storage service – suitable for unstructured data
Private cloud storage service – for more control over data and
Hybrid cloud storage service – for increased flexibility
Public cloud storage is the most widely adopted storage solution by the enterprises as both hardware and network are maintained by the cloud storage service provider, hence no maintenance worries for the enterprises.
There are hundreds of different cloud storage providers in the industry offering different solutions, but choosing the best cloud storage platform to store your data, depends on your business needs.
Read on to take an informed decision.
How to choose the best cloud storage service for my business?
- Determine how much space you need and what will be the frequency of your data storage i.e. daily, weekly, monthly etc.
- Determine characteristics you need in your cloud storage – scalability, control, bandwidth, multi-tenancy, latency, data availability and more.
- Determine the type of storage architecture your applications/instances require i.e. block storage or file storage or object storage.
There are many leading players in the global cloud storage service market, but the four leading ones are Amazon Web Services (AWS), Microsoft, Google and IBM. All of them have a variety of storage options. Each service has its own pros and cons depending on the purpose of using it and on the specific use case.
We have compared the block, object and file storage services in this article. Let’s delve into detailed public cloud providers’ comparison on the basis of their cloud storage offerings.
Comparison of AWS vs Azure vs Google Cloud vs IBM cloud storage options
Object based storage, as it is called, is a hierarchy free method of storing data as discrete objects, where each object includes a variable amount of metadata, the data itself and a unique identifying name that an application uses to retrieve the data.
Object storage is ideal for unstructured data. It offers metadata characteristics and is massively scalable.
Cloud storage providers offer three tiers of object storage, classified by how frequently data is accessed: hot for instantaneously accessible data, cool for infrequently accessed data and cold for archival and rarely accessed data.
The colder the storage, the cheaper it is.
Object Storage – Overview
|Service Name||S3||Azure Blobs||Google cloud storage||IBM Object storage|
|Hot||S3 Standard||Hot Blob storage||GCS||Standard|
|Cool||S3 Standard-Infrequent access||Cool Blob||Nearline||Vault|
|Cold (Archival)||Glacier||Archive Blob||Coldline||Cold Vault|
|Object limits||Unlimited||Unlimited||Unlimited||Unlimited perbucket,100 bucket limit|
|Object Size Limit||5 TB||4.75 TB (Block blobs)||5TB||10 TB|
|Encryption at Rest||Available SSE 256-bit AES||Default SSE 256-bit AES||Default SSE 256-bit AES/AES-128||Default SSE 256-bit AES|
|Encryption in transit||Uses CSE/TLS||Uses CSE/TLS||Uses CSE/TLS||Uses CSE/TLS|
|Pricing||AWS S3||Azure Blobs||Google cloud Storage||IBM Cloud Object storage|
AWS Object Storage
AWS offers different range of storage classes depending upon the use cases. S3 is the primary object storage platform of AWS. It offers S3 Standard-Infrequent Access for cool storage and Glacier for cold storage.
- Amazon S3 Standard– It is the storage option for frequently accessed data and is ideal for use cases such as cloud applications, dynamic websites, content distribution, gaming, and data analytics. It delivers low latency and high throughput.
- Amazon S3 Standard – Infrequent Access (Amazon S3 Standard – IA)–Storage option for less frequently accessed data like long-term backups and disaster recovery.
- Amazon Glacier– This storage system is highly durable and optimized for infrequently accessed data, or “cold data like end-of-lifecycle, compliance, or regulatory backups”. Here data is stored in archives for long term storage and is encrypted and immutable.
Azure Object Storage
Azure Blobs is the Microsoft’s object storage solution for the cloud. Blob storage is optimized for storing any type of unstructured (text or binary) data – images, videos, audio, documents and more. Azure storage features mutability, flexibility and data integrity of superior quality.
Blob storage is ideal for storing files for distributed access, serving images or documents directly to a browser, streaming video and audio, writing to log files, storing data for backup and restore, disaster recovery, and archiving, for analysis by an on-premises or Azure-hosted service.
Azure offers different storage tiers:
- Hot access tier: For data that is expected to be or in active use and staged for processing and eventual migration to the Cool storage tier.
- Cool access tier: This tier is intended for data that will remain in the Cool tier for at least 30 days. Short-term backup and disaster recovery datasets, older media content expected to be available immediately when accessed and large data sets.
- Archive access tier: This tier is intended for data that will remain in the Archive tier for at least 180 days and has capacity to tolerate several hours of retrieval latency.
Note: The Archive storage tier is only available at the blob level and not at the storage account level. Azure also offers Premium tier, ideal for workloads that require fast and consistent response times.
Google Cloud Storage
Google Cloud Storage (GCS) offers unified object storage for any workload. It offers four classes under high performance object storage and backup, and archival storage solutions. All four classes offer low latency and high durability.
- High performance storage (Hot): GCS offers multi-regional and regional storage for high frequency access data.
- Multi Regional storage enables storing of data that is frequently accessed around the world, such as serving website content, streaming videos, or gaming and mobile applications.
- Regional storage enables frequent access to data in the same region of Google Cloud DataProc or Google Compute Engine instances, like for data analytics.
- Archival Cloud storage (Cool & Cold): It offers Nearline and Coldline storage for low and lowest frequency access data.
- Nearline storage: Cool storage i.e. for the data that a business expects to access less than a month, but multiple times throughout the year. Ideal for back-up and serving long-tail multimedia content.
- Coldline for cold storage i.e. for data that business expects to access less than once a year. Best for disaster recovery, or data that is archived.
IBM Cloud Object Storage
IBM cloud offers flexible and scalable cloud storage with policy- based archive capability for unstructured data. This resilient cloud storage service is designed for data archiving i.e. for long term retention of infrequently accessed data, for analytics and backup, and web and mobile applications.
IBM offers four storage class tiers that come integrated with Asperaâ high-speed data transfer option that enables easy data transfer to and from Cloud Object Storage, and query-in-place functionality, making data analytics work for you.
IBM Cloud Object Storage class tiers:
Standard Storage for active workloads requiring high performance and low latency, and data that needs multiple and frequent access within a month. Usage scenario includes active content repositories, web content and mobile streaming, DevOps, analytics, and collaboration.
- Vault Storage for less active workloads requiring immediate, real-time access but infrequently, once a month or even less. Backup and digital asset retention are the common use cases for Vault Storage.
- Cold Vault for cold workloads, where data requires immediate, real-time access on demand but is primarily archived i.e. accessed a few times a year. Long-term backup, large data set preservation like scientific data, or older media content are the common use cases.
- The Flex Storage class tier is used for dynamic workloads (mix of hot and cold workloads) data based on access patterns. Common use cases include cloud-native analytics and cognitive workloads and user-generated apps.
File based storage, stores data in a hierarchical structure, and saves data in files and folders. There are many workloads or applications that rely on shared file systems i.e. they need a file system and access to shared files.
Cloud file storage is the service which enables data storage in the cloud and has an architecture based on common file level protocols like NFS (Network File System) and SMB (Server Message Block). It provides shared data access to servers and applications through shared file systems.
Cloud file storage is considered ideal for unstructured and semi structured data like spreadsheets, presentations and other file-based data and workloads like large content repositories, media stores, development environments and user home directories.
File Storage – Overview
|Service Name||EFS||Azure File Storage||Cloud file store||IBM File storage|
|Storage size||Petabytes of data||2 PB for US and Europe, 500 TB per storage account for all other regions including UK Max size of file shares= 5 TiB||63.9 TB per share||Storage (file share) in capacity up to 12 TB|
|Throughput||50 MB/s per TB of storage (can burst to 100 MB/s)||60 MB/s per file share||100 MB/s to 700MB/s per TB||Up to 48 IOPS|
|Replication||Multiple AZs in the region||LRS or GRS or ZRS||NA||Scheduled snapshots|
|Data at rest Encryption||Yes||Yes||Yes||Yes|
|Backup||EFS-to-EFS Backup Solution||Azure backup||Backups, snapshots, and instance failover aren’t available for Cloud Filestore||Snapshots|
|On premise support||(NFSv4.1 and NFSv4.0||SMB 3.0||NFSV3||NFS& SMB|
|Pricing||AWS EFS||Azure File Storage||Cloud Filestore||IBM File Storage|
AWS File Storage – EFS
Elastic File System (EFS) is the scalable file storage solution for AWS cloud. Amazon EFS is fully managed, easy to set up and has elastic storage capacity – your applications get on demand storage, whenever they need it. It is built to provide low latency, throughput, IOPS – all three needed for a broad range of workloads.
It provides file locking and strong data consistency and supports the Network File System version 4 (NFSv4.1 and NFSv4.0) protocol, i.e. integrates well with all the applications and tools of today.
With Amazon EFS, you get scalable file storage for use with Amazon EC2 and redundant data storage capacity across multiple availability zones. EFS also provides encryption support for file systems both in transit and at rest.
Elastic File Storage usage scenarios
Its general-purpose performance mode is ideal for latency-sensitive use cases, like web serving environments, content management systems, home directories, and general file serving.
Amazon EFS also supports developer tools, enterprise applications, container storage, media and entertainment processing workflows, database backups, and big data analytics workloads.
Azure File Storage
Azure Files is the Microsoft’s file storage service which offers fully managed file shares in the cloud. These shares are accessible from anywhere via the industry standard Server Message Block 3.0 (SMB) protocol.
Azure File Storage is cross platform and application compatible. It allows applications to mount file shares from anywhere, worldwide both from on premises deployments- Windows, Linux, and macOS or cloud.
Azure File Storage enables modern applications’ development via REST API protocol. You get hybrid flexibility with Azure File Sync which allows caching and synchronization of Azure File shares on Windows Servers for regional access.
Azure File Storage usage scenarios
Azure Files is designed to support a variety of different use cases-replace or supplement on-premises file servers, “lift and shift” applications to the cloud, simplify new cloud development projects like logs writing by cloud applications, shared application settings access and more.
Google File Storage
Cloud Filestore is Google’s managed file storage service for those applications that require a shared file system and a file system interface. It is in beta phase at present.
Google offers a fully managed network attached storage (NAS) service in the cloud.
It offers low latency, high throughput and high IOPS for file-based workloads, without effecting performance. GCP Filestore gives its users the freedom to tune their filesystem for a particular workload and mount Filestore file shares on Compute Engine VMs.
Google Cloud Filestore is integrated with Google Kubernetes Engine and with the rest of the Google Cloud portfolio.
Cloud Filestore is available as a storage option in the Google Cloud Platform (GCP) console.
Google file storage usage scenarios
Home directories, rendering workflows, applications migrations, web content management, media processing.
Service is in beta phase as of now.
Google Cloud Storage FUSE- an overview
FUSE is an open-source tool of Google which helps in the mounting of Cloud Storage buckets as file systems on Linux or macOS systems. With it, you can upload and download Cloud Storage objects vis standard file system semantics.
Despite having a file system interface, Cloud Storage FUSE is not like an NFS or CIFS file system on the backend.
IBM File Storage
IBM offers flash backed NFS based file storage which is fast and durable. IBM Cloud File Storage offers endurance tiers for general purpose workloads. It offers flexible IOPs customization i.e. its users can scale storage capacity as per workload demands and control total IOPs per storage volume.
There is a provision to create file shares in granular increments (from 1000 GB to 12000 GB) and also performance provisioning. IBM offers per-gigabyte pricing tiers up to 48k IOPS for their file storage with features including flash-backed storage, snapshots and replication, encryption for data at rest, volume duplication, expandable volumes and adjustable IOPS.
(These features are limited to the US, EU, Australia, Canada, Latin America and Asia Pacific regions)
Block storage is the data storage used in storage-area network (SAN) environments where data is stored in volumes, called blocks. Each volume or block acts like an individual hard drive, configured by the server administrator.
Database storage, File systems, RAID arrays are the common use cases for block storage.
Each of the providers break their block storage offerings into two categories: traditional magnetic spinning hard-drive disks, or newer solid-state disks (SSD), which are generally more expensive but have better performance.
AWS Block Storage-EBS
Amazon web services EBS (Elastic block storage service) is the block level storage for the long term persistent and quickly accessible data and to be used with EC2 instances. These Elastic Block Store volumes can be integrated with any instance that is running and is in the same Availability Zone.
AWS EBS offers two types of volumes: SSD – backed storage (next generation high performance drives) for transactional workloads and HDD – backed storage (traditional magnetic) for throughput intensive workloads.
- Throughput Optimized HDD designed for frequently accessed, throughput intensive workloads, Big data (Hadoop/HDFS ecosystem and Amazon EMR clusters), data warehousing applications (Vertica and Teradata), log and stream processing (Kafka and Splunk).
- Cold HDD designed for less frequently accessed workloads, colder data requiring fewer scans per day.
- General Purpose SSDfor a wide variety of transactional workloads, boot volumes, low-latency interactive apps, dev & test.
- Provisioned IOPS SSDdesigned for latency-sensitive transactional workloads, I/O-intensive NoSQL and relational databases like Oracle, Microsoft SQL Server, PostgreSQL and MySQL or Cassandra and MongoDB.
Azure Block Storage
Azure Managed Disks is the Block storage offering of Microsoft Azure. Azure creates and manages disk of your choice for Azure IaaS VMs. It manages storage accounts used for VM disks.
Azure Block storage offers four types of Managed Disks:
- Ultra SSD Managed Disks for most demanding workloads and extremely scalable performance, for data-driven applications, such as: SAP HANA, SQL Server, transaction-heavy workloads, complex analytical modeling, gaming, rendering and low queue-depth databases.
- Premium SSD Managed Disks for production and performance sensitive workloads, such as SQL Server, Oracle databases, Microsoft Dynamics, Microsoft Exchange Server, MySQL, Cassandra, MongoDB and SAP Business Suite.
- Standard SSD Managed Disks for cost-effective and consistent performance. Used for web servers, low-IOPS application servers, lightly-used enterprise applications and dev/test scenarios.
- Standard HDD Managed Disks for VMs running latency-insensitive workloads, for backup and archiving applications, for noncritical workloads and when production-level performance is not required.
Google Block Storage
Google Persistent Disk is the block storage for Google Cloud. It also offers both SSD and HDD storage for high throughput workloads and latency sensitive workloads. This storage can be integrated to instances that are running in either Google Compute Engine or Google Kubernetes Engine.
Its offerings include:
- Zonal persistent disk and Zonal SSD persistent disk.
- Regional block storage: Regional persistent disk and regional SSD persistent disk.
- Local SSD: High performance transient local block-storage.
- Cloud storage buckets: Affordable object storage.
IBM Block Storage
IBM Cloud offers iSCSI-based persistent Block Storage. This flash-backed storage is deployable and customizable from 25 GB to 12,000 GB capacity up to 48,000 IOPS. It can be provisioned and managed independently of compute instances.
IBM Block storage offers two IOPS provisioning options:
Endurance: It is designed to support various application needs. It offers pre-defined performance levels and features like replication and snapshots. There are four IOPS performance tiers for different application requirements:
- 25 IOPS per GB for workloads with low I/O intensity.
- 2 IOPS per GB for most general-purpose usage.
- 4 IOPS per GB for higher-intensity workloads.
- 10 IOPS per GB for the most demanding workloads.
Performance: This type of block storage is designed to support high I/O applications. It offers various IOPS rates (100 – 48,000) that can be provisioned with storage sizes ranging from 20 GB to 12 TB.
Block /Disk Storage – Overview
|Service Name||EBS||Managed Disks||Persistent Disk||Block Storage|
|IOPS/GB for SSD||GP2 SSD=3 to 10,000
Provisioned = 50 to 32
|Premium=80, 000 per second
Standard SSD= up to 500 to 2000 per disk
Ultra SSD= 100 to 16,000 per second
|Endurance tiers= .25 to 10 IOPS available up to 48k, with 12-TB Endurance volume|
|Availability SLA||99.99%||99.99% #based on the SLA of the underlying storage used and virtual machine to which it is attached.||99.95%||NA|
|Encryption at Rest||Available SSE 256-bit AES||Default SSE 256-bit AES||Default SSE 256-bit AES or AES-128||Default SSE 256-bit AES|
|Encryption in transit (instance to volume)||Yes||Yes||Yes||NA|
|Pricing||AWS EBS||Azure Managed Disks||Google Persistent Disk||IBM Block storage|
Please feel free to suggest more points to help us provide complete information.
Disclaimer: The information contained in this article is for general information purpose only. Product features are subject to change. This information has been sourced from the websites and relevant resources available in the public domain of the named vendors as on 20th November, 2018. Daily Host News makes best endeavors to ensure that the information is accurate and up to date, however, it does not warrant or guarantee that anything written here is 100% accurate, timely, or relevant to the website visitors.