AWS Storage Gateway<\/strong> is a <\/strong>hybrid cloud storage service<\/strong> that enables on premises access to virtually unlimited cloud storage by linking it to Amazon S3.<\/p>\n\n\n\n Resilio Platform does this and more: <\/strong>It\u2019s a software-only agent-based solution that runs on any device and reliably transfers data across any type of network, irrespective of bandwidth and latency. Resilio transfers and synchronizes files in real-time or on-demand and in any direction. You can transfer one- and two-way, many-to-one, one-to-many, or many-to-many.<\/p>\n\n\n\n As a storage gateway, Resilio provides a file interface to any type of object storage, including S3. You can selectively sync files on-demand with cloud storage in AWS or any cloud provider \u2014 as well as on-premise storage. This keeps data egress costs low through minimized and efficient transfer. <\/p>\n\n\n\n Resilio agents are installed directly on existing systems rather than provisioning VMs. The solution supports any OS and virtually any cloud storage provider, including AWS, Azure, GCP, MinIO, Wasabi, and more. Resilio’s speed, flexibility, and efficiency make it a superior solution for hybrid cloud, multi-cloud, remote work, data migration, and disaster recovery use cases.<\/p>\n\n\n\n In this article, we\u2019ll discuss how AWS DataSync and AWS Storage Gateway work, when to use one tool over the other, and when to use both. We’ll also explore how Resilio Platform provides a faster, more reliable, and more efficient solution for syncing and accessing data (any type of file, block, and object) stored on premises and in any S3-compatible cloud object storage. <\/p>\n\n\n\n Businesses such as 2K Games, Match.com, Delirio Films, Turner Sports, and more use Resilio Platform for hybrid cloud synchronization. To see Resilio in action, <\/em>schedule a demo with our team<\/em><\/a>.<\/em><\/p>\n\n\n\n AWS DataSync<\/strong> is useful for one-time <\/strong>data migrations<\/strong> from on-prem to the cloud, or for synchronizing data between two endpoints. <\/p>\n\n\n\n However, its point-to-point architecture introduces single points of failure and limits its performance to the slowest endpoint. It also can’t sync data in real-time or sync in multiple directions. These shortcomings make DataSync unsuitable for scenarios that require rapid sync to multiple endpoints \u2014 such as distributed workforces, file distribution, or transfers over high-latency, low-quality networks (such as edge ingest and sync).<\/p>\n\n\n\n AWS Storage Gateway<\/strong> is useful for accessing data<\/strong> stored as files, tapes, and storage volumes in AWS. <\/p>\n\n\n\n But it doesn’t work with other cloud storage providers, forcing you to use another storage gateway to access files in multi-cloud scenarios. It’s also not designed for efficient file access (i.e., minimizing data transfer costs), which can lead to unexpected charges on your monthly AWS bill.<\/p>\n\n\n\n Unlike <\/strong>AWS DataSync<\/strong>, Resilio Platform uses a P2P (peer-to-peer) architecture that provides blazing-fast, reliable, multi-directional sync.<\/strong><\/p>\n\n\n\n Organizations like Sunrise Productions<\/a> use Resilio to enable geographically distributed employees to collaborate on projects in real-time across AWS regions. <\/p>\n\n\n\n Resilio eliminates cross-region latency in AWS<\/a>, so changes made by users in one location are instantly reflected across all other endpoints in 4\u20135 seconds \u2014 roughly 10x faster than traditional point-to-point solutions like DataSync. Resilio can also perform horizontal scale-out replication and reach speeds of 100+ Gbps per cluster.<\/p>\n\n\n\n Unlike <\/strong>AWS Storage Gateway<\/strong>, Resilio provides access to files stored as objects in just about any S3-compatible cloud <\/strong>object storage<\/strong> (not just AWS).<\/strong> It increases efficiency by enabling you to choose which files to cache locally so you can free up space on on-premise servers while giving employees faster access to the files they need.<\/p>\n\n\n\n Overall, Resilio Platform provides file synchronization and access that is:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n Organizations use Resilio Platform to distribute and synchronize data for media workflows (CBS, Turner Sports), gaming (Blizzard, Wargaming.net), remote operations (McDonald\u2019s, Mercedes-Benz), and more. To see for yourself how Resilio Platform can provide blazing-fast synchronization and efficient access to files stored in AWS, <\/em>schedule a demo<\/em><\/a>.<\/em><\/p>\n\n\n\n Below, we describe how AWS DataSync and Storage Gateway work, the scenarios in which they should be used, and when you can leverage both solutions.<\/p>\n\n\n\n AWS DataSync works via Virtual Machine agents that are used to read and write data from storage systems. The agents can sync data from storage machines to Amazon S3, Amazon EFS, or Amazon FSx for Windows File Server (and across storage regions). You can monitor and automate the AWS DataSync migration service through AWS CloudWatch and AWS Lambda.<\/p>\n\n\n\n DataSync also includes a tool known as DataSync Discovery, which provides visibility into on-premise storage performance and utilization to help you simplify and accelerate data migration to the AWS cloud. Through automated data collection and analysis, it provides user-generated recommendations about which AWS storage services suit your needs.<\/p>\n\n\n\n You should use AWS DataSync when transferring data into Amazon S3 and you need:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n DataSync is also an appropriate solution when transferring from on-premise devices or Amazon S3 to AWS Elastic File System, as well as from on-premise blocks or tapes to AWS (as Storage Gateway can\u2019t perform these transfers).<\/p>\n\n\n\n AWS Storage Gateway operates via the Storage Gateway Appliance from Amazon that is hosted on a data center though you can also purchase and use a hardware appliance. Data can be stored in three types of interfaces:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n You should use AWS Storage Gateway if you want to extend your on-premise storage into the AWS cloud and provide on-prem users with low-latency file access.<\/p>\n\n\n\n Storage Gateway is also the better solution in certain data migration scenarios. For example, if you want to migrate files into Amazon S3, you don\u2019t need to filter files and folders, and\/or your team can code your own data validation logic (or you don\u2019t need data validation at all), then Storage Gateway can perform the migration and is 25% less expensive than AWS DataSync.<\/p>\n\n\n\n You can use AWS DataSync to migrate existing data to Amazon S3, and then use the AWS Storage Gateway to access the data sync with your on-premise devices.<\/p>\n\n\n\n Resilio Platform is a file synchronization software system that can also be used as an object storage gateway for files stored in any S3-compatible cloud storage.<\/p>\n\n\n\n If you\u2019re storing data in AWS services other than S3 (such as Amazon EFS, FSx, etc.), you\u2019ll still need to use DataSync and\/or Storage Gateway. But Resilio can provide fast, reliable, scalable data replication across Amazon S3 \u2014 as well as efficient, low-latency access for your on-premise file systems.<\/p>\n\n\n\n Note:<\/em><\/strong> For one-time and continuous <\/em>data migrations<\/em> into AWS from the edge, you should also consider using <\/em>AWS <\/em>Snowball<\/em><\/a> (which includes resources for <\/em>cloud computing<\/em>) and <\/em>AWS Snowcone<\/em><\/a>. <\/em><\/p>\n\n\n\n While you can use AWS DataSync to transfer and sync data in the AWS cloud, it\u2019s a point-to-point solution that suffers from many shortcomings, such as:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n Unlike AWS DataSync\u2019s point-to-point architecture, Resilio Platform syncs data in real-time using a proprietary peer-to-peer architecture that makes it fast, organically scalable, highly reliable, and multi-directional.<\/p>\n\n\n\n Every endpoint in a P2P environment can replicate data simultaneously, enabling Resilio Platform to provide some of the fastest replication speeds available.<\/p>\n\n\n\n While DataSync can only perform scheduled syncs, Resilio Platform can sync on a schedule, in real-time, and manually.<\/strong> Using optimized checksum calculations (identification markers are assigned to each file that changes when a change is made to the file) and notification events from the host operating system, Resilio can immediately detect and replicate changes made to a file in real-time.<\/p>\n\n\n\n Resilio also uses a process known as file chunking to break files down into multiple chunks that can transfer independently from each other. If, for example, you wanted to sync a 10 GB file across your environment, Resilio could split that file into five chunks of 2 GB each. <\/p>\n\n\n\n Endpoint 1 could share the first chunk with Endpoint 2. Endpoint 2 can then share that chunk with Endpoint 3, even before it receives the remaining four chunks. Soon, every endpoint will be working together to sync your system, resulting in sync that is 3\u201310x faster than traditional point-to-point solutions.<\/strong><\/p>\n\n\n\n Because DataSync can only sync point-to-point, it scales poorly. When replicating large amounts of data across many endpoints, a full synchronization of your system will take a long time and performance will be limited by the slowest endpoint.<\/p>\n\n\n\n Resilio\u2019s P2P replication architecture eliminates this issue and makes your environment organically <\/strong>scalable<\/strong>. <\/strong>Since every endpoint can take part in replication simultaneously, adding more endpoints only increases replication speed and available resources.<\/p>\n\n\n\n Resilio can perform horizontal scale-out replication, enabling it to:<\/strong><\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n Since replication can only occur point-to-point with DataSync, if replication between two endpoints fails, then it can delay synchronization for your entire environment.<\/p>\n\n\n\n But Resilio\u2019s P2P environment eliminates these single points of failure, since every endpoint can sync with any other endpoint at any time. If an endpoint or network goes down, the necessary files or services can be retrieved from any other endpoint in your system. <\/p>\n\n\n\n This enables Resilio Platform to provide Active-Active HA for disaster recovery \u2014 it minimizes data loss via backups and active\/passive or active\/active failover strategies. When recovering from a disaster (such as server failure), Resilio can utilize every endpoint in your environment and achieve sub-five-second RPOs and RTOs within minutes of an outage.<\/strong><\/p>\n\n\n Resilio can also dynamically reroute around outages or downed networks in order to find the optimal path and ensure your data always reaches its destination.<\/p>\n\n\n\n DataSync can only sync files one-way and two-way. But Resilio Platform can sync files in any direction, such as:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n One area where Resilio really outshines DataSync is with WAN transfer. WAN networks suffer from high latency and varying degrees of packet loss. When syncing data over WAN networks (such as geographically distributed cloud regions, on-prem devices, or to\/from remote locations), you need a solution that can optimize network utilization.<\/p>\n\n\n\n DataSync optimizes WAN transfers with an AWS-designed WAN transfer protocol that uses incremental transfers, sparse file detection, in-line compression, and multi-threaded connections between the local DataSync agent and the in-cloud components. While these methods reduce the size of data being transferred, they don’t efficiently manage\/optimize traffic over the network \u2014 which is important for maximizing transfer speeds over high-latency and low-quality networks.<\/p>\n\n\n\n Resilio Platform inherently performs incremental transfers (i.e., only replicating changed portions of files) and enables multithreaded transfers. But Resilio also utilizes a proprietary UDP-based WAN acceleration protocol known as Zero Gravity Transport\u2122 (ZGT).<\/p>\n\n\n\n ZGT reduces network traffic through advanced deduplication and transport efficiency to overcome latency and loss. It fully utilizes any network connection to optimize transfers via:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n But Resilio is especially useful for syncing data from edge devices. ZGT enables Resilio to fully utilize any type of network connection, such as VSAT, cell (3G, 4G, 5G), Wi-Fi, or any IP connection. It can reliably ingest data from the edge into the cloud, and immediately sync it across all cloud and on-prem endpoints. Clients such as Northern Marine Group<\/a> use Resilio to sync data across their fleet of geographically distributed sea vessels.<\/p>\n\n\n\n Shifo<\/a> provides over 3,000 remote communities with health data solutions. They use Resilio Platform to sync their servers across remote communities with underdeveloped network connections (such as Uganda). <\/p>\n\n\n\n “You can\u2019t imagine how difficult it was for us [prior to Resilio]: Delays, nothing being synchronized; now I don\u2019t hear any of those problems. Now we can focus on solving healthcare problems [instead of moving files].\u201d <\/em><\/strong><\/p>\n\n\n\n Learn more about how Shifo uses Resilio Platform to reliably sync data to remote, poorly-connected areas.<\/em><\/a><\/p>\n\n\n\n Resilio Platform also works as a storage gateway, allowing you to ingest, sync, and access files \u2014 through SMB (Server Message Block) and NFS (Network File Sharing) protocols \u2014 stored in any S3-compatible cloud object storage (such as AWS).<\/p>\n\n\n\n Many organizations that use AWS data storage don\u2019t properly calculate storage fees and are surprised by large bills due to egress charges. AWS users are charged egress fees within and across AWS cloud regions, as well as out of AWS. <\/p>\n\n\n\n\n
\n
\n
\n
\n
\n
\n
\n
AWS DataSync vs. Storage Gateway<\/h2>\n\n\n\n
AWS DataSync<\/h3>\n\n\n\n
AWS DataSync Use Cases<\/h4>\n\n\n\n
\n
\n
\n
\n
Storage Gateway<\/h3>\n\n\n\n
\n
\n
\n
AWS Storage Gateway Use Cases<\/h4>\n\n\n\n
When to Use Both AWS DataSync and Storage Gateway<\/h3>\n\n\n\n
How Resilio Provides Superior File Sync and Access<\/h2>\n\n\n\n
![\"Person](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/06\/TPN-banner-v3-1024x626.png\")
<\/figure>\n\n\n\nP2P Architecture: Fast, Reliable, Scalable Replication<\/h2>\n\n\n\n
\n
\n
\n
Blazing Fast Replication and Synchronization<\/h3>\n\n\n\n
![\"GIF](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/09\/image2.gif\")
<\/figure>\n\n\n\nOrganic Scalability<\/h3>\n\n\n\n
\n
\n
\n
Bulletproof Reliability<\/h3>\n\n\n\n
![\"Hot\/Live](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/04\/hot-dr-warm-dr-cold-dr-and-offsite-copy-699x1024.png\")
<\/figure>\n<\/div>\n\n\nMulti-Directional Replication<\/h3>\n\n\n\n
\n
\n
\n
\n
\n
WAN Acceleration: Fully Utilize Any Network<\/h2>\n\n\n\n
\n
\n
\n
![\"Resilio](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image2-e1693947776257.png\")
<\/figure>\n<\/div>\n\n\nCase Study: Shifo<\/h3>\n\n\n\n
Efficient, Low-Latency File Access<\/h2>\n\n\n\n
![\"How](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/04\/always-keep-on-this-device-option-761x1024.png\")
<\/figure>\n\n\n\n![\"Resilio](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image3-1024x776.png\")
<\/figure>\n\n\n\n![\"Deutsche](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/09\/Deutsche-aircraft-1.jpg\")
<\/figure>\n\n\n\n![\"Mutual](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image7.png\")
<\/figure>\n<\/div>\n\n\n