However, before we dive in, it\u2019s also worth noting that both Amazon S3 Transfer Acceleration and AWS Global Accelerator have some limitations that can\u2019t be avoided. Mainly: <\/p>\n\n\n\n
- \n
- Their data transfer speeds are limited due to their use of point-to-point transfers and TCP.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- They add complexity to your AWS service stack, which can make it much more difficult to manage and estimate your cloud costs.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- They\u2019re limited to the AWS ecosystem. You can\u2019t use either of these services to transfer data or manage traffic to other clouds (and in the case of S3TA \u2014 to other services besides S3).<\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
That\u2019s why, in the second part of this guide, you\u2019ll learn how Resilio Connect<\/a> \u2014 our WAN-optimized data transfer, sync, and file gateway solution \u2014 can help you get many of the benefits of <\/strong>S3TA<\/strong>, Global Accelerator, and other <\/strong>AWS services <\/strong>(like S3 Replication) in a much simpler and cost-effective way.<\/p>\n\n\n\n
In short, we\u2019ll show you how Resilio Platform can help your organization:<\/p>\n\n\n\n
- \n
- Optimize data transfer, sync, and replication speeds across any global AWS region or service<\/strong> (or use any storage for any cloud provider). Resilio Platform lets you transfer files of any size and type across any number of endpoints located anywhere in the world. Our software\u2019s P2P architecture and WAN optimization technology ensure you can do this with near-zero latency to keep files current in real-time across as many endpoints as needed. You can even use Resilio Connect in combination<\/em> with Global Accelerator to achieve extremely fast redirection between or across regions.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Control all aspects of data transfer, replication, file routing, file access, security, and more from a single place. <\/strong>Resilio\u2019s Management Console lets you centrally manage all aspects of file transfer and sync jobs through an easy-to-use console. You can set bandwidth priorities for agents, control and monitor performance, and set notifications. <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Get full deployment and <\/strong>data transfer<\/strong> flexibility.<\/strong> Resilio Platform lets you use any cloud provider (or a combination of providers). You can use our software to ingest data into the Amazon Web Services cloud, move it across regions and services, and transfer it to other providers like Azure and GCP, as well as on-prem storage and hybrid cloud environments.<\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
Companies in industries like gaming (Blizzard), media (Warner Brothers), and digital communications (Cisco) have used Resilio to achieve high-speed, reliable <\/em>data transfer<\/em>, sync, and replication. <\/em>Click here to book a demo with our team<\/em><\/a>.<\/em><\/p>\n\n\n\n
S3 Transfer Accelerator vs Global Accelerator: An Overview of Pros, Cons, and Use Cases<\/h2>\n\n\n\n
S3 Transfer Acceleration (<\/strong>S3TA<\/strong>) speeds up long-distance uploads and downloads to <\/strong>S3 buckets<\/strong>. <\/strong>It\u2019s an S3-specific feature, meaning it\u2019s not relevant in the context of other AWS services, like Amazon EC2, for example.<\/p>\n\n\n\n
S3TA automatically routes data through the closest AWS edge location using the Amazon CloudFront content delivery network and various network protocol optimizations. This speeds up transfers and avoids many of the issues that come with moving data over the public internet. <\/p>\n\n\n
\n![\"Key](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/s3-transfer-acceleration-vs-global-accelerator-4-1024x554.png\")
Source: <\/em><\/strong>AWS\u2019 documentation<\/em><\/a><\/figcaption><\/figure>\n<\/div>\n\n\n <\/div>\n\n\n\nThis makes S3TA a good choice for organizations that:<\/p>\n\n\n\n
- \n
- Have on-prem apps in distributed locations and use the S3 API to store data in a centralized bucket.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Have S3-powered apps and have end users distributed all over the globe.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Regularly transfer high volumes of S3 data between countries.<\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
Note:\u00a0<\/em><\/strong>You can learn more about this feature in our Ultimate Guide to\u00a0S3TA<\/a>.<\/em><\/p>\n\n\n\n
AWS Global Accelerator is a networking service that provides two fixed IP addresses to act as a fixed entry point to your app endpoints <\/strong>(like application load balancers, network load balancers, and EC2 instances). <\/p>\n\n\n\n
Like S3TA, this service also relies on the global network of AWS to deliver more consistent performance, compared to routing traffic over the public Internet. However, Global Accelerator isn\u2019t focused on transferring data between S3 buckets. Instead, its main purpose is to distribute traffic across multiple AWS regions.<\/p>\n\n\n
\n![\"Network](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/s3-transfer-acceleration-vs-global-accelerator-5-1024x654.png\")
Source: <\/em><\/strong>AWS\u2019 documentation<\/em><\/a><\/figcaption><\/figure>\n<\/div>\n\n\n <\/div>\n\n\n\nThis makes it a good choice for organizations that:<\/p>\n\n\n\n
- \n
- Want to manually route traffic to the nearest AWS region or achieve fast failover across regions (unlike S3TA which routes traffic automatically and doesn\u2019t cover the failover use case). <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Want to use custom routing to deterministically route traffic to a fleet of EC2 instances (which is especially useful in the media and gaming industries).<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Have enterprise firewall setups or IoT use cases and want to simplify allowlisting. <\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
Since S3TA and Global Accelerator serve different needs, they\u2019re not mutually exclusive<\/strong>. In fact, organizations often use them together, alongside other AWS services \u2014 like S3 Replication, AWS Snowball, and AWS Direct Connect \u2014 to ensure fast data transfers, maximum bandwidth utilization, and app reliability.<\/p>\n\n\n\n
The Limitations of S3TA and Global Accelerator<\/h2>\n\n\n\n
While S3TA and Global Accelerator are both useful services, they have some inherent limitations that can\u2019t be avoided:<\/p>\n\n\n\n
- \n
- Pricing unpredictability due to complexity. <\/strong>As we said, companies often use a myriad of AWS services to transfer, replicate, and sync their data quickly and efficiently (on top of other services like Lambda, VPC, Route 53 DNS, and RDS, for example). This makes the setup and management process incredibly complex. It also leads to pricing uncertainty, as each service comes with its own unique pricing, on top of the regular AWS egress fees<\/a> and data transfer charges.
<\/li>\n\n\n\n- Limited data transfer rates. <\/strong>S3TA and Global Accelerator rely on a point-to-point (client-server) model and use TCP for all transfers. As a result, the speed of each transfer is limited by the throughput between each endpoint. Plus, TCP often struggles with long-range transfers due to latency and packet loss. The client-server model also has a single point of failure, so one server going down can stop data transfer, sync, or replication across the entire environment.
<\/li>\n\n\n\n- Vendor lock-in. <\/strong>S3TA and Global Accelerator are built to keep users within the AWS cloud. You can\u2019t use them to move data or manage traffic across other cloud providers without buying separate software or spending lots of engineering hours on building your own tools.<\/li>\n<\/ol>\n\n\n\n
<\/div>\n\n\n\nIn the next sections, we\u2019ll show you how Resilio Platform can help you overcome these limitations.<\/p>\n\n\n\n
The Benefits of Using Resilio Connect<\/h2>\n\n\n\n
Resilio Platform is an agent-based, software-only solution that organizations use to:<\/p>\n\n\n\n
- \n
- Achieve lightning-fast data transfer, sync, and replication. Our solution is typically between 3-10x faster than other tools<\/strong>, thanks to its P2P architecture and proprietary UDP-based WAN optimization technology that delivers the fastest transfer speeds across locations over any distance. <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Handle massive data workloads, like transferring many large files to thousands of endpoints all over the world.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Remotely browse and manage transfers and data flows from a single, unified management console.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Synchronize web and app servers.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Backup files from remote offices.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Distribute software builds.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- And much more.<\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
Resilio Platform is also a vendor-agnostic solution.<\/strong> You can deploy it directly on your endpoint devices \u2014 like cloud instances, mobile phones, desktops, and servers \u2014 and manage everything from a user-friendly Central Management Console.<\/p>\n\n\n\n
Below, we\u2019ll discuss four benefits of using Resilio Platform over S3TA and Global Accelerator (or in some cases \u2014 as an addition to Global Accelerator) to transfer, sync, and replicate data.<\/p>\n\n\n\n
1. Speed & Organic Scalability<\/h3>\n\n\n\n
There are two fundamental challenges with the data transfer methods used by S3TA and Global Accelerator: <\/p>\n\n\n\n
- \n
- They\u2019re point-to-point<\/strong> (aka client-server<\/strong>).<\/li>\n\n\n\n
- They\u2019re TCP-based.<\/strong><\/li>\n<\/ul>\n\n\n\n
<\/div>\n\n\n\nWhen using a client-server model, one device acts as a hub that can receive and share data with any of the other devices. However, the clients can only share data with the hub and not with each other. <\/p>\n\n\n\n
As a result, the hub is a single point of failure<\/strong> and the whole process becomes reliant on cloud hopping, as data has to go through Client 1 to the hub, instead of just going from Client 1 to Client 2. Plus, the hub can only transfer data to one client at a time, so if one device has an outage, slow network, or another issue, the transfer to all other devices can be obstructed.<\/p>\n\n\n
\n![\"Client-Server](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/07\/client-server.gif\")
<\/figure>\n<\/div>\n\n\n<\/div>\n\n\n\nThe second issue has to do with the use of the TCP protocol for long-range data transfers. <\/p>\n\n\n\n
TCP works well for Local Area Networks (LANs) but struggles due to a number of factors when operating over distance, <\/strong>as even small levels of latency and packet loss degrade its performance. As a result, companies often can’t make the most out of expensive Wide Area Networks (WANs) and transfer data quickly all over the globe.<\/p>\n\n\n\n
Fortunately, Resilio Connect\u2019s unique P2P architecture and proprietary WAN optimization technology can help you overcome these issues.<\/p>\n\n\n\n
P2P Architecture: Speed, Built-In Scalability, and Reliability<\/h4>\n\n\n\n
Resilio Platform uses a unique P2P (peer-to-peer) architecture <\/strong>that lets all servers in your environment share data with each other. In other words, every server can act as a hub, which drastically speeds up data transfer and replication speeds. <\/p>\n\n\n\n
Resilio Platform also separates files into pieces and transfers them independently of each other <\/strong>(a process called \u201cfile chunking\u201d).<\/p>\n\n\n
\n![\"P2P](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/07\/p2p-vs-client-server.gif\")
<\/figure>\n<\/div>\n\n\n<\/div>\n\n\n\nThis combination of capabilities results in 3-10x faster transfer speeds<\/strong> compared to traditional point-to-point solutions. It also enables Resilio to sync and replicate data in any direction:<\/p>\n\n\n\n
- \n
- One-to-one <\/li>\n\n\n\n
- One-to-many <\/li>\n\n\n\n
- Many-to-one<\/li>\n\n\n\n
- Two-way<\/li>\n\n\n\n
- N-way (aka many to many). <\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
The N-way capability in particular makes Resilio Platform ideal for syncing large numbers of servers<\/strong>, as every server can sync across the entire environment. It\u2019s also great for remote work collaboration<\/strong> because anyone can share file changes with others instantly, regardless of their location.<\/p>\n\n\n
\n![\"Resilio](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/s3-transfer-acceleration-vs-global-accelerator-6-1024x512.png\")
<\/figure>\n<\/div>\n\n\n<\/div>\n\n\n\nBesides being faster, Resilio\u2019s P2P architecture also scales much better than traditional solutions<\/strong>.<\/p>\n\n\n\n
With client-server environments, performance often degrades when you add endpoints and try to move large volumes of data over long distances. Additionally, scaling up these environments is costly, because you need to:<\/p>\n\n\n\n
- \n
- Buy more hub servers as you add more clients.<\/li>\n\n\n\n
- Constantly find ways to balance the network load between the hubs.<\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
Resilio Platform is an organically scalable solution<\/strong>, so the more servers you add, the better it performs. For example, Resilio can synchronize data 50% faster than point-to-point solutions in a 1:2 scenario and 500% faster in a 1:10 scenario.<\/p>\n\n\n\n
Lastly, the P2P architecture makes Resilio Connect ideal for disaster recovery scenarios because it doesn\u2019t have a single point of failure<\/strong>. <\/p>\n\n\n\n
Our software can automatically route around outages and retrieve data from the nearest available device. As a result, data is always retrieved as fast as possible, enabling organizations to meet sub-five-second RPOs (Recovery Point Objectives) and RTOs (Recovery Time Objectives) within minutes of an outage.<\/strong><\/p>\n\n\n\n
Resilio can be used in different disaster recovery (DR)<\/a> scenarios, like:<\/p>\n\n\n\n
- \n
- Cold DR <\/li>\n\n\n\n
- Warm-site DR<\/li>\n\n\n\n
- Offsite copy<\/li>\n\n\n\n
- Hot-site DR<\/a><\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
![\"Hot\/Live](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/04\/hot-dr-warm-dr-cold-dr-and-offsite-copy-699x1024.png\")
<\/figure>\n<\/div>\n\n\n<\/div>\n\n\n\nThanks to its versatility, Resilio Platform can also be used in combination with Global Accelerator<\/strong>. <\/p>\n\n\n\n
For example, say you have file servers in one AWS region and want to sync them across another region. Resilio Platform can help you do that in real-time and achieve sub-five-second RTOs (which means you get \u201chot active\u201d copies of your data in both regions).<\/p>\n\n\n\n
From here, you can rely on Global Accelerator to balance users to either of the regions, depending on your needs. If one region goes down, Global Accelerator can send traffic to the other one, while Resilio continues to keep files synced on the back end. As a result, you get extremely fast redirection between or across regions, resulting in very low recovery times<\/strong>. <\/p>\n\n\n\n
WAN Optimization: Reliable and Fast Long-Range Data Transfer<\/h4>\n\n\n\n
Instead of using TCP, Resilio Platform relies on a proprietary UDP-based transfer protocol called Zero Gravity Transport (ZGT\u2122). This protocol is designed to maximize<\/strong> transfer speed across any<\/em> network and overcome the impact of latency and packet loss.<\/strong><\/p>\n\n\n\n
ZGT does this by:<\/p>\n\n\n\n
- \n
- Using a congestion control algorithm: <\/strong>This algorithm periodically probes the time it takes for the destination to receive a packet (i.e., the Round Trip Time or RTT). As a result, our software can maintain a uniform packet distribution rate by using a fixed packet delay. <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- \n
- Sending out interval acknowledgements and reducing unnecessary retransmission: <\/strong>Resilio Platform sends out acknowledgements after receiving a group of packets (instead of after receiving each packet, like most transfer solutions). It also retransmits lost packets once per RTT to reduce unnecessary retransmissions.<\/li>\n<\/ul>\n\n\n\n<\/div>\n\n\n\n
Put simply, Resilio Platform gives you a proprietary WAN <\/strong>optimization<\/strong> technology for all of your <\/strong>data transfer<\/strong> needs<\/strong> \u2014 including ingestion, distribution, synchronization, and cross-region replication<\/a> \u2014 regardless of your cloud region, cloud provider, and storage solutions.<\/p>\n\n\n\n
You can also use our speed calculator<\/a> to estimate how much time Resilio can save your organization, depending on your use case.<\/p>\n\n\n
\n![\"File](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/06\/file-transfer-time-calculator-example-1024x668.png\")
<\/figure>\n<\/div>\n\n\n<\/div>\n\n\n\nFor real-life examples of the power of our P2P architecture and WAN optimization technologies, check out the three case studies below:<\/p>\n\n\n\n
- \n
- Turner Sports uses Resilio Connect<\/a> to sync the files on the local server in the media truck to their SAN in Atlanta via a WAN.<\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
- Turner Sports uses Resilio Connect<\/a> to sync the files on the local server in the media truck to their SAN in Atlanta via a WAN.<\/li>\n<\/ul>\n\n\n\n
- Sending out interval acknowledgements and reducing unnecessary retransmission: <\/strong>Resilio Platform sends out acknowledgements after receiving a group of packets (instead of after receiving each packet, like most transfer solutions). It also retransmits lost packets once per RTT to reduce unnecessary retransmissions.<\/li>\n<\/ul>\n\n\n\n
- Using a congestion control algorithm: <\/strong>This algorithm periodically probes the time it takes for the destination to receive a packet (i.e., the Round Trip Time or RTT). As a result, our software can maintain a uniform packet distribution rate by using a fixed packet delay. <\/li>\n<\/ul>\n\n\n\n
- They\u2019re TCP-based.<\/strong><\/li>\n<\/ul>\n\n\n\n
- They\u2019re point-to-point<\/strong> (aka client-server<\/strong>).<\/li>\n\n\n\n
- And much more.<\/li>\n<\/ul>\n\n\n\n
- Distribute software builds.<\/li>\n<\/ul>\n\n\n\n
- Backup files from remote offices.<\/li>\n<\/ul>\n\n\n\n
- Synchronize web and app servers.<\/li>\n<\/ul>\n\n\n\n
- Remotely browse and manage transfers and data flows from a single, unified management console.<\/li>\n<\/ul>\n\n\n\n
- Handle massive data workloads, like transferring many large files to thousands of endpoints all over the world.<\/li>\n<\/ul>\n\n\n\n
- Limited data transfer rates. <\/strong>S3TA and Global Accelerator rely on a point-to-point (client-server) model and use TCP for all transfers. As a result, the speed of each transfer is limited by the throughput between each endpoint. Plus, TCP often struggles with long-range transfers due to latency and packet loss. The client-server model also has a single point of failure, so one server going down can stop data transfer, sync, or replication across the entire environment.
- Pricing unpredictability due to complexity. <\/strong>As we said, companies often use a myriad of AWS services to transfer, replicate, and sync their data quickly and efficiently (on top of other services like Lambda, VPC, Route 53 DNS, and RDS, for example). This makes the setup and management process incredibly complex. It also leads to pricing uncertainty, as each service comes with its own unique pricing, on top of the regular AWS egress fees<\/a> and data transfer charges.
- Have enterprise firewall setups or IoT use cases and want to simplify allowlisting. <\/li>\n<\/ul>\n\n\n\n
- Want to use custom routing to deterministically route traffic to a fleet of EC2 instances (which is especially useful in the media and gaming industries).<\/li>\n<\/ul>\n\n\n\n
- Want to manually route traffic to the nearest AWS region or achieve fast failover across regions (unlike S3TA which routes traffic automatically and doesn\u2019t cover the failover use case). <\/li>\n<\/ul>\n\n\n\n
- Regularly transfer high volumes of S3 data between countries.<\/li>\n<\/ul>\n\n\n\n
- Have S3-powered apps and have end users distributed all over the globe.<\/li>\n<\/ul>\n\n\n\n
- Have on-prem apps in distributed locations and use the S3 API to store data in a centralized bucket.<\/li>\n<\/ul>\n\n\n\n
- Get full deployment and <\/strong>data transfer<\/strong> flexibility.<\/strong> Resilio Platform lets you use any cloud provider (or a combination of providers). You can use our software to ingest data into the Amazon Web Services cloud, move it across regions and services, and transfer it to other providers like Azure and GCP, as well as on-prem storage and hybrid cloud environments.<\/li>\n<\/ul>\n\n\n\n
- Control all aspects of data transfer, replication, file routing, file access, security, and more from a single place. <\/strong>Resilio\u2019s Management Console lets you centrally manage all aspects of file transfer and sync jobs through an easy-to-use console. You can set bandwidth priorities for agents, control and monitor performance, and set notifications. <\/li>\n<\/ul>\n\n\n\n
- Optimize data transfer, sync, and replication speeds across any global AWS region or service<\/strong> (or use any storage for any cloud provider). Resilio Platform lets you transfer files of any size and type across any number of endpoints located anywhere in the world. Our software\u2019s P2P architecture and WAN optimization technology ensure you can do this with near-zero latency to keep files current in real-time across as many endpoints as needed. You can even use Resilio Connect in combination<\/em> with Global Accelerator to achieve extremely fast redirection between or across regions.<\/li>\n<\/ul>\n\n\n\n
- They\u2019re limited to the AWS ecosystem. You can\u2019t use either of these services to transfer data or manage traffic to other clouds (and in the case of S3TA \u2014 to other services besides S3).<\/li>\n<\/ul>\n\n\n\n
- They add complexity to your AWS service stack, which can make it much more difficult to manage and estimate your cloud costs.<\/li>\n<\/ul>\n\n\n\n
![\"Storage](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image3-1024x776.png\")
<\/figure>\n<\/div>\n\n\n![\"Resilio](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/07\/resilio-connect-overview-989x1024.png\")
<\/figure>\n<\/div>\n\n\n![\"Security](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image7.png\")
<\/figure>\n<\/div>\n\n\n