With the complex way that AWS (Amazon Web Services) structures their data transfer fees, it can be difficult to approximate your data transfer charges. Many organizations are surprised when they find that their monthly AWS bill is much higher than anticipated.
In particular, AWS cross-region data transfer costs can be difficult to assess due to the different rates and stipulations that apply. To assist you with your cloud cost optimization, we’ve put together this article that will describe how to calculate AWS cross-region data transfer costs and eight ways to reduce them — including by using our object file gateway solution, Resilio Connect.
To learn how to reduce your AWS data transfer costs now, please schedule a demo with us. Our solution engineers will show you a live demo and answer your technical how-it-works questions.
Resilio Connect provides highly efficient file replication and transfer for AWS or any other cloud. It can also be used as a file gateway to efficiently access files stored in the AWS cloud on S3 (or any other object storage platform).
As a cross-region data transfer solution, Resilio uses proprietary deduplication technology to efficiently transfer or synchronize only the changed chunks of a file. The solution scales out so many endpoints can be updated concurrently. So, not only are you reducing the amount of data being transferred, but you can also update many endpoints at once to reduce the total transfer time by up to 20x over conventional AWS transfer tools like AWS DataSync or S3 Replication.
When it comes to moving data to-and-from AWS, Resilio Connect can be used as a storage gateway: it is designed to efficiently retrieve (download, partially sync, or fully sync) files on-demand. This on-demand pull functionality gives you a handy tool to reduce egress traffic, and, thus, the associated costs of data egress. So, when it comes to cloud cost optimization, Resilio has a flexible approach. It gives you control over file movement by:
- Providing full control over data movement and caching so you can minimize egress traffic/costs: You can browse files on-demand and partially or fully sync files on-demand. You can fully or partially download files. You can selectively cache frequently used files (or any files you want) on on-prem devices while keeping infrequently used files in long-term cloud storage — freeing up space on on-prem devices so you don’t have to purchase more expensive equipment and can access frequently used files without getting charged. And, you can control which files are synchronized to which endpoints, as well as which files are stored locally on your on-prem devices.
- Minimizing data transfer during synchronization: Resilio only replicates the changed portions of files, so the minimum amount of data is transferred during syncs. You can also sync data in real-time or sync on a fixed schedule that’s designed to minimize egress traffic.
- Using smart routing: Resilio includes a feature known as smart routing, which enables you to pin traffic to the optimal network.
- Eliminating other cloud storage gateway costs: Resilio Connect is a hardware and vendor-agnostic solution that can be deployed on your existing infrastructure for use with any type of storage, so there’s no need to migrate your data or invest in new hardware. It provides an efficient and cost-effective way to access and manage all of your on-prem and cloud data from a single, unified location — making it an ideal hybrid/multi-cloud storage and disaster recovery solution.
“If we bought about 40 additional servers, it would have been pretty expensive. Resilio is a much cheaper solution for us right now.”
— Mario Jost, Systems Engineer at Mercedez-Benz
(Mercedes Benz AG synchronized 40 European dealership branches, without having to invest in expensive new hardware, using Resilio Connect. Learn more about how Resilio Connect helped them minimize operational expenses and reduce their OS deployment time by 95%.)
Organizations in retail (Exxon Mobil, KFC), gaming (Bungie, Larian Studios), tech (Match, Epson), media (WB, Sunrise Animated Studios) and more use Resilio Connect as an efficient, cost-effective hybrid and multi-cloud storage solution. To learn more about how Resilio Connect can help you minimize data transfer costs, schedule a demo.
What Is AWS Cross-Region Data Transfer?
AWS charges users for data transfer in and out of AWS services over a network. This can occur in various scenarios: data transfer out of AWS services, between AWS services (i.e., from an Amazon S3 bucket to an Amazon EC2 instance), and moving data across workloads, different regions, from/to an on-premises device, and between AWS and another cloud.
It’s important to understand how Amazon charges for these transfers and take those charges into account when constructing your solution in AWS. There are three factors involved in how AWS charges for data transfers:
- Which AWS services you’re transferring to and from (e.g., Amazon RDS, Amazon ElastiCache, Amazon Redshift, AWS Edge, etc.)
- How much data you’re transferring
- The distance the data is traveling (i.e., whether you’re transferring across Availability Zones or AWS Regions).
Cross-region (or inter-region) data transfer occurs when you transfer or access data across AWS Regions, and is generally more expensive than data transfers within the same region. The rates for cross-region data transfers are determined by the AWS services you’re using and the regions you’re transferring to/from.
8 Ways to Reduce Cross-Region Data Transfer Costs
Now that you understand how cross-region data transfer costs are incurred in AWS, you can use these eight methods to minimize data transfer and the associated costs:
1. Monitor Your Data Transfer and Charges
The first and most important step to managing data transfer costs is measuring and monitoring data transfer costs.
You can use the AWS Pricing Calculator to estimate the cost of data transfer in your AWS solution. You can also use a Cloud Intelligence Dashboard to help you visualize your data transfer and the associated charges, which helps you get a better idea of where your greatest expenses are occurring (learn more about how here).
2. Keep Your AWS Solution to as Few Regions as Possible
While there are good reasons for using multiple AWS Regions — such as business continuity (i.e., for disaster recovery), delivering a better user experience (i.e., storing data closer to end-users for quicker access), and legal requirements — you can minimize cross-region data transfer expenses by keeping your AWS applications to as few regions as possible and choosing the least expensive regions.
If business continuity is your aim, you can store your backup data in different Availability Zones within the same AWS Region (unless you’re trying to protect against a disaster that may affect an entire region, such as an earthquake).
As discussed earlier, transferring data out of some AWS Regions is cheaper than others. If possible, select the least expensive regions for your data.
3. Use a CDN, Such as AWS CloudFront
Content delivery networks keep data transfer costs low while also improving the experience for your end-users (by providing them with quicker access to data and applications).
Amazon CloudFront is a cost-effective CDN solution that moves your most-accessed data to the front of the AWS network, so end-users can access the data as quickly as possible. Data transfers from other AWS resources into CloudFront are free, while transfers from CloudFront to the internet are 20-40% cheaper than standard AWS data transfer pricing rates.
4. Use VPC Peering Connections
When connecting to AWS services from within AWS, you should use VPC gateway endpoints — rather than routing traffic over the internet — which enable intra-region communication to Amazon S3 and Amazon DynamoDB without being charged for the data transfer.
5. Use Amazon Elastic Block Store Optimized Instances
Amazon EBS optimized instances are EC2 instances that are optimized for EBS I/O. They help lower data transfer rates by reducing:
- The amount of data transferred over networks by improving the performance of network traffic.
- The time your workloads spend transferring data by enabling your workloads to run more efficiently.
- The amount of time that your workloads spend reading and writing data to and from instances.
6. Use Amazon Direct Connect
Amazon Direct Connect provides dedicated network connections between AWS and on-prem devices. Rather than using the internet, you should use Direct Connect to send data from AWS to your on-prem environment, as doing so minimizes the time your data spends in transit and reduces internet congestion.
7. Use Amazon S3 Transfer Acceleration
Amazon S3 Transfer Acceleration uses the CloudFront global edge network to:
- Reduce the distance your data needs to travel.
- Reduce the number of “hops” (i.e., the number of servers data needs to be transferred to in order to reach the end-user) between end-users and your S3 bucket.
- Decreasing the time your data spends in transit.
- Caching frequently used data at edge locations that are closer to end-users, thereby reducing the total amount of data that needs to be transferred.
8. Use an Efficient Replication and File Gateway Solution That Reduces Data Transfer Costs — Such as Resilio Connect
File gateway solutions allow organizations to store, transfer, and access files stored in the cloud. The way in which these solutions perform those capabilities impacts how much data is transferred and, accordingly, the costs associated with data transfer.
Resilio Connect is an object file gateway that is designed to efficiently store, transfer, and access files in a way that both enhances performance and minimizes cloud storage costs. In the next section, we’ll discuss how Resilio Connect helps you reduce cross-region data transfer costs (as well as other cloud storage costs) in order to save you money.
How Resilio Reduces Cross Region Transfer Costs
Resilio Connect provides an object storage gateway for files stored as objects in any type of cloud or on-premises storage profile. It enables organizations to manage all of their cloud and on-prem storage from a single, centralized location, while providing end-users with easy, low-latency access to files from a unified interface (that operates much like Microsoft Onedrive, so files can be browsed and accessed much as if they were on a desktop).
Resilio Connect is optimized for large, enterprise deployments. Through several capabilities that are unique to Resilio, it enables organizations to enhance performance and workflows (through quicker, more flexible access to data) while decreasing cloud storage costs by efficiently storing, transferring, and accessing files (Resilio works with files of any size, type, and number, but is especially good with large deployments and applications).
Resilio Connect can perform fast and efficient cross-region data transfer, such as region-to-region server replication (i.e., transferring or synchronizing data across as many endpoints or AWS data centers in different regions as needed) and region-to-region storage replication (i.e., replicating S3 buckets that span multiple data centers).
Resilio reduces AWS data transfer costs and assists with cloud storage cost management by:
- Providing full control over data movement so you can minimize egress traffic/costs (i.e., enabling you to selectively download, sync, and cache files as needed).
- Minimizing data transfer during synchronization.
- Managing all of your cloud and on-prem data from a single, unified location.
- Using smart routing.
- Eliminating other cloud storage gateway costs.
And, because Resilio uses a P2P (peer-to-peer) replication architecture and proprietary WAN acceleration technology, it can sync files in any direction, sync files 3-10x faster than traditional solutions, optimize file transfer over any network (including high-latency, loss-prone, low-quality, and consumer-grade networks), and reliably replicate files across your environment (Resilio has no single point-of-failure and it resumes file transfers until they’re complete).
Case Study: Mixhits Radio
Mixhits Radio uses Resilio Connect to synchronize music files for their music streaming service across all of their servers.
“If a music program updates in one location, file changes are detected and propagated across servers within 2 seconds. That rapid update and real-time synchronization has been a saving grace for us.”
Granular Control over Data Movement & Storage: Configure Data Transfer in a Way That Minimizes Costs
Resilio Connect gives you full control over how data is moved and accessed in your cloud and on-prem storage environment through a capability known as Transparent Selective Sync (TSS). The granular degree of control TSS provides enables you to configure your data movement parameters in a way that maximizes efficiency, minimizes costs, and optimizes remote and distributed workforce scenarios.
With TSS, you can browse and partially/fully sync files on-demand. Resilio only syncs changed portions of files, rather than syncing and transferring the entire file, which would incur more data transfer charges.
You can also control which files get synchronized to which endpoints, ensuring files are only delivered to relevant users and that data isn’t transferred unnecessarily. And, you can automate syncs so that employees don’t have to manually sync files and can focus on their work. You can program automated syncs to occur in real-time, or sync on a fixed schedule and create a sync schedule that minimizes data transfer (for example, by synchronizing all files at the end of the day once all changes to files have been made and finalized in order to avoid unnecessary or redundant syncs).
Resilio enables end-users to perform full or partial downloads, so they can download only the portions of files that they need without triggering egress charges (whereas other solutions force you to download entire files and folders). And, while most gateway solutions only cache recently accessed files locally, Resilio enables you to selectively cache any files you choose on local devices — so you can store frequently accessed files locally and store infrequently accessed files in long-term cloud storage. This enables you to free up space on your on-prem devices and access frequently used files without having to download them over the internet and get charged.
Resilio Connect also doesn’t suffer from the limitations of other replication options for AWS — such as AWS Datasync, Amazon S3 Replication, Amazon S3 Batch Operations Copy, and S3 Copy Object API. For example:
- File size limits: The maximum object size that Amazon S3 Batch Operations Copy or S3 Copy Object API can replicate is 5 GB per object. Resilio Connect can replicate objects of any size.
- Replicated file/object limits: AWS Datasync allows you to transfer up to 50 million objects per task (and just 25 million for syncing files between AWS services). Resilio’s engineers successfully replicated 450+ million files in a single job.
Reduce Data Management Costs through Unified Data Management
Resilio Connect is a flexible, versatile solution that you can use to manage all of your on-premise and cloud data from one unified location.
Resilio Connect is a hardware and vendor-agnostic solution that works with AWS and any other cloud storage provider (GCP, Wasabi, Azure, Backblaze, and more). It works with popular operating systems, such as Windows, Linux, Mac, Android, Unix, iOS, FreeBSD, and more. And, it can be deployed on virtual machines and hypervisors, such as Citrix and VMware.
With Resilio, there’s no need to invest in new hardware. You can install Resilio agents on your current IT infrastructure — servers, desktops, networks, and NAS, DAS, and SAN systems.
This kind of versatility means that, rather than investing in different gateways and management tools to manage your data stored in different locations, you can use Resilio Connect to manage all of your data from one centralized interface. This makes Resilio a perfect solution for hybrid and multi-cloud storage environments. Rather than using the proprietary gateways most solutions provide (e.g., AWS File Gateway for AWS, Starcloud for Wasabi, etc.), you can simply use Resilio for every platform — saving you money and making data management significantly easier.
Resilio’s Management Console gives you granular control over your data in every storage environment. You can use the console to:
- Manage S3 buckets and objects.
- Create, manage, and monitor (i.e., receive metrics and real-time updates on) replication jobs.
- Review logs for all replication jobs.
- Check the history of all replications and data transfer jobs.
- Create automated bandwidth policies that control how much bandwidth is allocated to each endpoint. You can create profiles that allot how much bandwidth each endpoint gets access to at certain times of the day and on certain days of the week — so you can optimize bandwidth utilization and control the associated costs.
- Tailor parameters associated with your storage stack, such as disk io threads, data hashing, and file priorities (i.e., which files get transferred/synced first).
- Use Resilio’s powerful REST API to integrate tools that your team is already using and configure any type of scripting and automation that your application requires.
Using Resilio to manage your data not only makes data management easier, it enables you to minimize and optimize all of the costs associated with data management across your entire environment.
Minimize or Eliminate Other Cloud Storage Costs
We’ve discussed the many ways that Resilio enables you to fully control and minimize data transfer costs. But, the way in which Resilio Connect is designed and operates also eliminates or reduces many of the other costs associated with cloud storage.
Native Security: No Need to Invest in Security Solutions
Many other file gateways don’t include native security features, forcing you to invest in 3rd party security solutions and VPNs.
But, Resilio Connect includes built-in security features that protect your data, including:
- End-to-end encryption: Resilio Connect encrypts your data at rest and in transit using AES 256.
- Cryptographic data integrity validation: Data integrity ensures your data always arrives at its destination intact and uncorrupted.
- Forward secrecy: Resilio uses one-time session encryption keys to protect each session.
- Mutual authentication: Before communicating with any endpoint (i.e., transferring data), Resilio requires that endpoint to provide an authentication key — ensuring data is only delivered where it’s supposed to go.
- Permission controls: Resilio gives you granular control over file access, so you can control who gets access to specific files and folders.
No Data Migration Required
Proprietary file gateway solutions only work with data stored in their proprietary cloud platforms. AWS file gateway, for example, only works for data stored in the AWS cloud. If your data is stored in another cloud, you’d need to migrate that data to AWS in order to use their file gateway.
But, Resilio works with any cloud storage service. You can keep your data in your current cloud platform and begin using Resilio without the need to migrate your data to a new platform and incur extra egress charges.
One Gateway for All Data
As discussed earlier, Resilio Connect can manage all of your data stored in all on-prem and cloud environments. So, in hybrid and multi-cloud storage scenarios, rather than investing in and using multiple file gateways to manage your data across different platforms, you can simply use Resilio Connect for everything.
No New Hardware Required
Resilio Connect also works with any type of on-prem storage device, so there’s no need to invest in new hardware.
You can also scale your data environment easily, organically, and cost-effectively. Most other hybrid cloud data storage solutions require you to have dedicated file servers in each office in order to access files. File access with these solutions requires an appliance using SMB (Server Message Block) or NFS (Network File System). Files retrieved from the cloud must first be delivered to these appliances, then to the end-user (who may be accessing them from home on a desktop or workstation). This incurs extra transfer charges.
But, Resilio’s P2P transfer architecture provides global file accessibility directly to each endpoint. Files can be accessed directly by the end-user, resulting in faster delivery of files and reduced data transfer costs.
To learn how Resilio Connect’s file gateway and replication solutions can help you minimize data transfer and cloud storage costs, schedule a demo.
