Microsoft Azure File Sync is a software-only solution for replicating and syncing Windows file servers with Azure Files. And while it works well for small-scale sync jobs, organizations with larger, more complex replication and sync requirements that need to sync at scale may want to consider an Azure File Sync (AFS) alternative (or complementary solution that can fill in the gaps).
Another big reason to consider an alternative to AFS is the need for multiple cloud solutions (AFS only works with Azure) and/or to maximize utilization of your existing IT environment (servers, storage, networks, etc.).
AFS’s largest challenges include:
- Azure only: AFS is not multicloud, meaning you can’t use it with other cloud solutions. You can only use it with Azure.
- Lack of flexibility: AFS can only be deployed on Windows, Linux, and macOS servers. And for cloud storage, it’s limited to Azure file storage. Businesses that are already using other storage solutions will have to migrate or use a more flexible sync solution that can be deployed on the storage they’re using.
- Slow sync topology: Like most file sync solutions, AFS uses a point-to-point file transfer topology. Transferring files point-to-point creates a bottleneck effect; if one point fails, the whole sync job grinds to a halt.
- Complexity of file sync management: Managing synchronization with AFS is easy for simple, homogenous sync environments. But as your sync environment grows — i.e., syncing large files, syncing more files, syncing to many endpoints — managing sync becomes more complex (i.e., requires more management endpoints).
- Poor scalability: AFS doesn’t handle sync for large deployments well. Not only does it require more management endpoints, but, with its point-to-point transfer architecture, AFS has trouble syncing large deployments in a timely manner.
- Poor efficiency: When a file is synchronized, only the changed portion of the file should be synchronized — and it should be synced efficiently across as many endpoints as needed without extra overhead. There should be options to control this — via schedule, on-demand, or real-time sync. AFS only syncs changes once every 24 hours in the Azure Cloud.
In this article, we’ll discuss several sync solutions that can be used as an alternative to or in conjunction with AFS — starting with a deep dive into our own file sync solution, Resilio Connect.
Resilio Connect is an agent-based replication solution that uses a P2P (peer-to-peer) transfer topology and WAN optimization to deliver fast replication (performance of 10+ Gbps per server) that always “just works” over any network (no matter how unreliable). It can be managed using a centralized Management Console, handle large replication jobs with ease (no limits on file size, and can easily sync large numbers of files and syncs quickly to many endpoints), and flexibly deployed on a variety of servers and/or any cloud storage platform. Organizations in media (Turner Sports, Warner Brothers), gaming (2K Games, Blizzard), tech (Match.com, Cisco), retail (McDonald’s, Mercedes-Benz), and more use Resilio Connect to automate sync jobs and enhance business workflows.
To see if Resilio Connect could be a good fit for you, schedule a demo or learn more here.
1. Resilio Connect
Resilio Connect uses P2P file replication and a proprietary WAN optimization transfer protocol to optimize file transfer and provide scalable, high-performance synchronization in any environment. It was designed for ease of management — it doesn’t require sync groups, needs fewer overall management tasks, and enables admins to monitor, manage, and automate all sync jobs in real-time from a single dashboard.
Resilio is also a flexible multicloud solution that runs on popular servers (in virtual machines, containers, or on physical systems) and uses any type of storage (NAS, SAN, DAS, or object storage). Moreover, Resilio Connect can be deployed anywhere: on-prem, in-cloud, or in a hybrid cloud. Unlike AFS, Resilio supports any cloud platform (including AWS FSx, Azure Files, Azure NetApp Files, and more). And because of its P2P architecture, Resilio scales easily and fluidly to handle deployments of any size (no limits on file size and tested at 400+ million files in a single job) without any complex management procedures.
Resilio Connect is an excellent alternative (or supplemental) solution to AFS due to its:
- High-performance replication: Resilio’s proven P2P file transfer technology & WAN acceleration enable it to synchronize files in real-time, in any direction 3-10x faster than traditional solutions.
- Organic scalability: Resilio has no limits on file sizes, can sync 250+ million files in a single job, and can sync to hundreds of endpoints in about the same time that it takes to sync two.
- Simplicity and ease of management: Managing, monitoring, controlling, and automating sync jobs is easy with Resilio’s centralized Management Console.
- Flexibility: Resilio can be deployed in a variety of configurations using a variety of server and cloud platforms. There’s no vendor lock-in and no need to migrate to a specific platform.
High-Performance Replication
Resilio’s high-performance replication capabilities are due in large part to two features:
- Real-time P2P replication
- WAN optimization
Real-Time P2P Replication
AFS, like most file transfer solutions, uses point-to-point transfer — either client-server transfer (where files are first sent to a hub server, which then replicates those files to every other server one by one) or “follow-the-sun” transfer (where one server replicates files to another server sequentially).
Point to point replication is slow, particularly when replicating large files or large numbers of files because replication must occur over and over again to each server sequentially. They also create bottlenecks — i.e., if replication fails or is impeded on one server, every other server must wait to receive files.
Resilio Connect’s replication technology is built on a proven peer-to-peer architecture that replicates files faster and more reliably than point-to-point systems. In a peer-to-peer system, every server can take part in replication at the same time.
For example, imagine you’re replicating between 10 servers, and Server 1 needs to sync a file with the other 9 servers. Resilio will break that file up into several chunks — using a process known as file chunking — so Server 1 can begin sharing those chunks with Server 2. Once Server 2 receives the first chunk, it can begin sharing it with another server even before it receives the rest of the file. In this environment, every server can take part in replication simultaneously, resulting in replication speeds 3-10x faster than traditional solutions.
Resilio Connect can also replicate files in real time. It uses notifications from the host OS and optimized checksum calculations (i.e., each file is assigned an identification marker that changes when a change is made to the file) to immediately detect and replicate file changes.
WAN Optimization Protocol
Resilio Connect also utilizes a proprietary WAN optimization protocol known as Zero Gravity Transport™ (ZGT) to accelerate file transfer over high-latency, lossy WAN networks.
Whereas most protocols send file packets at a fixed rate, ZGT uses a congestion control algorithm to constantly calculate the ideal send rate and create a uniform packet distribution over time. And rather than sending acknowledgements for every packet receipt, ZGT sends acknowledgements for groups of packets (interval acknowledgements) and retransmits lost packets once per RTT (round trip time) to decrease unnecessary retransmissions.
ZGT is also capable of dynamically rerouting around slow or failed networks in order to ensure files are successfully transferred to their destination. With the combination of P2P transfer and WAN optimization, you can fully utilize 100% of network bandwidth across any WAN.
And for deployments that require synchronization at the edge of networks (such as syncing footage filmed in a remote location or syncing data while at sea), ZGT enables you to sync across WANs in areas with limited network coverage.
Case Study: Northern Marine Group
Northern Marine Group uses Resilio Connect to synchronize data across their geographically distributed fleet of sea vessels.
“With Resilio, we can now proactively monitor these systems, validate ones that are likely to cause problems and fix them. So, we’re not just using it to keep these vessels up to date. We’re also using it to remotely administer these environments.”
Organic Scalability
One of the major benefits of Resilio’s P2P architecture is the ability to seamlessly scale your replication environment as your replication needs grow.
According to Microsoft’s deployment guide on AFS, it is tested with 100 million items (files and folders) per share. But they recommend keeping the number of items below 20 million or 30 million in a single workload. You’ll need to split your namespace into multiple shares if you start to exceed these numbers — though you can continue to group multiple on-premises shares into the same Azure file share if you stay roughly below these numbers.
But with Resilio Connect, there are no limits on file sizes for file transfers. And Resilio’s engineers successfully tested Resilio’s ability to sync 250+ million files in a single job.
As your replication environment grows, Resilio’s performance increases. Since every server can take part in replication, adding more servers only increases replication speed and bandwidth. In other words, more demand creates more supply.
Simplicity & Ease of Management
Managing sync groups and larger sync jobs can become complex with AFS. But with Resilio, managing, controlling, and automating your sync environment is easy.
With Resilio, there are no sync groups to manage and fewer overall management tasks. Once a server has a Resilio agent installed, you simply need to add servers and shares to your sync jobs.
You can use Resilio’s centralized Management Console to create, visualize, and automate all transfer and sync jobs, and monitor sync status via real-time notifications. The Management Console provides a unified view of your sync environment and enables you to control replication parameters, such as bandwidth utilization, disk io threads, file priorities, data hashing, packet size, and more. You can also use Resilio’s REST API to script any type of functionality and easily automate sync jobs.
End users also receive a unified, centralized view of files in your file system. And while Resilio doesn’t offer the full cloud-tiering capabilities of AFS, it does enable local caching, where files are stored locally in a dehydrated state (i.e., they don’t consume a lot of disk space) for easier, on-demand file access.
Case Study: Wargaming
Wargaming is a video game design company that uses Resilio Connect to distribute their video game builds (50+ GB files) across 20 offices over high-latency, lossy networks.
“Resilio met all our technical requirements, but was also very easy to deploy and integrate into our workflow. We also liked the fact that the simplicity carried over to transparent pricing and clear documentation.”
Flexibility (No Vendor Lock-In)
You can only use AFS on Windows, Linux, and macOS servers. On the cloud, you can only use Microsoft’s infrastructure. Organizations using other on-premises file servers or cloud-based platforms will need to migrate — a timely and expensive process.
But Resilio Connect is a vendor-agnostic platform that requires no migration and doesn’t lock you into any specific storage providers or other vendor types .
It can be used with:
- Multiple operating systems: You can use Resilio on Windows, Linux, FreeBSD, Unix, MacOS, and all major NAS solutions (Resilio also provides iOS and Android mobile deviceapps). Resilio can be deployed cross-platform, and in cloud, on-prem, and hybrid cloud environments. And you can use Resilio with any type of cloud for DFS and on virtualization platforms such as Citrix and VMware.
- Familiar tools: Resilio Connect pairs with familiar tools you’re already using, such as event processing solutions like Loggly and Splunk.
Resilio Connect can also be deployed in conjunction with AFS, as it natively supports Azure cloud storage (such as Azure Blobs, Azure NetApp Files, and Azure Disk Storage) to enhance its performance — e.g, rapidly sync data across sites and between Azure storage regions and on-prem sites, enable Active-Active high availability, reduce time-to-desktop in VDI deployments, and reduce recovery times in disaster recovery scenarios.
End-to-End Security
Resilio Connect also utilizes state-of-the-art security features to keep your data secure and uncorrupted, including:
- End-to-end encryption: Data is encrypted at rest and in transit using AES 256.
- Cryptographic integrity validation: With integrity validation, you can be sure your data always arrives at its destination intact.
- Mutual authentication: With mutual authentication, you can ensure your data is only delivered to designated, approved endpoints.
- Role-based permissions: You can create multi-level admins and control access to settings, jobs, and agents.
Transparent Pricing and Cost Considerations
As a sync solution for Azure Files, AFS consumes Azure Files resources. These resources, along with server licensing fees, will play a factor in the total cost of your AFS deployment.
Because there are multiple hidden cost-factors that can be difficult to estimate, calculating AFS costs (and predicting future costs as your replication needs grow) can be difficult. AFS cost factors you need to consider include:
- Server licensing fees: You must register your Windows File Servers with Azure’s Storage Sync Service. While the first server is free, each subsequent server costs a small monthly fee (check current server registration fee here).
- Storage utilization: AFS consumes storage space on your server endpoints. Storage costs will increase along with your replication needs (i.e., file sizes and number of files).
- Snapshot utilization: AFS takes share and file-level snapshots as part of regular usage. Although snapshot utilization is always differential, this can contribute in a noticeable way to the total Azure Files bill.
- Transactions from churn: AFS charges for transactions that occur when file changes are uploaded to the cloud, cloud tiering management, and data egress. Since the quantity and type of transactions is difficult to estimate, predicting associated costs is also difficult.
- Transactions from cloud enumeration: Cloud enumeration (i.e., when AFS discovers and syncs changes in the cloud) occurs once per day, and incurs a transaction cost.
Resilio Connect, on the other hand, offers a more transparent, straightforward pricing model. It enables you to sync any number/size/type of files across any distance (including WAN optimization) with no extra money being charged and no hidden, hard-to-calculate costs.
Resilio includes “smart routing” to keep traffic within Azure if needed in order to minimize data egress. And with Resilio’s Transparent Selective File Sync feature (which enables you to locally cache certain files in a dehydrated state that doesn’t consume lots of disk space), you can efficiently sync only the files you need down from Azure.
To learn more about Resilio pricing and get an estimate, contact our team.
2. GoodSync
GoodSync is a file sync and data backup solution that offers free and paid versions (making it a great option for small businesses) and can be deployed on Linux and Windows servers. Like Resilio, GoodSync encrypts data using AES 256. It enables you to remotely access data using one of the supported cloud file storage services (Onedrive, AWS, Google Drive, Office 365, Dropbox, Nasuni, etc.). And it increases replication speed by replicating only changed data in parallel threads.
3. SureSync
SureSync is a real-time file sync solution for Windows servers that can be used for file distributions, global file sharing, and disaster recovery. It speeds synchronization by using multi-threaded replication and remote differential compression (i.e., only replicating changed data).
To learn more about SureSync and competing solutions, check out our article on 4 SureSync alternatives.
4. Syncthing
Syncthing is an open-source P2P replication software that’s designed for Linux devices, but can also be installed on Windows, Mac OS X, FreeBSD, OpenBSD, and Solaris. It uses TLS encryption and cryptographic certification to secure data. And their open documentation enables users to learn how to implement, control, and optimize synchronization in their environment.
You can learn more about what to do when Syncthing goes out of sync here. And check out our article on 9 Syncthing alternatives for fast, reliable business sync.
5. PeerGFS
PeerGFS is a real-time replication solution that utilizes a hub-and-spoke architecture. It can be configured on multiple servers, cross-platform, on-prem, in the cloud, or in cloud-hybrid scenarios. It supports native file and object storage formats so, like Resilio, you can avoid vendor lock-in. PeerGFS protects data using Malicious Event Detection, which monitors files for suspicious activity.
6. LucidLink
LucidLink is an online file sharing and file streaming solution that enables media, engineering, and marketing companies to collaborate on files. Rather than syncing files across endpoints, LucidLink enables you to stream files on-demand. It also provides end-to-end encryption and works with any object file storage.
Check out our article on 5 LucidLink alternatives, where we break down some of LucidLink’s strengths and weaknesses and provide several alternatives and complementary solutions.
7. Signiant Media Shuttle
Signiant Media Shuttle is a media management service that provides fast file replication and protects data using TLS encryption. It works with any type of storage and can be deployed on-prem, in the cloud, or in hybrid-cloud scenarios.
When a file transfer fails midway, Signiant can perform a Checkpoint Restart and resume the transfer where it left off. Signiant also integrates with many media management workflow tools, making it an excellent solution for media companies.
Check out our article on 5 Signiant alternatives to learn more about Signiant and competing solutions.
8. DFSR
DFSR (Distributed File System Replication) is a free, command-line solution for syncing Windows servers. It can be deployed with, or in lieu of, AFS — though it experiences frequent reliability and performance issues that force many users to seek a DFSR alternative. Still, it provides fast synchronization by only syncing changed data. And it protects files using authenticated and encrypted procedures during replication.
Sync Files with Resilio Connect
Resilio Connect is an excellent file sync solution that can be used with or instead of AFS. It provides:
- Fast sync over any network: With its peer-to-peer replication methodology, real-time sync capabilities, and WAN optimization protocol, Resilio can quickly sync files (20+ Gbps per server) across your entire replication environment over any network.
- Reliable synchronization that “just works”: Resilio’s P2P architecture makes it incredibly fault tolerant, with no single point-of-failure. And its WAN optimization protocol enables you to reliably sync over even the most unreliable networks and in areas with little network connectivity.
- Flexible, easy-to-manage deployment: Resilio Connect can be deployed in a variety of setups (on-prem, cloud, hybrid-cloud), on a variety of servers, and on any cloud file service (including Amazon and Azure cloud storage). And Resilio’s centralized Management Console gives you granular control over how replication occurs, and enables you to easily monitor and automate replication in your environment.
- Organic scalability: Resilio’s P2P environment means that transfer speed and bandwidth increase as you add more devices. It’s designed to handle large sync deployments — syncing to many endpoints (can sync hundreds of endpoints in roughly the same time as it takes to sync two), syncing files of any size (no file size limits), and syncing large numbers of files (250+ million in a single job).
- End-to-end security: Resilio Connect’s state-of-the-art security features keep your data safe from interception and ensure files arrive at their destination uncorrupted.
To see if Resilio Connect could be a fit for you, schedule a demo or learn more here.
