Resilio Active Everywhere (formerly Resilio Connect) is a worry-free file replication solution that provides faster, more reliable transfer using a P2P transfer architecture, WAN optimization, and more. If you want to try replicating files with Resilio, you can get set up and begin replicating your Windows\u00a0file servers\u00a0in as little as 2 hours by\u00a0scheduling a demo<\/a>\u00a0with our team.\u00a0<\/em><\/p>\n\n\n\n When diagnosing problems with DFSR, the first step is to gain insight into your replication environment and narrow down potential causes of error and avoid wasting time trying erroneous solutions.<\/p>\n\n\n\n Unfortunately, DFSR provides little visibility into the state of your replication network. But the following methods should give you some information you can use to understand why your backlog isn’t clearing:<\/p>\n\n\n\n Use the command line “Get-DfsrState”<\/strong> to check the current replication state of your DFS replication group partners.<\/p>\n\n\n\n Use the command line “Get-DfsrBacklog” to see pending updates between two Windows-based file servers that are part of your dfs replication service. <\/p>\n\n\n\n To check DFS replication backlog status, install the dfsrdiag.exe<\/strong> tool by running one of the following commands from Powershell:<\/p>\n\n\n\n Use the following commands to check the connectivity in your Active Directory:<\/p>\n\n\n\n This yields any information that Active Directory has about DFS, the replication groups, and the folders it belongs to.<\/p>\n\n\n\n This makes the server check in with Active Directory. It should result in “operation succeed.”<\/p>\n\n\n\n This shows you what’s currently replicating.<\/p>\n\n\n\n To run a diagnostic report, start DFS Management, expand replication, then:<\/p>\n\n\n\n Before exploring other options, start by checking and eliminating the simplest potential causes of replication failure. This includes:<\/p>\n\n\n\n A common quick fix involves deleting and recreating the replication group:<\/p>\n\n\n\n DFSR can not sync files that are in use. So any files that are open won’t replicate. Try closing files and restarting replication.<\/p>\n\n\n\n The staging area<\/a> in DFSR is used to:<\/p>\n\n\n\n The default staging area quota for DFSR is 4 GB. But your staging quota must be, at a minimum, as large as the 32 largest files in the replicated folder. This is particularly true for initial replication, as it will make much more use of the staging area than day-to-day replication.<\/p>\n\n\n\n To find the 32 largest files and calculate the staging area, use the following commands:<\/p>\n\n\n\n This command will provide the file names and the sizes of the files (in bytes). Use it to identify the 32 largest files. Add them up to identify your minimum staging area quota.<\/p>\n\n\n\n This command will provide the size (in bytes) of the 32 largest files in a folder. Again, add them up to identify your minimum staging area quota. Then set your staging quota to this number.<\/p>\n\n\n\n These commands will help you find the minimum<\/em> staging area quota. But when it comes to setting your staging area quota, bigger is better (up to the total size of the replicated folder).<\/p>\n\n\n\n A high DFSR backlog can lead to problems that increase the backlog even more.<\/p>\n\n\n\n For example, if your DFS backlog has been growing for multiple weeks, you may have multiple versions of the same files waiting to be replicated. This can lead to conflict resolution errors where older versions of files win and outdated data is replicated. <\/p>\n\n\n\n This issue often occurs when admins roll out too many new replication folders at once, which can overload your hub server. To prevent this from happening, stagger rollouts so that initial replication is finished within a reasonable time frame.<\/p>\n\n\n\n Since DFSR operates on a client-server architecture, the hub server(s) is critical to its operation. Many replication environments are deployed with just a single hub server that must service many branch office servers. The hub server becomes overwhelmed and can’t perform all the tasks required of it, so file replication backs up.<\/p>\n\n\n\n When using DFSR with multiple branch office servers, you should have at least two hub servers. Not only does this protect you in the event that one hub server goes down, it also balances the load between servers and prevents replication delays.<\/p>\n\n\n Since every environment is unique, you’ll need to determine how many branch office servers a single hub server can service by monitoring your backlogs. To learn more about deploying hub servers, check out this Microsoft article on tuning replication performance<\/a>.<\/p>\n\n\n\n You may need to temporarily stop DFS replication on a particular replication group for one reason or another. In this event, be sure you:<\/p>\n\n\n\n DFSR must be running in order to read updates to the USN Journal. If lots of files are changed, added, or deleted while DFSR is stopped, it can cause journal wrap to occur.<\/p>\n\n\n\n A journal wrap occurs<\/a> when the change journal cannot store all entries and must overwrite old journal entries. In a large environment with lots of files or large files, it can take DFSR a long time to recover from journal wrap. During this time, replication can be very slow or not occur at all. Backlogs will build up during this process.<\/p>\n\n\n\n Ultimately, you\u2019ll have to confront the major issue with DFSR: it’s an unreliable replication tool that’s prone to errors. As your needs and replication environment grow, your DFSR replication issues will get worse. And if you\u2026<\/p>\n\n\n\n \u2026 Then you’ll need to find an alternate file replication solution.<\/p>\n\n\n\n Here\u2019s a list of the 5 fundamental reasons DFSR issues persist and how Resilio overcomes those challenges.<\/p>\n\n\n\n Transferring files over long distance WAN networks significantly increases the travel time and potential for packet loss. DFSR performs poorly over WAN networks because:<\/p>\n\n\n\n In a client-server transfer architecture, there is one sender (the hub server) and one receiver (the destination server). This means replication can only occur to one destination at a time \u2014 the hub server can’t transfer files to a second device until transfer is complete on the first. This can create a backlog of file transfers, especially when transferring large files or large numbers of files.<\/p>\n\n\n\n Resilio uses a peer-to-peer (P2P) transfer architecture. Every device in the replication network can send and receive data from every other device. And when transferring data among several devices, each device can also become a sender. In other words, once Device A receives a file packet, it can begin sharing that packet with other devices in the network \u2014 resulting in transfer speeds that are 3-10x faster than DFSR or any other client-server solution.<\/p>\n\n\n DFSR’s slow transfer architecture is made worse by the fact that it relies on TCP\/IP transfer protocol. TCP\/IP suffers from two issues that make it perform poorly over WANs:<\/p>\n\n\n\n a) How it responds to packet loss<\/strong><\/p>\n\n\n\n Due to its use of TCP\/IP (and other reasons), DFSR always assumes packet loss is due to channel congestion, and reduces transfer speed in order to reduce the load on the network.<\/p>\n\n\n\n But packet loss on WAN networks can occur for other reasons, such as a failure on the intermediate device. Reducing transfer speed doesn’t reduce packet loss and only results in greater delays (i.e., your backlog builds up).<\/p>\n\n\n\n b) How it ensures data delivery<\/strong><\/p>\n\n\n\n TCP\/IP ensures data gets delivered to its destination using confirmation packets. In other words, the sender sends a packet to a receiver, and the receiver returns a confirmation packet acknowledging receipt. Retransmission time (RTT) is the time it takes a packet to travel from sender to receiver or vice versa.<\/p>\n\n\n\n The RTT on a LAN (local area network) can be as little as .01 ms. But on a WAN, RTT can be 800ms or more. So using TCP\/IP over a WAN can significantly reduce transfer speed by up to 2 seconds for each new packet transfer.<\/p>\n\n\n\n While Resilio can use TCP\/IP over LANs, it uses our proprietary transfer protocol \u2014 Zero Gravity Transfer (ZGT \u2122) \u2014 on WANs.<\/p>\n\n\n\n ZGT \u2122 optimizes WAN transfer using:<\/p>\n\n\n\n As your enterprise organization and replication environment grow, DFSR’s performance will suffer when faced with:<\/p>\n\n\n\n a) Large files and large numbers of files<\/strong><\/p>\n\n\n\n DFSR is notoriously bad at transferring large files and large numbers of files. This is not only due to the fact that it uses a 1 to 1 client-server transfer architecture, but also because:<\/p>\n\n\n\n b) Lots of activity<\/strong><\/p>\n\n\n\n For those same reasons, DFSR also performs poorly when there’s lots of activity \u2014 i.e., many\/frequent file changes that need to be replicated. Every time you make a change to a file, the slow process of scanning folders and detecting changes has to begin again. The more changes you make, the bigger your DFSR backlog queue grows.<\/p>\n\n\n\n c) Replicating to more servers<\/strong><\/p>\n\n\n\n The more servers you add to your replication environment, the worse DFSR will perform.<\/p>\n\n\n\n DFSR’s 1 to 1 replication approach means that replication must complete to one server before it can begin on another server. This issue is compounded if one server is far away or on a slow network, as it forces all the other servers to wait until the slow transfer is complete.<\/p>\n\n\n\n DFSR also uses static IP: ports to establish connections to different machines. If a machine’s IP: port changes (or isn’t available), DFSR stops working and must be reconfigured by a human.<\/p>\n\n\n\n DFSR’s inability to adequately synchronize files to multiple destinations is one of the most common causes of a large backlog queue. Since most organizations sync files across multiple servers in multiple locations, using DFSR quickly becomes impractical.<\/p>\n\n\n\n Resilio provides fast replication at scale using:<\/p>\n\n\n\n Optimized checksum calculations<\/strong><\/p>\n\n\n\n A checksum is an identification marker for a file. When a file gets changed, the checksum also changes. Resilio uses optimized checksum calculations (as well as notifications from the host OS) to quickly detect file changes and replicate only the changed parts of a file \u2014 which reduces network load and increases transfer speed.<\/p>\n\n\n\n In the event of a network failure mid-transfer, Resilio can perform a checksum restart in order to resume transfer where it left off. When file transfers are interrupted on DFSR, it must start again at the beginning.<\/p>\n\n\n\n File chunking and omnidirectional file transfer<\/strong><\/p>\n\n\n\n With its P2P transfer architecture, Resilio can quickly replicate large files and large numbers of files across your entire replication environment. <\/p>\n\n\n\n Resilio Platform uses a process known as file chunking \u2014 transferring files in chunks. Once a server receives a file chunk, it can immediately share it with any other server in your network.<\/p>\n\n\n\n For example, Server 1 begins replicating file changes to Server 2. Server 2 can share that chunk with other servers as soon as it is received. Sync speed is significantly increased because:<\/p>\n\n\n\n This also means that Resilio is organically scalable. Since every device can share data with every other device in your network concurrently, adding more devices enhances performance \u2014 i.e. increases transfer speed and reduces the network load across your environment. On average, Resilio increases performance over conventional approaches by 10-20x, depending on the use case (such as number of files, network latency, bandwidth, storage capabilities, etc.).<\/p>\n\n\n\n For example, in the Resilio model, you can sync files between 2 devices in about the same time as it takes to sync files between 200 devices, with available network capacity. By contrast, in a traditional “point to point” or “client server” model, the maximum number of devices that can concurrently be synchronized is 2.<\/p>\n\n\n\n And Resilio Platform is a cross-platform solution. In addition to Windows servers, Resilio runs on Linux, macOS Server, BSD, and a variety of NAS-specific operating systems (Synology, TrueNAS, etc.).<\/p>\n\n\n\n The IT manager at an employee benefit company said: <\/em><\/p>\n\n\n\n \u201cWe went from spending at least 2 hours per night dealing with replication and troubleshooting issues with DFSR to spending about 1 hour per week monitoring jobs using Resilio.\u201d <\/em><\/p>\nRead more about how Resilio helped them achieve reliable and predictable replication<\/em><\/a>.<\/em><\/cite><\/blockquote>\n\n\n\n Many organizations require Active-Active High Availability. Active-Active HA refers to an environment where two different sites are actively serving users and both are available for redirection in case one goes down \u2014 which is useful for balancing network load (for unreliable network connections and VDI scenarios), quickly and accurately recovering data in disaster recovery scenarios, and quickly replicating concurrent data changes.<\/p>\n\n\n\n If you need Active-Active HA, DFSR is not a good option because it:<\/p>\n\n\n\n On the other hand, Resilio Active Everywhere’s bidirectional and omnidirectional synchronization makes it a perfect solution for Active-Active HA. If one site goes down, any client or group of clients can provide the service, each operating as nodes on the network. And file chunks get distributed across multiple endpoints concurrently. Not only does this balance the load between servers, it provides those in VDI scenarios more bandwidth and the ability to replicate file changes any time, anywhere.<\/p>\n\n\n\n According to the IT manager responsible for DFS replication at a Fortune 500 engineering and construction firm, <\/em><\/p>\n\n\n\n \u201cThere was simply no way to achieve true active-active high availability (HA) across sites, prior to Resilio. \u201cWe went from a 30-minute replication window using Microsoft DFSR to a 5-minute lead time to have all data synchronized across all sites with Resilio,\u201d they said. <\/em><\/p>\nRead more about it here.<\/em><\/a><\/cite><\/blockquote>\n\n\n\n And with WAN optimization, Resilio ensures data transfer occurs as quickly as possible over any network. Users access the servers closest to them, and Resilio dynamically routes around failures and network latency.<\/p>\n\n\n\n DFSR is a suitable replication tool for small environments with a few files. While we hope the DFSR solutions we compiled work for you, DFSR is highly problematic and you’ll eventually need a more reliable replication solution.<\/p>\n\n\n\nStart by Gaining Insight into the State of DFS Replication<\/h2>\n\n\n\n
Use DFS Command Lines<\/h3>\n\n\n\n
1) Get-DfsrState<\/h4>\n\n\n\n
2) Get-DfsrBacklog<\/h4>\n\n\n\n
3.) Use the dfsrdiag.exe tool <\/h4>\n\n\n\n
\u201cInstall-WindowsFeature RSAT-DFS-Mgmt-Con\u201d<\/strong><\/pre>\n\n\n\n
\"DFSDiag \/TestDFSIntegrity \/DFSRoot: \/Full.\"<\/strong><\/pre>\n\n\n\n
\"dfsrdiag backlog \/rgname:'Group Name' \/rfname:'folder name' \/smem:servername \/rmem:servername\"<\/strong><\/pre>\n\n\n\n
Check Your Active Directory<\/h3>\n\n\n\n
Command: $ DFSRDIAG dumpadcfg \/member:SERVERNAME<\/strong><\/pre>\n\n\n\n
Command: $ DFSRDIAG pollad \/member:SERVERNAME<\/strong><\/pre>\n\n\n\n
Command: $ FDSRDIAG replicationstate<\/strong><\/pre>\n\n\n\n
Run a diagnostic Report<\/h3>\n\n\n\n
\n
Try These 7 Common Solutions for Clearing Your DFSR Backlog<\/h2>\n\n\n\n
Solution 1: Check Network Cards and Antivirus Software<\/h3>\n\n\n\n
\n
Solution 2: Delete & Recreate the Replication Group<\/h3>\n\n\n\n
\n
Solution 3: Close Files before Syncing<\/h3>\n\n\n\n
Solution 4: Check Staging Area Quota<\/h3>\n\n\n\n
\n
1) Get-ChildItem c:\\\\temp -recurse | Sort-Object length -descending | select-object -first 32 | ft name,length -wrap \u2013auto<\/strong><\/pre>\n\n\n\n
2) Get-ChildItem c:\\\\temp -recurse | Sort-Object length -descending | select-object -first 32 | measure-object -property length \u2013sum<\/strong><\/pre>\n\n\n\n
Solution 5: Prevent High Backlog by Staggering Replication Folder Rollouts<\/h3>\n\n\n\n
Solution 6: Balance the Load on Hub Servers<\/h3>\n\n\n\n
![\"Diagram](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/09\/Is-Your-DFSR-Backlog-Not-Clearing-Try-This-1-1024x601.png\")
<\/figure>\n<\/div>\n\n\nSolution 7: Don\u2019t Stop DFSR for Too Long<\/h3>\n\n\n\n
\n
3 Reasons Why DFSR Issues Will Persist & How Resilio Overcomes Those Challenges<\/h2>\n\n\n\n
\n
1. DFSR Performs Poorly over High-Latency, Long-Distance WANs (Wide Area Networks)<\/h3>\n\n\n\n
\n
![\"GIF](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/09\/image2.gif\")
<\/figure>\n<\/div>\n\n\n\n
2. DFSR Scales Poorly<\/h3>\n\n\n\n
\n
\n
\n
3. DFSR can’t provide Active-Active High Availability<\/h3>\n\n\n\n
\n
\n
Use Resilio for Worry-Free File Replication<\/h2>\n\n\n\n