Bidirectional Sync (or two-way sync) is the ability to reliably sync files across 2 or more locations, preferably in real-time. Each location could have a few to many systems.
Most conventional approaches can only sync between 2 systems. But, using Resilio Connect, bidirectional synchronization can span as many systems and sites as needed, including the cloud.
One-Way Sync
Most conventional replication technologies offer one-way synchronization, from a single source (system 1) to a single target (system 2). Tools like Rsync, for example, can efficiently (depending on the payload, network, and other factors) sync one-way between 2 systems.
One-way sync is popular and serves a purpose.
Storage replication products (like Isilon SyncIQ and NetApp SnapMirror) offer one-way sync based on point-in-time snapshots initiated in their file systems. So for use cases like Disaster Recovery (depending on your RPO or RTO) or basic server-to-server or storage-to-storage backup, one-way sync may suffice. But if the customers’ DR goals are for faster RTOs and lower RPOs—and synchronizing files across more than 2 sites—these solutions may not be good fit.
Basic Point-to-Point Two-Way Sync
Other tools that do support bidirectional sync are Unison, Aspera Sync, and Microsoft DFSR. These are limited by their point-to-point (client-server) architecture, among other caveats like the prospect of file corruption or data loss.
These implementations limit the two-way sync to at most 2 systems. Thus, only those users and apps accessing those 2 systems will benefit from two-way sync. Worse, each of those systems is a performance bottleneck and single point of failure (SPOF).
At best, point-to-point two-way sync enables a classic hub-and-spoke: the hub is the source and each spoke the target. The benefit of the hub and spoke is that updates can be relayed from the hub (source) to the spokes (targets); and vice versa. This obviously does not scale. Worse, each spoke is land-locked from the other spokes; all traffic must go through the hub. So implementer beware: the hub is a performance bottleneck and a SPOF.
In general, point-to-point architectures are useful for certain use cases but greatly limit how you can move and sync data. (Here’s a good comparison of point-to-point vs. peer-to-peer.)
Other potential problems with point-to-point two-way sync tools include: poor reliability, poor management and monitoring, and lack of conflict resolution and diagnostics tools.
High-level Business Needs for Bidirectional Sync
Businesses and organizations of all types and sizes need their file-based data sets protected, current, and readily available when and where they are needed. Your business processes and workflows should determine when bidirectional sync is needed. In general, the most current files should be continuously available to users, applications, and workflows. For many shops, when data stops, business stops.
One popular use case for bidirectional sync is global collaboration across sites. Many customers use Resilio Connect for bidirectional sync to keep their files current and accessible across multiple locations. Blockhead Studios, for example, needs to keep large media files in sync between Australia and New Zealand. Resilio enables two-way sync massive media files across two sites for large teams.
One thing to note is that Resilio is designed to sync files, not different databases (such as CRM, Salesforce, mySQL, etc.). We have had customers sync their entire database files when virtualized in a file—but that is not a topic of discussion in this blog.
Reliable Bidirectional Sync at Scale
Resilio believes that bidirectional synchronization is a differentiated replication capability that should “just work” and not be limited to only 2 servers. Resilio’s solution for bidirectional sync is built on a peer-to-peer architecture. This highly reliable design enables multiple servers (or other endpoints running Resilio agents) to participate in the sync process.
Resilio Connect enables the following bidirectional sync characteristics:
- Reliability: The solution should always “just work”–and your files and deltas should be synchronized across any type of network and be resilient to failures.
- No SPOF: There should not be a single point of failure (SPOF). If one or more servers, storage devices, or network connections goes down, this should not break the replication process.
- Data integrity: There should never be data loss or data corruption during the two-way data synchronization, no matter how large the data set. If there’s a conflict accessing a file, the system should preserve data integrity. When files or deltas are in transit across the network, data integrity should also be preserved during the sync process.
- Automation: Replicating file changes and everything associated with the bidirectional synchronization process should be automated. Advanced solutions like Resilio Connect offer this in numerous ways: via the management UI, via scripting, or through an API.
- Real-time sync: Polling the file system or listening for file change notifications is a key enabler of real-time sync. All systems should be able to efficiently capture and replicate only the changed portion of the file during the two-way sync.
- Centralized management: It should be easy to configure, track, and monitor all synchronization jobs (and other replication jobs for distribution and ingest) centrally. Resilio offers an easy to use management console that provides visibility into job status and progress as well bandwidth control. Notifications through email or a webhooks connector is also a key capability.
- Conflict resolution: There should be straightforward policies, features, or best practices that enable conflict resolution and mitigates conflicts. Resilio publishes best practices to minimize the chance of conflicts and simplify resolutions.
- WAN optimization: Resilio offers native TCP as well as UDP-based WAN optimization for transfers and synchronization jobs to overcome latency and loss associated with WANs.
Resilio Connect gives you the flexibility to enable bidirectional synchronization across as many servers and sites as needed. This opens up opportunities for global file collaboration, real-time server sync, and active-active high availability across sites in support of HA and DR. Moreover, Resilio supports any type of storage: block, file, and object, from any vendor.
There are a number of bidirectional sync use cases and scenarios outlined in our recent blog, Reliable Bidirectional Sync for Files.
Please take a look or get in touch with our sales team to schedule a live demo on Resilio Connect!
