Maybe you\u2019ve heard of UDP in association with a variety of high-speed transfer products such as UDP blasters, Aspera, Signiant<\/a>, and FileCatalyst, among others. Resilio also uses UDP as part of our Zero Gravity Transport (ZGT) WAN Optimization solution<\/strong>. We\u2019ll cover that in more detail later in the piece. <\/p>\n\n\n\n For now, it\u2019s worth noting that while UDP offers advantages to moving data fast and in bulk across the Internet, it is not a solution; it\u2019s a protocol. <\/p>\n\n\n\n UDP doesn\u2019t require an ordered delivery of packets and supports a continuous packet stream. This makes it much more efficient and better at transferring over low-quality networks compared to TCP. However, UDP also carries reliability risks as there\u2019s no guarantee that all of your packets will arrive at their destination.<\/p>\n\n\n\n That’s why, if you want to take advantage of UDP\u2019s speed and efficiency for transferring your large files, you need to use a UDP transfer software<\/a> that includes critical features that overcome these issues.<\/p>\n\n\n\n Want to see how Resilio\u2019s proprietary WAN Optimization solution, ZGT, uses <\/em>UDP<\/em>to<\/em> transfer your <\/em>large files<\/em> over any network reliably, efficiently, and securely? <\/em>Schedule a demo with our team<\/em><\/a> to see Resilio ZGT in action!<\/em><\/p>\n\n\n\n In this article, we\u2019ll discuss how UDP file transfer works, as well as its features, benefits, and downsides. <\/p>\n\n\n\n We\u2019ll also discuss how our real-time file synchronization software system, Resilio Active Everywhere, provides fast and reliable\u00a0large file transfer<\/a>\u00a0and sync over any network using a proprietary file UDP-based protocol called\u00a0ZGT<\/a>\u00a0that overcomes the issues of alternative UDP solutions.<\/p>\n\n\n\n Resilio is the best UDP file transfer solution because it:<\/p>\n\n\n\n <\/p>\n\n\n\n Organizations in media, gaming, engineering, retail, tech, and more use Resilio Platform to sync and transfer large files over any network (WANs, LANs, edge, etc.). To learn how Resilio can provide fast, reliable sync for your organization, <\/em>schedule a demo with our team<\/em><\/a>.<\/em><\/p>\n\n\n\n To understand why some organizations turn to UDP file transfer, we should first review some use cases where large file transfer over WANs and unreliable networks is necessary. <\/p>\n\n\n\n Many of our clients require fast-paced work on large files across geographically distributed teams and need a solution that can quickly and reliably sync large files and file updates across multiple locations. These include organizations in:<\/p>\n\n\n\n <\/p>\n\n\n\n For these use cases, TCP-based protocols, such as the file transfer protocol (FTP) wouldn\u2019t be a good fit because they:<\/p>\n\n\n\n <\/p>\n\n\n\n For these reasons, the slow transfer rate and network congestion created by TCP protocols make speedy replication of large files across WANs (and unreliable networks) impossible. This prevents geographically distributed teams from collaborating quickly and efficiently in the scenarios outlined above, as well as in many other situations.<\/p>\n\n\n\n In this section, we\u2019ll cover the differences between TCP transfer, traditional UDP transfer, and Resilio\u2019s proprietary UDP-based transfer protocol known as Zero Gravity Transport\u2122 (ZGT) \u2014 which can transfer files 100x faster than traditional TCP-based protocols while being more reliable and efficient than most UDP-based protocols. <\/p>\n\n\n Try our <\/em>transfer speed calculator<\/em><\/a> to see how much time we can save you.<\/em><\/p>\n\n\n\n An acknowledgment occurs when a receiving device signifies to a sending device that it received a data packet. The way a transfer protocol handles acknowledgements can impact the speed and reliability of transfer.<\/p>\n\n\n\n As stated earlier, TCP protocols require the receiving device to acknowledge each packet before another can be sent. They also track packets to ensure that all packets were delivered to the receiver.<\/p>\n\n\n\n While this enhances the reliability of transfer (by ensuring all packets arrive at the receiving device), it also slows transfer speeds by increasing traffic over the network and making the sending device wait before transmitting more packets.<\/p>\n\n\n\n Traditional UDP protocols don\u2019t require any acknowledgements for received data packets. This reduces traffic over the network and allows the sender to continue sending data packets unrestricted. <\/p>\n\n\n\n But it also makes data transfer less reliable, as there\u2019s no way to confirm that an individual packet was received or that the entire file was delivered.<\/p>\n\n\n\n Resilio\u2019s ZGT transfer protocol uses interval acknowledgments to confirm packet receipt. Rather than requiring an acknowledgment after each received packet, ZGT sends acknowledgments in groups once per RTT. This combines the best of both worlds by minimizing traffic over the network, confirming packet receipt, and enabling a continuous data stream.<\/strong><\/p>\n\n\n\n In addition, Resilio Platform uses cryptographic data integrity validation to ensure that files reach their destination intact and uncorrupted<\/strong>. The data hashes and metadata are sent before the files are transferred, so the receiving device can confirm the integrity of each file. If any file is incomplete or corrupted, it\u2019s immediately discarded and scheduled for retransmission.<\/p>\n\n\n\n As stated earlier, packet loss (which happens when data packets don\u2019t reach their destination) is a common occurrence when transferring over long-distance WANs and unreliable networks. The way a transfer protocol responds to packet loss can impact transfer speed and reliability.<\/p>\n\n\n\n TCP protocols suffer from two issues with regard to packet loss:<\/p>\n\n\n\n First, TCP protocols require that lost packets be retransmitted after each acknowledgement. This increases traffic over the network and slows the rate of <\/strong>data transfer<\/strong>.<\/strong><\/p>\n\n\n\n The second issue is that TCP protocols treat large amounts of packet loss as a network congestion issue and reduce transfer speed in response. Since packet loss over WANs isn\u2019t indicative of network congestion, this response unnecessarily delays data transfer and impedes workflows.<\/p>\n\n\n\n UDP places no requirements on lost packet retransmission. While this can reduce traffic over the network and allow for faster transfer speeds, it also makes it unreliable (as there\u2019s no way to ensure all UDP packets are delivered to their destination).<\/p>\n\n\n\n ZGT uses a process known as delayed retransmission. Instead of sending lost packets after each acknowledgment, it retransmits lost packets in groups once per RTT. <\/p>\n\n\n\n Again, this provides the best of both worlds. It reduces unnecessary retransmissions and minimizes traffic over the network<\/strong>. And data is delivered reliably while ensuring bandwidth is utilized efficiently.<\/p>\n\n\n\n The rate at which data can be transmitted from a sender to a receiver is controlled by:<\/p>\n\n\n\n <\/p>\n\n\n\n Data flow control refers to controlling the amount of data sent to the receiver. Essentially, the receiver tells the sender the maximum speed at which to send data in order to avoid packet loss.<\/p>\n\n\n\n Congestion control is a mechanism for controlling the amount of data sent over a network. Rather than being dictated by the receiver, data transmission is controlled by how much traffic the network can effectively handle.<\/p>\n\n\n\n TCP includes options for flow control, such as sliding windows, congestion avoidance algorithms, and more. While these mechanisms can help reduce packet loss, they prevent TCP from fully utilizing the bandwidth of a given network. This makes TCP unsuitable for:<\/p>\n\n\n\n <\/p>\n\n\n\n Traditional UDP doesn\u2019t place any limits on flow control, allowing for faster transmission of data. But unrestricted data transfer can lead to network congestion and packet loss.<\/p>\n\n\n\n ZGT uses a congestion control algorithm to maximize bandwidth utilization of any network at all times.<\/p>\n\n\n\n The algorithm constantly probes the Round Trip Time of the network and analyzes the underlying network conditions (such as packet loss, latency, and throughput over time). It dynamically adjusts to these conditions<\/strong> in order to fully utilize the network, increase transfer speed, and maintain a uniform packet distribution over time.<\/p>\n\n\n\n This not only makes ZGT ideal for transfer over WANs, it also allows Resilio to fully-utilize any type of network connection \u2014 such as VSATs, cell (3G, 4G, and 5G), Wi-Fi, and any IP connection. So Resilio can be used for transfers over intermittent, unreliable edge networks, such as maritime file transfer<\/a>.<\/p>\n\n\n\n Whether they utilize TCP or UDP, most file transfer solutions use point-to-point replication architectures that inherently hinder their ability to transfer large files quickly and reliably.<\/p>\n\n\n\n Point-to-point replication occurs in one of two topologies:<\/p>\n\n\n\n <\/p>\n\n\n\n Both of these replication topologies are slow and unreliable, regardless of the transfer protocol being used.<\/p>\n\n\n\n The fact that replication only occurs between two servers at a time (i.e., one server to another) causes full synchronization of your entire environment to take a long time. This is especially true when syncing large amounts of data (i.e., large files or large numbers of files) or syncing across many endpoints. <\/p>\n\n\n\n Plus, synchronization is limited by your slowest endpoint.<\/strong> If any server in your environment is on a slow network, then it will delay synchronization for every other server behind it.<\/p>\n\n\n\n And both replication topologies introduce single points of failure. If any server in your environment goes down, it can delay sync for every other server. And if your hub server goes down in a hub and spoke topology, replication fails entirely.<\/p>\n\n\n\n Resilio Platform is one of the only file synchronization solutions<\/a> that uses a peer-to-peer (P2P) replication architecture.<\/p>\n\n\n\n P2P replication turns your environment into a mesh network where every endpoint can work together to sync files. It accomplishes this because:<\/p>\n\n\n\n <\/p>\n\n\n\n Resilio\u2019s P2P replication architecture enables it to overcome the limitations of traditional point-to-point solutions and provide file transfer\/synchronization that is:<\/p>\n\n\n\n The combination of file chunking and P2P transfer yields blazing-fast sync speeds, as every endpoint can sync files across your environment concurrently.<\/p>\n\n\n\n Imagine you wanted to sync a 100 GB file across five servers. Resilio can split that large file into five chunks (20 GB each). Server 1 can share the first chunk with Server 2. As soon as it receives the first chunk, Server 2 can share it with Server 3, even while it waits to receive the remaining file chunks. The same is true for every other server: they can immediately share file chunks while they wait to receive the rest of the file.<\/p>\n\n\n Resilio\u2019s P2P replication methodology enables it to sync files 3\u201310x faster than most point-to-point solutions<\/strong>.<\/p>\n\n\n\n As alluded to earlier, point-to-point replication solutions don\u2019t scale well. Every new endpoint you add to your environment causes full synchronization to take longer. If you need to sync 1 TB of data across 5 endpoints, that large data must sync with each endpoint one by one.<\/p>\n\n\n\n But since every endpoint in a P2P environment can work together to share files, adding more endpoints only increases the available resources (CPU, bandwidth, etc.) and replication speed \u2014 i.e., more demand inherently creates more supply.<\/p>\n\n\n\n This makes Resilio organically scalable, allowing it to quickly and reliably synchronize: <\/p>\n\n\n\n <\/p>\n\n\n\n And with Resilio, you can take transfer speed across large environments to another level using a process known as horizontal scale-out replication.<\/p>\n\n\n\n With scale-out replication, you can cluster endpoints together in order to pool their resources and linearly increase transfer speeds.<\/p>\n\n\n\n For example, since each Resilio agent can reach speeds of 10 Gbps, our engineers were able to cluster 10 agents together and achieve speeds of 100+ Gbps<\/strong>. You can cluster even more agents together to achieve even greater speeds, as there\u2019s no limit to how fast this can go.<\/p>\n\n\n While not directly related to Resilio\u2019s P2P architecture, Resilio has the ability to sync files in real-time. It uses notification events from the host OS and optimized checksum calculations (i.e., identification markers assigned to each file that change whenever a change is made to a file) to detect file changes as soon as they occur and immediately sync the changed portions of the file across your environment.<\/p>\n\n\n\n Resilio\u2019s P2P architecture also enables it to sync files in any direction, such as:<\/p>\n\n\n\n <\/p>\n\n\n\n N-way sync is a particularly powerful feature that\u2019s key to Resilio\u2019s ability to provide fast, reliable sync speeds for:<\/p>\n\n\n\n 1. Multi-site collaboration<\/strong><\/p>\n\n\n\n The combination of real-time sync and multidirectional sync is great for remote collaboration<\/a>. If any employee at any location makes a change to a file, that file change is immediately detected and synchronized across every other server in your environment.<\/p>\n\n\n\n 2. Disaster recovery<\/strong><\/p>\n\n\n\n Because it replicates data in real-time, Resilio can provide sub-five second RPOs (Recovery Point Objectives). And Resilio\u2019s P2P architecture with N-way sync effectively turns every endpoint in your environment into a backup server. This enables you to achieve Active-Active High Availability. And in the event of a disaster, every endpoint can work together to bring your system back online for RTOs (Recovery Time Objective) within minutes of an outage.<\/p>\n\n\n\n 3. Synchronization at the far edge <\/strong><\/p>\n\n\n\n When operating at the far edge of networks where internet connections are unreliable (and sometimes unavailable), you can use Resilio to transfer from agent to agent. For example, the shipping agency Ross Maritime has vessels that are often out of range of high-bandwidth connections, but are occasionally in range of other vessels. But Ross Maritime uses Resilio\u2019s P2P architecture to reliably deliver data<\/a> to those ships in a \u201cstore and forward\u201d fashion \u2014 i.e., sending data from ship to ship until it reaches its destination.<\/p>\n\n\n Since every endpoint in a P2P environment can share files directly with each other, there\u2019s no single point of failure.<\/p>\n\n\n\n If any device or network goes down, the required files or services can be retrieved from any other device in your environment.<\/p>\n\n\n\n Resilio\u2019s ZGT transfer protocol overcomes the limitations of UDP better than any other UDP-based protocol. But it also provides other capabilities that make it a superior file transfer and sync solution, such as:<\/p>\n\n\n\n <\/p>\n\n\n\n Reliable synchronization of mission-critical files is paramount for fast-paced, data-intensive workflows. As we\u2019ve discussed, other file sync and transfer solutions are unreliable because:<\/p>\n\n\n\n In addition to the reliability features of Resilio listed above, Resilio includes other features that enhance reliability, such as:<\/p>\n\n\n\n <\/p>\n\n\n\n Resilio Platform is built on open standards and universal protocols. This means that:<\/p>\n\n\n\n <\/p>\n\n\n\n Resilio is a highly-flexible, vendor-agnostic, agent-based solution that works by installing agents on each endpoint you want to replicate to and from. Resilio supports just about any: <\/p>\n\n\n\n 1. Device: <\/strong>You can install Resilio agents on desktops, laptops, file servers, NAS\/DAS\/SAN devices, mobile devices (Resilio offers iOS and Android apps), IoT devices, and virtual machines (such as VMware, Citrix, and Microsoft Hyper-V).<\/p>\n\n\n\n 2. Cloud storage platform:<\/strong> Resilio supports all major cloud object storage platforms, such as AWS, Google Cloud, and Azure Blobs. It also supports just about any other S3-compatible cloud object storage, such as Backblaze, Weka IO, MinIO, Wasabi, and more.<\/p>\n\n\n 3. Operating system:<\/strong> Resilio supports Windows, MacOs, Linux, Unix, Ubuntu, FreeBSD, OpenBSD, and more.<\/p>\n\n\n\n 4. Automation requirements:<\/strong> Using Resilio\u2019s powerful REST API, you can script any type of function and automation your job requires. Resilio scripts offer three types of triggers: before a job starts, after a job completes, and after all jobs complete. For example, in order to deliver security updates to their vessels at sea over intermittent satellite connections, a marine construction company used Resilio<\/a> to deliver the updates to each ship’s main server, and used scripts to have those servers immediately distribute the updates to each workstation on the ship.<\/p>\n\n\n\n 5. Workflow integration:<\/strong> You can use Resilio\u2019s REST API to integrate it with the tools your team is already using, such as development tools (e.g., TeamCity, Jenkins), and management tools (e.g., Microsoft SCOM, Splunk). For example, the documentary production company Delirio Films integrated Resilio with their media tools<\/a> and creative software (Adobe Premiere, DaVinci Resolve, and Avid Media Composer), various storage systems (Facilis Terrablock and OWC Thunderbay), and the LucidLink file streaming<\/a> solution.<\/p>\n\n\n This level of flexibility means that you can deploy Resilio on your existing IT infrastructure \u2014 at minimal cost and with minimal operational interruption \u2014 and begin replicating in as little as two hours. <\/p>\n\n\n\n Unlike many traditional and open-source UDP-based file transfer solutions, Resilio Platform is easy to manage and gives you granular control over how replication occurs in your environment.<\/p>\n\n\n\n While you can program Resilio via the command-line interface, Resilio\u2019s Management Console is a more user-friendly graphic interface that can be accessed from any web browser. Through the Management Console, you can control every aspect of replication across your environment, such as:<\/p>\n\n\n\n Data security is critical in all file transfer scenarios, but especially when transferring data over WANs and vulnerable edge networks. A major downside of most file transfer tools is that they don\u2019t provide native security features, forcing you to invest in 3rd-party security tools or VPNs.<\/p>\n\n\n\n Resilio protects your data end-to-end using built-in security features that were reviewed by 3rd-party security experts (TPN Blue certified and SOC2 certified), such as:<\/p>\n\n\n\n While UDP protocols are superior for transferring large files over WANs and unreliable networks, most UDP-based file transfer solutions\/protocols can\u2019t provide the speed and reliability that Resilio Platform can. Resilio\u2019s superior file transfer is due to its:<\/p>\n\n\n\n <\/p>\n\n\n\n\n
UDP File Transfer Use Cases<\/h2>\n\n\n\n
\n
\n
TCP vs UDP vs Resilio ZGT: Features, Benefits, Downsides<\/h2>\n\n\n\n
![\"Cross](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/cross-region-replication-Azure-3-1024x668.png\")
<\/figure>\n<\/div>\n\n\nAcknowledgments and Reliability<\/h3>\n\n\n\n
Transmission Control Protocol<\/h4>\n\n\n\n
Traditional User Datagram Protocol<\/h4>\n\n\n\n
Resilio Zero Gravity Transport<\/h4>\n\n\n\n
Lost Packet Retransmission<\/h3>\n\n\n\n
Transmission Control Protocol<\/h4>\n\n\n\n
Traditional User Datagram Protocol<\/h4>\n\n\n\n
Resilio Zero Gravity Transport<\/h4>\n\n\n\n
Data Flow and Congestion<\/h3>\n\n\n\n
\n
Transmission Control Protocol<\/h4>\n\n\n\n
\n
Traditional User Datagram Protocol<\/h4>\n\n\n\n
Resilio Zero Gravity Transport<\/h4>\n\n\n\n
Replication Architecture: Point-to-Point vs P2P<\/h3>\n\n\n\n
\n
\n
Blazing Fast<\/h4>\n\n\n\n
![\"GIF](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2022\/09\/image2.gif\")
<\/figure>\n<\/div>\n\n\nOrganically Scalable<\/h4>\n\n\n\n
\n
![\"Operations](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2024\/02\/operations-team-resilio-connect-management-console.png\")
<\/figure>\n<\/div>\n\n\nMultidirectional Sync and Real-Time Replication<\/h4>\n\n\n\n
\n
![\"Global](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/09\/remote-vessel-server-sync-1024x858.png\")
<\/figure>\n<\/div>\n\n\nNo Single Point of Failure<\/h4>\n\n\n\n
4 More Ways Resilio Provides Superior File Transfer and Sync<\/h2>\n\n\n\n
\n
1. Bulletproof Reliability Features<\/h3>\n\n\n\n
\n
\n
2. Vendor and Workflow-Agnostic Deployment<\/h3>\n\n\n\n
\n
![\"Resilio](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image3-1024x776.png\")
<\/figure>\n<\/div>\n\n\n![\"Resilio](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2024\/03\/remote-editors-via-WAN.png\")
<\/figure>\n<\/div>\n\n\n3. Centralized Management and Granular Control<\/h3>\n\n\n\n
\n
![\"Edit](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/cross-region-replication-Azure-5.png\")
<\/figure>\n<\/div>\n\n\n4. Built-In, State-of-the-Art Security<\/h3>\n\n\n\n
\n
![\"Security](\"https:\/\/blog.resilio.com\/wp-content\/uploads\/2023\/08\/image7.png\")
<\/figure>\n<\/div>\n\n\nUse Resilio Platform for Fast, Reliable UDP Transfer<\/h2>\n\n\n\n
\n