The Fastest Way to Move Files

Resilio Connect

The Fastest Way to Move Files

Introduction

Resilio’s technology offers the fastest delivery time across a broad array of network speeds  common to the modern Enterprise.  Our products combine peer to peer, rsync-like delta encoding, and WAN Optimization, three of the most powerful data transmission technologies, into one single solution.

 

The Early Internet

The early Internet was predominantly a peer to peer system.  It was a network populated with academics and researchers, and computers connected to this network were largely equal in that each contributed as much information as they received.

As the Internet matured, the client - server model came to dominate, especially with the advent of http and the World Wide Web.  In this system, roles were separated with consumers as “clients” connecting to “servers” somewhere on the network that would distribute content and data.  Serving a massive  audience required a  vast number of these servers.

As demand grows in this model, performance declines and fragility increases.  The same number of servers must  meet the needs of a larger number of clients.  Sharing the growing load naturally degrades the performance available to each client.   Additionally, such systems are inherently fragile.  With a single source of content at the servers, you introduce single points of failure that can result in complete downtime for an application.

Thus, technologies like Content Delivery Networks (CDNs) emerged to aggregate and multiplex server capacity across many sources of content.  An unpredictable burst of demand could be more easily shared and building capacity closer to clients improves performance.  Such innovations made the early client - server model a little more robust, but at considerable cost.

Yet despite the inefficiency, the client server model remains predominant today.  Examples of the client - server model in common use include most web content, search engines, cloud computing applications and even common tools like FTP and rsync.

 

Peer to Peer Systems

Peer to peer systems are fundamentally different.  In these systems, every “consumer” is also a “producer.”  Using the language of the client server model, each participant is both “client” and “server”.  In this way peer to peer systems become organically scalable.  As more demand emerges for any content, so does more supply.  As demand grows, the system becomes incredibly fault tolerant and actually gets faster, in sharp contrast to the client server model, which gets noticeably slower, and more fragile under the same circumstances.

To summarize this section, peer to peer has several advantages:

  • It is faster
  • It is more robust
  • It reduces the load on the server
  • It’s an efficient use of infrastructure

Peer to Peer in Action

Using a specific example, this section examines how peer to peer is always faster than any client-server architecture.

To explain the technology in a simple way, we’ll make the following assumptions. The file we want to transfer has five blocks. Each computer has a connection channel capable of sending one block per cycle.

We have a Sender that needs to send data to four devices A, B, C, and D (the Receivers).

  • Stage 1.

The Sender splits the file into independent pieces, and creates a meta-information data block that describes the pieces . In our example, the file consists of 5 pieces.

We will mark each piece as colored dot: Red, Green, Yellow, Blue & Black.

  • Stage 2.

The Receivers request the file meta-information. When Receivers get the meta-information, they know the file in question contains five pieces, and at the moment only the Sender has them.

  • Stage 3.

Each cycle, the Receivers randomly select one piece from the file and start downloading it.  This continues until every Receiver has every piece of the file.

So, How Many Cycles Would it Take to Transfer The Data?

  • Using P2P architecture: only 7 cycles
  • Using Client-Server architecture: 20 cycles

It takes ~3 times longer using client-server architecture.  The Receivers will download pieces only from the Sender. In the P2P case, most of the pieces are coming from other Receivers. Let's consider a real world example to demonstrate the differences.

 

Real World Example

Let's say we have a server and ten clients that are all connected over a 1 Gbps network. We need to send a 100GB file to all the machines.  A peer to peer solution will complete this job in 17 minutes. By contrast, the client-server approach will take more than an hour. As you increase demand on this system (either with larger files or more participants) the performance difference between these two models will grow.

 

Conclusions

If you need to synchronize data across servers, distribute data to several locations and endpoints and/or need to deliver data over unreliable networks, please contact us today for access to the world’s fastest data transfer solution.

 

Learn more about Resilio Connect.

Thank you for your form submission
error message
error message