The Fastest Way to Move Files
Resilio Connect / The Fastest Way to Move Files
Resilio’s technology offers the fastest delivery time across a broad array of network speeds common to the modern Enterprise. Our products combine peer to peer, rsync-like delta encoding, and WAN Optimization, three of the most powerful data transmission technologies, into one single solution.
The Early Internet
The early Internet was predominantly a peer to peer system. It was a network populated with academics and researchers, and computers connected to this network were largely equal in that each contributed as much information as they received.
As the Internet matured, the client - server model came to dominate, especially with the advent of http and the World Wide Web. In this system, roles were separated with consumers as “clients” connecting to “servers” somewhere on the network that would distribute content and data. Serving a massive audience required a vast number of these servers.
As demand grows in this model, performance declines and fragility increases. The same number of servers must meet the needs of a larger number of clients. Sharing the growing load naturally degrades the performance available to each client. Additionally, such systems are inherently fragile. With a single source of content at the servers, you introduce single points of failure that can result in complete downtime for an application.
Thus, technologies like Content Delivery Networks (CDNs) emerged to aggregate and multiplex server capacity across many sources of content. An unpredictable burst of demand could be more easily shared and building capacity closer to clients improves performance. Such innovations made the early client - server model a little more robust, but at considerable cost.
Yet despite the inefficiency, the client server model remains predominant today. Examples of the client - server model in common use include most web content, search engines, cloud computing applications and even common tools like FTP and rsync.
Peer to Peer Systems
Peer to peer systems are fundamentally different. In these systems, every “consumer” is also a “producer.” Using the language of the client server model, each participant is both “client” and “server”. In this way peer to peer systems become organically scalable. As more demand emerges for any content, so does more supply. As demand grows, the system becomes incredibly fault tolerant and actually gets faster, in sharp contrast to the client server model, which gets noticeably slower, and more fragile under the same circumstances.
To summarize this section, peer to peer has several advantages:
- It is faster
- It is more robust
- It reduces the load on the server
- It’s an efficient use of infrastructure
Peer to Peer in Action
Using a specific example, this section examines how peer to peer is always faster than any client-server architecture.
To explain the technology in a simple way, we’ll make the following assumptions. The file we want to transfer has five blocks. Each computer has a connection channel capable of sending one block per cycle.
We have a Sender that needs to send data to four devices A, B, C, and D (the Receivers).
- Stage 1.
The Sender splits the file into independent pieces, and creates a meta-information data block that describes the pieces . In our example, the file consists of 5 pieces.
We will mark each piece as colored dot: Red, Green, Yellow, Blue & Black.
- Stage 2.
The Receivers request the file meta-information. When Receivers get the meta-information, they know the file in question contains five pieces, and at the moment only the Sender has them.
- Stage 3.
Each cycle, the Receivers randomly select one piece from the file and start downloading it. This continues until every Receiver has every piece of the file.
So, How Many Cycles Would it Take to Transfer The Data?
- Using P2P architecture: only 7 cycles
- Using Client-Server architecture: 20 cycles