Like other configuration management tools such as Microsoft Configuration Manager, Jamf for Macs, Chef, and Puppet, Ansible is a great choice when extensible scripting and automation are desired for remote configuration and patch management. Ansible is an extensible automation platform–yet struggles transferring large files (such as packages and other bulky data sets) across wide-area networks (WANs) and VPNs.
Resilio Connect can be combined with Ansible to reliably, rapidly, and automatically transfer files across hundreds (or even thousands) of endpoints located anywhere. Like a CDN, Resilio can be used with Ansible to distribute and cache files at remote sites. This may be a one-to-many distribution or one-to-one, where clients at the remote sites pick up changes from a central repository. The benefits of using Resilio and Ansible together include speed, predictability, and reliability. Resilio, through its peer-to-peer (P2P) architecture, can distribute files in parallel to multiple locations. Combined with Resilio’s UDP-based WAN optimization technology, DevOps and IT teams can deploy their updates in fixed and predictable timeframes. Using Resilio Connect, it takes about the same to distribute files to hundreds of servers as it does to 2 servers.
Resilio Connect complements Ansible by offloading the heavy lifting of distributing large files for package delivery from the OS update process. Resilio excels at efficiently moving files in bulk across any type of network. Ansible excels at automating the process of configuration management.
Moreover, Resilio is cross-platform and can update many Linux, Windows, Mac systems and almost any cloud-based endpoint. The scripting and API capabilities of Resilio can also be used for Ansible. Resilio extends Ansible’s deployment automation capabilities and provides end-to-end reliability, flexibility (works well across many OS types and networks) and scalability for remote package delivery.
Ansible is a popular automation platform for configuration management, software provisioning, orchestration and application deployment. Here, we will discuss its essential components and their use in the DevOps setup. Lastly, we will look into some of its caveats and how Resilio Connect can help you complement Ansible’s capabilities for enhanced performance in distributing files across remote locations.
Of if you’d like to cut to the chase, please schedule a demo with us and we can show you Resilio Connect live in action–and discuss how large enterprise customers are using Resilio and Ansible together.
Background and components of Ansible
Michael DeHaan, a former Red Hat engineer, created Ansible in 2012. His primary aim was to design an open-source, agent-less configuration management tool that didn’t have the complexity of other such tools.
Later in 2015, Red Hat acquired Ansible as it gained popularity among DevOps professionals, largely due to its deployment automation and scripting capabilities. Since then, Ansible has become one of the industry’s most widely used automation tools, with a large and active community of developers and end-users. It uses SSH or Windows Remote Management (that allows PowerShell execution) to deploy required software to nodes.
Ansible is now maintained and developed by Red Hat and has been integrated into many of its products and services. Ansible Automation Platform is Red Hat’s enterprise IT automation solution which is subscription-based and comes in both managed and self-managed forms.
Some of the basic Ansible parts are:
Inventory: Ansible inventory refers to a text file or a set of files that mentions the list servers of nodes that Ansible can manage. The inventory has IP addresses and names of these servers, and it can be in various formats such as JSON, YAML and INI.
Playbooks: A playbook is a set of YAML files containing configuration management scripts that define how tasks will be executed on remote hosts. They simply tell the desired state of the system and Ansible runs those tasks to achieve that state.
Modules: Ansible modules are pre-written scripts that perform specific tasks on remote hosts. They can be used to do various operations like service management, package installation and file management and be called inside an Ansible playbook. Modules take some arguments as input and output values in JSON format after execution. Different vendors have created their own Ansible modules, for example, Cisco modules. You can also custom-create your module; the only hard and fast requirement is that it must return a JSON object.
Roles: Ansible Roles are a way to organize and reuse Ansible code. They allow you to group related tasks and files into a reusable structure. Roles are defined using YAML files with a predefined directory structure containing: defaults, vars, tasks, files, templates, meta, and handlers.
Ad-hoc commands: Ad-hoc commands execute simple, one-off tasks on remote hosts. They execute from the command line and can be helpful in quick fixes or troubleshooting.
Variables: Ansible uses variables to store and manipulate data. Variables can be defined at the inventory, playbook, or role level, and can be used to make playbooks more flexible and reusable.
Plugins: Ansible plugins are extensions that provide additional functionality to Ansible. They can be used to write custom modules, define custom inventory sources, and extend Ansible’s core functionality.
Handlers: Ansible handlers allow you to run operations on change. Oftentimes you may want a task to run only when a change is made on a machine. This is where ansible handlers come in handy. In short, handlers are tasks that only run when notified.
These components work together to provide a powerful automation platform for IT operations.
Basic Ansible features
Here are four essential features of Ansible:
There are two basic approaches utilized by automation processes in the Infrastructure as Code scenario:
Imperative: The imperative model focuses on how you do something.
Declarative: The declarative model focuses on what you want but not how to make something happen.
Ansible uses the declarative approach and allows you to write tasks focusing on what you want rather than how to achieve it. You can then map these tasks to some underlying procedural code. Ansible relies on YAML to create playbooks which itself is a declarative language. All the tools that utilise YAML are essentially creating a declarative interface for their users e.g. Kubernetes.
Ansible has an agentless architecture and you can begin managing your remote machines immediately without installing any agent software. One of the major problems of many configuration management solutions is one of “Managing The Management.”
Usually, software must be installed on remote machines before you can start managing them. During the updation process for your management software, these remote agents must be updated first. Most of these systems can’t self-update and many times compatibility issues come up between agent versions and the central management server.
Other significant benefits of agentless architecture are lower memory/CPU consumption and attack surface. When Ansible is not managing remote nodes, it does nothing on those nodes. This means there is no daemon to consume memory or CPU. Moreover, the chances of a cyber attack are reduced by not requiring any remote server agents. The only program you need to run is the OpenSSH daemon or WinRM service.
Ansible is a rich community of developers and users. Whatever task you try to automate, chances are someone has done it already. Ansible Galaxy is the repository where developers can contribute their Ansible modules, playbooks etc.
Another great feature of Ansible is its idempotent nature. Ansible defines idempotence as: “An operation is idempotent if the result of performing it once is exactly the same as the result of performing it repeatedly without any intervening actions.”
In mathematical terms, it can be defined as a function whose output doesn’t change when applied again to its own output: f(f(x)) = f(x)
Idempotence is highly beneficial for CI/CD platforms as it prevents duplicate records from being loaded, which can lead to erroneous insights and wasted storage space/bandwidth. Overall, it makes your data pipeline self-correcting.
For example, if a conventional script configures 5 VLANs, the same commands are executed 5 times each time the script is run. Instead, if there was an idempotent Ansible module in action, checking the existing configuration would be the first logical step, and each new VLAN being configured would be matched against the current configuration. Only if the new VLAN needs to be added (or a changed VLAN modifier) is a change or command actually pushed to the device.
Ansible use cases:
Network automation and configuration management
Ansible allows you to manage complete IT processes across physical, software-defined, and cloud-based networks. It uses a simple, secure, and robust agentless automation framework for IT operations and development.
Configuration management tools are fast becoming crucial for organizations with advanced networking setups. Introducing even minor misconfiguration errors or inconsistencies can lead to outages and other security and compliance problems.
With its feature-rich capabilities, Ansible lets you jump-start configuration management automation and share content with different teams for enhanced efficiency and collaboration. In short, previously siloed network administrators can now speak a shared language of automation.
Security Automation and endpoint protection
Ansible is used extensively for security automation purposes. You can use its vast library of modules, playbooks and roles to update and apply security patches, ensuring your servers are protected against all cybersecurity threats. Moreover, you can establish firewall rules and automatically analyze threats to endpoint devices with integrated Endpoint Protection Platforms (EPP) tools.
Ansible allows you to manage different tools and services as a whole and bring discipline to deployment challenges like multiple cloud environments, datacenters or applications with complex dependencies. Nobody deploys a single service on a single machine any more. Today’s IT brings complex deployments and complex challenges. Instead of just blasting out commands, we use orchestration techniques to perform things in a controlled manner, just like a flowchart.
DevOps: Ansible for deployment automation
The central aim of automating deployment operations is to speed up your ability to deploy software regularly. You achieve greater ease of deploying software by increasing deployment frequency. Beyond that, once you find errors in production, these same techniques are helpful in deploying the fixes. This means you can rectify a broken service at a much better speed.
Ansible is specifically designed to speed up the DevOps pipeline by automating the deployment of your applications to production environments. It has cemented itself as one of the most popular DevOps tools due to its powerful configuration management, automation and orchestration features for IT infrastructure. Ansible can quickly scale according to workload requirements.
One strength of Ansible is that it operates through SSH. A simplified deployment solution is possible without poking additional holes into your server or any other IT system. This allows you to save time and money by delivering builds to development, test labs, and partner sites in a streamlined manner.
With Ansible, you need three components: an Ansible server, a list of remote hosts (inventory) and a playbook. As you run your Ansible playbook, the central server will connect with each node using SSH. The tasks you want Ansible to execute are defined separately from the list of computers you want to configure.
The playbook will tell everything you want Ansible to perform on a particular endpoint or server. They are segregated into different sections. These sections include tasks, variables (vars), and handlers sections. Every task runs a corresponding Ansible module. Ansible has over 500 modules that easily cater to your most basic needs.
Ansible Synchronize module for software distribution and application deployment
One particular module of our interest is the Ansible Synchronize module. It is a wrapper around the rsync to make common tasks in your playbooks quick and easy. Rsync is a Linux utility for efficiently transferring and synchronizing files between endpoints by comparing the modification times and sizes of files.
Although it doesn’t provide the full power of rsync, it allows common operations to be performed easily. To use it, rsync must be installed on both the local and remote host. Currently, only a few connection types work with the Synchronize Ansible module: ssh, paramiko, local, and docker. Moreover, the connection for these must not need a password, as rsync does not provide a way to pass a password to the connection.
This post explains the strengths and weaknesses of Rsync and why Resilio Connect is a faster and more reliable alternative for software distribution.
What are the drawbacks of Ansible?
Ansible can pose a steep learning curve for users unfamiliar with Linux-style CLI. You have to learn multiple languages to achieve various tasks. Like YAML for creating playbooks and Jinja for creating templates; according to Ummer Ishtiaq of I2C Inc. While Ansible is written in Python, you may not need to know Python to use it.
Lack of corporate support for Ansible Modules:
Vendors have created various Ansible modules to facilitate user interaction with endpoints. However, most of these vendors don’t provide corporate-level support services if you are facing issues with these modules. At the most, they will offer troubleshooting guidelines on the discussion forums. This issue comes up even with the premium paid Ansible version called Ansible Tower. Overall, there is a lack of collaboration between Ansible and large enterprises.
Dealing with host unreachability:
While dealing with large deployments, we often face reachability issues with some hosts. Mostly, they are skipped and recorded somewhere, so the missed parts can be rerun as they become available. This issue doesnt come up with pull-based approaches.
Similar issues come up while working with handlers. This Ansible handler mechanism is designed to delay running a task at the end of the play, and it will be run once instead of multiple times as changes are detected. The problem is faced when a handler is flagged to be fired but has yet to get a chance to run since, e.g. the host is unreachable or shut downed. As this flag is not saved in a lasting store, the state is lost, and the handler may not run again even after rerunning the play as there is no change.
Ansible lacks scalability in certain situations, especially when dealing with many servers or complex automation workflows. Agentless architecture can come with its own set of problems.
One way to mitigate this would be to use the Ansible role reversal technique i.e. from push to pull. The whole role runs on the remote side when using Ansible pull, but the run is only limited to that host. But, running in the pull mode has consequences for which you must make your roles pull-friendly or sometimes separate pull-friendly parts from others. Surely these actions increase the complexity of working with Ansible.
Limited Support for Windows:
Although Ansible does support Windows systems, the support is not as comprehensive as the support for Linux systems. This can make using Ansible for Windows-based systems and workflows more challenging.
Using Ansible and Resilio Connect together
Ansible and Resilio Connect are tag team deployment automation partners when it comes to benefiting DevOps teams. Ansible can prepare servers for deployment (taking care of the dependencies), while Resilio Connect can be used for flexible, reliable high-speed file distribution. Resilio and Ansible are easily integrated through scripting. In fact, a scripting job can be created directly in the Resilio Connect management console. This capability enables Ansible functionality and rules to be built in around the file distribution and synchronization process. Ansible functionality can take place before or after a file distribution job completes.
When combined, deployment automation and synchronization of resources across multiple systems can be achieved seamlessly and across as many locations and endpoints as needed.
Let us discuss two scenarios where Resilio Connect and Ansible can be used together:
Distributing cyber security applications along with training content using Ansible & Connect.
- Resilio Connect helps distribute large binaries and other packages (such as security applications) along with any other file assets that may be needed (such as training videos and documents) to all the company laptops, desktops, and devices. The cyber security team can add and remove media files as required without worrying about the endpoints being fully synchronised.
- Ansible can handle the base application installation and compliance checks, informing the cyber security team of any hardware compatibility problems and more.
Setup of a Gaming firm distributed repository system
- Resilio Connect can work in a multi-site push configuration where the gaming firm may use it to distribute (push) gaming applications and content from a single remote source repository to multiple other distributed repositories. Once the repository is a distributed to a remote site, contributors and other users can mount and “pull” the repository on the LAN. This highly scalable process can serve many clients concurrently on the LAN.
- While Resilio provides the scalable file distribution component, Ansible ensures that gaming applications (and other updates) are installed how they should be. Consistency of configuration and compliance across all the involved environments can be achieved using Ansible.
Peer-to-peer (P2P) distributed architecture
Resilio Connect employs a peer-to-peer architecture which can spread the distribution load across multiple peers, concurrently. This enables efficient delivery and sync scenarios: you can easily synchronize new files from one-to-many and many-to-many servers in parallel without any bottlenecks. This parallelism is one of Resilio Connect’s key superpowers. In the case of a one-to-many distribution or synchronization job, performance can be scaled across peers, reducing the time it takes to update all of the files in the job. One benefit of this is predictability. You can distribute files to many endpoints in parallel in about the same time as it takes to distribute packages to a single endpoint.
Most conventional synchronization solutions, including the Ansible Synchronize module (based on rsync), transfer files using a client-server(or point-to-point) model. Each server is a bottleneck. Even when configured in a hub-and-spoke, the hub server is the bottleneck transferring files to client servers.
With Resilio Connect, changes can be replicated in any given direction: one-way, two-way, one-to-many, many-to-many and that too on the principle of real-time change detection.
Resilio Connect can speed up file transfer by 5-10x as it hashes files into multiple file chunks and then transfers each chunk to multiple destinations independently without relying on an upstream (hub) server. The load is distributed across multiple peers (multiple devices) in your network to speed up transfers.
When it comes to speeding up file distribution over WANs, Resilio Connect includes a built-in UDP-based capability called Zero Gravity Transport (ZGT). This WAN optimization offers complete predictability for moving large payloads across any type of WAN link, overcoming latency and packet loss. Connect works efficiently to obtain predictable transfers without needing dedicated software and/or hardware. ZGT speeds up sync across WAN by up to 100x over conventional file replication tools.
Resilio Connect agents support popular operating systems such as Windows, Mac, Linux, FreeBSD, Android, and various NAS platforms. Connect also supports popular virtualization platforms (Citrix, Microsoft, VMware), servers, storage, networks, and cloud providers. In fact, you can use any S3-compatible cloud object storage–as well as cloud file storage. It also works across all major web servers like Apache, Nginx, and Node.js.
Overall, Ansible and Resilio Connect combine to offer a complimentary solution for automated deployment. Ansible provides an extensible automation platform. Resilio Connects offers high-speed file delivery to distribute files to multiple sites and servers in parallel, across any IP network. Ansible includes a wide range of automation and configuration management features. Resilio Connect, through its P2P architecture, provides optimal file transfer and distribution for Ansible. Resilio Connect enhances Ansible’s software deployment capabilities, offering flexibility, scalability, reliability, and end-to-end security for high-speed data transfer, distribution, and synchronization.