It takes hours to execute terraform and ansible jobs. What if I want ansible jobs to start immediately after my infrastructure gets provisioned through terraform jobs? How would one hand over the monitoring of running jobs to other human resources? How would one get notified if a job fails? IT-Conductor Git/Terraform/Ansible integration to promote Application Infrastructure as Code is now GA per our Q1-2020 Features Announcement
A normal workflow for a firm that uses terraform/ansible would include the following manual activities
- setting up a jumpbox machine with terraform/ansible .
- pulling the terraform/ansible scripts to jumpbox from the repository .
- executing the terraform jobs in the terminal for provisioning and waiting for it to complete by manual monitoring.
- executing the ansible jobs in the terminal for installing stuffs after the completion of infrastructure provisioning and manually monitoring the jobs to complete.
IT-Conductor has taken these pain points into consideration and can configure multiple on-premise gateways that enable us to interact with the local environment. So we don’t need a jumpbox with pre-configured terraform/ansible. They are configured with portable versions on the fly through a gateway before running jobs. If the machine with one gateway was to fail, we can always transfer the jobs to another gateway.
We’ve also included the concept of process definition where multiple activities can be woven together so that they get processed serially. Activities can be anything like sql script, shell script , terraform script , ansible script.
Each activity has a self-refreshing web based log. The Ansible job collects each playbook task containing its own log, with the help of remote backend features. It’s just a matter of sharing the url of a running job if one wants to handover the monitoring to another resource. And that’s not required because we encourage automation to manual monitoring, letting IT-Conductor to notify you through mail/sms if the job fails/completes.
Using IT-Conductor as an Automation Engine
In this section we discuss how we can get the scripts from git repository, make terraform/ansible jobs out of the scripts, weave them together through process definition and finally automate it for provisioning/installation of SAP/HANA.
Ansible scripts integration to ITC from GitHub
https://github.com/IT-Conductor/automation-ansible
https://github.com/IT-Conductor/automation-terraform
IT-Conductor collects the github files and maps them as file objects.
Making a Terraform Job
For making a terraform job, we need the git repo project as shown in previous section, terraform backend object to capture terraform state file information, provider containing azure subscription/tenant information, and variables input to the terraform script.
Making an Ansible Job
Similar to Terraform Jobs, ansible jobs also need a git repo project along with execution config object containing variable data for installation process and inventory object or location of inventory file.
Weaving Terraform and Ansible Jobs together through Process Definition
IT-Conductor has the framework for automating different activities by weaving them together so that they can run serially. Process Definition is a template for processes which can be scheduled to run at desired time.
In the picture above the Process Definition template signifies that Terraform Activity “Provision HANA SCS APP Infrastructure” provisions the required infrastructure; when it completes Ansible Activity “Install HANA SCS APP” starts immediately for installing the configured SAP packages.
The terraform/ansible job can be fed input files and can generate output files. In the above flow, when the terraform job completes it generates output files as inventory, which are needed as an input file to the subsequent ansible job. The Process Definition supports this through implementation of the ‘Data Files’ feature.
In Process Definition one can add multiple data files that can be read/written by its activities.
In each activity, these data files are available for selection as input or output data files.
Running the jobs in Process
Along with scheduling the Process Definition, one can manually run it anytime by clicking Run/Instantiate.
One can simply name the process and then run it.
Then click object link on the popup.
From there one can go to the Process Flowchart:
And view job status:
For looking into each job progress, click on a particular job and then the execution log.
One can also view individual ansible tasks.
Overall, there are several advantages for using IT-Conductor as an automation server:
- IT-Conductor utilizes on-premises gateways as jump-servers alleviating the need to login remotely
- IT-Conductor layer of security is enforced to ensure the role-based access and attribution of executed activities
- IT-Conductor process graphical design, orchestration and scheduling capabilities are fully integrated
- IT-Conductor data that already configured/discovered, including host, application and cloud credentials and staged git repositories are seamlessly integrated into Ansible
- Dynamic Inventories based on dynamic IT-Conductor system and application groups
- Ansible task log recording is available through IT-Conductor native UI/automated delivery over email
