IT-Conductor Blog

Unifying the Git/Terraform/Ansible Workflow through IT-Conductor Agentless Platform

Posted by Dinesh Amatya on Oct 8, 2020 11:53:50 AM

It takes hours to execute terraform and ansible jobs. What if I want ansible jobs to start immediately after my infrastructure gets provisioned through terraform jobs? How would one hand over the monitoring of running jobs to other human resources?  How would one get notified if a job fails?  IT-Conductor Git/Terraform/Ansible integration to promote Application  Infrastructure as Code is now GA per our Q1-2020 Features Announcement

A normal workflow for a firm that uses terraform/ansible would include the following manual activities 

  1. setting up a jumpbox machine with terraform/ansible .
  2. pulling the terraform/ansible scripts to jumpbox from the repository .
  3. executing the terraform jobs in the terminal  for provisioning and waiting for it to complete by manual monitoring.
  4. executing the ansible jobs in the terminal for installing stuffs after the completion of infrastructure provisioning and manually monitoring the jobs to complete.

IT-Conductor has taken these pain points into consideration and can configure multiple on-premise gateways that enable us to interact with the local environment. So we don’t need a jumpbox with pre-configured terraform/ansible. They are configured with portable versions on the fly through a gateway before running jobs. If the machine with one gateway was to fail, we can always transfer the jobs to another gateway. 

We’ve also included the  concept of process definition where multiple activities can be woven together so that they get processed serially. Activities can be anything like sql script, shell script , terraform script , ansible script. 

Each activity has a self-refreshing web based log. The Ansible job collects each playbook task containing its own log, with the help of remote backend features. It’s just a matter of sharing the url of a running job if one wants to handover the monitoring to another resource. And that’s not required because we encourage automation to manual monitoring, letting IT-Conductor to notify you through mail/sms if the job fails/completes.

Using IT-Conductor as an Automation Engine

In this section we discuss how we can get the scripts from git repository, make terraform/ansible jobs out of the scripts, weave them together through process definition and finally automate it for provisioning/installation of SAP/HANA.

 

Ansible scripts integration to ITC from GitHub

https://github.com/IT-Conductor/automation-ansible

https://github.com/IT-Conductor/automation-terraform

IT-Conductor automation terraform-1

 

IT-Conductor collects the github files and maps them as file objects.

IT-Conductor collect the github files

Making a Terraform Job

For making a terraform job, we need the git repo project as shown in previous section, terraform backend object to capture terraform state file information, provider containing azure subscription/tenant information, and variables input to the terraform script. 

Making Terraform Job

Making an Ansible Job

Similar to Terraform Jobs, ansible jobs also need a git repo project along with execution config object containing variable data for installation process and inventory object or location of inventory file.

Making an Ansible Job

 

Weaving Terraform and Ansible Jobs together through Process Definition

IT-Conductor has the framework for automating different activities by weaving them together so that they can run serially. Process Definition is a template for processes which can be scheduled to run at desired time. 

Terraform and Ansible Jobs together through Process Definition

In the picture above the Process Definition template signifies that Terraform Activity “Provision HANA SCS APP Infrastructure” provisions the required infrastructure; when it completes Ansible Activity “Install HANA SCS APP” starts immediately for installing the configured SAP packages. 

The terraform/ansible job can be fed input files and can generate output files. In the above flow, when the terraform job completes it generates output files as inventory, which are needed as an input file to the subsequent ansible job. The Process Definition supports this through implementation of the ‘Data Files’ feature. 

 

In Process Definition one can add multiple data files that can be read/written by its activities.

Multiple data files in process definition

 

Process Data Files

In each activity, these data files are available for selection as input or output data files.

input and output data files

 

Provision HANA Infrastructure DataFileInputs

Running the jobs in Process

Along with scheduling the Process Definition, one can manually run it anytime by clicking Run/Instantiate.

instantiate process definition

One can simply name the process and then run it.

run process definition

 

Then click object link on the popup.

object process definition

 

From there one can go to the Process Flowchart:

modify process definition

 

And view job status:

view job status process definition

 

For looking into each job progress, click on a particular job and then the execution log.

modify provision HANA infrastructure

 

execution log job status

 

One can also view individual ansible tasks.

view individual ansible task

 

view ansible tasks

 

Overall, there are several advantages for using IT-Conductor as an automation server:

  • IT-Conductor utilizes on-premises gateways as jump-servers alleviating the need to login remotely
  • IT-Conductor layer of security is enforced to ensure the role-based access and attribution of executed activities
  • IT-Conductor process graphical design, orchestration and scheduling capabilities are fully integrated 
  • IT-Conductor data that already configured/discovered, including host, application and cloud credentials and staged git repositories are seamlessly integrated into Ansible
  • Dynamic Inventories based on dynamic IT-Conductor system and application groups 
  • Ansible task log recording is available through IT-Conductor native UI/automated delivery over email
Dinesh Amatya

Written by Dinesh Amatya

Topics: Automate, Terraform, Ansible, Git Integration

FREE IT-Conductor Trial to Monitor SAP

Posts by Topic

see all

Recent Posts

Subscribe to Email Updates