Benchmarking in the Cloud: Generating Cloud Baselines

  

Performance and configuration baselines are critical in understanding the current state of your application environment before migrating to the cloud. Without generating cloud baselines, measuring your cloud migration success becomes a challenge.

At the beginning of your cloud migration journey, it's crucial to have an established set of KPIs that lays the groundwork for measuring the success of your cloud migration efforts. Subsequently, application discovery helps you understand your enterprise environment by identifying and mapping all its components and interdependencies.

This article will cover how generating cloud baselines contributes to achieving the ultimate objective of migrating workloads to the cloud.


Table of Contents

What is Benchmarking in the Cloud?

Questions to Keep in Mind

Benchmarking Challenges

    1. Long Preparation & Execution Time for Manual Benchmarking

    2. Dynamic Nature of Distributed Systems

    3. Performance Isolation

Generating Cloud Baselines in IT-Conductor


What is Benchmarking in the Cloud?

Benchmarking in the cloud is a practice that measures and records the existing performance and configurations of an application environment. It is performed at an early stage of your cloud migration timeline. In the 5Ds of IT-Conductor's Cloud Migration Strategy, we call it the Distill stage, immediately following the Discovery stage, where application discovery occurs.

Cloud baselines and application discovery complement each other during the Validation stage, also known as the post-migration phase, where the cloud migration assessment occurs. Cloud baselines provide a reference point for evaluating the success of the migration, while application discovery ensures you have a complete understanding of your environment. Together, they facilitate a seamless transition to the cloud by offering insights into performance metrics, resource dependencies, proper sizing, and optimization opportunities.

Questions to Keep in Mind

When generating cloud baselines,  consider the following questions to help you plan, measure, and document the baselines of your existing monitored systems.

  • What are the peak values (lowest and highest) of the metric being measured?

  • What is the average value of the metric being measured?

  • What time range should be used when measuring the peak and average values?

  • How many data points should be used when measuring performance metrics?

  • What are the error rates of transactions/processes/metrics in the application and infrastructure domain?

When assessing user experience metrics, revisiting the business objectives outlined during the initial planning phase is important. From there, generate baselines that are aligned with these objectives. For instance, if your business aims to improve transaction processing time, you can measure peak and average process times for a defined number of transactions within high and low user activity time frames.

Benchmarking Challenges

So far, we've discussed why benchmarking is important in cloud migrations. It is also beneficial to consider the possible challenges you may encounter when generating performance and configuration baselines.

1. Long Preparation & Execution Time for Manual Benchmarking

Gathering data when generating baselines may take some time if you don't have a tool capable of exporting performance and configuration reports. Performing benchmarking manually means logging in to the device(s) directly, capturing data points from manual queries or GUI-based wizards, refining them to support your defined business objectives and established KPIs, and organizing them according to the structure of your environment.

2. Dynamic Nature of Distributed Systems

In distributed systems, various components operate interdependently, making them inherently complex and susceptible to changes. These changes can occur due to varying user demands, software updates, hardware failures, or scaling adjustments.

When migrating distributed systems to the cloud, its dynamic nature makes it difficult to predict how they will behave in the new environment. The performance observed in the on-premises environment may not necessarily translate directly to the cloud. This complicates establishing performance baselines, as data may quickly become outdated as conditions change. Therefore, ensure your systems are set up for monitoring before migration. This enables you to adapt real-time benchmarks to synchronize with the changing behavior of systems during and after the migration phase.

3. Performance Isolation

The behavior of an application in a distributed system can be affected by other applications and services running simultaneously. This makes it difficult to measure the performance metrics of each component. Without a tool capable of identifying these components individually, it becomes challenging to isolate their performance. This lack of isolation means that issues within one component can affect others, potentially decreasing service quality or overall system performance.

Generating Cloud Baselines in IT-Conductor

The typical performance baselines you must capture are the resource utilization metrics such as CPU/Memory, storage, and network. In IT-Conductor, you can capture those metrics from the service grid. You can quickly drill up/down components in the service grid to capture the metrics needed to generate performance baselines for your source environment.

Expanded Service Grid in IT-Conductor

Figure 1: Expanded Service Grid in IT-Conductor

Aside from these performance metrics, you need snapshots of your source application environment's last working configuration state. This will make it easier for you to revert to its last-known good state should you encounter issues in the deployment stage.

You can use the landscape discovery information from the Discovery stage to capture the different components running in your source environment.

SAP Landscape Discovery

Figure 2: SAP Landscape Discovery

You can export landscape inventory data with the compute, storage, and networking profiles.

SAP Landscape Report

Figure 3: SAP Landscape Report
 

IT-Conductor is also capable of generating customized reports depending on your requirements. For instance, in SAP landscapes, you can define what SAP transactions and jobs you need to measure and then generate a report for your source environment. Then, after the migration, you can generate the same report(s) for the resources in the cloud.

These baselines will be your starting point for comparison during the validation stage after completing the migration activity. You can use them to quickly perform targeted tuning or revert to the previous state if necessary.

In summary, baselines are generated from the Distill stage, and they provide important inputs for the subsequent activities such as: