Major American Airline

Business Snapshot

Efficiency increased

by 95% for test data creation with RPA

Business costs

reduced with extensive test automation

Reporting time

reduced by 75% with new visual dashboards

Overview

This major US airline operates both domestic as well as international flights around the globe. They have a complex business operation and run 500 applications with 15-20 major releases every two weeks. However, its manual testing approach allows the QA function to only cover 10% of these.

Since 2015, we’ve been working in partnership with the airline to automate their QA function testing and instil a culture of quality engineering throughout the software organization. Our work has saved the client millions of dollars and greatly accelerated the release of new software updates.

From a small project to apply test automation to one application portfolio, our partnership has grown to embed quality engineering across the airline’s entire software delivery organization.

A five-year journey from Quality Assurance to Quality Engineering

Our client is a major US airline operating flights domestically and around the globe. The client has an estate of over 500 applications that require constant development and maintenance. Owing to their complex business operations, there are a significant number of new releases every two weeks

Its small in-house QA team, was only able to provide release management assurance for 50-60 of the most business-critical applications, conducting mainly manual testing with the help of an external vendor. When we approached the company in 2015 with a proposal for test automation, they agreed to engage three ValueLabs QA professionals to assess what we could do.

Despite our small beginning, our relationship with the client has grown into a truly strategic partnership. Our five-year journey has seen the client evolve from manual quality assurance to a culture of continuous quality engineering across its entire software delivery organization.


Immediate improvements led to a long-term partnership

During our initial engagement, we increased the proportion of automated testing from 10% to nearly 30% for one application portfolio in just a few weeks. Building on that early success, we’ve worked continuously to elevate the speed and efficiency of quality assurance for our client, including

  • Creating an automated solution to the major challenge of test data creation
  • Building dashboards to provide actionable insights into test results and QA maturity
  • Introducing high levels of test automation across the entire application estate
  • Integrating security testing as part of a new DevSecOps model

Throughout the journey, we’ve consistently gone above and beyond to identify issues and propose solutions. We have been available and flexible for our client – especially during major increased activity. Today, we have a team of 236 highly motivated QA professionals and quality engineers working onsite and offshore, providing end-to-end delivery of QA and QE services across 95% of the entire applications estate.

Our methodology of working has enabled the client to release high-quality software faster proving to be cost-effective – even as COVID-19 brought about big and urgent changes to its passenger-facing processes. We’ve taken on our client’s objectives as our own, providing thought leadership and innovation that have increased the standing of the QA team within the client’s overall IT function. Our successful partnership is a prime example of our OneCompany® ethos in action.

Services Involved

Performance
Testing
Infrastructure
Management
Quality
Engineering
Robotic Process
Automation
User
Experience

Business Impact

Reducing test data creation time by 95% with Robotic Process Automation


Reduced test
creation time by 95%

Test data creation
in just two clicks

Increased testing capacity
without additional costs

Creating test data was one of the biggest bottlenecks to QA efficiency for our client. Every time a new process needed to be tested, a test engineer had to find data to test it on. A change to an online booking process, for example, would require test data like login credentials, flight data, and passenger data such as name and passport number.

With no central repository to pull the data from, engineers had to create it from scratch every time – a huge manual effort that took up 30% of the duration of each test. Our client and their existing vendor had tried to tackle the issue before using spreadsheets and scripts, but with little success. On noticing the problem, we immediately offered to build an automated solution using Robotic Process Automation (RPA).

We took this on as a ValueLabs initiative, securing budget and resources from our leadership to develop a solution at no cost to our client – a client-focused innovation approach we call The VC In Us. The resulting RPA solution reduced the time it took to create test data from an average of two minutes to an average of 20 seconds – a 95% difference in time.

Engineers can now create test data with two button clicks, as opposed to the complicated process of writing 10+ commands. Building the solution using existing tools from our client’s tech stack prevented additional licensing costs. Our automated solution caught the attention of our client’s senior IT team, who asked if it could scale beyond the 50-60 core applications assured by the QA team.

We demonstrated how we could apply this and other automation tools across all of the client’s circa 500 applications to vastly expand test capacity without increasing headcount. The conversation was a milestone in our partnership and saw us take on QA for release management for most of the estate.

Building dashboards to reduce test reporting times by 75%


Reduced time
taken to prepare performance test by 75%

Faster Analysis
and resolution of failures

Within client organisation
Elevated QA to a strategic level

As we became well-versed with our client’s applications, we started to notice areas of inefficiency with measuring QA activities across various application portfolios and testing types. In performance testing, for example, the client was using different tools for different tasks, so the overall results of performance tests were difficult to visualize.

We took the initiative to develop custom dashboards that unify data from the different tools for effortless evaluation. By configuring APIs and drawing all of the data into a SQL database, and using TIBCO Spotfire to analyze and visualize the data, we were able to provide insights such as the root causes of test failures, as well as trends in test results over time

In doing so, we reduced the time to visualize performance test results by 75% and provided actionable insights that could help to reduce failure rates in the future. Performance testing isn’t the only area where our dashboards have enabled our client to take an advanced approach to quality management. Another innovation we introduced is dashboards that provide a Quality Index Score (QIS) for every application portfolio.

They show the degree to which each application’s releases are tested, creating an at-a-glance guide to which applications and portfolios are most mature in terms of quality, and which need further attention.

The QIS dashboard was enthusiastically received by our client’s QA director, who used it to demonstrate key metrics to application portfolio owners – boosting the standing and reputation of QA within the organization.

Embedding quality engineering into CI/CD workflows


Reduced regression time
from days to hours across all portfolios

0ver 70% Automated
regression time, resources and freeing time

Transformed 80%
functional testers into skilled Quality Engineers

Throughout our five-year partnership with our client, we have been instrumental in embedding a culture of quality engineering across the entire software delivery function.

Beginning with the client’s 50-60 core apps and then extending to the 450+ other applications in its estate, we’ve introduced automation tools and practices that have saved vast amounts of time and effort and minimized the number of defects released into production. Our automation work has not only improved the quality of software releases but, also facilitated the client’s strategic transition in 2017 from a waterfall model to an agile way of working.

Test automation is now integrated into every stage of the CI/CD process, from initial test design through to regression testing – which is now 70% automated, with timescales reduced from days to hours. Throughout, we’ve taken the lead in advising our client on the best automation tools to use, and up-skilling our functional testers into quality engineers who are well-versed in applying the latest automation technologies.

Our testers are highly motivated and engaged in their work, this is attributed to delivering cost-effective and efficient solutions to the client. We were proactive and displayed skill in managing bottlenecks and backlogs with our client’s mobile application testing processes.

We proposed and implemented an automation solution at minimal cost to our client – establishing the cloud infrastructure, managing and maintaining mobile OS versions, automating test execution, and ensuring appropriate scripts are implemented.

Shortened delivery cycles by integrating security testing into DevOps processes


Reduced release cycle time
by 1 day with automation

Enabled faster
vulnerability identification and remediation

Reduced costs
Infrastructure for security testing

For any company that requires its online services and communications to be second to none, factors like web performance, in-depth analytics, and a consistent, intuitive UX are all vital elements for success. Our client’s continued relationship with ValueLabs ensures these capabilities are delivered – which means no more fragmented user experiences, slow loading times, CMS bottlenecks, or payment difficulties. Instead, the client has seen seamless customer experiences and streamlined internal processes from a partnership that continues to evolve.

Today, that partnership sees ValueLabs proactively identifying room for improvement in the client’s IT landscape, with a dedicated technical architect, project manager, dev and test lead overseeing regular technology and process improvements.

A five-year partnership that’s set to last


Right from the beginning of our five-year journey with our client, we set ourselves apart through our proactive approach, thought leadership, and client-focused mindset. In keeping with our OneCompany ethos, we have consistently taken on our client’s challenges as our own and found swift and effective solutions.

When our client started to run out of floor space for our growing team, for example, we leased a floor in the building opposite and had it up and running in just four weeks. When COVID-19 hit the airline industry and we had to downsize the team, our remaining team members worked around the clock in shifts to ensure vital passenger-facing functionality went live with no defects. Despite a change in leadership and approaches from other vendors, our client has chosen to stay with us for the long term in recognition of the significant value we continue to deliver.


We look forward to building on our successful five-year partnership and applying new quality engineering innovations, such as AI, across our client’s software operations.

Dharmendra Velivela, Senior Vice President, ValueLabs


Business Problems? Technology Challenges?

We’d love to help!

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close