As well as delivering finished technological solutions as detailed in our portfolio, we often undertake strategic and advisory work for many large companies, particularly as part of our FS practice. It’s often their policy to not explicitly name or endorse third parties who undertake this consultative work, and as such this case study has been anonymised.
The brief: Quality control for a leading payments organisation
The entire banking value chain has experienced massive change, both in the proliferation of new products and providers, and in overall growth. This is especially true in payment provision, where the leading payment organisations have consolidated away from the banks, growing so aggressively through M&A and tech investment that they now rival the banks in market capitalisation.
With this enormous progress comes significant growing pains, and we were approached by one such global payment processor to help them investigate and resolve some growing deliverability issues. They have experienced enormous organic growth, scaling in every direction over time, and acquired many smaller providers, meaning they now offer an increasingly diverse portfolio of products to partners and consumers.
It had been identified internally that a potential quality control problem was emerging from their struggle to efficiently meet the market demand for their services. The primary symptom of this was a lengthy and complicated testing process which dramatically slowed down delivery, meaning they regularly missed targets to ship at the desired cadence. As a result, they brought in Elsewhen as a strategic partner to clarify the underlying symptoms of the problem, and make recommendations around implementing positive change.
We immediately established to them that QA was a holistic horizontal concern, so although the client had issues in UAT we expected that the cause of those issues came from a variety of factors across the entire process. They just ended up in UAT, as you would expect.
They had good processes in place, but lacked the visibility to determine if they were happening, which always leads to the suspicion that they aren’t. Internally the bottleneck slowing down delivery was identified as in their user acceptance testing (UAT)—the last phase of the Quality Assurance (QA) testing process before deliverables see the light of day.
We were tasked with looking at the UAT phase, and help them to reshape it if necessary by telling them what they were doing wrong, and what needed to improve.
The context: Maintaining speed and momentum at scale
In many industries, it’s a perennial issue that the largest firms struggle to match the agility and release cadence of smaller companies. In the highly competitive payments space where our client operates, this is simply not a viable way to trade. They must continuously improve, test, and ship new features, and secure new partners with products that will make use of their underlying payments technology. Hot on the heels of these larger payment specialists and merchant services are the fintechs, where it’s estimated 40% now focus on digital payments in some aspect.
It is therefore crucial for the incumbents to rapidly implement their payments technology with partners to maintain their position in the market. They need to be able to adopt agile practices, and ship at regular and predetermined intervals.
This is not straightforward, because as well as maintaining consistency of service without downtime, they also have to navigate the different regulatory hurdles of the many markets where their technology is used. It’s also essential that when the partner is live, the payments provider always successfully and seamlessly handles the funds between the merchants’ and customers’ banks. To not do so would be a disaster.
As such, quality assurance takes on a particular relevance. Before partner solutions hit the market, it’s essential they are thoroughly tested strenuously in all aspects. UAT—the process of verifying that a solution works for the user—is the last line of defence in that testing. If there are any underlying problems ‘left’ or earlier in the testing process, they are eventually caught by UAT, even if the issue is not UAT related per se.
Our approach: Going wide and then deep in our assessment
Before we could make our recommendations on how to resolve the pain being felt at the UAT layer, or detail a plan for lasting improvement, we needed to conduct a detailed assessment on the current internal state of play. This meant mapping out the current stages and processes, the described fail/pain points, the reasons for them, and the resulting impact of these.
We knew that understanding how product was approached within the company would give us an insight into the symptoms presenting themselves at the UAT stage. We commenced a two-month investigation, which for the most part meant sitting down with the wider team. We spoke to as many people as we could—recording who we met, their role, and the area they worked in—and hearing their versions of reality. Creating a safe space for the many team members meant they could break down the problems as they saw them, and start to build trust for the eventual implementation of our recommended solutions.
As is common with large organisations, the teams had become siloed from each other, so these versions of events varied widely. A lack of strong leadership within the QA function to join the dots and push through collaboration meant this siloing ran deep.
The Challenge: A delivery and testing cycle affecting time to market
It was clear that the UAT team did have identifiable problems—they needed more in-house personnel to meet the demand which had developed into an over-reliance on contractors, and as suspected they lacked strong leadership which led to them becoming increasingly siloed from the rest of the testing process.
However, as UAT is the last line of defence in the testing process, we quickly identified that there were wider issues in the QA practice that went beyond the scope of just UAT, but ended up here. There were lots of manual, time-consuming activities taking place, with duplication of work, issues found late on, and either not enough test automation or test automation happening too late on.
There were good processes in place in QA—they were using the SAFe scrum framework for their delivery process—but the internal buy-in, and implementation for that process to work weren’t there. There were also fundamentals to these processes which weren’t being observed, and the various teams weren’t communicating with each other, or clear on who was responsible for what.
Our strategic recommendations: Invest, align and shift left
Based on our deep dive into the people and testing processes, we were able to make a series of recommendations and assist with the initial implementation. Of course many internal staff were vaguely aware of potential solutions already, but they needed a centralised and independent plan to elicit change through the organisation. The most pressing recommendation was that they needed to invest in their QA—investing in resources, skills and rethinking the structure of the many teams.
They needed to stop the ongoing brain drain from staff turnover, by reducing their over-reliance on contractors, instead making empowered and skilled hires in the areas of bottlenecks. They could then start to centralise the team, and bring empowered leadership and knowledge to open the lines of communication between teams.
In terms of the actual structure of the teams, we recommended that they better align UAT with the product owners and scrum teams, in the ratio of one UAT analyst for every two scrum teams. We also reintroduced and specified how the scrum process was supposed to operate.
We recommended they shift left i.e. earlier in their testing —meaning earlier collaboration, earlier business risk assessment, and earlier and more automation of testing, rather than the reliance on manual regression testing at the UAT stage.
The Outcome: Resourcing, automation and new processes in place
We were able to act proactively on their resourcing issue, helping them to hire the right people by putting together a detailed process to follow, and overseeing the hire of one very senior internal team member before the end of our tenure.
Our approach to the personnel issues we had identified was a combination of restructuring some existing teams to help centralise the test team, but also bring in leadership to take the strategy forward after we left. In this way we began the process of centralising, clarifying and ultimately promoting ownership within the company.
By reinforcing how the processes of how good testing should operate, we were able to provide the structure for better and earlier collaboration in the wider QA processes. This unlocked the potential for quicker testing that was still rigorous, where problems were caught earlier and by the right person, who was in turn empowered to resolve it.
The primary outcome of our recommendations was a much better QA process throughout, as well as key improvements felt at the UAT layer. For example, the introduction of sophisticated automated end-to-end regression reduces UAT effort by 87.5%, thereby speeding up delivery and directly tackling the core business challenge we were brought in to investigate.
