Over the years I have worked with organisations that traditionally have a silo’d approach to delivering changes to their business-critical IT systems. As these organisations embark on an IT transformation journey, they Invest heavily in the technology uplifts and migrations but struggle to standardise how these changes are managed and tested.
Typically, these projects are planned, resourced, and delivered using a team that is spread across multiple projects. The overall goal is the delivery of the project. The overall goal is not implementing a quality process. But why would that be an issue?
When employing an adhoc method to testing, the project team and core users and others are drafted in to ‘test’ the features of the new project. The project and organisation are then relying on their expert knowledge to ensure that any changes or new features work as designed, and don’t impact on other dependent systems. This may work for a small change, but small changes can have large impacts that may not be immediately obvious.
Informal or ad hoc testing also struggles to manage defects and understand where these issues arise from release to release. Are the same components or vendors the source of recurring issues? How will we know what systems and components will be impacted if we make these planned changes? How will we know if existing functionality will work after we make this change?
With no agreed formal approach, the test cases that are being executed are not being documented. This means that they are not repeatable by anyone except the SME who will be doing the test. It also means that the overall quality of projects is not quantifiable, so the team will have no real metrics from which to gauge project status from a quality point of view.
The main challenge is how to implement a consistent QA process across all the different projects. These projects may take very different approaches to QA and testing, some are mature and well managed, and others have no formal processes and rely on the availability of expert users to ‘test’ change in an adhoc fashion.
Implementing change in this manner regularly manifests as poor quality or buggy system, poor user experience and a large number of post deployment change requests, integration issues, defects, and a stressed project team.
Unfortunately, many organisations find out about their quality issues through their customers interaction and feedback, then need to manage the issues in public.
So, what strategies can be employed to mitigate these risks and prevent situations like this from happening? As any PM will know, what gets measured, gets managed. The typical KPIs or metrics for a QA and testing project are listed below. All of these contribute to measuring a projects ‘quality’ and inform the project team of issues that need to be addressed long before the product meets the end user.
- Total Number of Test Cases
- Tests Not Complete
- Tests Failed
- Tests Blocked
- Tests Not Run
- Tests Passed
When a testing process is in place and test cases are being managed and maintained, the QA team can then turn their attention to Automation.
Automation of testing and QA should be a high priority goal for any organisation. But it’s a long way off without a consistent QA approach.
Automation requires investment in the following:
- Selecting the correct automation framework for your technology stack.
- Training of the people to create automated test cases.
- Training in Source Control.
- Training in Continuous Integration (CI)
Mastering these key automation tasks will allow your team to harness the true power of automation.
Based on my experience of delivering managed services to our clients, I’ve identified the key benefits below.
- The QA team become SMEs on multiple systems across business units and will understand dependencies and integration points between systems.
- Consistent and managed process for test creation across all projects.
- Consistent reporting to stakeholders for all projects.
- Communications improves across the organization.
- Automation scripts for all projects will follow the same source control procedures.
- Solutions to problems on one project will be available to all projects.
- CI knowledge and pipeline set up will be applicable to all projects.
- Quality can be measured through consistent KPI tracking.
- QA Shared Services will increase QA productivity, efficiencies, and reusability of tool and solutions
A Shared Service approach will create a consistent and focused QA team that is aware of all current and future change requirements within your organisation and will implement a reusable and best practice QA framework to support delivery of those projects on time and within budget. A consistent approach to software Quality will ensure your organization is building a culture of Excellence.
