Everyone wants to automate their testing, right? But it’s not a small task to take on. Automation can be seen as a solution to many problems (we need to speed up our delivery time, manual testing is a bottleneck, etc etc) but it also can create more problems for you if you don’t spend the time and invest in the correct process. The IT landscape is littered with automation projects that floundered. Nothing hurts more than investing a large amount of time and money into a tool or framework that isn’t used and produces no return.
I don’t want to pretend that this post will cover every action that you need to take, but hopefully, it will give you a guide to making the right decisions and lay the groundwork for a successful outcome.
So let’s start with your current QA Process. the current culture and behaviors of your QA team need to be understood. If you don’t have a clear understanding of that, start there. What are the steps you currently go through to get your product out the door.
Most people say “we do Agile” and then when I say ‘ok describe your agile process’, they then describe a blend of waterfall and agile with a daily stand-up or some other blended process that has evolved over the years to deal with various challenges that the QA and Dev team have to handle. Your business is unique and your challenges are unique, the right process is the one that works for you. But you need to understand where you are now.
Do you have a budget? If not, then you need to get one. Automation is not a cheap endeavor and needs to be costed correctly. Not just the cost of implementing the framework but also maintenance and hardware/environment needs. All tasks take time and effort. So you need to do some homework and get your estimates and business plan together.
What problem am I Solving?
What’s driving the need for automation? Usually its speed. Speed of test execution. Speed of deployment. Speed of getting new features tested and ready to release, and then being able to repeat the whole process again, and in the case of CI/CD doing it multiple times per day.
So what to automate? Which tests are candidates? Do we just grab every test in our regression suite ( you do have a regression suite, don’t you? ) and start scripting?
This is one of the decisions you need to make early on, but to make a start and see a return on the investment early, start with a ‘smoke test’ containing the ‘business critical’ functionality that is stable and doesn’t change much from release to release. If you are choosing to automate less stable features, then your automation tests will require more maintenance to keep it up to date. Maintenance is one of the hidden costs in automation.
Tools
There are a huge selection of automation tools available now. I’m not going to start listing or comparing them. The web is full of reviews and comparisons, and you do need to do some homework, but essentially there are two camps. Open source or license/subscription based.
Typically the opensource (for example selenium) require higher technical skill level to implement and maintain. But have no upfront cost. The license/subscription tools are designed to appeal to the less technical QA teams. That doesn’t mean that you don’t need to have coding skills to get the most out of them. But they can get you started quicker. But just because you can start scripting earlier, doesn’t remove the need for planning and designing an overall automation framework.
All tools claim to provide a solution to you any problem. It is unlikely that one will be sufficient to cover all you need and all will require some level of coding to get the maximum from your scripts. Record and Playback is not going to be a robust solution to any complex application. This is especially true if your product/application is made up of multiple platforms and channels, has an internal legacy (desktop) systems, client facing e-commerce platforms and a mobile element. So be prepared to implement a blended hybrid framework.
How will your automation fit into the overall dev/QA process that you currently have? What are the triggers to start an automation script? Will it be a manual task for a QA engineer or integrated into your Jenkins pipeline?
Most tools now should integrate with the likes of cucumber to manage deployments and while this may be a manual task initially. Planning for it at the start will save you a lot of time and rework later on. Ideally, your automation should trigger when a build is deployed to a QA environment.
To sum up…
What is the overall QA process and where will automation fit? Tool selection, big list out there.
Cost, what’s our budget, are we going opensource or a licensed tool?
Time, what’s the schedule.
Environment, where will we script, test and run these scripts?
Skill sets, who will do the automation, do we have the resources available to assign to it? Are they the right skill set?
Planning, is this a project or a wish? 15% of the time and budget needs to into planning.
