As a provider of Software QA & Testing services, we’ve spent a lot of time implementing QA frameworks and automation solutions for our clients. Over the years we have encountered many situations, various issues and a wide selection of tools being used for testing and software QA.
With this wide sample of projects, we’ve seen patterns emerge where our clients are encountering similar issues, while also having similar goals. Here are a list of our top Eleven pitfalls that will hinder your QA and testing process from fully delivering to your needs. Hopefully this will help you avoid or resolve them.
This first section relates to test cases, and how they are written and managed, many of our clients want test automation implemented, but very often the fundamentals need to be addressed first. Namely what are we testing, why, what are the expected results and where do we maintain these test cases.
1. Disorganised Test Cases
- Automation projects that begin with poorly written, disorganised test cases (TC’s), that do not have clear and concise test steps have a reduced chance of success.
- Projects need to allocate the time and resources to ensure that the automation test team is provided with well written TCs, with clear and simple test steps.
- What goes in, comes out.
- Not everything should or needs to be automated, you need to decide what the goal of the automation suite is. Regression? Smoke Tests? Deployment Validation?
- Select TCs for automation – ensure the steps are well described and have detailed expected results.
- Prioritise those TCs.
2. Test Complexity
- Tests that include too many steps or try to test too many things in a single test case do not yield helpful information and are difficult to maintain.
- This approach will aid the automation team to create short, simple, reusable, and maintainable modules for the tests.
- Keep the test granular and specific.
3. Disorganised Test Suite
- A large, disorganised test suite can make it difficult to understand what is being tested and difficult to find individual TCs.
- Use module, repository and smart folders.
- Name repository items, modules and TCs intelligently.
- Use run configurations.
4. Using the Selected Automation Tool Inefficiently
- Ranorex has all of the inbuilt capability that will be needed, for Web, UI, Database and Mobile testing, however there is a learning process to fully utilize it.
- Avoid trying to use Ranorex like an IDE for Selenium, there is no need to do this and it introduces unnecessary complexity.
- Similar to creating clear and concise test cases, keep the Ranorex modules short and specific, avoiding overly complex and long modules that will increase the maintenance effort.
5. No Report Analysis
- Reports generated during test runs, need a process to review and action any defects or issues that are discovered.
- Reports can be delivered to stakeholders directly of created via the CI process to create a dashboard of test status in your test management tool. For example, Jira/Xray or Test Rail.
- Little to be gained from automating tests, if test reports are not analysed and investigated.
- Schedule time and resource for reviewing test reports.
6. No Defect Management
- What happens after a defect or issues are identified by your automated test?
- Each QA team needs to ensure they manage the defects lifecycle after it has been found, automated tests can create a defect directly to your defect tracking tool when a test fails, but is that the best approach?
- Inefficient defect reporting and tracking can result in defects being lost and making it into the next release
7. Lack of Source Control
- Automated tests are a software project and need to follow the same process that developer would use on an application development project.
- Source Control (GIT, GitHub etc) will enable your team to work on the same projects without overwriting or losing any work due to team members working on the same project code.
- Ranorex uses Magic Merge to help automation engineers merge their code without errors or loss.
- Inadequate source control can result in work being lost and testers spending too much time resolving conflicts.
- Agree, write, and implement a simple source control process.
- Automation engineers need to be training on that process.
8. Lack of Scheduled and / or Triggered Test Runs
- To get the maximum return on automated test suites, they must be run at the appropriate times and on the correct environments.
- Setting up Continuous Integration (Jenkins/Bamboo etc) servers / build tools can be challenging but is vital to give the test automation the power of all the DevOps solutions available.
- CI will allow the QA to schedule the tests to run at specific times or trigger the tests to run when a specific event has occurred. I.e. new code base deployed to QA environment.
9. No Code Reviews
- Follow best practice and agree a naming convention for modules
- All code has bugs – automated tests have defects.
- Are tests failing that should pass, or worse are tests passing that should be failing?
- Schedule regular reviews of the automated test cases and modules
- Strive for continuous improvement, through training and sharing of knowledge
10. No Automation Champion / Project Manager
- In many cases the automation is a task that is assigned to QA engineers that are not wholly focused on Automation. Their main task is manual testing of new features and changes to existing applications.
- This is a risk as Automation needs to be owned and managed like other projects.
- Without the leadership, automation projects can become unfocused and inefficient,
- Pressure to get new features tested can result in the best practices not being followed, short cuts being taken and the automation becoming a burden and not delivering the value that is promised.
- Manual Testing is still a vital activity for Test Automation, new features, exploratory testing, report analysis, writing test steps, selecting TCs for automation, etc
- But automation is a specific skill, with specific skill requirements.
11. Lack of Communication
- Ensure all stakeholders understand the role of automation and the value in time saving and increased test coverage during the development lifecycle
- Agree automation goals and objectives with the test team and all stakeholders
- Clearly
As the cadence of releases increases so too does the need to execute testing in a speedy and effective manner. Test Automation facilitates the efficient validation of your software using specialised tools to create and run tests.
Successfully implementing test automation ensures your business remains competitive in the rapidly changing field of software. Since 2005, CelticQA Solutions a leading provider of Strategic and Tactical software QA & Testing services have helped many organizations improve the efficiency and effectiveness of their test cycles.
Our Automation Accelerator Program has been tried and trusted by many companies such as Sartorius Stedim Bio, Aarons Inc, CRH, Boston Childrens Hospital and Shaw Industries. These programs are designed to condense the time for demonstrating the ROI and increasing the value of automation to the business. Each client is at different levels of technical maturity however we have consistently witnessed Test Automation deliver 80% reduction in testing cycles and increase the IT teams agility and capacity. Check out our case studies to find out more. We cannot guarantee this outcome for every client but if this is of interest, you can contact us at info@celticqa.com.
Test Automation is often seen as a capability however building organizational Automation capability should be viewed as a platform. Allowing high performing teams to deliver high quality products or services to the customer and increase operational effectiveness and efficiencies.
