Before you even start automating test cases you need to consider what are the right Automation/CI tools for your environment.
With so many open-source and licensing options to choose from this can be a daunting task.
Likewise, considering the skillset of your resources is important. There needs to be expertise in the tool if you are to demonstrate to the business the value that can be delivered by automation.
The cadence of software deployments is increasing. Everyone wants to deploy, deploy, deploy. Applications and systems are becoming more complex.
Whether you’re developing multi-functional software or services you will require a solid quality function throughout your development cycle to build a successful Automation/CI Framework.
Getting your QA team engaged early and often is critical. Catching potential defects/issues early in the cycle will save you time, money, and grief later.
Building a test case repository that is structured in a way that you can easily test your application under test (AUT) manually needs to be done first.
Your repository should be modularized to allow for the reuse of test functions/services.
This will minimize the maintenance of your test cases, increase reusability, and lead the way to a modular automation framework.
Remember, you want a framework that flows and is easy to maintain. Modularization of your test case repository makes it easier to create test plans for different testing cycles such as Functional, Integration, System (End-to-End), Regression, and Performance testing.
What is CI and CD?
Continuous Integration (CI) is a software development practice where members of a team integrate their work frequently, leading to multiple integrations per day.
Each integration is verified by an automated build, including tests, to detect errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
There’s often some misunderstanding about how Continuous Delivery and Continuous Deployment work together.
While Continuous Delivery is a software methodology that allows quality code to be produced rapidly with the goal of shipping new features, updates and patches to customers more regularly, Continuous Deployment takes things further. Each change that passes manual and automation tests is deployed to production automatically. Embracing development, version control, continuous integration, testing and automated deployments. Continuous Deployment is a logical end goal for Continuous Delivery, but it’s an approach that won’t necessarily be suitable for all operations.
Implementing test automation into CI process
Incorporating CI into your development and testing processes requires some basic steps and possibly changes to your operational practices. Let’s go through each one briefly.
Defining Test Cases:
Acquiring product knowledge, reviewing and creating test cases would be the first step before automation.
Defining an Automation Strategy:
Perhaps the single biggest problem with most test automation efforts is lack of a practical strategy. A practical test automation strategy is one that provides a pragmatic solution to address specific business needs with well-defined, measurable goals based upon realistic expectations.
Determine Automation Tool:
Identification of the right automation tool is critical to ensure the success of the testing project. Detailed analysis must be conducted before selecting your tool. The effort put into the tool evaluation process enables successful execution of the project. The selection of the tool depends on various factors such as:
- The application and its technology stack which is to be tested
- Detailed testing requirements
- Skill sets available in the organization
- License cost of the tool
Build Automated Testing Framework:
A test automation framework is a defined, extensible support structure within which the test automation suite is developed and implemented using the selected tool. It includes the physical structures used for test creation and implementation as well as the logical integration between components such as:
- Set of standards or coding guidelines, for example, guidelines to declare variables and assign them meaningful names
- Well-organized directory structure
- Location of the test data
- Location of Object Repository
- Location of common functions
- Location of environment related information
- Methods of running test scripts and location of the display of test results
Separate Automation Repository and Development Repository:
An automation framework should be treated as a separate software development project; it should have its own repository for code and build jobs.
If the changes build correctly, the automation tests should then run.
The tests show whether the changes affected the stability of the application under test (AUT).
Having separate build jobs would also remove false negative results by eliminating confusion as to whether the development project is broken, or the automation framework is broken.
Smoke Test Each Build:
When new builds are deployed, it should always be smoke tested. This is true for both manual testing and automation testing. This will quickly determine if the new build is ready for formal testing or not. Time is saved by having the CI system do the smoke testing, freeing individuals to work on other parts of the build.
Multiple Nodes with Multiple Configurations:
In order to take full advantage of a CI system, all product configurations should be reflected in the agent or nodes where the tests are run. If the end-user has the requirements of working on two browsers, there must be nodes with at least those two browsers covered. If the end-user requires the application to work on Windows 7 and Windows 8, with a minimum of 2GB of RAM, and for 32-bit and 64-bit machines. There should be at least four agents with the different configurations. The more agents available, the faster testing will complete if the tests are run in parallel.
Compare this to manual testing; testing everything on all configurations is tedious and time consuming.
What Types of Tests Should Be Run in CI?
The great thing about a CI system is that any tests that have been automated can be executed on demand. Depending on the scope of the changes being deployed you can determine what levels of testing need to be executed. Testing should cover GUI and database validations.
We work with clients in the USA, Ireland, and UK navigating them through the ‘Automation’ journey. Our purpose is to act as a sponsor and guide, ensuring they avoid potholes and reach their goals quickly and efficiently. We find breaking the journey into three phases works well. We call these phases QA Fundamentals, QA Automation kick start and QA Amplify. We can demonstrate the return from each phase, increasing the support and motivation for forward movement. It allows our clients QA team to expand their knowledge incrementally and increase their confidence.
We hope you have found this article informative, please like and share. We believe in sharing knowledge and experience. Driving dialogue and conversations is crucial to enhancing the role of QA within organisations.
Contact us at info@celticqa.com if you would like to continue this conversation.
