Test Automation Framework

What is a Test Automation Framework?

A test automation framework is a set of universal guidelines and rules used to produce beneficial results of the test automation activity. The automation framework brings together function libraries, test data sources, object details, and other reusable modules and methods. Automation framework makes testing consistent. Some of the guidelines are as follows:

• Rules for writing test cases
• Coding guidelines for creating test scripts
• Input test data templates
• Object repository management
• Log configuration
• Test result and reporting usage

What are Test Automation Framework development challenges?
Some of the challenges that you come across while using frameworks are:

• Ability to understand what needs to be achieved out of test automation
• Identifying requirements from multiple areas to design automation Framework
• Tool identification

What are the attributes of a good Test Automation Framework?

• Modular: The framework should be adaptable to change.
• Reusable – The commonly used methods or utilities should be kept in a common file which is easily accessible to all the scripts.
• Consistent – The test suite should be written in a consistent format by following the coding practices.
• Independent – The test scripts should be written in such a way that they are independent of each other.
• Logging – It is good to have implemented the logging feature in the framework.
• Reporting – Once the scripting is done, we can have the results and reports sent via email.
• Integration – Automation framework should be such that it is easy to integrate with other applications

What are good coding practices that should be followed while writing test cases for automation?

Certain good practices that you should follow when writing test cases are:

• Use comments at the appropriate places
• Maintain separate files for reusable functions
• Follow coding conventions
• Run scripts regularly

What are the different types of Automation Testing?

Different types of test automation are unit tests, GUI tests, and functional tests.

Unit tests are usually done during the development phase to reduce the bugs

GUI tests are done at the testing level, where tests are scripted to test how the user interface of an application behaves.

Functional tests are also done at the testing level to simulate the functional scenarios to test the application with provided input and for expected output.

What is data-driven testing?

The aim of data-driven automation testing is to simplify the testing process involving complex and huge data sets. In Data Driven Testing the test data includes input, expected output, and a result field. These fields are listed in files like CSV files, excel files, text files, XML files etc. These files are then fed to automation tool for execution, which in turn compares the expected and actual data. Then the obtained results are documented in the result field.

What are the important modules of Test Automation Framework?

• Test Assertion Tool: This module will provide assert statements for testing the expected values in the application under test.
• Data Setup: Each test case needs to take the user data either from the database or from a file or embedded in the test script.
• Build Management Tool: Tool that you can use to build a framework so that you can create test scripts.
• Continuous integration tool: Continuous Integration/Continuous Delivery tools are required for integrating and deploying the changes done in the framework at each iteration.
• Reporting tool: A reporting tool is required to generate a readable report after the test cases are executed. This way you can get a better view of the steps, results, and failures.
• Logging tool: The logging tool in the framework helps in better debugging of the error and bugs.

What do you think are the use cases where implementing automation is not suggested?

100% test automation is impossible. Manual testing is still necessary. Tests that cannot be automated are:

• Ad hoc testing – Here testing is done without preparation and writing test cases. During ad hoc testing, a QA specialist randomly tests the functionality of the system; his aim is to be creative, “break” the system and discover flaws.
• Exploratory testing – During exploratory testing, a QA specialist tests a product like an explorer, relying on his personal experience. After getting the idea of a product’s functionality, a tester designs test cases which he uses to further test the product. In exploratory testing, the tester needs to rely on his personal experience and design test cases in the testing process.
• User interface testing – Here, a QA specialist validates the properties and states of interface elements. Also, they make sure that design elements in the interface match elements in the final layout.

What are the advantages & disadvantages of using Automation Testing?

Advantages of automation testing are:

• Improves the reliability of tests
• Reduces maintenance cost
• Increases amount of test coverage
• Increases the speed of test execution
• Improves accuracy of the software tests

Disadvantages of automation testing include:

• Development and maintenance time is more
• The initial investment is high
• Skilled resources are required
• Environment set up is complex
• Debugging test scripts is difficult

What are the advantages of using an Automation Framework?

The advantages of using test automation framework, are:

• Re-usability of code
• Reliable recovery scenarios
• Maximum test coverage
• Low maintenance cost
• High Return of Investment(ROI) in the long run
• Minimal manual intervention
• Easy reporting capabilities

What are the different approaches to Test Automation?

Code-Driven Testing: Here the focus is mainly on test case execution to find out if the various sections of code are performing as per expectations or not. This testing approach is a popular method used in agile software development.

Graphical User Interface (GUI) Testing: Applications that have GUIs may be tested using this approach. Testers can record user actions and analyze them any number of times. Test cases can be written in a number of programming languages like C#, Java, Perl, Python etc.

Test Automation Framework: Framework is a set of guidelines used to produce beneficial results of the automated testing activity.  It brings together function libraries, test data sources, object details, and other reusable modules.

When is Automation testing useful? Which test cases to Automate?

It is impossible to automate all test cases, so it is important to determine which ones to be automated first. There are some top candidates like:

• Repetitive tasks are primary candidates. Not only are those tasks boring, but they’re often the ones where mistakes are most common.
• Tests that require multiple data sets. Rather than manually typing in all the information into fields, automate the process to read in information from a data source, and automatically type it into the respective forms.
• Tests that run on several different hardware or software platforms and configurations.
• Manually exporting bulk of data, crunching the numbers, and making detailed graphs is time-consuming. You can invest in a tool or automation strategy that will do the task for you.

When should you prefer Manual Testing over Automation Testing?

There are certain cases where manual testing is preferred over automation testing, like:

• Short-time projects: Though automated tests are aimed at saving time and resources, it takes time and resources to design and maintain them. For example, if you are building a small promotional website, it can be much more efficient to rely on manual testing.
• Ad-hoc Testing: In ad-hoc testing, there is no specific approach. It is a totally unplanned method of testing where the understanding and insight of the tester is the only important factor.
• Exploratory Test: This type of testing requires the tester’s knowledge, experience, analytical, logical skills, creativity, and intuition. So human involvement is important in exploratory testing.
• Usability Testing: Here the tester needs to measure how user-friendly, efficient, or convenient the software or product is for the end users. Human observation is the most important factor, so a manual approach is preferable.

What are the different types of automation frameworks that are available?

• Linear Scripting Framework: It is a basic level test automation framework which is in the form of ‘Record and Playback’ but in a linear fashion. This type of framework is mostly used to test small sized applications.
• Data-Driven Framework: It is used to create test automation scripts by passing different sets of test data. The test data which includes input, expected output and a result field are stored in files like CSV files, excel files, text files, XML files etc.
• Modular Testing Framework: Here the testers divide the application into multiple small modules and create test scripts individually. These individual test scripts are combined to make larger test scripts by using a master script to achieve the required scenarios.
• Keyword Driven Framework: In this framework, testers use a table format to define keywords or action words for each method. Based on the keywords specified in the excel sheet test, scripting is done and tests are executed.
• Hybrid Testing Framework: As the name suggests, this framework is the combination of two or more frameworks mentioned above. It attempts to leverage the strengths and benefits of other frameworks based on tester’s requirements.

Why use automation frameworks? What are the benefits that they offer?

Using automation framework avails a lot of benefits like:

• The framework enables consistency in testing. It provides a universal standard for testers to achieve the specific goals of their automated tests.
• Automation frameworks, when designed and implemented correctly, they deliver frequent and stable automated test code
• With a proper framework, the code is easier to maintain and reuse
• Even with limited knowledge of how the test case is set up, a tester can depend on the framework to refer to simple statements and implement the test cases with ease.