Test Automation Scripting Techniques

Introduction

In the below section I will be discussing scripting techniques used in various test automation projects. Advantages and disadvantages of implementing the techniques.

The various test automation scripting techniques are as follows
1. Linear
2. Structured
3. Shared
4. Data driven
5. Keyword driven

Linear scripting
Linear scripting approach is a simple record and playback used by a test
Engineer to automate a test flow/ test case of a system.
It can contain some redundant functions etc that may not be required at times.

Advantages
Test case is defined by script
Very low development costs
Programming skills required by testers are negligible

Disadvantages
A non-structured way of programming
Data is hard coded in scripts
Scripts are fragile, and hence maintenance costs are high
No sharing of scripts done and hence efforts may be repeated

Structured scripting
Structured scripting uses control structures in the scripts. These control structures enable testers to control the flow of the test script/ test case. The control structures are typically ‘if-else’, ‘switch’, ‘for’, ‘while’ conditions/ statements that help in implementing decision making in script, in perform some tasks iteratively and capacity to call other common functions that house
commonly needed functionality

Advantages

A structured approach to scripting
Test case is defined by script
Script maintenance required is less since they are robust, maintenance costs are relatively lower than liner scripting.
Planning is required to some extent

Disadvantages
Programming costs are slightly increased relatively to liner scripting
Intermediate programming skills are required by tester
Data is hard code in scripts
Knowledge of scripting language is required in addition to programming knowledge

Shared scripting
Shared scripting is one of techniques in which the scripts representing a application behavior is shared amongst other scripts. This means that the application under test common functionality is scripted as shared scripts and these scripts are then called by other scripts. This makes the scripts modular in terms of common functionality. Such scripts due to the functionality can also cross the AUT boundary and can be used in other software applications as well.

Advantages
Scripts are structured
Test case is defined by script
Development costs are reduced relative to structured scripting since the effort duplication is eliminated
Intermediate programming skills are required by tester
Script maintenance and maintenance costs are relatively lower than linear scripting

Disadvantages
More planning is required due to modularity of scripts

(To be continued......)

How to Meet the Customer Expectations

What is meant by success of every project? Delivering the end product to the client.

All projects work hard and deliver the end product, then why do only some projects are always appreciated and some are not? The project is appreciated by the client only when the team meets the expectations of the client. This article will help the teams to deliver the end product along with some useful information so that the expectations of the client are met.

Before discussing further always believe that the Clients feedback contributes the most when success of the project is taken in consideration.

Usually every customer has the following expectations:

1. On-time delivery: A customer expects that the end product is always delivered in time by the Off-shore team. Customer pre-plan his/her schedule, however the priority may change based upon the results delivered by the off-shore team. When the team maintain the time consistency the customer also gets ample time to prioritize his/her work instead of taking last moment decisions.
It is very hard to keep the delivery time always intact with the time scheduled by the customer.However, proper planning and team effort together can make this task easier.

2.Representation of the deliverables: Not only the work you perform should be good, the presentation of your work counts a lot, as all are more interested towards the decisions made by the team.
A status report should containing the following information:
a)The tasks performed by you-gives idea what are functionalities covered.
b)Comparative analysis with the previous build-This tell the graph of the product(Improvement/Worsen)
c)Make a list of all the bugs posted in the current build and identify the module that is most effected(which contains a lot of bugs)-This helps the customer to force the development team to work more on that particular module.
It is the responsibility of the QA to present there views and decisions on a release build so that the customer can finally decide whether to release or not.

3.Metrics: Though we have a bunch of metrics, the customer may not be required all the metrics. Identify the metrics that are required and provide the same. For example: For the release build, if the customer is provided with Test Coverage metric, this will increase the customers confidence level.

4. Mutual Relationship: The relationship will always be in a good health if 'Give and Take' policy is opted. Do not hesitate to work on a n Off day if your customer requests. At the same time, do not hesitate to ask for a compensatory Off when the team requires. Always Giving or Always Taking will pull down the health of the project.

5. Feed Back: A team should always request for Customer feed back in regular intervals. The feed back should be compared with the previous one and necessary steps should be taken by the team to overcome the issues mentioned by the customer.If you have performed extraordinarily and you customer fails to recognize, then identify the reason and try to overcome it.

Optimizing the time allocated for Regression Testing

Regression testing can be defined as process of testing changes to a programs to make sure that the older programming still works with the new changes.

Regression testing involves reuse of tests to manage risks of change. Usually the regression test is performed by repeating the exact same test as before. But in the case where the test bank is huge and the time allocated is not sufficient regression test should be properly planned ie prioritized. In such cases one should be able to isolate the test cases that are important and may be affected.

The following points can be considered to build the regression test bank:

1. Identiy the modules/functionalities where the bug fixes have been performed. These modules should be tested first as there is a change in the code. You can allocate 50% of your time for testing these modules.

2. Identify the modules/functionalities that are interrelated with the modules where the bug fixes have been done. Perform an Integration test for these modules. This can be allocated 25% of your time.

3. Do a high level smoke test on the rest of the modules. This can be allocated the rest 25% of the time.

The time allocation always differ upon the number of modules to be tested.

This way we will have a good test coverage and also we can ensure that we have performed a good regression testing.

Root Cause Analysis for errors Encountered

OVERVIEW:
We know that Debugging is the activity performed by the developers for fixing a defect. But how good it would if a tester can identify the cause before this process is performed. Identifying the statement that is causing error without having the code is not possible. But identifying the module or the area of the software that is causing the problem is very interesting. This activity is called Root Cause Analysis.It is performed by tester to identify why the error might have occurred.

BENEFITS:
The following are some of the benefits gained by this activity:
1.Developer can identify the module that is causing error easily. As, a result the chances of reassigning the bug to the tester is reduced.

2.He/She can make sure that similar errors have not missed. Quality of the software is improved as most of the errors are uncovered.

3. Testers knowledge on the internal structure of the software is improved-Reputation increases.

HOW TO:
While testing we encounter different types of errors. It is not possible to identify the cause for every error encountered. The following are some of the common errors for which the root cause can be analyzed directly:

1.Application Crashes: When ever an application is crashes the entry is written to the Application/System Events. Look into these logs so that you can identify the reason why the application has crashed.

2.Errors during installation: Before performing installation through an MSI make changes to the following:
HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Windows\Installer
Value Name: Logging
Type: REG_SZ
Data: voicewarmup
By enabling this registry MSI***.log file will be created in the temp folder. All the installation errors are logged into this file. Using this information you can identify the error occurred by the Error code.
Description for all the error codes can be found at http://support.microsoft.com/kb/229683

3.Fatal Error Exceptions: When you start a program you encounter error message similar to A fatal exception XY has occurred at xxxx:xxxxxxxx
Fatal exception errors are codes that are returned by a program in the following cases:
* Access to an illegal instruction has been encountered
* Invalid data or code has been accessed
* The privilege level of an operation is invalid

When any of these cases occurs, the processor returns an exception to the operating system, which in turn is handled as a fatal exception error. In many cases the exception is non-recoverable and the system must either be restarted or shut down, depending upon the severity of the error.

XY - the actual processor exception from 00 to 0F. (Note that the "h" that is listed after the 0E, 0F, and so on, in the explanations below is frequently omitted from the error message.) The xxxx:xxxxxxxx value represents the enhanced instruction pointer to the code segment; the 32-bit address is the actual address where the exception occurred.

4. Program Exceptions: When the program is not able to handle the process then program exceptions are thrown. For example: consider a text field which accept only alphabets. When the user enters an integer, the program should be able to handle this input properly by throwing a friendly message such as "Invalid Input" rather than crashing in between. These errors are clearly understood as the error contains word "Exception".

5. Memory Leaks: Every program should successfully unload from the memory when exited. If the system resources are not set to free after the execution of the program, the system becomes slow and the effect can be clearly observed observed in the Task Manager as the CPU cycles are continuously used by this program even after exit.

The more the tester gains the knowledge about the structure of the software, this activity becomes very easy.

Installation Testing

Installation testing is a part of testing that is performed on every product. Though it is the basic testing that needs to be performed on every software most of the testers take this testing very casually. Some testers always look at the front end (ie the messages displayed during the installation) rather than the back end( ie files that are getting installed).

It is responsiblity of the tester to ensure that the files/registries created during the installation process are successfully removed during the un-installation. How do I know the list of files that are created during the installation? What are the registries that are created? Does my software depend on any of the system files(dll etc) which are delivered with the Operating system or any other software? Does my software make any modifications to any of the existing registries?

These questions arrived in my mind while starting the testing of a software product. Some times I use to encounter with Errors during Installation. These errors forced me to do reverse engineering for the product. This helped me to understand the software in detail. The process I started to understand the flow of the software is as follows:

Every setup file for Windows is delivered as MSI or EXE file. The MSI(Microsoft Installer) contains the installation information along with the files. To know the files that are packed with the MSI I started using the tool "Orca". This tool gives me the details of the files that are packed in the msi(name of the file, size of the file etc). The next method to know the files that are created with the MSI of the product is to perform Administrator install through command line. This method will extract all the files into the target directory and as a result, one can easily identify the files that are packed with the MSI along with the path of the installation.

An exe can be built with different programming languaes. The other way is an self extractor/self installer exe that is built with iexpress.exe(inbuilt tool) in Windows. This kind of exe can be simply unzipped using 7zip.

There are also couple of tools like Process Manager developed by Microsoft which keeps a track of all the registries and files that are accessed/created/deleted while a process is running. This tool helps to build the data flow of the entire product.

Using these tools the Installation testing(including uninstallation) can be performed in an effective and efficient way.

For Newbies of Testing

Though Software testing is becoming very popular, I found that the effective way of testing different kinds of products is not shared. Having good experience in software testing especially for products like security suites I have decided to share my experience through this blog.

To start off with let me introduce to some facts for all the testers who are new to this field:
1. It is impossible for performing exhaustive testing of a product.
2. A black box tester may feel that there is no necessary of learning the concepts of a programming language-but these concepts will always help to understand the structure of a software though the code is not available.
3. Quality of the software should be the main aim of a tester.
4. Testing of a software is as difficult as development of a software- hence it requires more concentration.

The above facts should always be taken into consideration when one accepts testing as their career.