Friday, 16 December 2011

My Approach Towards Automation

Article written by Sreenivas Mothukuru

Anybody can develop software applications but only QA can better test and certify applications.

During QA validation test engineers perform various types of tests to ensure developed application is as per the requirements/specifications. The tests include manual checks to some extent and automation then onwards. Now days the term "automation" has gained lot of significant in Software Industry. Most of the Software professional’s know automaton as some kind of testing done at browser level. At Browser level automation QA engineers automate repetitive tasks and test operations performed on the web browser. Test Engineers have to constantly adapt and modify the script if the web page layout changes. Good number of tools such as QTP, Silk Test, Selenium, etc. support automation. But, are we doing good job testing web applications only at browser level? The answer is NO. This approach is like taking same pill for any illness! Choosing the right testing approach at right time saves lot of QA time and helps team identify bugs in advance. After all “Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives”.

Let us consider some real time scenarios …

a) A project has 10 automation (browser level) test suites covering 10 modules with total 300 cases. If all test suites are run one after another the tests will complete in around 150 minutes. If all suites execute well, with all PASS then no issues. With this approach we can identify failures only when some or other suite fails. What if 9th or 10th suite fail? We will come to know about the failures only at the end. This is a blind way of executing browser level automation tests. Unknowingly, most of the Test Engineers follow similar approach, which means, testers are unnecessarily spending their valuable time and expensive resources on failure cases, rather than identifying such cases in the beginning. A Test Engineer can better plan browser level tests if he/she knows the failure cases in advance. But how?

b) For a project assume that developers are making configuration changes to implement Apache level redirections for huge number of newly added web pages. As always, apache changes are very critical and for every small development change testers must ensure that all existing and newly created redirection rules work fine. Also, the same amount of testing is required in various environments i.e. QA, Stage and Production."To Err is human". It would be error prone and very laborious if checks are done manually. Automation is the right approach, but, automation at browser level consumes some amount of QA time. Because for every page request apache redirection will happen and automation checkpoint will execute only when entire page renders on the browser. We can quickly test apache level page redirections if any testing approach proves faster compared to browser level automation tests.

c) In most of the web applications form validations play a critical role in ensuring correct data is processed by the system. Interestingly, it was found that form validation cases consume lot of QA time during regression testing. If the logic to show error messages after form submission is controlled at only business layer (service API's) then what is the advantage in automating all form validation cases at browser layer?

Is there any testing approach which helps us
>> identify
failure cases in advance to better plan browser level automation tests?
>> quickly test apache level page redirections other than browser level automation tests?
>> test form validations faster compared to browser level automation tests?

The perfect solution to the above questions is Http Level Testing.

Any Test Engineer can do testing task, but only a few can perform intelligent testing. Trust me, intelligent way of testing doesn’t cost much, when carefully done make the tester’s life simple.

Http Level Testing:
Checking individual system components at http level which have HTTP interfaces (JSP, ASP, CGI, PHP, AJAX, Servlets, HTML Forms, XML/SOAP Web Services, REST, etc). Http level tests can be used as a test harness to create a suite of [HTTP level] automated functional, acceptance, and regression tests.

Since the tests are done directly at HTTP level and http level scripts doesn’t care about
- Downloading all the embedded objects (i.e. images, advertisements, js, css, etc.,)
- Rendering issues
- HTML/JS changes on the page

Http level tests are not the replacement to browser level tests. In some scenarios http level tests work better compared to browser level tests. One must be intelligent enough to decide when manual, http and browser level tests are apt during SDLC.


In recent past, I performed some tests at both browser level and http level and below are my observations.

Module \ Method of Execution
Selenium (Execution Time)
Http Level (Execution Time)
Module A (Form Validation)
7 Minutes 1 Second
2 Minutes 5 Seconds
Module B (Basic Functionality)
3 Minutes 47 Second
57 Seconds
Module C (Apache Redirections)
2 Minutes 5 Second
25 Seconds
Total Execution Time
12 Minutes 53 Seconds
3 Minutes 45 Seconds

Total execution time for 3 scripts at browser level is 12 Minutes 53 Seconds.
Total execution time for 3 scripts at http level is 3 Minutes 45 Seconds.

After some R&D interestingly I found that http level tests run 70% faster compared to browser level tests. In the word of Information Technology where time is money, effectively managing the time using intellegent testing approaches will defintely result in greater productivity.

Question: "Is there any testing approach which helps us identify failure cases in advance to better plan browser level automation tests?"
Answer: Http test scripts can be used to do initial checks to know the state of AUT (applications and web services). The outcome of http level tests must govern what browser level automation test suite(s) to kick off. This approach helps test engineer to know the failure modules/cases in advance and target applications which are up and running.
Question: "Is there any testing approach which helps us
>> quickly test apache level page redirections other than browser level automation tests?
>> test form validations faster compared to browser level automation tests?
Answer: Since the tests are done directly at HTTP level and http level scripts doesn’t care about "Downloading all the embedded objects (i.e. images, advertisements, js, css, etc.,)", "Rendering issues" and "HTML/JS changes on the page", apache level redirection and form validation cases can be tested much faster compared to browser level tests.

To become an expert at http level testing one should know http protocol, http request & response formats/types, a few add-ons (Firebug, Fiddler, etc.) to monitor http request & response code and the attitude to learn & implement new concepts.
----------------------------------------------------------------------------------------------
I hope the time you spent on reading this blog didn't disappoint you!
Please share your feedback on this topic and let me know if you have any questions.
----------------------------------------------------------------------------------------------

4 comments:

  1. Do you have more info on http level tests? Scrips that perform http level test. I am quite interested in the apache redirects testing using http level tests.

    ReplyDelete
  2. Yes. I'll frame the content (in continuation to this ...) and share information based on the response to this topic.

    Thanks,
    Sreenivas

    ReplyDelete
  3. Hey,
    Check my blog to know about http level tests for blogger site. Example shows 2 apache redirection cases and 3 form validation cases.

    ReplyDelete
  4. It is really a great work and the way in which you are sharing the knowledge is excellent.Thanks for your informative article

    selenium training in bangalore|

    ReplyDelete