Towards Developing a Selenium Based GUI Automation Test Pro: a Comparative Study

Towards Developing a Selenium Based GUI Automation Test Pro: a Comparative Study

Ms. Kadambari Metha.

1. Student, Department of computer Engineering, MGMCET,Mumbai University

   Kamothe-Panvel.

   Prof. Sachin Chavan.

   Co-Guide, Department of computer Engineering, MGMCET,Mumbai University

   Kamothe-Panvel.

   Prof. Manoj Patil.

   Guide, Department of Computer Engineering

   Datta Meghe College of Engineering, Mumbai University Navi Mumbai,India.

   Abstract The goal of this paper is to review different automation tools available for GUI automation like Selenium and QTP for web-based applications. Testing automation tools enables developers and testers to easily automate the entire process of testing in software development. This paper will help us to gain understanding of the GUI automation and recent attempts and trends in Selenium based GUI automation tools as well as QTP. Selenium is a software tool which works with different browsers, operating systems, various programming languages etc. Whereas on the other hand QTP is losing its business because of its limitation such as it support only windows environment, limited cross browsing functionality and high licensing costs. The paper will provide the comparison of Selenium and QTP based on different criteria. The comparison will lead us to state the need to develop new Selenium based automated test pro which will overcome the disadvantages of both Selenium and QTP. It aims to reduce the time of testing & human errors and to increase the efficiency and quality of the software product.

   KeywordsAutomated testing framework; GUI testing ;Manual testing; QTP; Selenium;Web application testing

   1. INTRDUCTION

      As the delivered product may contain defects every software development group tests its products before delivery. Test Engineers try their best to catch the defects before delivery of the product with the different testing methodology like manual but they end up in vain. To overcome drawbacks of manual testing method, automated testing is the used to test the product with increase in efficiency. The task which are highly impossible and which requires more time with manual testing can be easily repeated using automated tests. Automated software testing improves test coverage, the accuracy within short interval of time at low maintenance cost. The main advantage of test automation it improves the long- term efficiency of a software teams testing processes at low cost and its quite quick compared manual testing.

      Manual tasks are often involved when performing the acceptance test of web applications. Manually doing the task, the operating personnel human error can occur easily. It requires more time as well [1]. The web-based applications were containing more and more platform formation and more and more Complex business logic, along with the characteristics of the quick regeneration and short release

      cycle [2]. This basically requires the web application test cases to be of comprehensiveness, expansibility and efficiency. The Browser carrying on web applications also was emerging in diversification and pluralism [3]. The complexity and workload is also increased for the web application testing. As to test the application functionality of the web applications it needs to conduct repetitive testing which becomes quite tedious and thus we go for test automation in place of regression testing. To gain control over and to monitor the execution of the test cases, the test automation framework which is a strategic approach is useful [11].

      Pointing at some problems mentioned above of manual testing, the paper presents why the test automation framework is necessary for testing web-based application.

      Various test automation tools, such as Selenium, QTP WinRunner Load Runner, Robot, SilkTes are used widely for web application testing [4]. Selenium can be said to be the most comprehensive and cost effective of open source web test automation tools, because of its well support to the web application which employed the BackBase framework [5]. HP Quick Test Professional automation is straightforward and easy to understand. QTP is an end-to-end and complete software test automation solution that has built-in data table interface and Object Repository.

   2. LITRATURE REVIEW

      I have undergone literature review phase and evolved with a problem statement with the help of work that has been published till today in the area of automation framework design for web based application. Most published research on test automation frameworks presents case studies or feasibility studies.

      The paper [6] describes a test automation framework using AutoIt tool and an Excel Sheet as a Data Container for test data [02]. The framework proposed was tool specific and the disadvantages of using excel sheet as a data container were not

      covered, as excel sheet representation is not suitable for more complex and dynamic web applications.

      A paper [7] describes a browser based framework for testing UI of Web applications. The author concluded that the time required to create a test cases using browser agnostic UI framework can is reduced by 50% as compared to a manual testing approach..

      A paper [8] describes designing of the automation framework for testing web applications based on the Selenium and JMeter. We can use the automation framework to efficiently improve the reusability and extensibility of automated test.

      Another paper [11] explained Selenium Remote Control (RC) as a test tool that can be used with the web browsers and supports six high level programming languages. Selenium- java-client-driver-1.0.1 is used in this framework for driving the browsers using java. A client driver of Selenium RC provides only basic functionalities. Another effective method can be design for expanding the functionalities that creates a customized logger file and generation of customized test summary report by implementing procedure which will capture different actions performed by user automatically.

      The paper [9] states that one can select a testing tool based on the type of application need to be tested, budget, and the efficiency required.

      The paper [10] describes automated software testing has become necessity of companies because it saves both time and money.

   3. TERMINOLOGY

      As automation testing is more flexible and powerful than manual testing, different companies are trying to develop different automated testing tools for various web-based applications. There are two types of test tools.

      • Open source test tools

      • Commercial test tools

      1. Open source test tools

         Open source tools are typically created as a shared effort in which programmers improve the code and offer the developments within the community. These tools are usually available without any charge. There are many tools available for free of cost download such as Sahi, Slim, Jmeter, Selenium etc. In this paper we are concentrating on Selenium.

         1. Selenium

            Selenium was initially developed by Jason Huggins and released in 2004 for testing web-based applications. Selenium IDE is add-on for the Firefox web browser and its open source suite of tools. Selenium IDE can be easily downloaded from internet. Web development community mainly uses Selenium IDE to perform automated testing of web-based applications. The main criteria for selecting a particular automated testing tool is depends the type of application we are testing and the cost of using it. Our main motive is to perform black box

            testing o the web based application. Selenium supports a test domain specific language (DSL) to execute test cases in many software development programming languages such as Groovy, python, Php, C#, java, ruby, and Perl.

            Components of Selenium:

            1. Selenium IDE

               It is an add-on for Mozilla Firefox which is used for tracing the user actions and also play-back. Software testers can also debug and edit the test cases in above mentioned software development programming languages as Selenium IDE is an integrated development environment. Latest version of Selenium IDE is 2.5.0 released on 01/Jan/2014.

            2. Selenium RC

               It is a client server architecture / solution for executing linear / regression /integration tests. Selenium RC receives the commands from the Selenium server and tests are carried out via the browser. Traced different actions performed by user on web application can be obtained from the Selenium IDE in any supported software development programming language and then that actions can be imported to eclipse for test case/suite execution. It supports client drivers such as Perl, Php, .net, java, python, ruby and the Selenium server.

               Advantages of Selenium:

               1. Its open source, the tool is available for free of cost for download.

               2. Test cases can be coded in various popular high level programming languages, such as Perl, .Net, PHP, Ruby, Java and Python.

               3. User Acceptance Test is carried out with Selenium.

               4. Test execution can be done in parallel with other work so no other dedicated machine is required.

                  Limitations of Selenium

                  1. Does not support the windows application, supports only browser based application.

                  2. File uploads from local machine is not supported.

                  3. Partially supports dialog boxes

                  4. As it is an open source, it has no official technical support.

      2. Commercial test tools

         Different automation testing tools such as QTP, QA Test, QA Load, QA Wizard Pro, Win Runner, Test complete, Silk Test, Silk Performer are available for performing Functional, Performance Testing, int his paper we are considering QTP test tool.

         1. Quick Test Proffestional (QTP)

      Quick Test Professional is an automation tool which has graphical interface to record-playback user actions. QTP is commercial automation tool; trial version of QTP can be downloaded from the official web site of HP. Functional test and regression test automation can be done using automated testing tool QTP which addresses every major web-based

      application and windows applications as well as application environment. Applications developed on java platform such as Java applets, multimedia objects on applications, Visual Basic 6 applications and .NET frame work applications can also be tested using QTP. It identifies the objects in the software user interface or a web page and when performing desired operations (such as keyboard events or mouse clicks); Object properties like name or handler ID can be capture by QTP.

      HP QTP uses VBScript scripting language to specify the test procedure, to manipulate the objects and to control the application under test. Users may need to manipulate the VBScript for more sophisticated results. HP Quick Test Professional is mainly used for UI Based test case automation. Although QTP supports the UI based test case automation it can be used for automate some Non-UI based applications also. Test cases can be generated for non-UI based applications such as database and file system operations.

      Advantages of QTP:

      1. It is easy even for a non-programmer to learn QTP, to create test cases.

      2. Excellent Object Identification process / mechanism.

      3. QTP uses VBScript which is English-like and very easy to understand and gain expertise.

      Limitation of QTP:

      1. Support for Windows environments only

      2. Support limited browser types and versions

      3. High licensing and add-inns costs

      4. Execution time is more as compared with open source tools

   4. COMPARISION BETWEEN SELENIUM AND QTP

      Based on certain features of both Selenium and QTP, the comparison can be done as follows:

      FEATURES	SELENIUM	QUICK TEST PROFESSIONAL
      Cost for Using	Its free of cost. It is freely available on internet. So, theres no licensing or renewal cost for this tool.	As it is commercial tool, it is licensed and very Expensive, Ten user license costs approx. 60L [9].Trial version is available.
      Application support	It supports only Web- based Applications.	It Supports web-based application and windows based applications. It also supports add-ons, but user needs to purchase license for them.
      Programming Language support	Supports Java, Perl, .Net, Ruby, Python and C#.	Scripts can be developed only in VBScript or JavaScript.
      Various operating systems/platforms supported	Supports Windows and PC/MAC/UNIX Platforms.	QTP supports only Windows.
      Programming skills	Programming skills are must to execute test cases using Selenium.	QTP uses VBScript which is English-like and very easy to assimilate and become expertise.
      Difficulty level	Selenium needs quite	QTP is quite easy to learn

      TABLE1 COMPARISION BETWEEN SELENIUM AND QTP

      	a bit of expertise and needs lots of focus.	in a short time. We have plenty of resources to learn QTP.
      Execution Speed	Execution speed is little faster as compared to QTP.	Execution Speed is little slower as compared to Selenium.
      Dialog Box support	Partially supports	Supports all major kinds of dialog boxes
      File upload (system) support	Not possible	Supports all types
      Creation of Scripts	Not powerful. Because many actions are not recorded by the IDE and have to be manually entered	Powerful than Selenium
      Multithreading	It is possible with Selenium to run	Cannot run multiple threads/instances
      	multiple instances of
      	the application on
      	different browsers at
      	the same time.

   5. PROBLEM STATEMENT

      Different models for automated GUI testing and web application testing are already been deployed and developed by Researchers and Developers. Our main motto and specific contribution is to develop a single model on the top of the Selenium.

      Sauce labs have created a Sauce builder which is a free testing tool based on the open-source Selenium testing framework.

      To create new projects, package, class an existing system requires knowledge of core programming languages. Sauce Labs have tried to simplify and improve the cross-browser testing process for development [12]. Benefits of Sauce Builder include:

      • Build Selenium tests with zero programming Sauce Builder automatically writes Selenium scripts that manifest user actions performed by navigating through web based application.

      • Export results in any language of your choice

        It uses same language as your development team and application, which may include HTML, Java, Grooy, C#, Perl, PHP, Python and Ruby, etc programming languages.

      • Eliminate bugs faster Use and share instant video playback of your tests in action with your teammates.

      • Remove test infrastructure headaches – Sauce Builder allows to either run tests in the cloud with access to all the browser / operating system combinations supported in the super scalable Sauce onDemand service or locally in Firefox.

      Though sauce builder provides a significant progress in Selenium based testing framework, it still fails to provide a solution to reduce the script creation time. Some of the weaknesses of sauce builder are listed below:

      1. High script creation time.

      2. No option for uploading files from local machines.

      3. No option for importing data from excel and database

   6. PRAPOSED SYSTEM

      There is a space to develop Selenium based testing framework that retains the features of the existing solutions. Proposed system aims to automate testing process and faster bug elimination. We can decrease the script creation time in order to make the automation framework more efficient and powerful, making it java based allows us to run it on any other platform. The most important merit of the automated tests is the reusability. Additionally the test maintenance is easy.

      Selenium and QTP both has some advantages and limitations as discussed above. Proposed system will try to develop automation framework for web based application by integrating features of Selenium and making it free-ware.

      A Proposed system will have following features:

      1. This tool will support latest version of Selenium IDE.

      2. Development will be based on java so that can be used on any platform.

      3. Reduced script creation time.

      4. Will support data driven testing framework.

      5. Test case flow wise report, HTML reports

   7. CONCLUSION

Selenium and QTP are dominating the market of web based GUI automation frameworks. QTP being more efficient, supposed to be the first choice. But because of its high cost only large industries can afford it. Whereas Selenium, an open source tool, is becoming more popular among testers. But as compared to QTP, it is less user-friendly, requires core java programming knowledge and supports limited browsers. These limitations offer to develop the Selenium based GUI automation test pro that will make the proposed tool more user-friendly, covering more browser support. The tool will try to incorporate maximum features from both Selenium and QTP making it more efficient and reliable

ACKNOWLEDGMENT

We wish our sincere gratitude to MGM college of Engineering, Kamothe – Panvel, India for providing the facility to carry out the work.

REFERENCES

1. Ying Wu, Automation Testing Framework for web base on web base on Selenium. Infomation Techonlogy[J],2011,9:187- 188

2. YanQin. Lin, Function Auto Testing for web application, 2010

3. McMahon, C. , History of a Large Test Automation Project Using Selenium 2009,8

4. LinLi. Fu, JunQiu.Dai, JinHe. Liu, Auto Test Solution for Web Application[J] 2010,4(39):23

5. http://www.oschina.net/search?scope=blog&q=Selenium

6. James M. Slack, System Testing of Desktop and Web Applications,[J], Volume 9, No. 3 August 2011.

7. C.Merchant, M. Tellez, and J. Venkatesan, A Browser yu6Agnostic Web Application UI Test Framework

8. Fei Wang,Wencai DU, A Test Automation Framework Base on WEB, 2012 IEEE/ACIS 11th International Conference on Computer and Information Science

9. Harpreet Kaur1, Dr.Gagan Gupta , Comparative Study of Automated Testing Tools: Selenium, Quick Test Professional and Testcomplete [J]

   ISSN : 2248-9622, Vol. 3, Issue 5, Sep-Oct 2013, pp.1739-1743

10. Pro Manjit Kaur, Raj Kumari, Comparative Study of Automated Testing Tools: TestComplete and QuickTest[J] (0975 8887) Volume 24 No.1,

    June 2011

11. Navaraj Javvaji, Anand Sathiyaseelan, Uma Maheswari Selvan, Data Driven Automation Testing of Web Application using Selenium, STEP- AUTO 2011,pp 32-37.

12. http://www.eweek.com/c/a/Application-Development/Sauce-Labs-

    Delivers-Seleniumbased-Sauce-Builder-257660/

13. www.saksoft.com/PDF/White_Papers/Selenium.pdf