Adventures in Capybara presentation

This presentation is from my talk at  NYC Metro Selenium Users Meetup held on 17 Nov 2011. 

View more presentations from Mona Soni

BDD and Agile testing

         BDD has gain popularity among many Agile shops in past few years. It has helped teams to work more collaboratively and with focus on end user expectations and business goals.

        BDD with ubiquitous language gives the power to business and development teams to define the tests in business domain language. The Give…When…Then format let's you write the tests in readable english.

           Given I am a registered user

           When I enter my credentials

           Then I am on the user account page

    Above scenario gives a very clear picture of the tests intent without diving into the underlying technology and tools used.
   
      BDD framework gives the testing team an opportunity to provide their inputs to the story even before the development is kicked off. QAs can write BDD scenario before the story with Product team which can then be used for automation tests. This establishes a very cohesive model where acceptance criteria are translated into a well functioning feature drive by BDD scenarios. It proves to be very successful as the automation of the features is done right with the development and the QA cycle doesn't lags behind the development cycle. BDD scenarios bridges the gap of documentation, lack of which is mostly feared by big organisations while adopting Agile. It covers for both requirements and manual tests which are difficult to maintain and often loose their worth in long running and big projects.

     Given the wide acceptance of BDD in projects, frameworks are available for most of the popular technologies used, here are the few popular ones :

•  Cucumber            : mostly used for ruby projects
•  Jbehave                : Java
•  Cucumber-JVM   : Cucumber compatible for Java
•  Spinach                : new kid, mostly ruby
•  Jasmine                : JavaScript tests 
•  SpecFlow             : .Net
•  Specs2                  : Scala

      

Internet Explorer testing for Agile projects

Agile projects with continuous and frequent changes have many challenges. Being a QA, one of the major challenges I have seen in most projects while testing a web application is keeping up with the Internet Explorer versions and its users. Which I would think is faced by any Agile team who have huge group of end users and more than 50% of them being IE users.
As the below graph shows the browser usage trend for past few years, the number of IE users is gradually decreasing. But its still the browser preferred by masses.

Let's list out some of the main blockers with IE testing that a QA usually faces during web testing:

1.  Internet Explorer implements CSS commands differently than other browsers. Which calls for different implementation for IE browser usability, resulting in extra testing effort every time there is a front end change.
2.  IE being comparatively slow needs more than usual time to test.
3.  IE versions : different IE versions have different problems and requires version specific fixes.There needs to be put in extra testing effort for different versions.
4.  These days Linux and Mac OSx operating systems being the preferred development platforms, QAs also need to move to these platforms to be in sync with the team and to deploy the applications in support environments. There have been no working installations available for IE on these operating systems. The testing team needs to either work on VMs which are slow or have dedicated IE testing machines. Which eats up lot of time and resources.
5.  Most of the popular browsers Firefox, Chrome and Safari have extensions and plugins(firebug, cookie editors..) available which are very helpful in verifying and debugging the application code. Which are not available in IE, I have been using Fiddler but its not advanced as compared to other browser plugins.
6.  In agile projects there is always more stress on automation. Most of the projects I have seen are able to automate upto 75-80 % of the testing effort, covered under unit tests, integration tests and functional tests. For automation one of the biggest challenges is to run the tests against IE browser. Some of the popular opes source tools as Selenium-Webdriver do support running tests against IE. But the very same tests that run against Firefox cannot be used. As the css implementation is diff for IE the tests needs to be rewritten and they are comparatively fragile and slower on IE.

I am sure every project team will have their own set of problems and there have been efforts to improve IE testing. QA teams can do a improve upon with IE testing and improve the user experience on IE and reduce the bugs found in production.

1. Most of the manual testing should be done on IE browser and the automation tests should have high coverage which can be run against Firefox browser. TDD is good practice to have better test coverage with a cohesive development and QA team. 
2.  Larger teams should divide the testing effort such that QAs pair on the stories and test the same functionality on different browser.this would result in knowledge transition without dedicating extra time and effort. Given there are two individuals, they will have different perspectives and will result in better edge case testing.
3.  There are various websites that provide you different versions and different browsers that can be used to test specific applicationwww.browserstack.com
4.  Instead of running entire test suite on IE, a subset of tests which would serve as a health check of website against IE can be run on a daily basis.
5.  Teams should keep a regular updated browser usage pattern for their application handy or on a board and accordingly prioritize their cross browser testing.

Hopefully there would be better tools and technologies for IE that would relieve the pain from software development teams leading to better user experience. Till then we need to get workarounds to improve quality of our softwares.