Estimation and testing

After more than a decade since Agile Manifesto was published there is a wide spread adoption of agile practices in software industry. Although it had been more prevalent in small and start up companies but in last 2-3 years even banks and big companies have been learning the agile ways. With this increase in agile popularity there is a surge in failures attributed to agile methodology. As part of a research it was established that how unsuccessful Agile has proven to be, the key points from the report are available here. But there are reason for these failures, a great article "How To Fail With Agile:" explains possible causes behind these failures and how teams can avoid these pit falls.

    Being a QA on various agile teams I have observed that even for successful agile projects, teams find it difficult to extend agile practices to the test/QA teams. These teams usually have their dev teams following agile but their QA team will still be in waterfall. The reason being, QAs are not able to catch up with the dev team and are lagging behind to cover up with automation and manual testing.

   One of the important reasons for this gap is the way teams do estimation. All agile teams follow story or feature estimation as part of their planning process. There are varied number of ways that one can do estimation which can represent either complexity or time. In scrum methodology estimation activity is called planning poker and usually the stories are estimated in fibonacci numbers series : 1, 2, 4, 8.... or they could be S, M, L, XL sizes. During iteration planning meetings (IPM) developers throw these numbers for a given story to get estimation. 

 

But a story other than development work involves testing phase which could be unit tests, acceptance tests, exploratory tests and regression tests. Since developers write unit and acceptance tests they are included in the estimates but the work done by QAs is not considered in these estimates. There could be argument that testing work can be done in parallel with development activities and hence we need not allocate extra time for this. But as part of TDD and BDD developers also share the responsibility of writing tests and setting up test environments. Then there are other testing processes that involves exploratory testing, setting up test data, fixing automation tests and other testing activities that vary with each story. Some times there are stories where QA effort might be more that dev effort. 

The best way to make sure that QA teams don't end up being bottleneck for releases they should be included in planning meetings and should be given power to vote & estimate QA effort involved in stories. The QA team might not be bale to contribute much initially but as they get experienced in the estimation process the would be able to get accurate estimates. The QA estimates need not be added to dev estimates instead they should be averaged out and you will get a true velocity of the team.

Testing at start ups

Testing and beyond at startups from Mona Soni

Effective BDD with Cucumber

After attending couple of events in last few months I came across the reality of our BDD world. Most of the teams that use cucumber or any other BDD tool have adopted them as QA tools. There developers or business or product people never look at these tests and are sole responsibility of QA team. Does this sounds familiar? do you do it the same way? is cucumber killing you?

        BDD is an evolution of TDD and by definition it aims to help focus development on the delivery of prioritized, verifiable business value by providing a common vocabulary that spans the divide between Business and Technology. There are various tools and frameworks that can be used to implement BDD and cucumber is by far the most used.

Before you decide that cucumber isn't the right option for you, lets analyze different reasons that make  using cucumber less effective:

1. Test written by QA team only: The biggest advantage of BDD is in enforcing more collaboration amongst different roles in projects. When developers are not writing a single feature test, then there is no behavior drive development and the collaboration is not achieved. The tests in this scenario will be regression tests and not inline with the system design. 
2. Test written only by Developers: Again we loose the collaboration aspect here but what we lose more is the user intent. QAs bring the user aspect to software testing, with QAs input we can avoid writing scenario that end user will never face and hav better user experience implemented in our system.
3. Product Owners: In an ideal scenario in BDD world we expect BAs or Product managers to write feature tests with QAs. But I haven't seen this model being successful anywhere. I haven't seen any product manager buying this idea ever. The best option to get the business intent into our tests is to have clear deliverables and acceptance criteria in our user stories. These can then be translated into feature tests by QAs and developers.
4. Regression suite: Cucumber tests are very valuable in providing feedback and they majorly serve the purpose of testing various integration points in your system. Using them as regression suite by testing all possible negative use cases will not be fruitful. The well known test automation pyramid recommends the amount of integration tests should be around 5-15 % of entire tests and this applies to cucumber tests as well. 



Automation Pyramid

5. Java script testing: With the ever rising popularity of selenium-webdriver the amount of java script testing in cucumber testing is also very much in practice. I have even seen team duplicating there entire tests to run against IE. Just because the tool supports these features doesn't guarantees return if used at large scale. Java scripts test in selenium have been historically brittle and will continue to be. The effective way is to use a Java script testing framework like Jasmine and few end to end tests in Selenium to test the UI.
6. Test Case documentation: While writing BDD tests we don't add comments to our tests as the feature tests serves the purpose of documentation. Doesn't this applies to your manual test cases? Teams use test case management systems to document there test cases, with BDD you should get rid of that as well. Cucumber features are very effective way to manage test cases that are regularly updated without any extra effort.

BDD is a great practice to make your development cycle effective and cucumber is an amazing framework for writing tests and collaboration. You just need to do it right.

Brush Strokes

I have been in love with colors since my school days. In school it was more for the competition and prizes. There was a time when these prizes got annoying as there was not enough room to display them in our living room, given I was sharing the space with my sister's trophies. Eventually all those trophies landed up in a box packed nicely to be never opened again. After school it was college functions and I still remember the 10'*10' painting that we made it was incredible.Wish I had a pic of it with me. But now after all these years painting is now a means to relax and calm down. 

  I am happy that I was able to paint few pic in recent years unlike my sister who is better than me in painting but busy enough with her cute daughter. I am not that good to paint something imaginary but good enough to take inspiration from a click. I have experimented with what I have been painting and the results are interesting.

This is my first attempt on canvas and I loved doing it. I wanted to add some bright colors to my living room and what better than your own work. 

This is a pic of girl performing Rajasthani dance Ghoomar

I have painted a lion face and a tiger separately when I was in school and really wanted to do them again. Although these Lion and Tiger look kind of cute than wild!

This scene is from a hilly area in northern India and one of my friends posted this on facebook and I liked the snowy mountains the most and tried to create the same effect here.

This scene is again from a facebook share, the dark forest view seemed quite challenging to create and was tempting enough to try.

This scene of a village in Punjab, India was from a wall hanging in my room. I have always drawn village scenes since school and are my favorite.

Being a Rajasthani I have grown up with these scenes but missed them while I was away for work. Trust me women carrying these pots for mile to get water is still part of daily life in Rajasthan villages.

As much as I love wearing traditional clothes I enjoy painting women with these dresses and the fall effect I get to create in their skirts. This was form a 2010 calendar and depicts the pre historic era of central India.

Choosing to be a tester!!

          During interviews I have always come across candidates who wanted to be developers but landed up in QA somehow or they started with QA considering it to be an easy job and want to move to development as they gain experience. I have always been supporter of changing profiles, trying different roles but this one not so much. As those I have seen taking this path are mostly doing this because either they think developers have better roles then testers or they think development is too difficult. According to me if you are working as a tester and believe in any of these two then you probably don't know your art well enough.

          I didn't chose being a tester or would say that was the only road I had seen in my initial career. And I am happy that it out to be the right choice. But what about those who have choice? Given that the average developer to tester ratio is around 4-5 developers per tester, there are good number of testing jobs in the market. And the testing community is definitely in need of people who are passionate about their work. I am highlighting some points that I like most about being a QA:

1. The Role : I prefer to be a QA then tester, here is a great article by Alan S Koch on Testing vs Quality Assurance. All the projects that I have worked my role was never limited to software testing. As QAs we are responsible for varied tasks ranging from software testing, writing automation code, creating test systems, data analysis, requirement analysis, release planning, risk mitigation, iteration planning, reporting, training and many more that varies with teams. We are the gatekeepers, our role is to assure quality of the whole product which is not limited to navigating through the application and writing code.
2. The Work : As a QA my daily work involves both manually testing the functionality and writing automation code. I was once working with a business analyst who had recently moved from QA role, he confessed that he missed writing code. QAs understand the functionality and usability of the system, sometimes more then the business analysts. I have always been excited about exploratory and manual testing as much as automation aspect. I have seen people jumping to automation in the first week of their job, which always wonders me that how in the world can anyone do automation without understanding the application? These are the people who say that manual testing is boring but because of their lack of application knowledge are not able to get good code coverage. 
3. Quality : I have never been a bug hunter, not that I don't find bugs. My aim of testing had never been finding bugs but to assure quality of the system. For me bug count has never been the metrics for evaluating the performance of testing team. Instead how early the testing team can provide feedback once the work is done is the true performance measurement. This approach works really well and dramatically reduces the number of bugs found close to release or in production. Earlier the issue is found and fixed lower is the cost incurred and better is the quality of the system.  
4. The Team : I have always enjoyed working in collaborative teams where I can talk to anyone at any point during my work. Creating protocols for communications reduces the work throughput. QAs always seek answers for their never ending questions to developers and product managers. Teams where QAs are supported well tend to perform better and are more efficient. But its the responsibility of testing team to make sure they are not bugging people with irrelevant question s which can affect the trust in the testing team in long run.

 My experience as QA/tester had been amazing and it's definitely a great role with lot of opportunities in the software industry. Its the responsibility of us who have been in this profession for long time to help and encourage new talent to learn the art of software testing.

CAST 2012 - Day 2

Another day full of learning, fun and meeting amazing people. I have never been around so many smart people where every body is talking about testing. Talking to different people coming from different countries with completely different ways of working but the same goal - how as QAs we can help our organizations and people around us? These conversations have helped me clear my thoughts and re-affirm that the way I am working in my projects is the right approach. 

          Day 2 had more of workshops lined up then day 1, which makes it more difficult to chose which sessions to attend. I was introduced to Anna Royzman the day before as she also is based out of New York and she told about her workshop on Thought Provoking Leadership. I decided to attend her workshop for the first half of the day. In this workshop divided in different groups we used the Empathy maps from Game-Storming principle to understand the stake holder for a product and his needs as a stake holder. 

         We used the travel ads product that we have at IntentMedia to draw the empathy map in our group and we considered the customer who sees the ads as stake holder. I learned about our actual end user here more than I could in more than a year that I have been working at IntentMedia. 

           Post lunch and post the iPad draw(I didn't win :( ) the keynote was delivered by Elisabeth Hendrickson. She talked about the theme of the conference 'Thinking Tester' and I couldn't have agreed to anyone else more than her on the topic that testing is dead. She started by explaining the testing in current perspective as  'Any activity that yields empirical evidence about the extent to which our intentions, our implementation and the actual business needs are aligned'. Most of the people think that testing is just checking but a code is tested when its checked and explored. Exploratory testing is a very important aspect of testing. Therefore a 'Thinking Tester' has following traits: analytical, relentlessly curious, observant, skeptical, empiricist, critical thinking, investigator. The next point that she mentioned was very true with how I think of my work and have been doing it for past 5 years. We in our jobs as tester are not only testers, we play different roles of Product Owner, Programmer, Architect, Project Manager and many more. She closed her talk which I believe explains how the testing community can continue to grow and play that important role in organizations: Testing is not Dead. But the Context continues to evolve, and so do we. Great close to one of the best talks I have ever attended, here are her slides for reference.

            The second part of the day was again a workshop that helped me understand better my work as a service to various stake holders. Lynn Mckee held the workshop on topic Thinking about Testing as a service. During this workshops in our group we brainstormed about what we think Testing and Quality Assurance are. Often the terms that we used daily in our lives are often misunderstood and continue to be ambiguous. Next steps was to understand various stake holders for testing service in a project. These stake holders as we discussed usually are Programmers, Project Manager, Product Owner, Fellow QAs, Share Holders, Customer, Operations and many more. As QAs we directly or indirectly offer our services to each of these stake holders and this makes our job more crucial and important. Lynn has a great website which provides huge resources that would be helpful to any tester to improve his daily activities.

         During CAST welcome note on the first day it was mentioned that in all other conferences evenings are very dull and everybody retards to their rooms but at CAST they have to ask people to go back to their rooms as it was late at night. Through tweets I found that even the first day there were people till 1:30 AM playing testing games. Therefore I decided to stay back the second day to find out more about these testing games and was there past 11 PM. 

We played different games for more than 4 hrs, it was fun and yes there was so much learning from these games.

CAST 2012 - Day 1

The day started with a quick introduction and welcome note. Other conferences that I have attended  the Q&A time is always around 5-10% of session time but at CAST after each talk speakers are given 25 minutes for Q&A. The total time for a session is 75 minutes and if groups wants the discussion can be further extended. This made all the sessions very interactive and led to great discussions.

 The next surprise was facilitators role,  I haven't come across any conference with such an amazing session facilitation. I always thought facilitators job was to just introduce the speaker but today that belief was shattered. At CAST delegates are given three different colored numbered cards which they can use to ask questions. During a session they show a red card if there is an emergency for example speaker cannot be heard or a green card for any explanation but no questions. In the Q&A session when a green card is shown facilitators notes down all the no.s in the order hands are shown and takes the questions in that order. If someone has a question or comment on current thread a yellow card is shown. This helps a lot and conduces a healthy and consistent discussion without any one hijacking the thread and gave everybody a chance to talk in depth about a topic.

  First half of the day I attended the workshop Brainstorming for Testers by Karen Johnson. This was a very interactive workshop with interesting exercises and was based on the Game-storming principles. Game-storming is a set of practices for facilitating innovation in the business world. A facilitator leads a group towards some goal by way of a game, a structured activity that provides scope for thinking freely, even playfully. Karen talked about Beautiful Testing her book, mental locks to overcome as a tester, the importance of notes taking, how mind mapping is helpful in brainstorming process and the importance of asking questions in software testing. We did few exercise which were fun and learning, let's brief these:

1. First exercise we were given a piece of paper and each one wrote whatever they could think of work related or personal stuff. This exercise helped us to align our thoughts and gave a clarity about each topic. I would recommend  doing this whenever you feel you have too much in your head as this definitely helped me clear my mind before other exercises.
2. For second exercise we divided ourselves into groups and had to come up with a testing problem.  After this we were given a deck of testing heuristics. Test heuristics are kind of cheat sheet to use any time during testing, for example : cidtestd = Customers, Information, Developer relations, Team, Equipment & Tools, Schedule, Test Items, Deliverables. James Bach has been advocating use of these mnemonics in testing and recommends writing your own. We used these mnemonics to apply to our testing problem and come up with a prospective solution.
3. The third exercise was using Phoenix Checklist to solve our testing problem. Phoenix Checklist which was originally developed by the CIA is a thinking tool that gives you multiple creative options for problem solving.  Following are a few examples of the questions on the checklist:

           * Why is it necessary to solve the problem?
           * What is the unknown?
           * What is it you don’t yet understand?
           * What isn’t the problem?

Post lunch was the key note on 'Re-Thinking Management…Re-thinking IT' by Tripp Babbitt and few emerging topics. Next session was Foundations of Facilitation and the Tester’s Environment by Chris Blain & Ben Kelly. They talked about how as a testers we can facilitate the team in such a way as to allow the team to solve their own problems. Few points to be considered are:

Positive Environment: Always emphasize on improvements rather than blaming for an incident in the team. Create an environment free of fear and encourage people to try new things without being scared of failures. John Cleese talk on closed and open mode is very helpful in defining the working environment for maximum creativity : 

• Closed Mode: Purposeful, highly productive, but not creative. Good for getting things done. Usually the default Mode at Work. Should be the right approach when you have a specific task with high priority to be completed in a constrained time.
• Open Mode: Playful, curious, fun, humorous, relaxed, contemplative without goals. Relieves the team of pressure and provides opportunity for creativity and innovation.

   Difficulties faced when driving change: Credibility, hierarchy, power dynamics, Apathy and fear. Changing the physical space brings a big change, in our own office having more open space and our little dogs running around relieves the tension and creates fun environment. Now I will get some sleep for the next fun filled day at CAST.