The last part of the four-hour tester is about bug reporting. I’m a bit sad that it’s already the last part. The work on this has been interesting and informative. It was really helpful to focus on the different skills which are important for a tester. Additionally, the hints and tips which were given in the briefing, e.g. the FCC CUTS VIDS tours, the cheat sheet or the PEW-heuristic are pretty helpful and I will keep them in mind. Thanks to Helena Jeret-Mäe and Joep Schuurkes for this great idea, and to my mentors for the great support.
The exercise is to write a bug report about an old bug in Google Calendar, using Michael Bolton’s PEW (Problem Example Why)-heuristic:
Google Calendar invites undesired people to personal meetings
If you create an event in the Google Calendar and add an email address into the title, the calendar from this email-account will get this event added. The user won’t get an email notification of it, like they do if they are invited normally (add guests), but they will get a meeting reminder pop-up. If you use the quick for events, Google Calendar asks for confirmation to send the meeting invite. If you use the full interface, there is no warning.
It seems like this bug is present only on the web application and not on Android phones.
The problem seems to be mainly for Gmail addresses; some non-Gmail addresses also seem to be affected, but not all. Right now it cannot be determined which non-Gmail addresses would receive the item in their calendar and which not.
If you delete this suspicious calendar item, the “Cancellation notification” is emailed whether the user received the original invite or not.
Open Google Calendar on web and create a new event via the full interface (not the quick add pop-up). Add an email address into the event title. Add a date and time, save the event, then check the calendar from the email you added into the event title. Your own appointment where you did not add any guests but only an email address into the title, is now shown in the calendar for the email address in the title.
If you want to set a personal event for yourself, where you just want to add an email address as a note for yourself, other people can see what you wrote in your personal event, because they get this event added to their calendar. They also receive your private Gmail address. Furthermore, they could associate your Gmail account to your Google+ account and so maybe access personal data you don’t want to share with them.
As a suggestion, the user should at least be notified that other people will receive an event invite. Users would expect that only guests added via the “add guests” will be added/informed.
The first two questions from the evaluation should be answered immediately after doing the exercise:
- How does separating Problem, Example and Why help in writing a clear and easily understandable bug report?
If you follow this heuristic, there are a few advantages. The first one is that the mnemonic reminds you of the most important points to include.
Also, the heuristic automatically structures your report. If you write the report without a heuristic, you will maybe go back and forth between topics.
- Compare the blog post with your own bug report. Which one does better on which of the three aspects?
I think that mine is better in terms of the problem description because it is more focused. The aim of the blog was to exemplify the problem a great deal and not to just describe it – the blog contains images and videos.
The third question should be answered after a few hours or days:
- Read the bug report you wrote. Focusing only on the bug report evaluate it on
- clarity of the example
- communicating why this bug is a bug
I think the report is pretty understandable. As I mentioned before the “PEW” structure helps to get a quick overview and focus on the main content.
The example is reproducible, because I gave a step by step navigation. It should be clear why this is a bug from my point of view, because I wrote about the actual behavior and my expected behavior. Furthermore, I wrote about some risks, which could happen if the bug doesn’t get fixed.
Presentation with my mentors
It was not easy to analyze my own evaluation. As I already learned from testing psychology: Nobody has an independent perception for their own work. But I tried my best to do it as neutrally as possible and was curious for the feedback from my mentors.
The first point noted by my mentors was that my problem description is too long. I should focus on the main points and maybe leave information like “not on mobile” for a section on additional information.
The example part could be improved by writing it in bullet points or a numbered list. This would be helpful for developers and other testers to get a quicker overview and reproduce the steps more easily.
For the why we are in an agreement that a continuous text is the best solution, because it’s a personal opinion which may require some explanation.
Finally we discussed whether it’s ok to make assumptions about how the defect happens or how to fix the behavior. Opinions differ on this, but we decided that it can be fine to make an assumption (which we say clearly is one) and a friendly suggestion about how to alter the behavior.