The four-hour tester – Part IV
The next chapter from the four-hour tester is about note taking. I didn’t really expect this skill as a part of the four-hour tester, but after finishing the exercise and evaluation I agree even more with Helena and Joep that note taking is a necessary skill for a tester. The exercise was about checking features from Google Calendar, and taking notes while testing. They recommend taking more rather than fewer notes, and offer some helpful tips:
- Label the page with a title and a date
- ? - question
- ! - suspicious, bring to attention
- B - bug
- light bulb - idea for further testing
- OK - looks good to you
In the following I will present my results for this exercise.
Below are the notes I took while testing, so they are in quite a raw form.
Selected Feature: Appointment
Exercise for note taking with google calendar appointments 12/09/2016
- B – If blank typed into time it changes weirdly
- ? – First event color is isolated from the others, why?
- ! – Why are invite others and see guest list disabled if I pick modify event?
- OK – Title: Looks fine, there is a default title and it is removed if you click in
- OK – Add guests: Seems to be okay – invalid addresses are noticed
- OK – Visibility: Seems to be okay, there is a description for everyone and all are enabled
- Light bulb – Non-existing days like 02/29/2017, check what the application will do
- OK – Discard function: Seems to be ok. You get a pop-up reminder if you leave the event without saving, and if you still want to discard, nothing gets saved
- ! – Add Notification: Why is the default changing to 0, if I select weeks?
- OK – Select calendar
- ! - It’s possible to save events without date and time
- OK – Repeat: Pop-up how you want to the repeat to work
- OK – All day checkbox: Enable, if selected time is getting removed
- OK - Add attachment: Pop-up with “select file” appears
- OK – Video Call: Is getting added and removed, if selected.
- ! – Sorting: “Where” not sorted alphabetically
- OK – Find time: Switches into “calendar view”
Now that the exercise is done, we can move on to the evaluation.
1. Right after finishing the testing and note taking, start the timer for 3 minutes and write a summary of your testing based on your notes.
Summary for note taking exercise
- On the 12/09/2016 I (marvinb) tested the feature “appointments” from Google Calendar for 20 minutes.
- I used Firefox (Version 45.5.0) on the operating system Windows Server.
- Maybe there is a bug when adding spaces in the event time, I’d recommend checking this with high priority.
- Other suspicious points are: User rights, Add Notifications, save untitled events and the sorting for the place for the appointment “where”. These points need to be also checked.
- The specification about the event colors need to be checked, I’m not sure if there is a bug or just an understanding problem.
- The other features of the application that I tested seem to be OK.
For more details like which “other features” (please see last bullet) I have tested, and which behaviors I would expect from the marked- (bug, suspicious and question) features, I haven’t enough time to add it on the summary (3 minutes’ time-limit).
2. Return to your notes the next day. How easy is it get an overview of what you did? How easy is it to understand? Can you reproduce/retrace your steps?
It wasn’t very easy to get a complete overview from my notes, because they weren’t very sorted and structured. For some notes I was missing some information to be able to understand them again. I think I could reproduce all/most of the steps I have done, since the features aren’t that complex.
3. How can you improve your note taking next time?
I should write down my notes in a more structured way. Maybe it will help to gather all questions, suspicious, bug, light bulb, and OK under their respective categories. Another way to structure the notes would be to structure them like the application is structured. It could help to gather some features from the application into groups and structure the notes in this way.
I should write a few more words for each note, even if I think I will understand it with fewer words, because I won’t.
Presentation with my mentors
As presenting my results to my mentors I got some interesting hints.
1. The first one was that I should write down how long I tested. This is helpful for others who read my notes, to get a feeling for the tests I have done. E.g. finding five bugs in 10 minutes is different than finding five bugs in three days.
2. The second hint was, that I could gather my notes with bullet points to make them quicker and easier to read for other people.
3. The final one was more a correction than a hint. I had written that “the rest of the application looks fine”, which is a really terrible sentence for a testing report. What I wanted to say was that the other features I tested seemed OK. The sentence I chose could have been wildly misinterpreted!
All three hints/corrections were already implemented in the results from the exercise above.
Another aspect we talked about, which is more useful for test reporting than note taking is that the tester should add their feelings about the application and the testing they did. This can be useful information for reporting and debriefing.
As a final extra task, I created a personal template for a test report sheet to help me in the future.
I liked the categories for note taking in this exercise and will definitely use them in my note taking. What I additionally learned from this exercise and evaluation is that if you think “these few words should be enough for me to remember what I meant here”, you won’t!
Template: Test report sheet
Areas which weren’t tested
Feelings after the testing