The four-hour tester – Part III

20.04.2017
Von: Marvin Bodziuk

The third chapter from the four-hour tester is called test design and it follows on from the modeling chapter. In the first step, you choose 2 or 3 areas from the Data Tour; I chose the Appointment and Task features. From the briefing of this chapter we know that it’s now time “to think about what and how exactly you are going to test. That thinking is called test design.”

The exercise uses Elizabeth Hendrickson’s Test Heuristics Cheat Sheet. The idea for this task is to use the Data Type Attacks section “to test the selected areas in Google Calendar.” The time limit is set to 20 minutes.

We discussed a small deviation to this exercise with my mentors: that I would first try to create my own ideas before using the cheat sheet. In this post, I’ll mostly present my own ideas since the cheat sheet is freely available for everyone to look at.

I created a Mind Map to show the structure of my ideas.

 

 

 

 

I created a few categories (based on the application structure) for which I wrote some test ideas down. After writing them all down I went through the application and tested the behavior for my ideas.

For the appointment, I came up with these tests:

Create

  • What if I type e.g. 11/31/2016 or something similar and the day “doesn’t exist”?
  • What happens if I save an empty event?
  • What happens if I save an event without a date specification?

Attachment

  • What’s the maximum size for uploads, what happens if you exceed it?
  • Which data types are accepted for an upload, what if you pick a forbidden one?
  • Does drag & drop work? Check both ideas above for drag & drop
  • Is there a timeout if the connection is too slow?
  • Does the connection for Google Drive work?

 Add guests

  • Are invalid Email-addresses noticed?
  • Are the user rights set correctly (i.e. only those I picked are available?)
  • Does this function work if guests don’t have Gmail?

 Discard/Return

  • Do I get a notification if I click discard, but didn’t save my event?
  • If I click discard, does the application really “discard” the event?
  • Check both ideas above for the return button
  • If there is a pop-up that I didn’t save my event and I cancel the pop-up, what does the application do? Save or Discard?

Quick Add

  • What if I type tomorrow or next week or something similar and the day “doesn’t exist” (e.g. 02/30/2017)?
  • Do the relative date types like “tomorrow” work? Which other relative dates are also accepted by the application?
  • Do the relative date types like “tomorrow” work? Which other relative dates are also accepted by the application?
  • Can I change quick add events afterwards?
  • If I discard a quick add does the application really “discard” the event?
  • If I discard a quick add without saving, does the application send out a notification?
  • What happens if I save an empty quick add?
  • What happens if I save a quick add without a date specification?

At this point 15 minutes were already over (quicker than expected). Since I wanted to spend 5 minutes on test design for the tasks area, I moved to this feature.

  • If you delete tasks do they really get deleted? If yes, does the undelete function work? (Is the complete task restored?)
  • Does the sorting-function work? What happens with tasks that don’t have a “due date”?
  • What happens if you save empty tasks? Or some with only blanks in?
  • What happens if the “due date” is in the past? Pop-up notification that it’s invalid? Or does it get accepted and maybe get marked in red or something similar, to notice that it’s overdue?
  • Does the shifting-function (into other calendars or task-lists) work? Are any data lost while shifting?
  • Is it possible to discard a task? Or is every task saved?
  • Does the reverse-function work?

After creating some test ideas for the application, I executed some of the easy and quicker ones. 

Test execution

For the execution I tested some of my ideas e.g. selecting a non-existing date. It isn’t possible when using the date selector. Because if an invalid date is typed, the next valid date is chosen.
In this way I found out how the application reacts.

 

Evaluation

  1. How did the cheat sheet help you in elaborating your test idea(s) in more detail?

    • Mostly I had some specific areas and scenarios on my mind, which I wanted to test. The sheet helped me to elaborate them.

  2. How did the cheat sheet help you create different tests? Which tests seemed to yield more interesting results than others? Why do you think that is the case?

    • The sheet shows different “Attacks” which I could test on the application. More interesting for the Google Calendar were time and date and strings “Attacks”. Especially leap days, always invalid days, daylight saving and different formats.
    • For most of them Google could handle the attacks, and just correct your “mistake”. E.g. if you pick an invalid day (9/31/2016) Google skips to 10/1/2016.
    • In the string section, the most interesting result was the leaving the field blank. If you type a blank at 11:30pm between the 1 1 Google changes the time to 1:01pm.

     

    For this observation I made two screenshots. After trying it out a few times, I can’t see the rule behind this change.

    1 Before typing blank

    2 After typing blank between "11"

  3. Considering the modeling and the test design exercise, in what ways were the two exercises the same? In what way were they different?

    Different:

    • In the modeling exercise I was focusing mostly about what to test.
    • In the test design exercise I was focusing mostly about how to test and what I expect from the application when I test the things I want to test.
    • The cheat sheet is testing “Attacks” on the application, and I was testing different things e.g. if the applications works correctly from the user’s view.

    The same:

    • In both exercise I was using the application and their features to create ideas
    • The testing/exercise is focused on 2 or 3 areas

After presenting my outcome for this chapter to my mentors, they asked me how I would give a summary to a Product Owner for this exercise.
I would report that my testing improved my confidence about the quality of the test object. The tests I tried yielded reasonable, consistent results. I would report that I can’t say anything about the attachments because I didn’t execute any tests for that area. The one question I have remaining is how the rule works to create a time if there is a space in the time.
Once we’d discussed this, we talked about debriefing exploratory testing sessions and how lists of bug numbers might not be as helpful as a conversation about the testing and the value of the testing. I’ve also saved the cheat sheet for future reference


INDIVIDUELLE SOFTWARE

KONTAKT | IMPRESSUM | DATENSCHUTZ | AGB