RSS-Feed der Blog-BeiträgeDie Blog-Einträge sind nur in englischer Sprache verfügbar.
Improving planning activities in agile teams - from board to screen
24.01.2012 13:47 von Alexandra Schladebeck
One of the most exciting (and challenging) aspects of being agile is planning user stories. They should bring small packages of customer value, yet be well-described for developers and also have acceptance criteria for testing. Sometimes easier said than done!
Pain points
Over the years, our team has got quite good at defining user stories that are visible and value-bringing for the customer. We've even got good at making them nice and "thin" so that we can see progress quickly and aren't dependent on everything being ready to make it worthwhile. Nevertheless, like every team, we have our pain points. For us these are most often forgetting details (what needs to be tested, what aspects of the development can't be forgotten when implementing this story, etc.) We also sometimes found it difficult to see whether everything was going to plan in terms of time or not. This was often compounded by "sneaky stories" joining the sprint after it had started, essentially increasing what we had to do in the same amount of time. (An entry on why sneaky stories are dangerous anyway is here).
Throwing out the board
About three sprints ago, we decided to change some fundamentals of how we were doing things. Up until then, we'd been using a storyboard.
On the one hand, this was nice and easy to use and it was very visual. However, there were some aspects of our process for which it wasn't supporting us, and some aspects in which it was helping us to be bad...
The board was a bad influence
First of all, cards are just so temptingly easy to add to the storyboard once the sprint has started. There was no automated collection of how many story points we had planned, no way of seeing that this had happened, and no way of seeing how often within a sprint. Also, if you know you can change things, maybe you don't plan so well in the first place.
Lack of details
The cards themselves were also problematic. They let us write nice user-stories, but never really had enough room to write individual development activities (update model, show new info in problems view, documentation etc), or acceptance criteria. Details sometimes got left out, sometimes forgotten. At the same time, we had no good way of managing a product backlog where we could always go back and see what we'd already planned, what we'd estimated and what other things are of interest.
On a purely logistical level, our wall was also getting too small for a growing team.
Time for a tool?
So, three sprints ago, we decided to try out a tool we use at the company for other aspects of our work (booking hours, calendars, project planning, writing invoices etc) which proclaimed to have support for agile processes. We were naturally somewhat sceptical, because we thought that it would limit us in our work. Nevertheless, one of the great things about agile teams is their willingness and ability to try out new things, so we launched a pilot sprint for the tool.
Three sprints on, and we're actually really enjoying it and gaining benefits from specific areas of additional structure it brings us.
Hierarchy: from story to technical details
First of all, epics, stories and activities are created and edited as hierarchical structures. We can keep the customer perspective we've got so good at thinking of while also specifying more technical activities for the developer, so our stories are more well-rounded and also well-estimated. Because everything we've done gets saved into the backlog, we don't have to do any work twice. If we've discussed a feature, written partial or complete stories for it, done mock-ups etc, then those data are still all there later. We can use old stories to help us plan new ones, and we can go back to not-yet-fully-planned stories and carry on where we left off - even if it was a while ago because something else got prioritized instead.
Prioritization
Speaking of prioritization, that's become easier too. Without a well-maintained central list of what the customers want (we have multiple sources for requirements), then it's easy to prioritize something more highly for an upcoming sprint and forget that other things are maybe just as (if not more) important. Now, we see exactly what we're leaving out to include something else and can make sure that that's a decision the customer(s) want(s) to make.
Keeps us on the straight and narrow
At the beginning of the sprint, we add product backlog items to the sprint backlog in the tool. Once the sprint has started, it's very much a "holy" object. An automatic burn-down chart gets created every day, and we have a representation of our old story board on our screens. It is "difficult" to add new items to the sprint, and impossible to do so without noticing that our planned story points have increased. So we don't do it. This has advantages for our transparency with our customers as well - we can tell them exactly when a sprint is beginning, and exactly what it will contain.
Tool must support the team
Now, working with the tool wasn't the only change we made - we also started doing more regular, but shorter, planning meetings and changing the way we work with known or new issues within a sprint. So our general planning has changed a bit too, but even here its nice to know that the results of our planning are safe in the backlog, with all the information it contains.
I wouldn't want to say by any stretch of the imagination that the tool is perfect, or even that our way of working with it is always ideal. Nor would I want to challenge the idea that tools are secondary to interaction. But used in the right way, a tool can help a team to improve. We've now had three good consecutive sprints, and that's worth celebrating 
.
Automated testing topics : BIRT reporting for quality information over time
19.01.2012 15:38 von Alexandra Schladebeck
Background
Back in early 2011, when we were deciding where and how to split GUIdancer into two tools, we put a lot of consideration into what we wanted to achieve with our Open Source activities. We firmly decided that Jubula users should have all they need to write, execute and analyze tests. We wanted to have a good, well-rounded tool as our Open Source contribution, not something that is missing necessary features. Nevertheless, we did want to save some nice features for GUIdancer, the idea being that for a small amount of money, you can add some nice aspects to your testing project and process.
As it turns out, many of the features we kept closed source and as part of the commercial tool are things that are likely to become nice-to-haves once you’ve got more than just a few tests up and running. Once tests start getting bigger, then aspects like Test Style (like Checkstyle) and Mylyn integration really start getting interesting. And if you’re using Jubula in CI processes and your tests are gaining in importance, then it would be nice to have reporting capabilities (you know, for the managers ;) ) and to get some information on code coverage. If you’ve already used Jubula successfully in one project, then maybe you’ll think about starting testing even earlier next time with the Model-Based approach.
But these are things that come later, and we often get asked the question by newer users – what can I do with all of this? Well, in a short blog series, we’re going to describe how the various aspects of GUIdancer are designed to be used, and how they help us with our work in the Jubula project and in customer projects. This first entry will look at how we use BIRT reports in our daily, weekly and monthly work.
GUIdancer and BIRT – the basics
Jubula contains various options to analyze single test runs – a test result report appears dynamically during test execution, and individual test reports can be reopened including information on error-types and screenshots. This is indispensable in a test tool – I need to be able to see what went wrong so it can be fixed. However, what this doesn’t allow me is any kind of view over time. Questions such as:
- How many tests ran successfully / failed / didn’t run this week / month?
- How many tests have been added over the last period of time?
- How has the code coverage developed since adding the new tests?
can’t be answered with single test runs; they need to be cumulated and displayed in an easily readable manner. This is the aim of the BIRT integration in GUIdancer.
The existing BIRT reports
When we added the BIRT feature, we added some example reports that we imagined we would like to use and that may be useful for customers. New reports can be added by anyone who wants to, but it is nice to have a selection out-of-the box. Of the reports that we offer, there are three that we use very extensively. They have become such an integral part of our project that it’s worth describing how we use them and what they tell us.
Report 1: Comments
The report we use the most frequently is the “Comments” report. The first activity our team has every morning is to analyze the test results. As we have in excess of 70 GUIdancer / Jubula Test Suites (functional tests, performance tests, tests for our actions, tests for starting AUTs – all multiplied by a total of 5 platforms!), we wanted something that would help us to report the test results at our stand-up meeting. For any Test Suite that failed overnight, we use the “Edit Comment” function in the Reporting Perspective to add a short description of the reason for the failure to the test run. This also gives us a comparison over time and lets us find patterns in even sporadically occurring errors.
Once the comments have been entered, the “Comments” report is generated. The report displays a table of all failed Test Suites in a specified period of time (we choose “yesterday”) that don’t have the word “BROKEN” in their name (we have specific “BROKEN” Test Suites containing tests that show known, uncritical bugs). For each Test Suite, the comment is shown in the table. The report is printed and brought to the stand-up meeting so that any necessary action can be discussed.
Report 2: History and Coverage
The second report that we use on a regular basis shows the amount of expected Test Steps versus the amount of executed Test Steps as well as the Code Coverage value as three points for each test run. Again, by entering a time frame (we usually take 2 weeks for our weekly “Show&Tell” meeting, but sometimes look over a whole year for a better comparison), we can see how the tests are progressing. The report below shows a period of two months during which new tests are frequently added (the red line), the code coverage improves (the green line), and after an initial period of stability, tests often failed (the blue line). This pattern improved on the day the report was printed however (the last point on the graph).
This report is one of my favorite ones, because it can potentially show us so much. One the one hand, the amount of expected Test Steps can show how many tests have been added recently, and what sort of an effect it has had on the Code Coverage (I’m looking forward to writing the blog entry on Code Coverage, because it is fascinating to try and decide whether it actually tells us anything, but that’s another story…).
On the other hand, the amount of executed Test Steps can give us some hints about the type of errors we’ve been having:
- drops to almost 0 can either mean that something went terribly wrong at a central place (no database connection could be established for example), or that the error handling is bad (a small error resulted in the whole test stopping)
- a zig-zag graph is usually a good sign of test environment problems: sometimes the tests run, sometimes they don't. Test environment problems are particularly nasty - nobody really wants to feel responsible for them, yet they can be just as damaging as software errors or errors in the test specification.
- A flat graph means that nothing has been changed. No new tests, possibly no new versions of the software, and no new fixes. It's important to analyze why - in this example, it was simply because the tester was on holiday and there was no cover for him!
- Small dips in the executed test steps can either mean relatively uncritical errors, or a good error handling.
- Reports that never manage to get to 100% execution also tell us something about our reaction times. The status quo should always be that we are executing 100% of our expected tests – otherwise we have gaps in our knowledge. This means reacting quickly in the team, and it also means that our tests have to be able to keep up with the pace (I wrote this entry on test discipline a while ago). Errors will happen, so there should never be any questions about why 100% isn’t achieved constantly. But we can get a feeling for our general stability if we’re seeing too many drops that take too long to fix.
Report 3: Histogram
This report gets used to publish our test results on the Jubula website. It currently has a couple of advantages over the History report – it can show days when tests didn’t run at all (broken build, environment problems), and it also splits the Test Steps into successful / failed / not-executed. Otherwise, it shows a pretty similar view of the world to that of the History except that the colors are nicer :).
This first diagram shows tests that are generally stable, but no new steps are being added.
This diagram, on the other hand, shows a test for a problematic area of the software that our team decided make a priority. We wanted to add more tests and fix any issues they found quickly. This shows the last year. We've steadily been adding more and more tests, making them work by fixing bugs, then adding more tests. As you can see, at the time of writing, we're entering a stage of fixing ;)
Other reports
We tend to add new reports as we realize we need them to support our process. The possibilities are endless – percentage of static waits in the tests and test duration are just two that come to mind. At the moment though, we feel quite at home with the three I’ve described. What we’d like to hear is how you use reports in GUIdancer, if you do already. Contact us with your examples and receive a goodie to say thank you!
Until next time
The next entry in this series will be about Code Coverage, so stay tuned. In the meantime, happy testing!
Eclipse Stammtisch on 9th February in Braunschweig
16.01.2012 15:28 von Alexandra Schladebeck
|
Even though it's getting a little late to wish people a Happy New Year, I'd like to do it anyway on behalf of all of us here at BREDEX! We're starting what will hopefully be an exciting year of development, testing and Eclipse events with a regional meet-up (Stammtisch) in our offices on the 9th February at 7pm. |
|
We're pleased to welcome Dr. Christian Bartelt from the Technical University in Clausthal as the speaker for the first Stammtisch of the year. He'll be talking about the Cooperative Modelling project that he is involved in at the university. After the talk, we'll head on over to our local restaurant, the Knochenhauer, to chat over some food and a beer or two 
As always, registration and arrival details are on the BREDEX website. If you can't make the Stammtisch, you can already start planning your participation in the spring series of Demo Camps - contact us if you'd like to give a demo!
Agile Testing Days 2011 in Potsdam
21.11.2011 16:21 von Alexandra Schladebeck
Last week I was pleased to be able to attend the Agile Testing Days 2011 in Potsdam. If you're working on an agile project or in a testing role, I can only recommend it. It really is the number one opportunity to get together with a great community, gather ideas, help others and relight your testing fire ;)
My week started with Gojko Adzic's Specification By Example workshop on the Monday. Gojko has a great range of stories to exemplify the way different teams have adapted the approach to suit them, and I'm sure I wasn't the only one who went away with new ideas of how we can derive scope from goals and ensure that everyone has the same understanding of the system and the new features.
The second day for me was very much about test automation. I attended one talk on specification by example through the GUI, and then presented a Jubula demonstration in the afternoon to give some insights into how we fit into agile processes. The feedback from the participants was very useful, and we're talking about incorporating some very high level description possibilities into the tool to make it even more accessible to business users. Watch this space!
The second evening was the main social event of the week, giving everyone the chance to chat in a relaxed (and very entertaining!) atmosphere. It was great to see Gojko receive an award, and to chat with people about their processes, problems, solutions and experience. That kind of evening really sums up this kind of event for me - I've come back with a better understanding of people and processes, a list of books I should really get around to reading, and with a renewed enthusiasm for all things testing. I'll be demonstrating that enthusiasm later on this week at the Eurostar conference in Manchester - see you there!
Arbitrary check constraint in Oracle Database!
16.11.2011 14:26 von Holger Klene
Data integrity is a core feature of a database. The promise: It doesn’t only allow structured queries (> SQL) but also represents a logically consistent data-state at any given point in time. Transactions help to bundle changes into atomic operations (though they might take hours to complete). On commit, a range of conditions can be checked to guarantee the integrity of the internal structure.
A few relatively simple constraints are already built in:
- a column does not contain null values (Not-Null)
- a column does not contain a value twice (Unique)
- a set of columns uniquely identifies any row (Primary key)
- all values of a column point to another table (Foreign key)
- the values of a row are within a certain range (Check-constraint)
It’s up to the developer to translate business rules into those checks. Most are quite obvious, once you’re familiar with relational modeling. But some rules are more complex, sometimes too complex to be expressed by the simple constraints.
Examples:
a) Foreign key cycles: Table PUPIL has a foreign key to table CLASS. CLASS has a foreign key back to PUPIL, to nominate one representative as spokesman. How do we guarantee that the spokesman is from that particular CLASS?
b) Defaults per category: If there is no explicit CLASS table and the PUPILS only have a VARCHAR column for the class, the spokesman will likely be modeled as a NUMBER(1) column IS_SPOKESMAN. A combined unique constraint over both columns can guarantee that there are never two spokesmen within the same class. But how can we verify that each class has exactly one spokesman?
c) 3 dependent tables: ANIMAL, CAT and DOG each with an ID-column. CAT.ID and DOG.ID point to the ANIMAL.ID to represent the inheritance that any DOG or CAT is an ANIMAL. How can we make sure that no ANIMAL is left after deleting a DOG?
d) How can we check that CATs and DOGs don’t share the same ANIMAL?
As you can see, there are quite a few cases where the built-in database constraints won’t suffice. On the other hand it’s not possible to check everything that comes to mind, as checks slow down any write operation. Most often an additional check is justified either by the potential loss or by the developer time to fix the data later on. Sometimes it’s too late anyway and you have to deal with the corrupted data. But if you're in the situation where you have to write some SQL to fix a problem, it’s a good idea to add a check to ensure that it won’t happen again.
For the example problems this might look like:
a)
select * from CLASS c inner join PUPIL s on c.fk_spokesman = s.id where s.fk_class != c.id;
b)
select * from (select p.class, sum(p.is_spokesman) as numSpoke from PUPIL p group by p.class) where numSpoke != 1;
c)
select * from ANIMAL where ID not in ((select ID from CAT) union all (select ID from DOG))
d)
(select ID from CAT) intersect (select ID from DOG);
If the necessary columns are covered by indices, the checks will be reasonably fast; fast enough that the end user won’t detect any difference. Just one question remains: Who’s going to run the checks? Answer: A trigger!
create or replace trigger <TRIGGER_NAME> after insert or update or delete on <TABLE_NAME> declare i integer; begin select count(*) into i from <CHECK_CONDITIONS>; if i > 0 then RAISE_APPLICATION_ERROR(-20123, i || ‘ rows invalid’); end if; end;
This straightforward approach has one small drawback. For example b) you cannot change the spokesman ever again. If you set the old spokesman to zero first, the check will immediately complain that there has to be at least one. If you set the new spokesman to one first, the check will immediately complain that there cannot be two spokesman for the same class. Both values must be updated in one single update statement. Good luck trying to convince any persistence layer like Hibernate to do this for you!
What we need is the trigger to fire only once on commit and not once per update statement. Fortunately we can work around this limitation by introducing an intermediate table:
Create table RESULTS ( ID integer primary key, VIOLATION integer ); alter table RESULTS add constraint NN_VIOLATION check (violation = 0) deferrable initially deferred;
The trigger would end accordingly:
... -- Maintain a single row per Check within the RESULTS table. delete from RESULTS where id = -20123; insert into RESULTS(id, violation) values(-20123, i); end;
A single transaction can update the violation count multiple times. Only the last update before the commit to write back a zero into column violation is crucial. Otherwise the deferred constraint NN_VIOLATION will stop the transaction altogether and thereby prohibit entry of invalid data. The trigger must always run on insert, update and delete. Inconsistency might have been introduced by previous statements that get deleted again before the commit.
In addition, multiple triggers on several tables can reuse the same RESULTS table. Sadly almost the same trigger might have to be copied to multiple tables if they are all involved in the check. E.g. example a) must not only listen to changes of CLASS.fk_spokesman but also to changes of PUPIL.fk_class. But the actual check-code can be moved to a function in a package to facilitate its maintenance.