Presentation: Work Agile with VS2010
Traceability in VS2010
Always an interesting question, how does traceability work in VS2010?
To often asked without any context, or only asked from a requirements perspective. Still way too open to answer in a tweet. So, a post to canalize those questions a little bit more…
[see the image below for the numbers]
you can take two directions when talking about traceability. The first is focused on “work”, the work items repository part of TFS [on the left side of the green dotted line]. with work related traceability we can answer questions like “did we completed all tasks for that requirement?” [for more see the red list]. Information which is very important for every roll in the Application Lifecycle. A tester want to know if he can start testing, so he want to see if the developer is ready. A project manager wants to plan the work. Developers want to know if designers are ready to start with a stabile set of needs… and many more. All work related information, and information which can be tracked/ traced by using work item and its repository.
VS2010 main capability for work related traceability are the TFS work item repository [see image below from MSDN] and linking of work items, setting up a hierarchy.
We can breakdown requirements, or user stories, in tasks. 
tasks that needs to be executed to get the requirement done. These tasks can have a parent-child relation [and other see Working with Link Types on MSDN]. Giving us the information we need in any kind of way, for example reports.
So far the work related traceability. The other kind of traceability is based on artifacts, things we make during the application lifecycle, I will call them ALM artifacts. ALM artifacts are used to create the solution/ application. For example the source, XAML and configuration files which make the solution. But, also the diagrams which we created to make a correct, consistent and good communicated application architecture, we use them to drill down from the needs to code. And also test cases belong to the things we make during the application lifecycle [I can imaging when you get confused now, test cases are within VS2010 work item types, but they definitely belong to the artifacts section].
Use case – user story what is the difference… I often use both [see this piece of MSDN documentation for modeling requirements]. User stories are work item types and use cases are ALM artifacts. So, they are a great bridge between the ‘work-world’ and ‘artifact-world’. The good thing is the capability of VS2010 to link diagrams, and other model elements to work items. [How to: Link Work Items to Model Elements], this give us the capability to create a trace.
From the description and diagrams of the requirements, the often called problem domain, we have to make a big jump into the solution domain.
Diagrams are created like the component diagram and layer diagrams visualizing the high level pieces of the solution with the dependency and interfaces between these components and layers. [used this image in this post; the modeling world].
It is also a big gap for traceability, to get this a little bit better a solution can be the replaying of scenarios written down in the user stories and drawn in activities as described in this post; VS2010 Modeling; Create Lifeline from Component.
A trace back from component/layer to requirement can be a link from model element to the user story work items in where he is used. Never did this, it’s a manual process and probably people will forget the links, maybe with some kind of notification this would be valuable…
An interesting VS2010 capability is the connection between the layer diagram and the sources. This makes a trace possible between the high level design and the sources. See image below.
Because test cases are work item types in VS2010 they can be connected to any other ALM artifact, giving us the possibility to connect for example test cases to use case diagrams. Easier to accomplish and to maintain is the connection with test tasks and corresponding user story. But this doesn’t answers questions like; “which code is touched by this test”. VS2010 answers this question with Test Impact, see “Determining Which Builds Have Bug Fixes, New Features, or Requirements”, “Recommending Tests to Run That are Affected by Code Changes” and “Developing Tests from a Model”.
Another interesting traceability scenario can be build with test case generation, see this ‘old’ Model Based Testing beta 1 video. [some more info on MBT can be read on Rob’s blog]
Next, 
all have a work related task. So, not only traceability within the two sections is possible also traceability between the two sections is possible. answering questions like; “which task created this line of code?”… creating this link / trace is a manual task but it can be controlled by a check-in policy.
So, when someone asks you the question; “what about traceability in VS2010?” you now can canalize this and guide him/ her to the real traceability question they want to have answered [and tell the solution]….
< probably more to come on this interesting topic, also because the extensibility model of the diagrams is powerful enough to create your own required traceability between artifacts with notifications and work item linking… very very interesting >
Quick reference poster--the VSTS Rangers “scrumified“ process
from: Certified Scrum Developer course … we can finally remove the duck tape :)
The Accentient Scrum Developer course is an intensive five day experience for whole teams of developers. The course teaches teams how to turn product requirements into potentially shippable increments of software using the Scrum framework, Visual Studio 2010, and modern software engineering practices. Attendees will work in self-organizing, self-managing teams using a common instance of Visual Studio Team Foundation Server 2010 to achieve this goal.
03: Agile Testing Business Whitepaper.
My colleague's in Ireland wrote a great paper about Agile Testing practices…
‘ At its heart, agile is all about quality and focusing on the customer. Testing therefore has a significant role to play within an agile development environment and can actually strengthen the output of the process.’
Part 00: Agile Testing with VSTS 2010 and TMap Part 01: User Stories 02: Management, preserving and organization of manual test cases within VSTS 2010. 03: … this post 04: … [probably] 2.1 test case management practices
02: Management, preserving and organization of manual test cases within VSTS 2010.
- Part 00: Agile Testing with VSTS 2010 and TMap
- Part 01: User Stories
- [current post, removed the Agile Testing blablabla and part pre-fix]
This post belongs to the series of “Agile Testing with VSTS 2010 and TMap” although this topic has a wider scope, in used software delivery methodology you have to think about test case management.
A small introduction.
First a small introduction to give some direction on the needs of test case management.
The drawing below I made during a discussion with several TMap experts and explains how test cases are created and executed in a simple situation.
From top to bottom and left to right.
 | Teams start brainstorm over user needs, create stories and write these down on cards together with acceptance criteria.
Within VSTS 2010 this can be done in the work item type ‘user story’ [see previous post]. |
 | After prioritizing the stories, teams start the iterations by brainstorm on implementation tasks and test tasks.
Again, within VSTS 2010 this can be done in the work item type ‘user story’ [see previous post]. |
  | Next, the sources and test case are going to be created, developed. Within VSTS 2010 source files in the IDE and than added to version control. Test cases are added as work item to TFS. |
 | The source files are build, unit tested, labeled, packaged and finally deployed on a test environment where the manual test cases are executed against this deployment |
All the artifacts created by tasks are under version control, except test cases.
All the sources and corresponding automated tests like, unit tests, load tests, web tests and codedui tests are under version control. Getting the latest version of the sources also gets the latest version of these tests. But, also when using the more sophisticated capabilities of version control, like branching and merging, the tests also have this characteristics.
Manual test cases different, they are work items and aren’t under version control they have a history. Belong [often] to a user story. So, it gets interesting when making the above scenario more real live with sources and automated tests under version control with branching and merging. See drawing below…
- User stories, sources and test case are created [top left].
- [red line] A branch is made from the first version to start the development of an other version.
- User story 2 [center] is added to this branch [new/ added user needs], also sources are added to this branch and test cases are created.
- Finally version 2 is build and the test case of user story 2 are executed [bottom right].
Dotted red line; while there was a branch created from the sources, no branch was created from the test cases of user story 1 [top left], resulting in a version 2 with only partial tested functionality.
This branching scenario is very simple, there are some more complicated ones as you can read in Microsoft Team Foundation Server Branching. This 2006 MSDN story has also some nice branching and merging strategies.
The need for test case management within VSTS 2010.
This branching, merging and the decoupled characteristic of test cases is one of the reasons why we need test case management. We manually need to organize and manage our test cases in such a way that every iteration, version, service pack, hot fix and release is sufficient tested and to prove that its well tested.
As described in the paper Test Cube principle [pdf]:
A good test set (= collection of test cases) is of priceless value. During the realization of an application it is the basis for the reporting concerning test coverage and progress. In a maintenance situation it is a source of knowledge concerning the application and a source of ready-made test cases for regression tests.
In the next post we will have a look / deep dive in the capabilities of TFS 2010, MTLM [Test and Lab manager] how they fulfil the needs of test case management and in what way the TMap process template helps this process.
One part of the solution and even an agile one: automate the manual test cases as quick as possible and put them under version control. Next post more…
Agile Testing with VSTS 2010 and TMap: Part 01-User stories
Previous post:
- Part 00: Agile Testing with VSTS 2010 and TMap
- … [current post]
User stories breakdown...
This post is about the work breakdown of user stories during the planning phase of an iteration.
[image from the documentation about VSTS 2010 on MSDN, this page]
During the planning phase of the project, also called iteration 0 [first blue piece], user stories are collected / brainstormed / defined /… in VSTS this information is collected in the new work item type ‘user story’ [image below].
During the planning of the iteration developers the team start to breakdown the user stories [which are selected for that iteration] in implementation tasks. Within VSTS this is done in the implementation tab. The new 2010 functionality of hierarchies between work items is used for this.
Another task also executed during this phase is the creation of test cases. Within the TMap methodology this is described in the planning phase. Where you create the test plan for that iteration, based on the risk class [see user story work item] and discussion with the customer the necessary test techniques are allocated for the user story and functional area. See the initial work items for iteration 1 in the image below, added during the unfolding of the TMap process template.
So, not only the implementation tasks needs to be determined also the test tasks needs to be allocated during the planning phase of an iteration. These test tasks are specific to testers like ‘create test cases for area …. based on the test technique <… decide based on risk and customer contact>’
List of different test techniques…
- Data combination test (DCoT)
- Data cycle test (DCyT)
- Decision table test (DTT)
- Elementary comparison test (ECT)
- Error guessing (EG)
- Error testing (ET)
- Process cycle test (PCT)
- Real life test (RLT)
- Semantic test (SEM)
- Syntactic test (SYN)
- Use case test (UCT)
So, just like the implementation tasks also the test tasks are a link type, have hierarchy with, the user story. [this is TMap process template specific]
We end up with a list of tasks for a user story, implementation tasks and test tasks… testers and developers are going to execute these tasks together in parallel, during the iteration we have: implemented sources for the user story and test cases which are ready for execution. This user story is finished when: every implementation task is fulfilled, all test cases are successful executed and … the tester hasn’t got any open tasks, so all test cases are created.
When giving the tester/ the team a place where they can record their testing tasks, testing is really going to be a first class citizen of the lifecycle. beside this benefit the connection between test activities, risk, user story and test cases gives a great opportunity for reports based on test effort, risks and implementation, later on more on this…
With this this lightweight addition to the user story work item we got a closer to several Agile Testing key characteristics, for example:
Customer involvement in writing tests. –> just like the developers, testers breakdown there work discusing it with the customer An iteration is ready when the tests succeed. –> and all tests are created..!
Agile Testing with VSTS 2010 and TMap: Part 00
Part, because I’m planning several posts around this topic.
00, because this is the ‘setting the stage’ post…
While in the Agile manifesto is written: “Individuals and interactions over processes and tools” I will cover in the coming posts information how individuals and there interactions are supported by tools and processes.
Several characteristics of agile testing [in random order].
- Lightweight implementation of tools and processes.
- Find bugs as early as possible.
- Don’t do things twice, just enough testing.
- Short feedback loops to the team.
- Customer involvement in writing tests.
- An iteration is ready when the tests succeed.
- There needs to be space for exploratory testing.
- Focus on automation.
- Repeatability is a key success factor, frequent testing.
- Pair testing.
Tool and processes support should focus on these characteristics. At this moment the tool support with VSTS 2010 for the tester is getting really mature [see this post]. So, beside the agile support for the developer like TDD, refactoring, continuous builds, guidance and many other things [see this post and image below].
Also the tester wants / needs this kind of support, beside an easy connection with all the capabilities developers have he needs the capabilities to execute his work in an agile team…
So, what practices needs to be in place for the tester within a VSTS 2010 agile project? just randomly start using the offered tools isn’t the right direction. In the coming posts I will capture some thoughts, practices and lessons learned from around the world…
As a start some useful agile testing links: