VS2010 TMap Testing Template | The Test Infrastructure
Initial Work Items #2 Setting up and maintaining the test infrastructure, beside the Initial Work Items #1 which describe / help create the Master Test Plan, also activities for setting up the environment must take place at the early stage of the project.
The test infrastructure consists of the facilities and resources necessary to carry out the testing satisfactorily. A distinction is made between the facilities for test execution (test environments), for supporting the testing (test tools) and for the day-to-day work of the testers (workplaces).
While it doesn’t look like rocket science, just setting up an environment, most projects lack a good structure for these activities resulting in time loose and poor quality.
The TMap for VS2010 process template helps with setting up the infrastructure by providing 6 initial work items, with specific guidance, discussing the TMap activities within VS2010 ,which covers Lab Management and Test Manager.
Together with the initial work items and the guidance also three checklists are added to help.
VS2010 TMap Testing Template | Initial Work Items #1
During the unfolding of the TMap for VS2010 process template, several pre-defined work items are created for the test organization. These work items helps the test organization to start structured test process with a Master Test Plan.
When starting a testing effort first several things needs to be in place… we need to understand the assignment, determine what we are going to test what are the risks, etc… the first four steps in the image below.
Beside these steps also decisions needs to be taken about what to test where, you don’t want to loose money by testing things twice. As described in the TMap Next book:
The type of system and/or development approach and/or test policy determine which forms of which test levels are best used. For iterative system development, for instance, a thorough acceptance test is less obvious. This is because the quality from the user perspective has already been tested in previous test levels. However, for package implementations there is a far greater emphasis on a thorough acceptance test. The risks here are focused on the implementation of the package in the organization, a typical acceptance test aspect.
So, topics covered by the master test plan can be:
All these activities are described in detail in the TMap Next book, and you also get these steps when using the TMap for VS2010 process template, as initial work items.
These Master Test Plan activities got descriptive guidance attached to it. Accessible by using Microsoft Test Manager 2010, Visual Studio 2010 or the TFS 2010 Web Access tool, helping every role, to understand the assignment, the activities and test goals.
[Microsoft Test Manager 2010 | Initial Master Test Plan Work Items with guidance]
[Visual Studio 2010 | Initial Master Test Plan Work Items with guidance]
[ TFS 2010 Web Access | Initial Master Test Plan Work Items with guidance]
So, when the project starts the guy / girl with the Test Manager role has to assign the different tasks to the right team members to complete the Master Test Plan.
In a following post [probably not the next one] some Word and Excel templates which support these tasks and can be found at the project portal. First a planned post about initial work items for planning the infrastructure.
If you want to keep up to date about TMap testing and the VS2010 process template, send me an email with your name and organization
. Beside this blog and our the TMap codeplex site we have a community of users, to exchange ideas, usages stories and additional updates of the template en testing practices.
Visual Studio 2010 Team Foundation Server Requirements Management Guidance
Another delivery from the VS ALM Rangers is public available… the Requirements Management Guidance.
Very useful guidance, with a small contribution from me with the Enrich VSTA 2010 Use case diagram with SketchFlow Screens post. [see the Requirements Elicitation part].
Traceability in VS2010
Always an interesting question, how does traceability work in VS2010?
To often asked without any context, or only asked from a requirements perspective. Still way too open to answer in a tweet. So, a post to canalize those questions a little bit more…
[see the image below for the numbers]
you can take two directions when talking about traceability. The first is focused on “work”, the work items repository part of TFS [on the left side of the green dotted line]. with work related traceability we can answer questions like “did we completed all tasks for that requirement?” [for more see the red list]. Information which is very important for every roll in the Application Lifecycle. A tester want to know if he can start testing, so he want to see if the developer is ready. A project manager wants to plan the work. Developers want to know if designers are ready to start with a stabile set of needs… and many more. All work related information, and information which can be tracked/ traced by using work item and its repository.
VS2010 main capability for work related traceability are the TFS work item repository [see image below from MSDN] and linking of work items, setting up a hierarchy.
We can breakdown requirements, or user stories, in tasks. 
tasks that needs to be executed to get the requirement done. These tasks can have a parent-child relation [and other see Working with Link Types on MSDN]. Giving us the information we need in any kind of way, for example reports.
So far the work related traceability. The other kind of traceability is based on artifacts, things we make during the application lifecycle, I will call them ALM artifacts. ALM artifacts are used to create the solution/ application. For example the source, XAML and configuration files which make the solution. But, also the diagrams which we created to make a correct, consistent and good communicated application architecture, we use them to drill down from the needs to code. And also test cases belong to the things we make during the application lifecycle [I can imaging when you get confused now, test cases are within VS2010 work item types, but they definitely belong to the artifacts section].
Use case – user story what is the difference… I often use both [see this piece of MSDN documentation for modeling requirements]. User stories are work item types and use cases are ALM artifacts. So, they are a great bridge between the ‘work-world’ and ‘artifact-world’. The good thing is the capability of VS2010 to link diagrams, and other model elements to work items. [How to: Link Work Items to Model Elements], this give us the capability to create a trace.
From the description and diagrams of the requirements, the often called problem domain, we have to make a big jump into the solution domain.
Diagrams are created like the component diagram and layer diagrams visualizing the high level pieces of the solution with the dependency and interfaces between these components and layers. [used this image in this post; the modeling world].
It is also a big gap for traceability, to get this a little bit better a solution can be the replaying of scenarios written down in the user stories and drawn in activities as described in this post; VS2010 Modeling; Create Lifeline from Component.
A trace back from component/layer to requirement can be a link from model element to the user story work items in where he is used. Never did this, it’s a manual process and probably people will forget the links, maybe with some kind of notification this would be valuable…
An interesting VS2010 capability is the connection between the layer diagram and the sources. This makes a trace possible between the high level design and the sources. See image below.
Because test cases are work item types in VS2010 they can be connected to any other ALM artifact, giving us the possibility to connect for example test cases to use case diagrams. Easier to accomplish and to maintain is the connection with test tasks and corresponding user story. But this doesn’t answers questions like; “which code is touched by this test”. VS2010 answers this question with Test Impact, see “Determining Which Builds Have Bug Fixes, New Features, or Requirements”, “Recommending Tests to Run That are Affected by Code Changes” and “Developing Tests from a Model”.
Another interesting traceability scenario can be build with test case generation, see this ‘old’ Model Based Testing beta 1 video. [some more info on MBT can be read on Rob’s blog]
Next, 
all have a work related task. So, not only traceability within the two sections is possible also traceability between the two sections is possible. answering questions like; “which task created this line of code?”… creating this link / trace is a manual task but it can be controlled by a check-in policy.
So, when someone asks you the question; “what about traceability in VS2010?” you now can canalize this and guide him/ her to the real traceability question they want to have answered [and tell the solution]….
< probably more to come on this interesting topic, also because the extensibility model of the diagrams is powerful enough to create your own required traceability between artifacts with notifications and work item linking… very very interesting >
VS2010 Modeling; Create Lifeline from Component
From the VS2010 MSDN sequence diagram documentation…
Relationship to other diagrams
You can use UML sequence diagrams together with other diagrams in several ways.
Lifelines and types
The lifelines you draw in a sequence diagram can represent typical instances of the components or classes in your system. You can create lifelines from types, and types from lifelines, and show the types on UML class diagrams and UML component diagrams. For more information, see Classes and Lifelines.
Beside we can create lifelines from/for classes we also can create lifelines for/from components… which is actually a very interesting feature.
Context menu on a component shows the create lifeline option [not available when clicking on a part ]
After the creation of several lifelines, you can draw the interactions between the components. There isn’t functionality like the bi-directional method creation in the class-sequence diagram interaction for the component-sequence diagram. Kind of logic, a component only has interfaces.
Why would you want to do this, create sequence diagrams for components?
While the component diagram 
is a static view of your system the sequence diagram is dynamic, it visualizes the flow in your application, the sequence of actions that occur in the system. So, you could use sequence diagrams to validate your system, your component design. Take a scenario, a requirement scenario, and create a sequence of actions from in a sequence diagram 
. By doing this you create a kind of simulated behavior of the component structure, which you can check on correctness and consistency.
The scenarios used for this can be found in the use case diagrams 
, which describe the actions that take place on/in the system. Now describes a use case diagram not really a scenario, you better can use an activity diagram for that 
, which can be based on the use case diagram as a kind of use case realization. Now the interesting thing is, testers use activity diagrams to create test cases, they have specific technologies to extract the right amount and type of scenarios [see http://robkuijt.nl/index.php?entry=entry080423-135750 ], these scenarios we can use to validate our component design which actually closes the loop.
It would be even more interesting and powerful if we not only could link sequence and component diagrams but also use case and activity diagrams making notification when an activity changes and a scenario can’t be run anymore on the purposed component… hard to accomplish, but interesting, versioning would be hard…
Quick reference poster--the VSTS Rangers “scrumified“ process
from: Certified Scrum Developer course … we can finally remove the duck tape :)
The Accentient Scrum Developer course is an intensive five day experience for whole teams of developers. The course teaches teams how to turn product requirements into potentially shippable increments of software using the Scrum framework, Visual Studio 2010, and modern software engineering practices. Attendees will work in self-organizing, self-managing teams using a common instance of Visual Studio Team Foundation Server 2010 to achieve this goal.
VSTS 2010 Test Edition at the TMap® Dag 2009 [Dutch]
Theorie en praktijk wisselen elkaar af met onderwerpen als TMap NEXT®, ketentesten, security testen, Agile testen en nog veel meer actuele onderwerpen.
Tijdens de TMap® Dag 2009 wordt het nieuwe boek TPI® NEXT gelanceerd! Een compleet herziene versie van het TPI® model waarin voor het eerst de business doelen écht een ssentiële rol spelen.
WS8 -- De TMap NEXT® Proces template voor Visual Studio Team System
Gerard van der Pol, Microsoft
Visual Studio Team System ondersteunt uw procesbenadering voor software ontwikkeling door het gebruik van Proces Templates. De guidance en werkwijze kunnen hierdoor transparant worden ondersteund door de in het project gebruikte tooling. Voor de nieuwe 2010 release
van Visual Studio Team System is er een proces template in de maak rondom TMap NEXT®.
Deze sessie introduceert de TMap NEXT® proces template en laat zien welke voordelen er te verwachten zijn door deze te hanteren. Tevens zullen we laten zien op welke wijze de template aansluit op de bestaande en nieuwe test functionaliteit in Team System.
Programma [pdf]
Datum: 17 november 2009
Locatie: Hotel van der Valk, Vianen
Aanmelden: www.sogeti.nl
…and another Team system MVP in the Netherlands
Pieter just pinged me that he is awared as Team System MVP…
He already updated his LinkedIn page:
but not his blog :-)
great news… congratulations Pieter!!! [lets go skiing during the mvp-summit!?]
How to keep you fingers on the keyboard within VSTS 2010 Architecture Edition
Keyboard junkies have a hard time with all that dragging and dropping of shapes and connectors around within VSTS 2010. So here are some tips for these junkies how to keep you fingers on the keyboard.
New default DSL Tools behavior…
Adding shapes to the design surface allows you to directly start typing, for example after adding a class shape to the design surface lets you start typing the class name. Hitting 'Enter’ will confirm the value and F2 lets you edit it again.
More interesting, in the compartments of the class shape hitting Enter twice adds an attribute or operation. So, type a name for an attribute, hit Enter to confirm the value and hit Enter again for adding a new attribute.
Very simple and also very quick for the creation of classes.
Fast diagram editing with shortcuts…
With the default behavior you still have to drag and drop class shapes on the design surface. To eliminated this piece of mouse handling you can create your own keyboard shortcuts. Cameron pointed me to this environment settings in Options dialog screen. The architecture team has created a very nice an clean overview of all the actions you can execute in this dialog. So, adding your own shortcut keys is very easy. Below I added the shortcut key “Alt-C” in the context of the Logical Class Diagram for adding Class shapes to my design surface.
I also added the shortcut keys for adding Attributes and operations, just because setting the focus to the compartments isn’t that easy with shortcuts…
And when this is still to hard for you, you can buy a drawing tablet.. with a nice tablet overlay.
and you will look as happy as this guy…
although it just looks like he isn’t that comfortable with it…
02: Management, preserving and organization of manual test cases within VSTS 2010.
- Part 00: Agile Testing with VSTS 2010 and TMap
- Part 01: User Stories
- [current post, removed the Agile Testing blablabla and part pre-fix]
This post belongs to the series of “Agile Testing with VSTS 2010 and TMap” although this topic has a wider scope, in used software delivery methodology you have to think about test case management.
A small introduction.
First a small introduction to give some direction on the needs of test case management.
The drawing below I made during a discussion with several TMap experts and explains how test cases are created and executed in a simple situation.
From top to bottom and left to right.
 | Teams start brainstorm over user needs, create stories and write these down on cards together with acceptance criteria.
Within VSTS 2010 this can be done in the work item type ‘user story’ [see previous post]. |
 | After prioritizing the stories, teams start the iterations by brainstorm on implementation tasks and test tasks.
Again, within VSTS 2010 this can be done in the work item type ‘user story’ [see previous post]. |
  | Next, the sources and test case are going to be created, developed. Within VSTS 2010 source files in the IDE and than added to version control. Test cases are added as work item to TFS. |
 | The source files are build, unit tested, labeled, packaged and finally deployed on a test environment where the manual test cases are executed against this deployment |
All the artifacts created by tasks are under version control, except test cases.
All the sources and corresponding automated tests like, unit tests, load tests, web tests and codedui tests are under version control. Getting the latest version of the sources also gets the latest version of these tests. But, also when using the more sophisticated capabilities of version control, like branching and merging, the tests also have this characteristics.
Manual test cases different, they are work items and aren’t under version control they have a history. Belong [often] to a user story. So, it gets interesting when making the above scenario more real live with sources and automated tests under version control with branching and merging. See drawing below…
- User stories, sources and test case are created [top left].
- [red line] A branch is made from the first version to start the development of an other version.
- User story 2 [center] is added to this branch [new/ added user needs], also sources are added to this branch and test cases are created.
- Finally version 2 is build and the test case of user story 2 are executed [bottom right].
Dotted red line; while there was a branch created from the sources, no branch was created from the test cases of user story 1 [top left], resulting in a version 2 with only partial tested functionality.
This branching scenario is very simple, there are some more complicated ones as you can read in Microsoft Team Foundation Server Branching. This 2006 MSDN story has also some nice branching and merging strategies.
The need for test case management within VSTS 2010.
This branching, merging and the decoupled characteristic of test cases is one of the reasons why we need test case management. We manually need to organize and manage our test cases in such a way that every iteration, version, service pack, hot fix and release is sufficient tested and to prove that its well tested.
As described in the paper Test Cube principle [pdf]:
A good test set (= collection of test cases) is of priceless value. During the realization of an application it is the basis for the reporting concerning test coverage and progress. In a maintenance situation it is a source of knowledge concerning the application and a source of ready-made test cases for regression tests.
In the next post we will have a look / deep dive in the capabilities of TFS 2010, MTLM [Test and Lab manager] how they fulfil the needs of test case management and in what way the TMap process template helps this process.
One part of the solution and even an agile one: automate the manual test cases as quick as possible and put them under version control. Next post more…