Dev/Test Lab Management on Azure, what is your test infrastructure

Before you can start using a Dev/Test Lab on Azure you have to think about what is in the lab. It can be a complete production environment (not that obvious) or just a single machine with an app and everything in between.

  INFRA

Example Enterprise Dev/Test Infrastructure.

From lower left, a datacenter with databases and an erp system. The ERP system is connected with the cloud data layer via a queue and some internal Apps. Identity in both environments are the same via AD replication. the data tier has a high available SQL server connected with several internal API’s (1,2 and 3) and external API’s which are consumed by the high available App for internal users. A common Dev/Test infrastructure for an enterprise.  

Cloud Dev/Test infrastructure.

It is pretty hard to come up with a reasonable test infrastructure for a system under development. You have to think about the different test coverage the system under test needs. Not all test scenario's require the same infrastructure. For example a chain test has different infrastructure needs as a UI test run for a sprint or a load test or a acceptance test. Probably you will need multiple test infrastructure configurations for one system. The benefit of using Azure for it, is that you can have these different flavors.

For example in the above drawn Dev/Test infrastructure when you need to run some Functional UI tests on the App, you don’t need the whole Dev/Test infrastructure with all the expensive machines. The App with a small stub should be enough for UI testing.

The availability set can be removed too for UI testing only (see Redundant). The same for the internal API’s when you want to validate them, only the API’s are needed with a stub database. When a complete end to end chain test needed to be done, than all machines and infrastructure components should be in place.

The description of your Dev/Test infrastructure can exist out of Azure images with pre-installed software, an Azure Resource Manager definition file and some script files to configure the environments. When you have a cloud only solution you will probably have enough on a ARM file, when you need to mimic a complete on premise environment you need all and probably more.

The nice thing about using Azure Resource Manager for your Dev/Test infrastructure is that you can group (link, nest) different ARM definition files together. So you can breakdown pieces and group them when needed.

image

ARM template: https://msdn.microsoft.com/en-us/library/azure/dn790564.aspx

So you can make separate ARM definition for the UI environment, the API environment and all other components and group them together when needed. in this way making a collection of Dev/Test ARM definitions per test type / component under test.

Redundant.

An infrastructure for acceptance tests is in most situation used by real users or the customer, to validate if it fits. You defiantly don't want this system to drop availability during the test, in opposite to test executed by team members. So when you setup a test environment for acceptance testing you need to make it redundant. Where in other situations where the team itself is involved a cheaper single instance will work. When you Lab also supports acceptance test take this in mind.

Normally for functional or chain testing a Dev/Test environment can look like this, no high availability or redundancy. Much cheaper as the double installment.

not redu

Chain of systems.

To setup an environment on Azure for a chain of systems is pretty hard, especially when these systems in production run on premise. For cloud only scenario's even with some service providers it is easier. For the 'on premise chain' Dev/test lab in the cloud, every system needs to be analyst if it fits/ will run on Azure and if it has additional dependencies with on premise systems. A solution when not all systems needs to be re-created on Azure is connect the to be tests environment to the on premise chain of systems, if there are test systems ready on premise or off course external systems. batch jobs, schedules, identity connections with external systems are the challenges.

To start analyze the parts to decide if and how they are created on Azure. Make an ARM definition per part, which can be grouped together later on in a single one. It is up to the test knowledge to decide what needs to be part of the chain and what can be validate in isolation (with or without mocks/ stubs).

Mocks and Stub.

It is really nice to have the capability to stub a system or environment. It is even nicer to have a complete library of images and ARM definitions for your Dev/Test lab. In that way the team can plug in any stub in their lab. There are tools dedicated for this purpose, making it easier for you Dev/Test lab to provide isolated tests and cheaper lab environments.

image

and read: Top Tools to Help You Mock Web Services

Evolving systems.

As with everything also systems evolve overtime. Not only with updates but also with complete new configurations due to architectural changes. Changes can come from development, from vendors, from maintenance from everywhere. Pretty challenging to keep your Dev/Test lab in sync with these changes.

The Immutable Infrastructure pattern is a possible solution for this, also known as Phoenix systems (see Fowlers post). Immutable systems can be recreated again and again from their description. Automatic, they can rise again in the same state. Changes to these systems are done in the description only, which can be used for your Dev/Test lab environments.

The technology to accomplish this is getting more and more mature and should be on the adoption list of the on premise operation organization. New cloud systems development must have this immutability adoption from start, manual changes to systems are not done anymore. For Dev/Test environments these ARM descriptions, script files and, or images are great for reuse.

When all this isn't in place yet, than the Dev/Test lab is a good start for adoption and learning these technologies and processes. Meanwhile keep a manual process in place to keep your Dev/Test environments in sync with production.

There are still no good tools in place which can read an environment and make the proper description of it.

UI Tests.

A different environment flavor in the Dev/Test lab environments. There must already be a running system from which you can use from the different Client configurations or one client when only the functionality needs to be validated.

UI Stub

A challenging with UI tests is that Azure only delivers Client operating systems with a MSDN Azure subscription (see the coming licenses post).

Knowledge.

To create the proper Dev/Test lab environment for your system it is a must that people with test and people with infrastructure knowledge work together with the cloud specialist in creating the environment description. The process of creating such a description is never a one shot success and will always be an iterative process. (see the coming knowledge post)

Closing

The ideal situation is off course when you can reuse the definition used for production, either a written definition (probably not up to date) or a definition created with immutability in mind.

But, it isn’t necessary that always you have to have a production equal Dev/Test environment, it depends on the test type and component under test. When making use of ARM and the nesting capabilities a more cost efficient Dev/Test environment on Azure can be created, used, deleted and recreated again  by the team.

Comments (1) -

Great and Informative Article.
In similar lines will be Interested in an article for "Load \ Regression" test infrastructure that can be developed using Azure and ARM.

Pingbacks and trackbacks (2)+

Add comment