3 Challenges when Testing from and on the Cloud.

When adopting the Cloud for your test work it comes with several challenges. Looking at the different usages flavors of the cloud for teams, read: Cloud usage flavors for Development and Test teams, then every flavors has its own challenges.

Challenge 1: Team Workspace in the Cloud – Test Environment On Premise.


The scenario is that the team adopted Team Member Workspace on Azure, but the infrastructure (test environments) are staying on-premise.

The planning, preparation, specification and execution of the test work takes place on the team member workstation which is hosted in the Cloud.

The challenges is that the execution of test are triggered from the public cloud and the test runs on the private data center. There needs to be a connection between the team member workstation and the test environment. The same counts for automatic test runs.

Solution: make a network connection between Azure and the On-Premise datacenter. Often a very long discussion with the internal infrastructure people is needed to get this working.

You can create a Site-to-Site VPN, for connecting datacenters to an Azure site or Point-to-Site VPN, when connecting a single machine to an Azure site.



Challenge 2: Acceptance Test Infrastructure in the Cloud – Production On Premise.


the scenario, all environments (development and test) are on Azure except production, this one still lives on-premise. The challenges is to get the test and acceptance (on Azure) environments as equal as possible to the production (on prem) environment.

Solution: The best solution is the usages of the same installation scripts for production as for acceptance and test. Make PowerShell DSC scripts for and re-use them for every environment.

there is also a need for stubs to make testers need to focus on writing isolated tests, which will make more systems valid. Still not all tests can run on Azure Test Infrastructure (performance and load tests won’t give valid results).

the worst case scenario is that an additional pre-production environment needs to be created on-premise for tests which couldn’t be validate in the cloud.

see: http://blogs.technet.com/b/privatecloud/archive/2013/08/30/introducing-powershell-desired-state-configuration-dsc.aspximage


Challenge 3: ALM Infrastructure in the Cloud – test, acceptance and production on- premise


The scenario: The team consumes their ALM infrastructure from the Cloud and use on premise workstations.

Actually from a technical perspective there isn’t a challenge in this scenario. The only need is reliable https connection between the Test case repository (ALM Infrastructure ) and the test environments.

Interesting in this scenario can be the test data. There is probably a security reason why the environments (test, acc and prod) are in the local datacenter. When the test cases, test data and test results are stored on Azure, the security regulations probably also count for that data.

Often production data is used as test data resulting in a no go for this scenario. A solution is to anonemize the test data.


these where just three challenges for the three different components of Cloud usages for Dev and Test on Azure.

There are many more challenges, maybe in another post.

Add comment