Visual Studio Team Services Logic Apps continuous integration and deployment

..or “The code to my INTEGRATE2017 session”.

So, INTEGRATE2017 just ended (actually at the point this post is out it ended about a month ago). Fantastic event! If you’ve never been, in short it’s an integration focused event running over the period of a couple of days in London featuring Microsoft product group and community speakers. Regardless if you were there this year or not I very much encourage you to be there next year (or catch the US version in October).

I had a session: “Logic Apps continuous integration and deployment using Visual Studio Team Services”. In it I showcased a process for doing just what the title says: A process for developing Microsoft Azure Logic Apps solutions with GIT source control hosted in VSTS and using the build and release management capabilities to package and deploy Logic Apps, Integration Account artifacts; schemas and maps, as well as Functions.

The video and slides are available on the INTEGRATE2017 site (or will be soon) and the slides are also available from me here.

What I’d also like to share is the sample code and scripts that I used, available here. As well as stepping through the actual Visual Studio Team Services configuration step by step with screenshots and configuration strings.

The primary purpose of creating this process is the requirement to have a repeatable build, release and deployment pipeline that would look the same, work the same, and be configured the same way for all developers in the team over time building a lot (think in terms of 1500+) integrations based on Logic Apps. One of the goals is also to limit the amount of work each developer has to do when building an individual integration and offloading some of that to the build and release steps, like for example creating a resource group template for deploying the developed Azure Function or adding the code to enable diagnostics in every Logic Apps resource group template. There are many more steps and actions we have taken along those same lines, and perhaps I will have more blogposts about it, however I will try not to make a mess of the red thread of this post by getting side tracked with those. Much like I tried to simplify some things and keep my talk focused I will with this post.

As far as the actual build, release and deploy process much can be improved and made even easier than it is. With a few simple tweaks to the build and release definitions (as long as you follow a naming convention) things can be easily changed to use more of the built-in variables so that the definitions be easily cloned from one integration to another without any changes. The name of the build or release definition is all that will change and all that is needed. Not all integrations will include a Function, nor schemas and maps, and if so these steps can simply be removed.

Overview

So first of, the process. It is covered in more detail in the slides as well as the video, so go there for more coverage. Here I am just pasting an image of it to have it as a reminder of what we are trying to achieve.

The Visual Studio solution

Also. before we dive into what the VSTS process looks like, let’s also take a look at the VS.Net solution and project structure so that we know what we are trying to, in the end, deploy. Again, the code is available here.

To sum it up, we have:

• A Resource Group Template project, containing the Logic App and the API Connection(s)
• A Enterprise Integration Pack aka Integration Account project containing two schemas and a map, and also the xslt that is the result of compiling the map.
• An Azure Functions project

Except for what is contained within the Visual Studio solution for a specific integration it should also be noted that we have a structure in source control that contain some of the other shared artifacts that we will need and use in the build and release definitions. Let’s briefly go through those as well.

First of I have my scripts folder.

For the purpose of this walkthrough it contains two scripts:

• Deploy-AzureIntegrationAccount.ps1, which is a help script to deploy schemas and maps into an integration acount from a folder.
• Enable-AzureRmDiagnosticsSettings.ps1, which is a help script to enable diagnostics logging, in this case to an Microsoft Operations Management Suites workspace.

I will not go into detail on any of these scripts here. Just not what they are and what they do.

I also have a shared ResourceGroupTemplates folder.

That for the purpose of this walkthrough contains two files, of which one is significant.

• FunctionAppTemplate.json, which contains a generic Azure Resource Group Template to deploy a Functions App. This is so that this template does not need to defined each and every time an integration is developed, merely the unique parameters to it is needed.
• EXAMPLE FunctionApp.env.parameters.json, which contains an example of the unique parameters that each Function App needs to supply. If you look at the Visual Studio solution we have and the Functions App project you can see that it holds two parameter files. One for test and one for prod.

So, now that we have looked at what we are going to deploy, let’s look at the Visual Studio Team Services build and release definitions.

Visual Studio Team Services build definition

The main purpose of the build step and the build definition is to create a package for the release step to release to an environment. Not all of the project types that we are working with can be built by VSTS, for example the Functions project (which is also the reason why the xslt is needed in the source controlled project). Except for the fact that ideally it would be best if we could actually build them – the build itself being a validation that the code contained within the project has passed that quality check point – we do no need to build them to be able to deploy them.

The build definition contains 6 steps, well, seven actually.

1. (Get the code from source control)
2. Build the Resource Group Template solution – the Logic App
3. Copy the Schemas and Maps artifacts
4. Copy the Functions
5. Copy the Shared Functions Template
6. Copy the Scripts
7. Finally. Publish all of the prepared artifacts and publish them so that they are available to be used by the release definition.

And since I’ve called this continuous build and not only build, we also have a trigger for when this build should be triggered. In this case setup to trigger when changes are made to the master branch (most often in our process by completing a pull request).

The way I see things, most of the steps are self explanatory in their names, so I’ll simply go through the steps with a screenshot and a textual representation of their significant configuration with no more explanation then that. If I do not show parts of the configuration then that’s because there is no configuration made to those sections and the defaults are in use.

Build solution

Project: **/INT0001_ProcessPurchaseOrder/*/*.deployproj

Clean: checked

Copy Schemas and Maps

Source Folder: $(Build.SourcesDirectory)/INT0001_ProcessPurchaseOrder/INT0001_ProcessPurchaseOrder_Artifacts

Contents: **/?(*.xsd|*.xslt|*.ps1)

Target Folder: $(Build.ArtifactStagingDirectory)/artifacts

Copy Functions

Source Folder: $(Build.SourcesDirectory)/INT0001_ProcessPurchaseOrder/INT0001_ProcessPurchaseOrder_Functions

Contents: **

Target Folder: $(Build.ArtifactStagingDirectory)/functions

Copy Shared Functions Template

Source Folder: Shared/ResourceGroupTemplates

Contents: FunctionAppTemplate.json

Target Folder: $(Build.ArtifactStagingDirectory)/functions

Copy Scripts

Source Folder: Shared/Powershell/scripts

Contents: *

Target Folder: $(Build.ArtifactStagingDirectory)/scripts

Publish Artifact

Path to Publish: $(Build.ArtifactStagingDirectory)

Artifact Name: output

Artifact Type: Server

That’s it for the build definition. Let’s now look at the release definition.

Visual Studio Team Services release defintion

The purpose of the release definition is to take the created by the build (as described above in the build definition step) and use the artifacts within to deploy the build to an environment.

The release definition consists of 5 steps.

1. Deploy Integration Account Schemas and Maps
2. Deploy FunctionsApp template (aka create the “Functions application container”)
3. Deploy Functions (aka use Web Deploy to deploy the Functions project I built)
4. Deploy Logic Apps
5. (Run a Powershell script to enable Azure diagnostics and ship them to my OMS workspace)

The last step isn’t needed. I added it to show of another small thing I consider a best practices.

Before we look at each step involved, let’s again look at the continuous aspect of it. In the release definition I have my trigger configuration set to enable Continuous Deployment, meaning that as soon as a new artifact version is available (as soon as a new build completes) a new release will be created.

As you can see I have defined two environments. Test and Prod. For Test I have deployment set up so that as soon as a new release is created a new deployment to that environment is automatically triggered. For Prod it is configured as manual. For test there is no approval needed, but for prod I have also setup approval so that when someone does request a deployment to be made to prod it must first be approved, or Bruce will get angry (which is a reference to something I said during the presentation if you haven’t seen it, meaning that not everyone should be allowed access to production in this scenario).

I have it setup so that any project administrator can approve the deployment, but you can create your own groups or point to specific individuals directly as well.

Now for the tasks that will be triggered once a deployment is made to an environment (I will give screenshots only for test, but you can quite easily figure out what it would have looked like for prod – the steps are all the same).

Deploy Integration Account schemas and maps

(Azure Powershell)
Connection Type: Azure Resource Manager
Azure Subscription: (In my case INTEGRATE2017) This would be your Test or Prod (or whatever) subscription.
Script Type: Script File Path
Script Path: $(System.DefaultWorkingDirectory)/$(Release.DefinitionName)/output/scripts/Deploy-AzureIntegrationAccount.ps1
Script Arguments: -rootPath ‘$(System.DefaultWorkingDirectory)/$(Release.DefinitionName)/output/artifacts’

Deploy FunctionsApp template

(Azure Resource Group Deployment)
Subscription: (see comment on subscription in previous step)
Action: Create or update resource group
Resource Group: (in my case $(Release.DefinitionName)-test) If you deploy to test and prod in different subscriptions then you can just leave this as $(Release.DefinitionName), provided that’s what you want.
Template location: Linked artifact
Template: $(System.DefaultWorkingDirectory)/$(Release.DefinitionName)/output/functions/FunctionAppTemplate.json
Template parameters: $(System.DefaultWorkingDirectory)/$(Release.DefinitionName)/output/functions/INT0001Functions.test.parameters.json

Deploy Functions

 

App Service name: INT0001Functions-test (here, if you deploy test and prod to different subscriptions you must still have different suffixes since a globally unique name is required – this name must also be the same as the name given in the parameters file in the previous step)
Package or folder: $(System.DefaultWorkingDirectory)/INT0001_ProcessPurchaseOrder/output/functions
Publish using Web Deploy: Enabled

Deploy Logic Apps

(Azure Resource Group Deployment)
Action, Resource Group, Location, Template location as before.
Template: $(System.DefaultWorkingDirectory)/INT0001_ProcessPurchaseOrder/output/INT0001_ProcessPurchaseOrder/LogicApp.json
Template parameters: $(System.DefaultWorkingDirectory)/INT0001_ProcessPurchaseOrder/output/INT0001_ProcessPurchaseOrder/LogicApp.test.parameters.json

Enable Diagnostics

Even though this step is optional, let’s look at it anyway for completeness.

(Azure Powershell)
Script Type: Script File Path
Script Path: $(System.DefaultWorkingDirectory)/INT0001_ProcessPurchaseOrder/output/scripts/Enable-AzureRmDiagnoticsSettings.ps1
Script Arguments: -resourceName $(Release.DefinitionName)

Wrap Up

That’s it. I think. There are alot of moving parts and I am sure there could be some additional explanations required depending on your previous experiences and knowledge. But the solution is all here. The links to the code and the explanation of the VSTS configuration. If you have any further questions please feel free to contact me.

As a follow up, in case you have API Management in your solution as well and would like to use VSTS for it as well, have a look at the VSTS pipeline described by Mattias Lögdberg here.