Automating Deployments using Infrastructure as Code with Pulumi
Microsoft Azure allows you to quickly deploy and destroy resources in the cloud using Microsoft’s massive, scalable data centers. It is tempting to quickly deploy a solution in minutes that normally would take weeks to complete. However, when deploying solutions in a large enterprise company, it gets disorganized quickly and can lead to various issues related to security and cost management.
I prefer to adhering to a controlled deployment framework, where users have access to specific resource groups under a subscription. We can manually deploy our changes to a developer/sandbox resource group and then replicate these changes, using programmatic deployments, to support a controlled deployment with a review workflow.
Here, we give all developers and engineers full control of their own resource group and let the DevOps pipeline use a separate authentication token to deploy changes to test, production and so fourth. Administrators can then get limited access, such as read only, to these groups.
Infrastructure-as-Code
To make the flow work, I use a product called Pulumi, which is built on the open-source platform Terraform. Pulumi handles versioning and recreation of Azure resources (as well as other cloud providers) and can show a preview of what changes will occur.
For example, we can create a SQL database, get the connection string from the database and use that in a connection string or key vault when creating a web application that relies on that database. Later on, we can scale up/out the resources and Pulumi will detect that they are already created, so a reconfiguration will happen.
You build the logic for Pulumi using code, and many languages are supported; such as Python, TypeScript etc. I use C# using Visual Studio Code, as it’s familiar to me. Let’s walk through a simple standard setup.
Creating a basic Pulumi project
We need Visual Studio Code installed. I develop this on a Mac, so I also install .NET 5 SDK and .NET 3.1 SDK, Git extensions so we can push to our repo, and PowerShell for automating the configuration.
For VS Code, make sure you can run Pulumi from the Terminal.
- Launch VS Code.
- Open the Command Palette (⇧⌘P) and type ‘shell command’ to find the Shell Command: Install ‘code’ command in PATH command.
- Restart the terminal for the new $PATH value to take effect. You’ll be able to type ‘code .’ in any folder to start editing files in that folder.
From here on, you need to create a Pulumi account, which is free for individuals. Then we start the setup process, which is documented well on the ‘Getting Started’ section of the Pulumi website.
For Mac, there are a few extra steps, so I need to run the Homebrew installer for Apple Silicon on my M1 Mac. Open Terminal and run:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
We then use the homebrew package manager to install Pulumi:
brew install pulumi
I then create a new .NET C# project for Pulumi and open it in Visual Studio Code. When the Pulumi asks for input, just use defaults for now.
cd ~/Documents
mkdir PulumiSample && cd PulumiSample
pulumi new azure-csharp
code .
Now the project is created and we have opened the project in VS Code. Open the MyStack.cs class and look at the code. This will create a resource group and a storage account.
Return to the terminal and enter ‘pulumi up’, which results in:
(base) tobiaslekman@Tobias-MacBook-Pro PulumiSample % pulumi up
Previewing update (dev)
View Live: https://app.pulumi.com/lekman/PulumiSample/dev/previews/1fc5a038-c147-433a-8c1a-f7aa3d582fa5
Type Name Plan
+ pulumi:pulumi:Stack PulumiSample-dev create
+ ├─ azure:core:ResourceGroup resourceGroup create
+ └─ azure:storage:Account storage create
Resources:
+ 3 to create
Do you want to perform this update? [Use arrows to move, enter to select, type to filter]
yes
> no
details
This will preview the changes and ask you to confirm to create the resources. You can go ahead and make the change, but afterwards delete the resources using:
pulumi destroy
The Pulumi documentation covers most things you need to know about creating resources using the code framework.
Handling configurations with Pulumi
If we pass the configuration between many developers/engineers then we will probably need some configured settings objects.
Notice that the Pulumi project contains a file named Pulumi.dev.yaml. This will contain all our properties and settings. For argument’s sake, let’s say that I want resource group names to be set specifically. Pumuli adds a wildcard ending to resource names to support upgrades, but this can be hard coded using the Name property of the resource type as:
var environment = "dev";
var product = "pulumisample";
var resourceGroupName = $"rg-{product}-{environment}";
var location = "uksouth";
// Create an Azure Resource Group
var resourceGroup = new ResourceGroup(
resourceGroupName,
new ResourceGroupArgs
{
Name = resourceGroupName,
Location = location,
Tags = { { "environment", environment } }
});
By adding “Name” to the cration arguments, we can control the exact name of the resource group. But since this is not shared between configurations, then I want it to be based off the configuration file. I run these commands in the terminal:
pulumi config set pulumi:disableAutoNaming "true"
pulumi config set azure:environment "dev"
pulumi config set azure:productName "pulumisample"
The config file is now updated as:
config:
azure:environment: dev
azure:location: uksouth
azure:productName pulumisample
pulumi:disableAutoNaming: "true"
We can read the configuration values from the stack using the following code change:
var config = new Config("azure");
var environment = config.Require("environment");
var product = config.Require("productName");
var resourceGroupName = $"rg-{product}-{environment}";
var location = config.Require("location");
The “require” option makes sure that the value is present and an exception is thrown if it’s missing.
If you now run a “pulumi up” and compare it to the previous results, you will see that the resource group name has changed.
Sharing configurations between developers
We also want some of these settings to remain secret, such as connection strings and passwords. We can use Pulumi to encrypt specific values and store these in configuration files. To do this, let’s create a sample password as:
pulumi config set keys:sqlPassword "ThisIsALongPasswordForSql123456789" --secret
This results in the following output.
config:
azure:environment: dev
azure:location: uksouth
azure:productName pulumisample
keys:sqlPassword:
secure: AAABAMjVPK7BXEq7nYza1VSmIl9Qr1nlWhty3Ho18tQxCpHsNJlWyW5PuXpTzIAUy9O7dG620419OESF/3HUBAOu
pulumi:disableAutoNaming: "true"
The password is encrypted inside the Pulumi service, so if you pass the yaml file to a build pipeline or other service, then it is decrypted fine as long as you are logged in to the same Pulumi account. You can access this value in the stack using the following code:
var keyConfig = new Config("keys");
var sqlPassword = keyConfig.RequireSecret("sqlPassword");
When a new developer gets the project, this will not work. To make the correct change, the developer has to have their own Pulumi account and run ‘pulumi login’ in the terminal. Then, to make things easier, I create a PowerShell script that is checked in to the same Pulumi project as:
# Script used to set environment specific variables
# Update prefix below to your account name instead of 'lekman'
pulumi stack select 'lekman/PulumiSample/dev'
# Set to the end part of your resource group, for example for John Doe:
pulumi config set azure:environment: "devjnde"
# Set the SQL password to a complicated and long password, use https://www.avast.com/random-password-generator
pulumi config set keys:sqlPassword "PASTE generated long password" --secret
The developer can then set these variables and the config file is updated. As the developer should use their own GIT branches for changes then this will work well.