Ultimately since I currently write all images to a storage account in Azure, I decided that I’ll continue to leverage that and create an Azure PostgreSql instance. In order to develop the API I needed to haev PostgreSql installed locally, so instead of doing a whole installation I thought Docker was the answer.
So the issue, on Windows (I’m running Windows 11, but this may be an issue for earlier vesrions), I was getting a weird error message:
C:\ProgramData\DockerDesktop must be owned by an elevated account
This error occurs when Docker Desktop’s installer detects that the C:\ProgramData\DockerDesktop folder is not owned by an elevated (administrator) account. It’s a security measure introduced to prevent privilege escalation vulnerabilities. The issue often happens if the folder already exists with incorrect permissions or ownership, possibly from a previous installation attempt.
My solution as I had not attempted to install Docker was to open an elevated prompt and delete and recreate that folder:
cd C:\ProgramData\DockerDesktop
Remove-Item DockerDesktop -Recurse -Force
New-Item -Name DockerDesktop -ItemType Directory
After that re-run the installation and everything should go, if however you already had Docker installed, it might be enough to just take ownership and assign Administrators. If you want to do that via PowerShell:
$dockerData = "C:\ProgramData\DockerDesktop"
$owner = New-Object System.Security.Principal.NTAccount("Administrators")
$dir = Get-Item $dockerData
$sd = $dir.GetAccessControl()
$sd.SetOwner($owner)
$dir.SetAccessControl($sd)
Your mileage may vary on that one.
]]>My own work has progressed nicely as well, I have now created several GitHub Actions that should really help me in the long run. As always you can find the code living in the Mod-Posh Organizatin. I’ve got 4 Actions down and I think at least 4 more to go, not including things I pull out and create seperate actions for. For example, I’m working on an Action that will setup my environment for building and deploying, part of that required pulling a version from a PowerShell module or a C-Sharp project file. I have moved that into a seperate Action dedicated to just getting the verison, this allows the Action I’m working on to be solely dedicated to that aspect. Basically anything that can be calculated or passed into the Action by some other means should be. The Action should only do the thing it was written to do and not anything beyond that.
As I work through these, I’m beginning to realize that they can also be written to work with Azure DevOps as Pipeline Templates. As I do a lot of work in both environments this may lend itself very nicely to that work as well.
Finally, I’m starting to work on a BuildAutomation project that is akin to the Github Actions I’m working on. But this project is a PowerShell module that will allow me to really pare down my psake file. This BuildAutomation module will encapsulate the code in that script now, into easy to use cmdlets. Not everything will be able to be removed, but I’m hopeful that I can pull out a large percentage of that code. I may also spin up a few PowerShell script modules for BlueSky and Discord for the notifications that I use.
I hope you have a great week ahead, and I’ll write more later!
]]>In other more pertinent news I have begun to start the conversion from my local psake environment over to Github Actions. My first stab was writing the in Python, while not terribly complex, that’s a muscle I don’t really want to stretch, so I’ve decided to re-write them in PowerShell. This is in line with what I typically write in for my personal and business code. Since I’m starting to consolidate all that code into the Mod-Posh Organization, I decided to move them from my personal repositories to that org.
My assumption was that I could unlist the Github Action and then publish it from the new repository. This was incredibly inacurate on my part, what needs to actually happen is to just transfer the repository. When you transfer the Github Action repository to a new owner/organization it updates everything for you on the back-end. Long story short, I didn’t do that for the first one, I’ve now been on a Github Support journey, where the action can’t be published because of a naming conflict, but when you search for that name in the marketplace it doesn’t exist.
Name must be unique. Cannot match an existing action, user or organization name.
I am not aware of the backend process, but my guess is there is a database entry someplace that contains the name of my action and it’s all messed up. So in order to prevent you from facing this issue, please just transfer the repo, that’s what I’ve done with the remaining Github Action repos and no issue at all.
Once things have been sorted I plan on writing about the actions and how to set them up and use them. For the most part these will mimic my current workflow which may or may not be yours, but hopefully I will have built enough flexibility in so that you can also use them either all together or individually.
Have a great week!
]]>In terms of nuget.org it’s not to terribly difficult and after a few hit and miss attempts I was able to put together a nice little one-liner that returns all the code that I have published there. It was simply a matter of finding the right endpoint and knowing what to ask.
(Invoke-RestMethod -Method Get -Uri "https://azuresearch-usnc.nuget.org/query?q=owner:jeffpatton1971") |Select-Object -ExpandProperty data
@id : https://api.nuget.org/v3/registration5-semver1/patton.githubclient/index.json
@type : Package
registration : https://api.nuget.org/v3/registration5-semver1/patton.githubclient/index.json
id : Patton.GithubClient
version : 2.3.0
description : A C# Dll library for working with the Github Rest API
summary :
title : Patton.GithubClient
licenseUrl : https://www.nuget.org/packages/Patton.GithubClient/2.3.0/license
projectUrl : https://jeffpatton1971.github.io/GithubClient/
tags : {Github, GithubApi}
authors : {Jeffrey Patton}
owners : {jeffpatton1971}
totalDownloads : 1694
verified : False
packageTypes : {@{name=Dependency}}
versions : {@{version=1.0.0; downloads=339; @id=https://api.nuget.org/v3/registration5-semver1/patton.githubclient/1.0.0.json}, @{version=1.1.0; downloads=327;
@id=https://api.nuget.org/v3/registration5-semver1/patton.githubclient/1.1.0.json}, @{version=1.1.1; downloads=341;
@id=https://api.nuget.org/v3/registration5-semver1/patton.githubclient/1.1.1.json}, @{version=2.0.0; downloads=336;
@id=https://api.nuget.org/v3/registration5-semver1/patton.githubclient/2.0.0.json}…}
vulnerabilities : {}
@id : https://api.nuget.org/v3/registration5-semver1/retrosheet/index.json
@type : Package
registration : https://api.nuget.org/v3/registration5-semver1/retrosheet/index.json
id : RetroSheet
version : 2.0.0
description : A collection of models that represent the data structures in Retrosheet data files.
summary :
title : RetroSheet
tags : {}
authors : {Jeffrey S. Patton}
owners : {jeffpatton1971}
totalDownloads : 118
verified : False
packageTypes : {@{name=Dependency}}
versions : {@{version=2.0.0; downloads=118; @id=https://api.nuget.org/v3/registration5-semver1/retrosheet/2.0.0.json}}
vulnerabilities : {}
@id : https://api.nuget.org/v3/registration5-semver1/modposh.logger/index.json
@type : Package
registration : https://api.nuget.org/v3/registration5-semver1/modposh.logger/index.json
id : ModPosh.Logger
version : 1.0.0
description : A Logging library for .Net and PowerShell
summary :
title : ModPosh.Logger
projectUrl : https://github.com/mod-posh/Logger
tags : {}
authors : {Jeffrey S. Patton}
owners : {jeffpatton1971}
totalDownloads : 55
verified : False
packageTypes : {@{name=Dependency}}
versions : {@{version=1.0.0; downloads=55; @id=https://api.nuget.org/v3/registration5-semver1/modposh.logger/1.0.0.json}}
vulnerabilities : {}
Asking the PowerShellGallery was a little more difficult, where the nuget api is version 3 and there is decent documentation for it, the PowerShellGallery is an older version of nuget. It seems really to be geared almost exclusively to the tools you are meant to use to interact with it and the only documentation I’ve found is an unofficial documentation page. Fortunately I was able to get something to work.
Invoke-RestMethod -Method Get -Uri "https://www.powershellgallery.com/api/v2/Packages()?`$filter=Authors eq 'jeffrey@patton-tech.com'"
id : https://www.powershellgallery.com/api/v2/Packages(Id='AzDevOps',Version='1.0.0')
category : category
link : {link, link}
title : title
updated : 2022-12-31T11:17:41Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='AzDevOps',Version='1.0.1')
category : category
link : {link, link}
title : title
updated : 2022-12-31T17:37:49Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='AzDevOps',Version='1.0.2')
category : category
link : {link, link}
title : title
updated : 2023-09-30T19:03:41Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='AzDevOps',Version='2.0.0')
category : category
link : {link, link}
title : title
updated : 2023-12-19T22:54:04Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='AzDevOps',Version='2.1.0')
category : category
link : {link, link}
title : title
updated : 2024-01-08T14:30:19Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='Pipelines',Version='1.4.0')
category : category
link : {link, link}
title : title
updated : 2024-01-04T18:07:26Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='Pipelines',Version='1.5.0')
category : category
link : {link, link}
title : title
updated : 2024-01-04T18:07:26Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='Pipelines',Version='1.6.0')
category : category
link : {link, link}
title : title
updated : 2024-01-04T18:07:26Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='Pipelines',Version='2.0.0')
category : category
link : {link, link}
title : title
updated : 2024-01-07T16:15:17Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshAdoTask',Version='1.0.0')
category : category
link : {link, link}
title : title
updated : 2024-01-01T07:32:03Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='1.0.0')
category : category
link : {link, link}
title : title
updated : 2022-11-24T03:40:04Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='1.1.0')
category : category
link : {link, link}
title : title
updated : 2022-11-24T03:40:04Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='2.2.0')
category : category
link : {link, link}
title : title
updated : 2022-11-26T07:25:57Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='2.3.0')
category : category
link : {link, link}
title : title
updated : 2022-12-05T14:59:18Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='2.4.0')
category : category
link : {link, link}
title : title
updated : 2022-12-05T14:59:18Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='2.4.1')
category : category
link : {link, link}
title : title
updated : 2022-12-08T03:34:27Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='2.4.2')
category : category
link : {link, link}
title : title
updated : 2022-12-08T10:34:28Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='3.0.0')
category : category
link : {link, link}
title : title
updated : 2022-12-20T21:09:53Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='3.1.0')
category : category
link : {link, link}
title : title
updated : 2022-12-20T21:09:53Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='3.1.1')
category : category
link : {link, link}
title : title
updated : 2022-12-20T21:09:53Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='3.2.0')
category : category
link : {link, link}
title : title
updated : 2022-12-20T21:09:53Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='PoshMongo',Version='3.2.1')
category : category
link : {link, link}
title : title
updated : 2024-01-04T21:52:28Z
author : author
content : content
properties : properties
id : https://www.powershellgallery.com/api/v2/Packages(Id='RetroSheetModule',Version='2.0.0')
category : category
link : {link, link}
title : title
updated : 2024-01-01T08:42:04Z
author : author
content : content
properties : properties
The main difference here, is while Nuget has an Owner property, PowerShellGallery only has an Authors property. Turns out I’ve used 3 different Author names over the years! So step one will be to re-push all the packages so they all share the same Author and then see if we can consolidate the data. The PowerShellGallery returns all packages as discreet id’s, there is a Properties property that will allow this list to be sorted, grouped and filtered.
At any rate this little post is just to remind me that I’ve actually done this, so we can come back to this later and try to create some sort of landing page that would accumulate downloads and such.
Have a great Monday! (if that’s possible)
]]>Monday was a holiday, and honestly I didn’t even realize that, actually had to look it up on the HR page at work, so I put in an hour or so before knowing I didn’t need to! This is rather par for me, I don’t often notice holidays outside of a few, and I guess New Year’s Day was never really one I paid attention to! Over the weekend and into Monday I did a lot on a new PowerShell module that should really help with some of the things I need to do for work.
Pipelines is the name of the module, and the goal is really to make it easier to work with either [Azure Devops pipelines(https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops)] or Github Actions. I wrote a YamlSerializer that converts the pipeline classes into yaml, those can then be written to a file so that you can programatically create a pipeline. This will fill a very niche need of my own, not sure how popular it may be for anyone else, but I’m very proud of it, it’s my first dive into creating a more feature rich C# application/module for PowerShell.
As part of that I am wanting to step-up my Local Automation game. I’ve re-visited a few of those configurations and spent a significant amount of time learning and using the Github GraphQL API. I was thinking about a way to automate some of what I was doing and report it back into a project or pulling down unassigned cards from a project and creating a TODO out of those. I need more time learning this, but I may see about writing something up in the near term on using the Github GraphQL API.
Wednesday was spent learning about Logging Providers, enoguh so that I have started my own Log Provider, Logger. It’s a simple Provider that should write to console or a log file, and includes some basic log rotation capability. As part of a first release I wanted to have at least the basics of console/file writing enabled, additionally there should be tests as part of the project. This is the first C# project I’ve put up that incorporates a Test project. I’m using Nunit tests to make sure the provider covers the basics. The best part of this for me is that it works with the current release of PowerShell wihtout too much work at all, this has been an issue of mine for a while now, I’m sure it’s possible with Nlog or other Providers, but I’ve never really had much luck getting that sorted out and working for me.
My Thursday was filled with work meetings and a lot of waiting on other people so I could progress. I spent time between reading and working out the Unit tests for the log provider. Friday was a very good day, again a LOT of meetings, but progress was made. I wrote a lot of documentaiton for work about an overly complex process that really shouldn’t be and I added a C# project to my Local Automation repo.
I’m also thinking that it may be time to convert a lot of my automation tasks into Github Actions. Thinking about how my automation work for local development, a lot of that can also be done via workflows in Github. Most of my psake tasks could just as easily be steps in a workflow. As I think about how I use pipeline technologies in Azure Devops, it’s possible that I could create those tasks as individual Github Actions and then tie them all together in a larger workflow that performs most of the same functions as my psake does locally. Considering most of my work is C# and .Net, it turns out I will need to figure out Docker as well. Not sure if that’s good bad or ugly at the moment, but I guess we shall see.
I understand that this has been a long winded post, but if you’ve stuck through to the end, congratulations you made it! I hope you all have a wonderful weekend and I look forward to what comes next week!
]]>In the world of Azure DevOps Pipelines, both ‘stageDependencies’ and ‘dependencies’ play a pivotal role in orchestrating the flow of your pipeline. Let’s distinguish between them and understand when to use each.
In the first example, the utilization of ‘stageDependencies’ orchestrates a sequential order between the ‘Build’ and ‘Deploy’ stages. The ‘Build’ stage, encapsulating the application’s compilation, acts as a prerequisite for the ‘Deploy’ stage. This dependency ensures that deployment only proceeds when the build is successfully completed, providing a structured and controlled progression through the pipeline.
stages:
- stage: Build
jobs:
- job: BuildJob
pool:
vmImage: 'windows-latest'
steps:
- script: echo "Building the application"
- stage: Deploy
dependsOn: Build
jobs:
- job: DeployJob
pool:
vmImage: 'windows-latest'
steps:
- script: echo "Deploying the application"
The second example showcases the role of ‘dependencies’ within a job. Here, ‘SecondTask’ depends on the successful completion of ‘FirstTask’ within the same job. This setup ensures a sequential execution of tasks, where the output of one task serves as the input for another. ‘dependencies’ within a job prove instrumental in maintaining task order and inter-task dependencies. The ‘name’ property of tasks is instrumental in defining dependencies between tasks within a job or template. To ensure a smooth dependency chain, it is vital to place the ‘name’ property strategically.
jobs:
- job: ExampleJob
pool:
vmImage: 'windows-latest'
steps:
- task: Task1
name: FirstTask
script: echo "Executing the first task"
- task: Task2
name: SecondTask
script: echo "Executing the second task"
dependsOn: FirstTask
In addition to managing dependencies, understanding how to handle variables and parameters is crucial for building flexible and reusable pipelines.
Moving beyond dependencies, our third example introduces the strategic use of variables at the pipeline level. The ‘solution’ variable holds the name of the solution file and is accessed in the ‘BuildJob.’ This centralized definition of common values simplifies maintenance and ensures consistency across the pipeline, showcasing the power of variables in maintaining a coherent and adaptable pipeline structure.
variables:
solution: 'MySolution.sln'
stages:
- stage: Build
jobs:
- job: BuildJob
pool:
vmImage: 'windows-latest'
steps:
- script: msbuild $
Parameters allow you to make your pipeline more dynamic by accepting values from external sources. They are defined at the job or template level using the ‘parameters’ keyword. Our final example introduces the concept of parameters at the job level. The ‘ExampleJob’ accepts a parameter named ‘inputParameter’ with a default value. Parameters provide flexibility, allowing external inputs to customize job behavior. This adaptability is crucial for handling diverse scenarios, making pipelines more dynamic and responsive to changing requirements.
jobs:
- job: ExampleJob
pool:
vmImage: 'windows-latest'
parameters:
- name: inputParameter
default: 'defaultValue'
steps:
- script: echo "Using parameter: $"
By mastering the use of ‘stageDependencies’ and ‘dependencies’ for managing dependencies and understanding the proper handling of variables and parameters, you can create efficient, flexible, and reusable Azure DevOps Pipelines. The examples provided should serve as a guide to help you navigate these concepts seamlessly in your CI/CD processes. Happy coding!
]]>As a seasoned C# and PowerShell developer, you’re likely familiar with the challenges of managing templates and their output variables in different pipelines within Azure DevOps. In this article, we’ll explore a common issue: how to pass outputs between discreet pipeline files, which isn’t directly supported by Azure DevOps. We’ll delve into a practical workaround involving the use of the Azure DevOps REST API to update a variable group, enabling seamless access across pipelines.
Azure DevOps pipelines are powerful, but managing template output variables across different pipelines can be challenging. You might find that passing outputs directly between separate pipeline files isn’t supported out-of-the-box.
It’s crucial to understand that the scope of output variables in Azure DevOps is limited to the job in which they were set. This means that accessing output variables across different pipeline files directly is currently not supported in Azure DevOps. This limitation can pose a challenge when you need to share data or outputs between distinct pipelines.
jobs:
- job: Build
steps:
- script: |
# Your build steps here
displayName: 'Build'
- job: Deploy
steps:
- script: |
# Your deploy steps here
displayName: 'Deploy'
To address this limitation, you’ll need to employ alternative methods for sharing data across multiple pipelines. Three commonly used approaches are:
One effective method is to leverage variable groups in Azure DevOps. Variable groups allow you to define a set of variables that can be used across multiple pipelines. By updating the variable group using the Azure DevOps REST API, you can dynamically adjust the values of these variables, ensuring they are accessible in various pipelines.
- task: PowerShell@2
displayName: 'Update Variable Group'
inputs:
targetType: 'inline'
script: |
$url = "$(System.CollectionUri)$(System.TeamProject)/_apis/distributedtask/variablegroups/{variableGroupId}?api-version=6.0-preview.2"
$token = "YOUR_PERSONAL_ACCESS_TOKEN"
$headers = @{
"Content-Type" = "application/json"
"Authorization" = "Basic $(System.AccessToken)"
}
$body = @{
variables = @{
'VariableName' = 'NewValue'
# Add more variables as needed
}
} | ConvertTo-Json
Invoke-RestMethod -Uri $url -Method PUT -Headers $headers -Body $body -UseBasicParsing
YOUR_PERSONAL_ACCESS_TOKEN with the generated Personal Access Token.{variableGroupId} with the ID of the variable group you want to update.You can leverage Azure Redis Cache and Azure Storage Account to store and retrieve output variables in your pipelines. This approach provides a reliable and scalable solution for sharing data across pipelines.
- task: PowerShell@2
displayName: 'Use Azure Redis Cache'
inputs:
targetType: 'inline'
script: |
# Connect to Azure Redis Cache
$redisHost = "your-redis-host.redis.cache.windows.net"
$redisPort = 6380
$redisKey = "$(YourRedisCacheKey)"
$redisConnection = [Microsoft.Azure.Commands.RedisCache.Models.RedisCacheManagementClient]::new($redisHost, $redisPort, $redisKey)
# Set a value in Redis Cache using an output variable
$myValue = "$(OutputValue)"
$redisConnection.StringSet("myKey", $myValue)
Storing output variables in an Azure Storage Table can be a valid approach, depending on your specific use case and requirements. Azure Storage Tables are a NoSQL data storage service provided by Microsoft Azure that can store large amounts of structured data.
steps:
- powershell: |
# Define your output variable
$outputVariable = "Hello, Azure Storage Table!"
# Authenticate with Azure using a service principal or other method
# For simplicity, this example uses a Personal Access Token (PAT)
$token = "YOUR_PERSONAL_ACCESS_TOKEN"
az login --output none --service-principal --username http://azure-cli-2016-08-05-14-31-15 --password $token --tenant YOUR_TENANT_ID
# Define variables for storage account and table
$storageAccount = "yourstorageaccount"
$tableName = "yourtablename"
# Create a new entity in the table with the output variable
az storage entity insert -t $tableName --account-name $storageAccount --partition-key "PartitionKey" --row-key "RowKey" --property @{"OutputVariable"="$outputVariable"} --if-none-match "*"
displayName: 'Store Output Variable in Azure Storage Table'
How to pass variables from one pipeline to another pipeline in azure devops:
This discussion suggests using a variable group to store values that you want to control and make available across multiple pipelines. Variable groups can also be utilized for storing secrets and other values that might need to be passed into a YAML pipeline. It’s important to note that this method is more suitable for sharing static values across builds and release pipelines.
Example:
Consider a scenario where you have a variable that represents a version number. If you need to ensure this version number is consistent across multiple pipelines, using a variable group would be an effective approach.
variables:
- group: MyVariableGroup
Share variables across stages in Azure DevOps Pipelines:
This thread focuses on how to share variables across stages within a single Azure DevOps Pipeline. It mentions that output variables may now be used across stages in a YAML-based pipeline. This capability allows you to pass useful information, such as a go/no-go decision or the ID of a generated output, from one stage to the next.
jobs:
- job: Build
steps:
- script: |
# Your build steps here
echo "##vso[task.setvariable variable=GeneratedOutput;isOutput
=true]OutputValue"
displayName: 'Build'
- job: Deploy
dependsOn: Build
steps:
- script: |
echo "$(Build.GeneratedOutput)"
displayName: 'Deploy'
Passing variables between release pipelines in azure:
This post explains that output variables are created by the pipeline and referenced by the other tasks in the pipeline. It emphasizes that these variables cannot be defined statically, and their value can only be known after running the task in the pipeline.
jobs:
- job: Build
steps:
- script: |
# Your build steps here
echo "##vso[task.setvariable variable=GeneratedOutput;isOutput=true]OutputValue"
displayName: 'Build'
- job: Deploy
dependsOn: Build
steps:
- script: |
echo "$(Build.GeneratedOutput)"
displayName: 'Deploy'
In conclusion, we covered different scenarios where sharing data or variables is essential in the CI/CD process. It’s important to note that while Azure DevOps provides mechanisms for sharing data within a pipeline, passing variables directly between different pipeline files is currently not supported. As a workaround, methods like updating a variable group or utilizing external services or storage can be employed to achieve the desired outcome.
It’s essential to carefully evaluate your pipeline architecture and choose the approach that best suits your specific requirements. By adopting these workarounds, you can effectively manage template output variables across Azure DevOps pipelines, ensuring smooth and reliable CI/CD processes.
URL References:
In modern application development, managing secrets and sensitive configuration data is crucial. Azure Key Vault provides a secure and efficient way to store these sensitive values. In this article, we’ll explore how to integrate Azure DevOps pipelines with Azure Key Vault using templates and variable groups.
Before we begin, ensure you have the following:
Variable groups in Azure DevOps allow you to define a set of variables that can be reused across multiple pipelines. When combined with Azure Key Vault, you can securely store and reference sensitive information.
Pipelines > Library > Variable groups.+ Variable group to create a new group.To access secrets from Azure Key Vault in your pipelines, you’ll need to link your variable groups:
Pipelines > Library > Variable groups.Link secrets from an Azure key vault as variables.Now, your variable group is linked to the Key Vault, allowing you to reference secrets in your pipeline.
For more detailed information, refer to the official documentation on Variable Groups.
Azure Key Vault integration with Azure DevOps pipelines allows you to securely store and use sensitive information in your CI/CD process. To learn more about this integration, refer to the official documentation on Azure Key Vault integration in Release pipelines.
Azure DevOps templates provide a way to define reusable, parameterized pipeline configurations. When combined with variable groups and Key Vault, you can streamline your CI/CD process.
Create a new YAML file in your repository (e.g., azure-pipeline-template.yml). This template will define the structure of your pipeline and accept parameters for customization.
parameters:
- name: environment
displayName: 'Environment'
type: string
default: 'dev'
- name: buildConfiguration
displayName: 'Build Configuration'
type: string
default: 'Release'
jobs:
- job: BuildAndDeploy
displayName: 'Build and Deploy Job'
pool:
vmImage: 'windows-latest'
steps:
- script: |
echo "Building for $ environment with configuration: $"
# Add your build and deployment steps here
In this template, we define two parameters: environment and buildConfiguration. These parameters allow for customization when the template is used in a pipeline. Note how we reference the parameters using $ and $.
In your pipeline YAML file (e.g., main-pipeline.yml), reference the template and pass the desired values for the parameters. Additionally, declare your regular variables using name/value pairs.
variables:
- name: regularVariable
value: 'This is a regular variable'
- name: MyRegularVariable
value: 'This is another regular variable'
- group: 'MyVariableGroup' # Reference to your variable group
trigger:
branches:
exclude:
- '*'
pr:
branches:
include:
- '*'
jobs:
- template: azure-pipeline-template.yml
parameters:
environment: '
prod'
buildConfiguration: 'Debug'
In the above example, the template is referenced, and the environment parameter is set to 'prod' and buildConfiguration is set to 'Debug'. Note how we have defined our Variables and Variable Group references, when including both in your Variable block you must reference un-grouped variables using name/value syntax, as defined in the documentation.
If you encounter any challenges while working with variable groups and Azure Key Vault integration, you’re not alone. Here are some Stack Overflow threads where developers have discussed and resolved similar issues:
Storing and Retrieving Secrets in Azure Key Vault with Variable Groups in Azure DevOps
Azure Release Pipeline: Azure Key Vault Task vs Variable Groups
Authorize Button when Linking Variable Group to Azure Key Vault in Azure DevOps
These threads cover a range of issues and solutions related to integrating variable groups with Azure Key Vault. Feel free to explore them for additional insights and solutions to potential challenges you may encounter.
Integrating Azure DevOps pipelines with templates and variable groups linked to Azure Key Vault provides a robust solution for managing sensitive information in your CI/CD process. By following the steps outlined in this article, you can enhance the security and efficiency of your deployment pipeline.
For more detailed information, refer to the official documentation on Variable Groups and Azure Key Vault integration in Release pipelines.
]]>Navigate to the Epic:
Create a Filter for Done Tasks and Bugs within the Epic:
Click on the “Create Issue Filter” icon (looks like a funnel) at the top right of the epic window.
In the filter creation window, use the following JQL query:
"Epic Link" = EPIC-1234 AND issuetype in (Task, Bug) AND status = Done
Replace EPIC-1234 with the key of your epic.
Click “Save As” to save this filter with a descriptive name like “Done Tasks and Bugs in Epic”.
Use the Filter in a Board or Search:
Go back to your board or use the search functionality in Jira.
Apply the filter you just created by selecting it from the list of available filters.
Exporting the Report:
Once you have applied the filter and can see the list of done tasks and bugs, locate the export option (typically represented by an icon with an arrow pointing outwards or the word “Export”).
Select the format you prefer for your export (e.g., CSV for a spreadsheet, PDF for a document).
Save the file to your preferred location on your computer.
First off I want to set expectations, I write a lot of code and have been writing code for a very long time now. Do not take what I’m about to say as the best way to approach the problem, this is how I work day to day and it’s worked extremely well for me.
I’ve never really given any in-depth thought about how I approach a development problem. Perhaps the best way for me to explain it is to give an example of a recent problem I’ve been working. Last week I wrote about an issue I face with Azure Devops, where you can only grab the latest release of a nuget package regardless if the package has been flagged as “release” or “pre-release”.
My process starts researching the problem, in this instance I needed to validate that I didn’t have something configured incorrectly, or that I was using the tool in the wrong way. From what I was able to discover there was nothing misconfigured in Azure Devops and I was using the nuget command in the correct way. As I have been working on a PowerShell module for Azure Devops, I was aware of the API and focused my research on the API and how it would respond. After an hour or so of reading and poking at the API with PowerShell I came to the conclusion that the “release” or “pre-release” monikers had no impact when it comes to a repsonse. Everything was returned as “type” release, which mean that my “pre-release” view was basically ignored.
The next step for me is to then take the big picture problem and break it down into smaller more manageable pieces. How do I get from the name of a package to the latest release version that I need? In Azure Devops the packages are stored in an Artifact Feed. Those feeds have Views and Packages are associated with one or more of those Views. Each package has a specific version that goes along with it.
So I have my set of smaller steps:
In reality for me, there are several sub-steps/tasks that go along with each step, for the purposes of this post we’ll stick to the high-level steps. Again this may or may not be the best way to do things, but this is how I approach the problem.
I have existing code that I can use as a reference, and then I just work with the API documentation and the command-line until I get something that works close to what I want. It’s at that point that I formalize the snippets I have into a function and then I iterate on that code until I feel it will always return what I expect it to return. The code itself is very straight forward and has never been truly complex, and this I think leads to the answer to this question.
Identify the problem you are facing, determine if there is a way to break it down into discreet steps and finally work out each step until it gives you the desire result. I know that sounds very simplistic, but that’s really how I approach the problem. I don’t try to solve the entire problem at once, I break it down into smaller pieces and then work on each piece until I get the desired result. I then take those pieces and put them together into a larger solution. What language you use makes little difference, the process is the same and all the languages functionally work in the same way.
Honestly this is how I approach most problems, not every problem I work can be solved with code, but the steps are always the same. I hope this helps, and I hope you have a great day!
]]>