Configuring Agent Pools and Understanding Pipeline Styles in Azure DevOps

This blog is inspired by a Microsoft Lab with similar tittle

INTRODUCTION
Welcome to our latest exploration into the dynamic realm of Azure DevOps! In today's blog post, we delve into the crucial aspects of configuring agent pools and unravel the intricacies of pipeline styles. Navigating the ever-evolving landscape of DevOps requires a deep understanding of these fundamental components, and we're here to guide you through the process.

Whether you're a seasoned DevOps engineer or just starting your journey, mastering the configuration of agent pools and comprehending various pipeline styles is essential for streamlining your development workflows. Join us as we unravel the mysteries behind these integral concepts, providing insights, best practices, and actionable tips to empower your Azure DevOps journey. Let's embark on this knowledge-packed expedition together!

The post will cover the creation and management of agent pools, as well as an understanding of different pipeline styles. It will also discuss the significance of these concepts in the context of DevOps practices and their impact on the efficiency of software delivery.

Let start by defining the relevant concepts.

Azure DevOps
Azure DevOps is an end-to-end software development platform that supports Agile, DevOps, and other continuous development practices. It provides a range of tools and services for requirements and project management, version control, automated builds, testing, release management, and reporting. Azure DevOps aims to help teams create and improve products at a faster pace than traditional software development approaches, fostering a collaborative culture and integrating with a variety of other tools and services. In simple terms, it's a comprehensive solution that enables teams to efficiently manage the entire software development lifecycle, from planning and coding to testing and deployment, all in one place.

What is agent pool and its use in Azure DevOps?
An agent pool is a collection of one or more agents that can be used to run builds and deployments in Azure DevOps. Agents are the machines that execute the tasks defined in pipelines. Agent pools are used to organize agents based on their capabilities, such as operating system, software, or hardware configurations. This allows teams to manage and allocate resources more efficiently and to ensure that the right agents are used for the right jobs. Agent pools can be created and managed from the Azure DevOps organization or project settings. They can be used in both build and release pipelines and can be either self-hosted or provided by Microsoft as hosted agents.

Azure DevOps Pipelines
Azure DevOps Pipeline is a service that allows you to automatically build, test, and deploy your code to various platforms and cloud environments. It supports continuous integration (CI) and continuous delivery (CD) practices, enabling you to automate the entire software delivery process. Azure Pipelines can be used with any language or platform, and it provides cloud-hosted agents for building and deploying applications. It offers features such as parallel jobs, native container support, and flexible deployments to Kubernetes, serverless, or virtual machines.

Pipelines Configurations Supported by Azure DevOps
Azure DevOps supports YAML-based pipelines and classic UI-based pipelines configurations. The classic UI allows users to define and manage pipelines using a graphical interface, which can be a preferred option for those who are not familiar with or prefer not to use YAML syntax. However, Microsoft has been encouraging the use of YAML pipelines due to their portability, version control integration, and ease of sharing.

In this exercise, we will be using the YAML-based pipeline editor, which is a tool provided by Azure DevOps to author and edit pipelines using YAML syntax.

Prerequisites:

• Azure Account with subscription

• Microsoft Edge or an Azure DevOps supported browser.

• Set up an Azure DevOps organization

• Git for Windows

• Visual Studio Code.

Objectives
By the time you conclude this learning journey with us, you will be able to do the following in Azure DevOps:

1. Implement YAML-based pipelines.
2. Implement self-hosted agents.

Here we go!

Step 1: Setting up an Azure DevOps Organization

• Login to Azure portal at www.portal.azure.com.

• On the global search bar, type in Azure DevOps select Azure DevOps Organizations.

• One the Azure DevOps Home page, click on My Azure DevOps Organizations.

• On the Azure DevOps Organizations page, click on Create new organization.

This will take you to a new Signup page, click on the Continue button if you do not wish to switch directory.

• On the next page, enter a globally unique organization name, the location you intend to host your projects and the authentication character displayed.

We can see the new organization we created and can now start creating our projects.

Step 2: Create a New Project

• Here we will be creating a project named eShopOnWeb.

• Enter the project name and click on Create project

• Our project landing page!

Step 3: Import our project files from a Git repository to Azure DevOps

• On the project landing page, on the left pane under your project name, click on Repos>Files, Import a Repository. Select Import.

• On the Import a Git Repository window, paste the URL of the Git repository where your application is located. For this exercise, we will be using a project provided by Microsoft. Here is the URL https://github.com/MicrosoftLearning/eShopOnWeb.git and click Import

The project in Azure DevOps Repos Environment

CONFIGURE A YAML-BASED PIPELINE

Step 4: Create an Azure DevOps YAML pipeline

• Create a template-based Azure DevOps YAML pipeline.

• On the EShopOnWeb project page, in the vertical navigational pane on the left side, click Pipelines.

• Click the Create Pipeline button - if you don’t have any other pipelines created yet or click New pipeline to create an additional new one.

• On the Where is your code? pane, click Azure Repos Git. Since we have imported the code there. _You can select other locations based on where you have your code. _

• On the Select a repository pane, click EShopOnWeb. Which is our project.

• On the Configure your pipeline pane, click Existing Azure Pipelines YAML File.

• On the Select an existing YAML file, select main for the Branch, and /.ado/eshoponweb-ci-pr.yml for the Path and click Continue

• This sample pipeline is a basic .NET application Build pipeline, which does the following:

• A single Stage: Build

• A single Job: Build

• 3 tasks within the Build Job:

• Dotnet Restore

• Dotnet Build

• Dotnet Publish

resources:
  repositories:
    - repository: self
      trigger: none

stages:
- stage: Build
  displayName: Build .Net Core Solution
  jobs:
  - job: Build
    pool:
      vmImage: ubuntu-latest
    steps:
    - task: DotNetCoreCLI@2
      displayName: Restore
      inputs:
        command: 'restore'
        projects: '**/*.sln'
        feedsToUse: 'select'

    - task: DotNetCoreCLI@2
      displayName: Build
      inputs:
        command: 'build'
        projects: '**/*.sln'

    - task: DotNetCoreCLI@2
      displayName: Test
      inputs:
        command: 'test'
        projects: 'tests/UnitTests/*.csproj'

    - task: DotNetCoreCLI@2
      displayName: Publish
      inputs:
        command: 'publish'
        publishWebProjects: true
        arguments: '-o $(Build.ArtifactStagingDirectory)'

• On the Review your pipeline YAML pane, click the down-facing caret symbol next to the Run button, click Save. This action creates the pipeline.

The created pipeline

MANANGE AZURE DEVOPS AGENT POOLS

Step 5: Configure an Azure DevOps self-hosting agent

In the Azure DevOps portal, in the upper right corner of the Azure DevOps page, click the User settings icon, depending on whether or not you have preview features turned on, you should either see a Security or Personal access tokens item in the menu, if you see Security, click on that, then select Personal access tokens. On the Personal Access Tokens pane, and click + New Token.

• On the Create a new personal access token pane, click the Show all scopes link and, specify the following settings and click Create (leave all others with their default values):

Name = EShopOnWeb
Scope (custom defined) = **Agent Pools** (show more scopes option below if needed)
Permissions = **Read and manage**

• On the Success pane, copy the value of the personal access token to Clipboard and click the Close button.

• On the Personal Access Token pane of the Azure DevOps portal, click Azure DevOps symbol in the upper left corner and then click Organization settings label in the lower left corner.

• To the left side of the Overview pane, in the vertical menu, in the Pipelines section, click Agent pools.

• On the Agent pools pane, in the upper right corner, click Add pool.

• On the Add agent pool pane, in the Pool type dropdown list, select **Self-hosted, in the Name text box, type **proj-2420-pool and then click Create.

• Back on the Agent pools pane, click the entry representing the newly created proj-2420-pool.

• On the Jobs tab of the proj-2420-pool pane, click the New agent button.

• On the Get the agent pane, ensure that the Windows and x64 tabs are selected, and click Download to download the zip archive containing the agent binaries to download it into the local **Downloads*8 folder within your user profile.

To Register the Agent

• Start Windows PowerShell as administrator and in the Administrator: Windows PowerShell console run the following lines to create the C:\agent directory and extract the content of the downloaded archive into it. By running these lines of code:

cd \
mkdir agent ; cd agent
$TARGET = Get-ChildItem "$Home\Downloads\vsts-agent-win-x64-*.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem

• In the same Administrator: Windows PowerShell console, run the following to configure the agent:

.\config.cmd

-Follow the prompt, specify the values of the following settings:

Enter server URL = https://dev.azure.com/Project-24201

That's the URL of your Azure DevOps organization, in the format https://dev.azure.com/**organization_name**, where organization_name represents the name of your Azure DevOps organization.

Enter authentication type (press enter for PAT) = **Enter**

Enter personal access token = **The access token you recorded earlier in this task**

Enter agent pool (press enter for default) = **proj-2420-pool**

Enter agent name = **az400m03-vm0**

Enter work folder (press enter for _work) = **Enter**

(Only if shown) Enter Perform an unzip for tasks for each step. (press enter for N) WARNING: only press **Enter** if the message is shown

If the above prompt is not displayed, then continue with your configuration.

Enter run agent as service? (Y/N) (press enter for N) = **Y**

enter enable SERVICE_SID_TYPE_UNRESTRICTED (Y/N) (press enter for N) = **Y**

Enter User account to use for the service (press enter for NT AUTHORITY\NETWORK SERVICE) = **Enter**

Enter whether to prevent service starting immediately after configuration is finished? (Y/N) (press enter for N) = **Enter**

• At the end of registering the agent, you will get successful notification as shown below.

Back to the Azure DevOps portal

• In the web browser window displaying the Azure DevOps portal, in the upper left corner, click the Azure DevOps label.

• From the list of projects, click the tile representing your EShopOnWeb project.

• On the EShopOnWeb pane, in the vertical navigational pane on the left side, in the Pipelines section, click Pipelines.

• On the Recent tab of the Pipelines pane, select EShopOnWeb and, on the EShopOnWeb pane, select Edit.

• Here we provide the detail of the new agent we registered to the YAML-based CI pipeline.

• On the EShopOnWeb edit pane, in the existing YAML-based pipeline, replace line 13 which says vmImage: ubuntu-latest designating the target agent pool with the newly created self-hosted agent pool.

• On the EShopOnWeb edit pane, in the upper right corner of the pane, click Save and Run. This will automatically trigger the build based on this pipeline.

You can check the option to Enable system diagnostic

• In the Azure DevOps portal, in the vertical navigational pane on the left side, in the Pipelines section, click Pipelines. Depending on your system setup, the pipeline might prompt you for permissions. Click Permit to allow the pipeline to run.

• On the Recent tab of the Pipelines pane, click the EShopOnWeb entry, on the Runs tab of the EShopOnWeb pane, select the most recent run, on the Summary pane of the run, scroll down to the bottom, in the Jobs section, click Phase 1 and monitor the job until its successful completion.

• When the process concludes, you will be notified of the status of your pipeline as successfully built.

Thank you for your time, kindly subscribe and follow our page for more insightful articles.