Although uncommon, there are valid reasons to have a monorepo - a single git repository for multiple projects, for example migration projects. Until yesterday, I thought this was not possible in Azure DevOps.

A colleague informed me it’s possible to rename the file something other than /azure-pipelines.yml. From there I figured out how to accomplish create multiple Azure DevOps YAML pipelines in a monorepo.

In this tutorial, you will learn how to:

• setup a root pipeline
• setup 2 pipelines in subfolders and triggered by changes in those folders.
• rename pipelines in DevOps UI
• use triggers
• change working directories

The full example is available here
https://github.com/julie-ng/azure-devops-monorepo →

I. Project Structure and YAML files

Let’s imagine we have the following setup:

  .
  ├── README.md
  ├── azure-pipelines.yml
  ├── service-a
  |── azure-pipelines-a.yml
  │   └── …
  └── service-b
      |── azure-pipelines-b.yml
      └── …

In a standard Azure DevOps project, you have a single azure-pipelines.yml file in your project root folder. In our project, we will have 3 different pipeline files:

• azure-pipelines.yml
• service-a/azure-pipelines-a.yml
• service-b/azure-pipelines-b.yml

These files will be very similar to your standard YAML pipelines, with two small exceptions: triggers and working directories. We’ll cover those later. First we will add the pipelines.

Step 1 - Add the Pipelines

When you create a new DevOps pipeline, select the repository and on the “Configure your pipeline” page, select “Existing Azure Pipelines YAML file”, which will open up this overlay on the right:

You want to go through this process 3 times, each time selecting a different YAML file. In the image above, I have chosen /a/azure-pipelines.yml, which is original filename before I renamed it later.

Step 2 - Rename your pipelines

By default, Azure DevOps names your pipelines per GitHub user/org and repository name, so you will end up with 3 pipelines named similar to this:

• julie-ng.azure-devops-monorepo
• julie-ng.azure-devops-monorepo (1)
• julie-ng.azure-devops-monorepo (2)

Not very helpful. Find more options button and select “Rename/move”.

I’ve chosen the following names:

• azure-devops-monorepo (root)
• azure-devops-monorepo (Service A)
• azure-devops-monorepo (Service B)

II. Triggers and how this works

Normally pipeline runs when anything changes. These three pipelines are defined so they only build when their respective files change. We accomplish this with trigger path definitions.

Root Project must exclude paths

Note the root project excludes our subdirectories. This means that a change to service-a/readme.md will not trigger our root pipeline.

trigger:
  paths:
    exclude: # Exclude!
      - 'service-a/*'
      - 'service-b/*'

Sub-projects must include paths

We have two sub-projects with their own pipelines. We have to adjust each appropriately so it only runs when the sub-project’s code changes:

trigger:
  paths:
    include: # Include!
      - 'service-a/*' # or 'service-b/*'

Now you have your multi-pipeline monorepo setup! But you are not finished. There are reasons why the monorepo setup is not common. While it is acceptable to choose this path, you should understand the disadvantages and caveats.

III. Caveats

Be Aware of Other Triggers

There are many reasons for a pipeline to be built. In fact, the official docs name four different types of events that can trigger build pipelines:

Trigger	Description
CI triggers	Git Push
PR triggers	Pull Requests
Scheduled triggers	Schedules defined in Cron format
Pipeline triggers	Pipelines can call each other
Manual	A human clicks a button

I add manual runs to make it 5. Although we are limiting path triggers to our subfolder, the when is partially determined by external factors. This means that a pull request that conceptually only affects service B may unintentionally trigger a build of service A.

If you are used to committing and pushing incomplete changes, you may have an unusual number of broken builds. A common symptom is seeing multiple commits in a row that start with “update….” This danger also applies to the separate repo use case. But in a monorepo case it is made worse. The danger here is that a developer or team gets used to red or broken builds and stop reacting to them. So it’s important to be disciplined across your entire team when committing and pushing your changes.

I haven’t tried it. But theoretically you should be able to separate schedules for the pipelines.

Building A or B or both?

Let’s say you have a commit history that looks like this:

0883cf8  b: change number 3 (47 minutes ago) <=== git push
2896d9c  a: change number 5 (49 minutes ago)
3fa6757  root: add newlines to readme (49 minutes ago)

First off, both pipeline A and pipeline B will run and they will run with the files from the working tree at 0883cf8.

In this example, a developer first made changes to service A and then later to service B. Because the changes were pushed together, the azure-pipelines-a.yml pipeline runs with files not from 2896d9c but from the future 🤯.

This means if you actually have dependencies outside of that include path in the triggers: property, you may experience unexpected build results. It seems unlikely in our example. But what if you had such a project structure?

  .  
  ├── service-a
  |   |── pipeline-a.yml
  │   └── …
  ├── service-b
  |   |── pipeline-b.yml
  │   └── …  
  └── common-components
      |── pipeline-c.yml
      └── …

Then you would be more concerned. This is a trade-off that comes with monorepos. Builds may be accidentally triggered and you should prepare for that. If you’re working in teams, make sure it’s very transparent what everyone is working on.

Keep your Working Directory in Mind

To illustrate this caveat, service B is a Node.js project. Although our YAML file for service B sits in the correct subfolder, the working directory will still be the root. If you try to run npm install without changing directories, it will fail because there is no package.json in the root.

We can change this by using the workingDirectory key in the YAML:

- script: npm install
  workingDirectory: service-b/

Unfortunately workingDirectory is only available under steps:, which means you cannot set it once on the whole pipeline, but rather for every task, script, etc. You can make this less painful by using a variable like in my code sample. See the official docs: YAML Reference for details and further limitations in the YAML syntax.

IV. Conclusion

If you have good reason to use a monorepo and want to setup multiple Azure DevOps pipelines, you can. But remember that you lose some sense of control over when and what you are building in your CI pipeline. So if you march down this path, over-communicate within your team, keep your commits squeaky clean, and carry on.