Skip to content

Deploy Cosmo Tech workspace

Remember

This guide focuses on explaining each object individually.
A complete deployment workflow is provided in the Examples section, where all components are combined in a practical scenario !

Before proceeding, ensure that you have selected the correct platform and project.
If unsure, contact your Babylon administrator for the available options.

In this example, we will use the following identifiers:

  • context_id: test
  • platform_id: dev
  • state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a

To deploy a complete Cosmo Tech workspace, you can declare its configuration in yaml files corresponding to specific deployment type.

Each file contains general information about the deployment:

Important

In this tutorial each Cosmo Tech object, values defined directly within the corresponding YAML file.
However, in the Examples section, you will see that we use a centralized variables.yaml file, which defines all required variables and is then referenced across the YAML configurations !

API Organization

Organization.yaml 
kind: Organization
namespace:
  remote: true   # false by default
  state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a
  context: test
  platform:
    id: dev
    url: https://dev.api.cosmotech.com/phoenix/v3-0

The deployment configuration must define the following keys:

  • kind
    Specifies the type of resource to deploy.
    Accepted values: Organization, Solution, Workspace, WebApp.

    ⚠️ The resource type must always start with a capital letter.

  • namespace
    Provides metadata that uniquely identifies the deployment, including:

    • state_id
    • context_id
    • platform_id

  • remote
    A boolean flag indicating whether the state should be stored locally only (false, default) or both locally and in the cloud (true)

  • spec
    Defines the resource configuration. The details are specified under the payload section.

For example, in an Organization deployment file:

Organization.yaml 
kind: Organization
namespace:
  remote: true   # false by default
  state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a
  context: test
  platform:
    id: dev
    url: https://dev.api.cosmotech.com/phoenix/v3-0
spec:
  payload:
    name: My new Organization
    security: {{security}}

All keys in this file can be templated with {{}} syntax for objects and "{{}}" for strings, as security section of this file. Corresponding values must be stored in _variables.yaml_ file at the same level that your project See 👉 Examples:

security:
  default: viewer
  accessControlList:
    - id: user1@email.com
      role: admin
    - id: user2@email.com
      role: editor
    - id: user3@email.com
      role: viewer

API Solution

This is how the solution deployment file is structured

Solution.yaml 
kind: Solution
namespace:
  remote: true   # false by default
  state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a
  context: test
  platform:
    id: dev
    url: https://dev.api.cosmotech.com/phoenix/v3-0
metadata:
  workspace_key: "Project1"
  selector:
    organization_id: "{{services['api.organization_id']}}"
spec:
  payload:
    key: "demosolution"
    name: "My Solution Name"
    description: "My solution description"
    repository: brewery_for_continuous
    version: latest
    tags:
      - brewery
    runTemplates:
      - id: "run_id"
        name: "Standard simulation"
        csmSimulation: AzureWebApp/AzureWebApp_Simulation
        run: true
        preRun: true
    parameters:
    parameterGroups:
    security:
      default: none
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

The metadata section defines deployment specific attributes. It is required in the following resource files:

  • solution.yaml
  • workspace.yaml
  • webapp.yaml

The workspace_key parameter is a mandatory field within the metadata section

  • Purpose:
    Identifies the target workspace
    For example, when Babylon provisions resources such as a Dataset (ADX) or an Event Hub, the generated name follows the convention:
    <organization_id>-<workspace_key> # Ex : o-rv0h6dd492w8-testppdprojectwork

  • Constraints:

    • Must always be defined.
    • Cannot be left empty.

  • ⚠️ Failure Condition:
    If workspace_key is omitted or empty, the deployment will fail.

The selector block is used to specify the target organization under which the resource will be deployed.

  • Parameter: organization_id

    • Purpose:
      Defines the unique organization in which the solution (or other resource) will be created.

  • Behavior:

    • Babylon will associate the deployment (e.g., a solution, workspace) with the specified organization_id.
    • Typically, this value is injected dynamically from the Babylon state (services['api.organization_id']).

API Workspace

The Workspace configuration may include additional parameters required to provision external services such as:

  • Power BI workspaces
  • Azure Event Hubs
  • Azure Data Explorer (ADX) databases

These parameters are defined under the sidecars section, specifically within the azure key.

Workspace.yaml 
kind: Workspace
namespace:
  remote: true   # false by default
  state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a
  context: test
  platform:
    id: dev
    url: https://dev.api.cosmotech.com/phoenix/v3-0
metadata:
  workspace_key: "Project1"
  selector:
    organization_id: "{{services['api.organization_id']}}"
    solution_id: "{{services['api.solution_id']}}"
spec:
  sidecars:
    azure:
      powerbi: # <--- powerbi section
        workspace:
          name: "My workspace Powerbi Name"
          reports:
            - name: Report Name A
              type: dashboard
              path: "powerbi/myreportA.pbix"
              tag: "myReportATag"
              parameters:
                - id: "ADX_Cluster"
                  value: "https://{{services['adx.cluster_name']}}.westeurope.kusto.windows.net"
                - id: "ADX_Database"
                  value: "{{services['api.organization_id']}}-{{workspace_key}}"
          permissions:
            - identifier: "user1@email.com"
              rights: Admin
              type: User
            - identifier: "user2@email.com"
              rights: Contributor
              type: User
            - identifier: "user3@email.com"
              rights: Viewer
              type: User
            - identifier: "<guid>"
              description: "Object Id of Service Principal WebApp"
              rights: Admin
              type: App
      adx:                   # <--- adx section
        database:
          uri: "https://{{services['adx.cluster_name']}}.{{location}}.kusto.windows.net"  # URI Azure Data Explorer Cluster
          create: true
          retention: 365
          permissions:
            - type: User
              email: "user1@email.com"
              principal_id: "412f3fad-3ce3-588s-994c-2a36bccaa0b2"
              role: Admin
            - type: User
              email: "user2@email.com"
              principal_id: "987d3fad-3ce3-588s-994c-2f5s4de8ddd5"
              role: User
            - type: App
              description: "Cosmo Tech Platform <platform_name> For <tenant_name>"
              principal_id: "{{services['platform.principal_id']}}" # Object ID of Platform Enterprise Application
              role: Admin
          scripts:
            - id: "demoscript"
              name: Create.kql
              path: "adx/scripts"
      eventhub:                    # <--- Eventhub section
        consumers:
          - displayName: adx
            entity: ProbesMeasures
          - displayName: adx
            entity: ScenarioMetadata
          - displayName: adx
            entity: ScenarioRun
          - displayName: adx
            entity: ScenarioRunMetadata
        connectors:
          - table_name: ProbesMeasures
            consumer_group: adx
            connection_name: ProbesMeasures
            database_target: "{{services['api.organization_id']}}-{{workspace_key}}"
            format: JSON
            compression: Gzip
            mapping: ProbesMeasuresMapping
          - table_name: ScenarioMetadata
            consumer_group: adx
            connection_name: ScenarioMetadata
            database_target: "{{services['api.organization_id']}}-{{workspace_key}}"
            format: CSV
            compression: None
            mapping: ScenarioMetadataMapping
          - table_name: SimulationTotalFacts
            consumer_group: adx
            connection_name: ScenarioRun
            database_target: "{{services['api.organization_id']}}-{{workspace_key}}"
            format: JSON
            compression: None
            mapping: SimulationTotalFactsMapping
          - table_name: ScenarioRunMetadata
            consumer_group: adx
            connection_name: ScenarioRunMetadata
            database_target: "{{services['api.organization_id']}}-{{workspace_key}}"
            format: CSV
            compression: None
            mapping: ScenarioRunMetadataMapping
  payload:
    key: "Project1"
    name: "My Workspace Name"
    description: "Workspace for solution"
    solution:
      solutionId: "{{services['api.solution_id']}}"
    useDedicatedEventHubNamespace: true
    sendScenarioMetadataToEventHub: true
    sendInputToDataWarehouse: true
    sendScenarioRunToEventHub: true
    webApp:
      url: "https://{{services['webapp.static_domain']}}"
      options:
        disableOutOfSyncWarningBanner: true
        charts:
          workspaceId: "{{services['powerbi.workspace.id']}}"
          dashboardsViewIframeDisplayRatio: 1.8686131386861313
          scenarioViewIframeDisplayRatio: 3.2
          logInWithUserCredentials: false
          dashboardsView:
          scenarioView:
        instanceView:
          dataContent: null
          dataSource: null
        datasetManager:
        menu:
    security:
      default: none
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

The path to existing Power BI reports can be specified under the sidecars → powerbi → workspace → reports section.

Example

    reports:
      - name: Report Name A
        type: dashboard
        path: "powerbi/myreportA.pbix" # <--- 
        tag: "myReportATag"
        parameters:

How Babylon Handles Power BI Report IDs

Automatic Power BI Report ID Retrieval

Babylon can now automatically retrieve and manage Power BI report IDs during deployment.
This eliminates the need for manual copy-paste of report IDs, as was required before.

How it works

  • Each imported Power BI report must be assigned a unique identifier, called a tag.
  • Babylon uses this tag to map the report ID.
  • Once mapped, the report ID can be referenced in the following sections:
    • dashboardsView
    • scenarioView
  • This referencing will be accomplished using a second variable called reportTag in dashboardsView and scenarioView sections, as illustrated in the example below. This variable should correspond to the Power BI report tag you intend to use. Therefore, Babylon will handle everything automatically.
Example 
    sidecars:
      powerbi: 
        workspace:
          reports:
            - name: Report Name A
              type: dashboard
              path: "powerbi/myreportA.pbix"
              tag: "myReportATag" # Here, you should add the tag corresponding to this Power BI report
            - name: Report Name B
              type: dashboard
              path: "powerbi/myreportB.pbix"
              tag: "myReportBTag"

    dashboardsView:
      - dynamicFilters:
        title:
          en: Report Name A
          fr: Report Name A
        report_tag: "myReportATag"

    scenarioView:
      - dynamicFilters:
        title:
          en: Report Name B
          fr: Report Name B
        report_tag: "myReportBTag"

With this configuration:

  • The tag (e.g., myReportATag, myReportBTag) acts as a stable reference.
  • Babylon automatically resolves and injects the corresponding Power BI Report ID into the deployment files.
  • These IDs are also persisted in the Babylon state, ensuring consistency across environments.

Auto-injected Report IDs in workspace.yaml

    dashboardsView:
      - dynamicFilters: []
        reportTag: "myReportATag"
        title:
          en: "Report Name A"
          fr: "Report Name A"
        reportId: "03729d49-c423-4bf5-bb85-681449b56710"

    scenarioView:
      - dynamicFilters: []
        reportTag: "myReportBTag"
        title:
          en: "Report Name B"
          fr: "Report Name B"
        reportId: "9c275c7a-d390-40a0-bc75-b9c5c8093986"

Stored in Babylon State

    powerbi:
      dashboard_view:
        myReportATag: 03729d49-c423-4bf5-bb85-681449b56710
      scenario_view:
        myReportBTag: 9c275c7a-d390-40a0-bc75-b9c5c8093986

All ADX scripts must be placed inside the adx/ folder of your project structure See 👉 Examples

Permissions Reminder

Certain operations may fail if Babylon doesn't have the necessary Azure permissions.

To automatically create Azure Data Explorer (ADX) databases, Azure Functions, or Event Hubs, Babylon must have at least the Contributor role on the target resource group.

🛡️ Owner role is required if Babylon also needs to assign roles to Azure resources during the deployment.

If you security policy doesn't grant such access to Babylon, these operations must be done manually.

Static Web App

To deploy a Static Web App, you can either:

  1. Create a new Azure App Registration handled automatically by Babylon.
  2. Use an existing App Registration requires manual configuration.

This behavior is controlled by the create key in the sidecars.azure.app section:

  • If create: true → Babylon will create a new App Registration automatically.
  • If create: false → You must manually provide:
    • client_id → The Azure App Registration Client ID.
    • displayName → The name of your App Registration.

Example

    azure:
      app:
        create: false
        use:
          displayName: thisismyappforcontinuous
          client_id: "3d0531b1-d23b-4baf-98be-a764c0a42f00"
        principal_id: "{{services['app.principal_id']}}"
        add_to_powerbi: true
        payload:
          displayName: thisismyappforcontinuous
          signInAudience: AzureADMyOrg

Requirements

Deploying a Static Web App also requires:

  • A GitHub repository containing your application code.
  • A destination branch that Babylon will use for deployment.

Quick start:

  1. Create a new GitHub repository
  2. Configure your branch <BRANCH> with source code, for example: Azure sample webapp

Commands to configure your branch

git init

echo "# empty_webapp" >> README.md

git add README.md

git commit -m "first commit"

git branch -M <BRANCH>   # e.g main 

git remote add origin git@github.com:<YOUR_GITHUB_REPOSITORY>.git

git remote add upstream https://github.com/Cosmo-Tech/azure-sample-webapp.git

git remote set-url upstream --push "NO"

git fetch --all --tags --prune

git checkout -B <BRANCH> <SOURCE_TAG>

rm -r .github/ .git-hooks

rm -r config.json   # ⚠️ Remove if exists

git add .

git commit -m "first commit"

git push origin <BRANCH> -f

Now that your web app GitHub repository is configured, you can use this webapp.yaml file to deploy a new static web app:

Webapp.yaml 
kind: WebApp
namespace:
  remote: true   # false by default
  state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a
  context: test
  platform:
    id: dev
    url: https://dev.api.cosmotech.com/phoenix/v3-0
metadata:
  workspace_key: "Project1"
spec:
  sidecars:
    github:
      organization_name: Cosmo-Tech
      repository_name: <YOUR_GITHUB_REPOSITORY>  # e.g azure-webapp-test-brewery-webapps
      branch: <BRANCH>  # e.g main
    powerbi:
      group_id: <changement>
      settings:
        properties:
          POWER_BI_SCOPE: "https://analysis.windows.net/powerbi/api/.default"
          POWER_BI_AUTHORITY_URI: https://login.microsoftonline.com/common/v2.0
          POWER_BI_WORKSPACE_ID: "{{services['powerbi.workspace.id']}}"
          POWER_BI_CLIENT_ID: "{{services['app.app_id']}}"
          POWER_BI_CLIENT_SECRET: "{{secret_powerbi}}"
          POWER_BI_TENANT_ID: "{{services['azure.tenant_id']}}"
    azure:
      app:
        create: false
        use:
          displayName: thisismyappforcontinuous
          client_id: "3d0531b1-d23b-4baf-98be-a764c0a42f00"
        principal_id: "{{services['app.principal_id']}}"
        add_to_powerbi: true
        payload:
          displayName: thisismyappforcontinuous
          signInAudience: AzureADMyOrg
          spa:
            redirectUris:
              - http://localhost:3000/sign-in
              - https://<custome_domain_of_your_static_web_app>/sign-in
          requiredResourceAccess:
            - resourceAppId: "{{services['platform.app_id']}}"
              resourceAccess:
                - id: "{{services['platform.scope_id']}}"
                  type: Scope
      function:
        url_zip: "https://github.com/Cosmo-Tech/supplychain-azure-function-dataset-download/releases/download/2.1.10/artifact.zip"
    config:
      REACT_APP_APPLICATION_INSIGHTS_INSTRUMENTATION_KEY: "{{services['webapp.insights_instrumentation_key']}}"
      REACT_APP_ENABLE_APPLICATION_INSIGHTS: "{{services['webapp.enable_insights']}}"
      REACT_APP_APP_REGISTRATION_CLIENT_ID: "{{services['app.app_id']}}"
      REACT_APP_AZURE_TENANT_ID: "{{services['azure.tenant_id']}}"
      REACT_APP_COSMOTECH_API_SCOPE: "{{services['api.scope']}}"
      REACT_APP_DEFAULT_BASE_PATH: "{{services['api.url']}}"
      REACT_APP_ORGANIZATION_ID: "{{services['api.organization_id']}}"
      REACT_APP_WORKSPACES_IDS_FILTER: ""
      REACT_APP_APP_VERSION: ""
      REACT_APP_ORGANIZATION_URL: "{{services['api.organization_url']}}"
      REACT_APP_DOCUMENTATION_URL: https://cosmotech.com
      REACT_APP_SUPPORT_URL: https://support.cosmotech.com
  payload:
    name: "my-webapp-for-continuous"
    location: westeurope
    properties:
      repositoryUrl: https://github.com/<YOUR_GITHUB_REPOSITORY> 
      branch: <BRANCH>  # e.g main
      repositoryToken: "{{github_secret}}"
      buildProperties:
        appLocation: "/"
        apiLocation: api
        appArtifactLocation: build
    sku:
      name: Standard
      tier: Standard

API Connector

Currently, to create a new connector, we use the API endpoint.
Below is an example of how you can create it using a YAML configuration file.

connector_azure_storage.yaml 
key: connector_azure_storage
name: Azure Storage Connector
description: Connector for Azure Storage. Read all data in a container with a prefix
  and write the data in CSV for a ScenarioRun
version: "1.1.2"
repository: cosmo-tech/azure-storage-simulator-connector
tags:
- Azure Storage
- Babylon
url: https://github.com/Cosmo-Tech/azure-storage-simulator-connector
ioTypes:
- read
azureAuthenticationWithCustomerAppRegistration:
azureManagedIdentity:
parameterGroups:
- id: parameters
  label: Parameters
  parameters:
  - id: AZURE_STORAGE_CONNECTION_STRING
    label: Azure Storage Connection String
    valueType: string
    default: "%STORAGE_CONNECTION_STRING%"
    envVar: AZURE_STORAGE_CONNECTION_STRING
  - id: AZURE_STORAGE_CONTAINER_BLOB_PREFIX
    label: Azure Storage Path in the form container/path
    valueType: string
    envVar: AZURE_STORAGE_CONTAINER_BLOB_PREFIX

API Dataset

To deploy one or multiple datasets, one yaml file is needed by dataset. Four sourceType of datasets are available:

  • ADT - creates dataset from ADT
  • AzureStorage - creates dataset with Azure Storage
  • File - creates dataset from a local file
  • None - creates an empty dataset
Dataset.yaml 
kind: Dataset
namespace:
  remote: true   # false by default
  state_id: 8db6069e-e05f-42e6-b6d6-56dde124516a
  context: test
  platform:
    id: dev
    url: https://dev.api.cosmotech.com/phoenix/v3-0
spec:
  sidecars:
    azure:
      dataset:
        storage:
          local_path: # needed if you want to upload your local dataset to AzureStorage
        file:
          local_path: # needed for datasets with sourceType File
  payload:
    id: # mandatory if you want to launch an update, without id a new dataset will be created ; if you want a new dataset, leave this field empty
    name: Apply dataset
    description: Creating dataset with nothing but update
    sourceType: None | AzureStorage | ADT | File
    source:
      path: # path to the folder in AzureStorage, mandatory if sourceType is AzureStorage
        # and no local file is provided
      location: # mandatory field if sourceType is ADT: path to dataset stored in ADT;
        # if sourceType is AzureStorage, default value is set to organization
        # container, you can edit this field if you want to use a dataset from
        # another container
      name: # field used for sourceType AzureStorage, by default is set to storage account
        # name referenced in state; edit it if you want to use a dataset from another
        # account
    security:
      default: viewer
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

Note current usage

  • To create a dataset, we currently use the API endpoint to register a new dataset.
  • Connector ID (must be also created via API see API Connector section)
API INPUT 
  name: mydataset
  description: Some description of your dataset
  tags:
    - dataset
    - File Storage
  connector:
    id: c-q5qv7xcsqv64
    parametersValues:
      AZURE_STORAGE_CONTAINER_BLOB_PREFIX: "%WORKSPACE_FILE%/default_parameters/<mydataset>/data.xlsx"
  security:
    default: editor
    accessControlList:
      - id: user1@email.com
        role: admin
      - id: user2@email.com
        role: editor
      - id: user3@email.com
        role: viewer

Babylon project structure

Project folder must have the following structure:

Tree

├── variables.yaml
├── project
│   ├── adx
│   │   └── scripts
│   │       └── Create.kql
│   ├── connector_azure_storage.yaml
│   ├── organization.yaml
│   ├── powerbi
│   │   ├── report1.pbix
│   │   ├── report2.pbix
│   │   └── report3.pbix
│   ├── solution.yaml
│   ├── webapp.yaml
│   └── workspace.yaml
└── README.md

Launching the deployment Macro command

After filling all deployment files, you can launch the following command:

Babylon Macro apply

babylon apply project/

Babylon will create and deploy all resources and save it in the state except for datasets.
Keeping this information in the state simplifies modification of the resources as you can edit one of the project deployment files and relaunch babylon apply command. It will update existing resources or create missing ones, for example, in case when Babylon was granted more rights between two apply commands.

You can also specify different variable files when launching the babylon apply command.

To do this, use the --var-file option.

Example

babylon apply project/ --var-file variable_file_1.yaml --var-file variable_file_2.yaml

Remember

If you don't specify a variable file, Babylon will use the default variable file variables.yaml

Executing babylon apply on a Single Object API

Babylon now supports deploying or updating a single object API using the apply macro command.
This enhancement makes it easier to maintain and update only the specific object that has changed, instead of running the macro for all objects following best practices for efficient deployments.

Command syntax

> babylon apply --OBJECT --var-file variable_file_1.yaml project/
As objects, Babylon accepts the following:

--organization       Deploy or update an organization.
--solution           Deploy or update a solution. 
--workspace          Deploy or update a workspace.
--webapp             Deploy or update a webapp.
--dataset            Deploy or update a dataset.

Executing babylon with --payload-only option

The --payload-only option allows Babylon to deploy or update an object API without triggering the deployment of associated Azure services defined in the deployment file.
This is particularly useful for incremental updates where you only need to modify lightweight parts, such as adding permissions in the Object ACL or updating the payload. Fully redeploying these parts can be time consuming.

However, this option should not be used when there are new Power BI reports, ADX database permissions, or other changes related to azure section, as those require a full deployment.

💡 Best practice for using --payload-only

Use this option only when you have changes limited to the payload itself, avoiding unnecessary redeployment of Azure section.

> babylon apply --OBJECT --payload-only --var-file variable_file_1.yaml project/
As objects now, Babylon accepts the following:

--solution           Deploy or update a solution pyload only. 
--workspace          Deploy or update a workspace pyload only.