Deploy Cosmo Tech workspace¶

Remember

This guide focuses on explaining each object individually.
A complete deployment workflow is provided in the Examples section, where all components are combined in a practical scenario !

Before proceeding, ensure that you have selected the correct tenant.

In this example, we will use the following identifiers:

context_id: test
tenant_id: dev
state_id: 8db6069e

To deploy a complete Cosmo Tech workspace, you can declare its configuration in yaml files corresponding to specific deployment type.

Each file contains general information about the deployment:

Important

In this tutorial each Cosmo Tech object, values defined directly within the corresponding YAML file.
However, in the Examples section, you will see that we use a centralized variables.yaml file, which defines all required variables and is then referenced across the YAML configurations !

API Organization¶

kind: Organization
namespace:
  remote: true   # false by default

The deployment configuration must define the following keys:

kind
Specifies the type of resource to deploy.
Accepted values: Organization, Solution, Workspace, WebApp.

⚠️ The resource type must always start with a capital letter.
namespace
Provides metadata that uniquely identifies the deployment, including for now just:
- remote
  A boolean flag indicating whether the state should be stored locally only (false, default) or both locally and in the cloud (true)
spec
Defines the resource configuration. The details are specified under the payload section.

For example, in an Organization deployment file:

Organization.yaml

kind: Organization
namespace:
  remote: true   # false by default
spec:
  payload:
    name: "{{organization_name}}"
    security: {{security}}

All keys in this file can be templated with {{}} syntax for objects and "{{}}" for strings, as security section of this file. Corresponding values must be stored in variables.yaml file at the same level that your project See 👉 Examples:

security:
  default: viewer
  accessControlList:
    - id: user1@email.com
      role: admin
    - id: user2@email.com
      role: editor
    - id: user3@email.com
      role: viewer

API Solution¶

This is how the solution deployment file is structured

Solution.yaml

kind: Solution
namespace:
  remote: true   # false by default
metadata:
  selector:
    organization_id: "{{services['api.organization_id']}}"
spec:
  payload:
    key: "demosolution"
    name: "My Solution Name"
    description: "My solution description"
    repository: brewery_for_continuous
    version: latest
    sdkVersion: '10.4.0'
    alwaysPull: true
    url: 'https://webapp.com'
    tags:
      - brewery
    runTemplates:
      - id: "run_id"
        name: "Standard simulation"
        csmSimulation: AzureWebApp/AzureWebApp_Simulation
        run: true
        preRun: true
    parameters:
    parameterGroups:
    security:
      default: none
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

The metadata section defines deployment specific attributes. It is required in the following resource files:

solution.yaml
workspace.yaml

The selector block is used to specify the target organization under which the resource will be deployed.

Parameter: organization_id
- Purpose:
  Defines the unique organization in which the solution (or other resource) will be created.
Behavior:
- Babylon will associate the deployment (e.g., a solution, workspace) with the specified organization_id.
- Typically, this value is injected dynamically from the Babylon state (services['api.organization_id']).

API Workspace¶

The Workspace configuration may include additional parameters required to provision external services such as:

Power BI workspaces
Azure Event Hubs
Azure Data Explorer (ADX) databases

These parameters are normally defined under the sidecars section, specifically within the cloud key.

However, this is not the case in the current release, because Cosmotech API v5 does not depend on any external cloud resources for the moment.

Workspace.yaml

kind: Workspace
namespace:
  remote: true   # false by default
metadata:
  selector:
    organization_id: "{{services['api.organization_id']}}"
    solution_id: "{{services['api.solution_id']}}"
spec:
  sidecars:
    cloud:
  payload:
    key: "Project1"
    name: "My Workspace Name"
    description: "Workspace for solution"
    solution:
      solutionId: "{{services['api.solution_id']}}"
    useDedicatedEventHubNamespace: true
    sendScenarioMetadataToEventHub: true
    sendInputToDataWarehouse: true
    sendScenarioRunToEventHub: true
    additionalData:
      webApp:
        solution:
          runTemplateFilter:
          defaultRunTemplateDataset: null
        charts:
          workspaceId:
          logInWithUserCredentials: false
          scenarioViewIframeDisplayRatio: 4.514285714
          dashboardsViewIframeDisplayRatio: 1.610062893
          useWebappTheme: true
          dashboardsView:
          scenarioView:
        menu:
          supportUrl: 'https://support.cosmotech.com'
          organizationUrl: 'https://cosmotech.com'
          documentationUrl: 'https://portal.cosmotech.com/resources/platform-resources/web-app-user-guide'
        datasetManager:
    security:
      default: none
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

API Dataset¶

You can include any parts you need from your CSV files, but make sure all CSV files are located in the root of the deployment directory.

Dataset.yaml

kind: Dataset
namespace:
  remote: true   # false by default
spec:
  sidecars:
  payload:
    id: # mandatory if you want to launch an update, without id a new dataset will be created ; if you want a new dataset, leave this field empty
    name: Apply dataset
    description: Creating dataset with nothing but update
    tags:
      - brewery
    parts:
      - name: part1
        sourceName: customers.csv
    security:
      default: viewer
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

API Runner¶

This is how the runner deployment file is structured

Runner.yaml

kind: Runner
namespace:
  remote: true   # false by default
spec:
  sidecars:
  payload:
    id: # mandatory if you want to launch an update, without id a new dataset will be created ; if you want a new dataset, leave this field empty
    name: "{{runner_name}}"
    runTemplateId: "{{run_template_id}}"
    solutionId: "{{services['api.solution_id']}}"
    solutionName: "{{solution_name}}"
    runTemplateName: "{{runTemplate_name}}"
    ownerName: "{{owner_name}}"
    security:
      default: viewer
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

Cosmo Tech Web App¶

To deploy a Cosmo Tech Web App, you can:

Using the Terraform module
For the moment, the web app is not managed by Babylon. It is deployed using a Terraform module, but this will be supported in a future release.

If you need to deploy the web app, refer to the README file in the Terraform web app module see 👉 Terraform.

Babylon project structure¶

Project folder must have the following structure:

Tree

├── variables.yaml
├── project
│   ├── Organization.yaml
│   ├── Solution.yaml
│   ├── Workspace.yaml   
│   ├── Runner.yaml
│   ├── Dataset.yaml
│   └── Workspace.yaml
└── README.md

Launching the deployment Macro command¶

After filling all deployment files, you can launch the following command:

Babylon Macro apply

babylon apply project/

Babylon will create and deploy all resources and save it in the state except for datasets and runner.
Keeping this information in the state simplifies modification of the resources as you can edit one of the project deployment files and relaunch babylon apply command. It will update existing resources or create missing ones, for example, in case when Babylon was granted more rights between two apply commands.

You can also specify different variable files when launching the babylon apply command.

To do this, use the --var-file option.

Example

babylon apply project/ --var-file variable_file_1.yaml --var-file variable_file_2.yaml

Remember

If you don't specify a variable file, Babylon will use the default variable file variables.yaml

Executing babylon apply on a Single Object API¶

Babylon now supports deploying or updating a single object API using the apply macro command.
This enhancement makes it easier to maintain and update only the specific object that has changed, instead of running the macro for all objects following best practices for efficient deployments.

Command syntax

> babylon apply --OBJECT --var-file variable_file_1.yaml project/

As objects, Babylon accepts the following:

--organization       Deploy or update an organization.
--solution           Deploy or update a solution.
--workspace          Deploy or update a workspace.
--dataset            Deploy or update a dataset.
--runner             Deploy or update a runner.

Executing babylon with `--payload-only` option¶

The --payload-only option allows Babylon to deploy or update an object API without triggering the deployment of associated Azure services defined in the deployment file.
This is particularly useful for incremental updates where you only need to modify lightweight parts, such as adding permissions in the Object ACL or updating the payload. Fully redeploying these parts can be time consuming.

However, this option should not be used when there are new Power BI reports, ADX database permissions, or other changes related to azure section, as those require a full deployment.

💡 Best practice for using --payload-only

Use this option only when you have changes limited to the payload itself, avoiding unnecessary redeployment of Azure section.

> babylon apply --OBJECT --payload-only --var-file variable_file_1.yaml project/

As objects now, Babylon accepts the following:

--workspace          Deploy or update a workspace pyload only.