Deploy Cosmo Tech workspace¶

Remember

This guide focuses on explaining each object individually.
A complete deployment workflow is provided in the Examples section, where all components are combined in a practical scenario !

Before proceeding, ensure that you have selected the correct tenant.

In this example, we will use the following identifiers:

context_id: test
tenant_id: dev
state_id: 8db6069e

To deploy a complete Cosmo Tech workspace, you can declare its configuration in yaml files corresponding to specific deployment type.

Each file contains general information about the deployment:

Important

In this tutorial each Cosmo Tech object, values defined directly within the corresponding YAML file.
However, in the Examples section, you will see that we use a centralized variables.yaml file, which defines all required variables and is then referenced across the YAML configurations !

API Organization¶

kind: Organization
namespace:
  remote: true   # false by default

The deployment configuration must define the following keys:

kind
Specifies the type of resource to deploy.
Accepted values: Organization, Solution, Workspace, WebApp.

⚠️ The resource type must always start with a capital letter.
namespace
Provides metadata that uniquely identifies the deployment, including for now just:
- remote
  A boolean flag indicating whether the state should be stored locally only (false, default) or both locally and in the cloud (true)
spec
Defines the resource configuration. The details are specified under the payload section.

For example, in an Organization deployment file:

Organization.yaml

kind: Organization
namespace:
  remote: true   # false by default
spec:
  payload:
    name: "{{organization_name}}"
    security: {{security}}

All keys in this file can be templated with {{}} syntax for objects and "{{}}" for strings, as security section of this file. Corresponding values must be stored in variables.yaml file at the same level that your project See 👉 Examples:

security:
  default: viewer
  accessControlList:
    - id: user1@email.com
      role: admin
    - id: user2@email.com
      role: editor
    - id: user3@email.com
      role: viewer

API Solution¶

This is how the solution deployment file is structured

Solution.yaml

kind: Solution
namespace:
  remote: true   # false by default
metadata:
  selector:
    organization_id: "{{services['api.organization_id']}}"
spec:
  payload:
    key: "demosolution"
    name: "My Solution Name"
    description: "My solution description"
    repository: brewery_for_continuous
    version: latest
    sdkVersion: '10.4.0'
    alwaysPull: true
    url: 'https://webapp.com'
    tags:
      - brewery
    runTemplates:
      - id: "run_id"
        name: "Standard simulation"
        csmSimulation: AzureWebApp/AzureWebApp_Simulation
        run: true
        preRun: true
    parameters:
    parameterGroups:
    security:
      default: none
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

The metadata section defines deployment specific attributes. It is required in the following resource files:

solution.yaml
workspace.yaml

The selector block is used to specify the target organization under which the resource will be deployed.

Parameter: organization_id
- Purpose:
  Defines the unique organization in which the solution (or other resource) will be created.
Behavior:
- Babylon will associate the deployment (e.g., a solution, workspace) with the specified organization_id.
- Typically, this value is injected dynamically from the Babylon state (services['api.organization_id']).

API Workspace¶

The Workspace configuration may include additional parameters required to provision external services.

These parameters are normally defined under the sidecars section.

One of the major changes is the replacement of the ADX database with PostgreSQL by default. Babylon now automatically creates a dedicated PostgreSQL schema for each workspace, which can be used to store runner results. As a result, the sidecars section now references postgres instead of ADX.

PostgreSQL Schema Creation with Kubernetes Job¶

With the new workspace configuration, Babylon leverages a Kubernetes job to automate the creation of a PostgreSQL schema for each workspace. This is defined under the sidecars.postgres.schema.jobs section:

When create is set to true, Babylon will execute the specified Kubernetes job (k8s_job.yaml) located in the postgres/jobs directory. This job is responsible for initializing the PostgreSQL schema required by the workspace. This approach ensures that each workspace has its own isolated schema, improving data management and security.

Workspace.yaml

kind: Workspace
namespace:
  remote: true   # false by default
spec:
  sidecars:
    postgres:
      schema:
        create: true # false by default
        jobs:
          - name: k8s_job.yaml
            path: postgres/jobs
  payload:
    key: "Project1"
    name: "My Workspace Name"
    description: "Workspace for solution"
    solution:
      solutionId: "{{services['api.solution_id']}}"
    useDedicatedEventHubNamespace: true
    sendScenarioMetadataToEventHub: true
    sendInputToDataWarehouse: true
    sendScenarioRunToEventHub: true
    additionalData:
      webapp:
        solution:
          runTemplateFilter:
          defaultRunTemplateDataset: null
        charts:
          workspaceId:
          logInWithUserCredentials: false
          scenarioViewIframeDisplayRatio: 4.514285714
          dashboardsViewIframeDisplayRatio: 1.610062893
          useWebappTheme: true
          dashboardsView:
          scenarioView:
        menu:
          supportUrl: 'https://support.cosmotech.com'
          organizationUrl: 'https://cosmotech.com'
          documentationUrl: 'https://portal.cosmotech.com/resources/platform-resources/web-app-user-guide'
        datasetManager: 'removeToDisableDatasetManager'
        datasourceFilter: []
    security:
      default: none
      accessControlList:
        - id: user1@email.com
          role: admin
        - id: user2@email.com
          role: editor
        - id: user3@email.com
          role: viewer

Cosmo Tech Web App¶

To deploy a Cosmo Tech Web App now is possible with Babylon.

Babylon now supports deploying web applications using Terraform as backend. This integration allows you to provision and manage web app infrastructure as code directly through Babylon, streamlining the deployment process.

For more details on how to deploy the web app, refer to the README file in the Terraform web app module: 👉 Terraform Module.

Babylon automatically clones the Terraform web app repository and updates the terraform.tfvars file with all required variables. These variables are retrieved from the webapp.yaml configuration file below.

webapp.yaml

kind: Webapp
namespace:
   remote: true
spec:
   payload:
      cloud_provider: "azure"
      cluster_name: "{{cluster_name}}"
      cluster_domain: "{{cluster_domain}}"
      tenant: "{{tenant}}"
      webapp_name: "{{webapp_name}}"
      organization_id: "{{services['api.organization_id']}}"
      azure_subscription_id: "{{azure_subscription_id}}"
      azure_entra_tenant_id: "{{azure_entra_tenant_id}}"
      powerbi_app_deploy: false

Babylon project structure¶

Project folder must have the following structure:

Tree

.
├── babylon.log
├── project
│   ├── Organization.yaml
│   ├── postgres
│   │   └── jobs
│   │       └── k8s_job.yaml
│   ├── Solution.yaml
│   ├── Webapp.yaml
│   └── Workspace.yaml
├── terraform-webapp
└── variables.yaml

Launching the deployment Macro command¶

After filling all deployment files, you can launch the following command:

Babylon Macro apply

babylon apply project/

Babylon will create and deploy all resources and save it in the state.
Keeping this information in the state simplifies modification of the resources as you can edit one of the project deployment files and relaunch babylon apply command. It will update existing resources or create missing ones, for example, in case when Babylon was granted more rights between two apply commands.

You can also specify different variable files when launching the babylon apply command.

To do this, use the --var-file option.

Example

babylon apply --var-file variable_file_1.yaml --var-file variable_file_2.yaml project/

Remember

If you don't specify a variable file, Babylon will use the default variable file variables.yaml

Executing babylon apply on a Single Object API¶

Babylon now supports deploying or updating a single object API using the apply macro command.
This enhancement makes it easier to maintain and update only the specific object that has changed, instead of running the macro for all objects following best practices for efficient deployments.

To support this, Babylon introduces two new options: --include --execlude

Option	Description
`--include`	Specifies only the objects to be included in the action.
`--exclude`	Specifies the objects to be skipped during the action.

include

babylon apply --include organization project

🚀 Deploying Organization in namespace: dev
    → Loading configuration from Kubernetes secret...
    → No existing organization ID found. Creating...
    ✔ Organization o-nl8pgn5lpwd created

📋 Deployment Summary
    • Organization Id : o-nl8pgn5lpwd

✨ Deployment process complete

exclude

babylon apply --exclude workspace project/

🚀 Deploying Organization in namespace: dev
    → Loading configuration from Kubernetes secret...
    → No existing organization ID found. Creating...
    ✔ Organization o-nl8pgn5lpwd created

🚀 Deploying Solution in namespace: dev
    → Loading configuration from Kubernetes secret...
    → No existing solution ID found. Creating...
    ✔ Solution sol-6069lgex2xz created

📋 Deployment Summary
    • Organization Id : o-nl8pgn5lpwd
    • Solution Id     : sol-6069lgex2xz

✨ Deployment process complete