Getting started with Babylon¶

This is a guide to getting started with Babylon. You'll learn how to install, run, and experiment with the Babylon.

Azure requirements¶

Requirements

The following elements are required before you can use Babylon on an Azure subscription

Fully set up Cosmo Tech Platform installed on your Azure Tenant
App registration for Babylon

Babylon uses a Service Principal to interact with the API and other resources. To do so, Babylon needs to be registered as an application in the Azure Active Directory.

This App Registration will be used to authenticate and get the required tokens to connect to your Platform. Rights on the App Registration depends on the type of connection you want to make.

We recommend defining this app registration as a mobile and desktop application in Azure configuration with redirection URL (mostly for authenticating against Azure services.

See official documentation :

* https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-auth-code-flow
* https://learn.microsoft.com/en-us/entra/identity-platform/msal-client-application-configuration
* https://learn.microsoft.com/en-us/onedrive/developer/rest-api/getting-started/msa-oauth?view=odsp-graph-online

The App registration needs the following API permissions:

API Permissions

Microsoft Graph
🔹 Application.ReadWrite.All (Application)
🔹 Application.ReadWrite.OwnedBy (Application)
🔹 Group.ReadWrite.All (Delegated)
🔹 User.Read (Delegated)

Platform API
🔹 Platform.Admin (Application)

For those concerned with specifics Azure policies, all Microsoft Graph's permissions can be deleted excepted User.Read (Delegated), but this implies that some operations would be handled manually. The platform API permission is mandatory.

Important

Babylon App registration needs at least Contributor role on Cosmo Tech Platform Resource Group.

Get Babylon from source¶

Requirements

🐍 Python 3.10+
📦 Pip 22.2+
🌱 Git (latest version)

We will go through the process of getting a version of Babylon before the installation.

Get sources from git

You can download the latest version of Babylon using git clone command

Latest Version

git clone git@github.com:Cosmo-Tech/Babylon.git babylon
cd babylon

Get the specific version from git

If you want a specific version of Babylon (e.g. 3.0.0)

Specific Version

git clone -b 3.0.0 git@github.com:Cosmo-Tech/Babylon.git babylon
cd babylon

Install Babylon¶

You can install babylon globally in your system

Example

python3 -m venv ~/babylonenv
source ~/babylonenv/bin/activate
pip install .

Now that you have a fully functional installation of Babylon, you can check the next steps to learn how to start running commands.

Setup Babylon¶

Babylon use a Vault service and provides a group of commands that can be used.

The first thing to do in order to check if Babylon is working properly:

babylon --help

Setup Environment Variables¶

The Vault service is required some environment variables before interacting with Babylon CLI To do so, Contact your Babylon admin to get service URI and token.

Important

You can retrieve those values directly from the cluster
See 👉 Retrieve Vault URL and Token.

When you retrieve these values, you need to set the following environment variables

Example

export BABYLON_ORG_NAME="vault_org_name" # cosmotech
export BABYLON_TOKEN="vault_root_token"  # hvs.CLxxxxxxxxxxxxxxx
export BABYLON_SERVICE="vault_url"       # https://example.api.cosmotech.com

Configuration¶

At this point, you need three variables to perform Babylon commands.

Variable	Description	Example
`context_id`	Project name (string of your choice, no special characters)	`project1`
`platform_id`	Platform ID (e.g., dev, staging, prod)	`dev`
`state_id`	State name (string of your choice, no special characters)	`state1`

⚠️ Variable Constraints

context_id and state_id can be any string of your choice, but they must not contain special characters.

If you are on Linux, you can generate a new UUID with uuidgen:

sudo apt update
sudo apt install uuid-runtime -y
# run uuidgen command 
uuidgen
# Example output:
0475231d-3c7c-4505-af38-558041240d3d

platform_id represents the platform identifier (e.g., dev, staging, ...).
⚠️ It must be the same platform_id you used when initializing the Vault configuration.

Example Vault config command you can See 👉 Usage :

python main.py config write --resource all --use-azure --engine v1 --platform-id dev

To initialize Babylon with these values, run:

babylon namespace use -c <context_id> -p <platform_id> -s <state_id>

Generated Configuration

This command creates a file called namespace.yaml
Location: ~/.config/cosmotech/babylon/namespace.yaml
It contains the current context for your deployment

You can now test Babylon by running a simple command:

babylon api organizations get-all

If you have already created an organization, you should see its details in the output.

Example List Organizations

> babylon api organizations get-all

[
  {
    "id": "o-55dsz6e51n8y8j",
    "name": "Organization name",
    "ownerId": "e0035649-0f12-4221-9631-3519704816c1",
    "services": null,
    "security": {
      "default": "none",
      "accessControlList": [
        {
          "id": "example.test@cosmotech.com",
          "role": "admin"
        }
      ]
    }
  }
]

The initial configuration is fetched from Vault and then persisted in two different backends to ensure consistency and recovery. This persisted configuration is referred to as the Babylon state.

Where Babylon state is stored

The state is saved in two places:

Remote backend (Azure Storage Blob) → babylon-states
Local backend (YAML file) → ~/.config/cosmotech/babylon/state.<context_id>.<platform_id>.<state_id>.yaml

State file specification¶

If the state file does not exist, the first Babylon command you run will initialize a new state and persist it in Azure Storage and local file.

Note

To enable persistence in Azure Storage, you must set the parameter remote: true.
(This will be explained in detail in the Deploy Workspace tutorial just keep it in mind for now)

Babylon State Structure¶

The Babylon state is a structured YAML file composed of multiple sections.
At a high level, you will find three main entries:

context:
files: []
id: 25075b92-fe8e-4952-9e9b-53360dacf369
platform:

The last one contains following keys with some information already prefilled with data from vault:

Azure Container Registry

services:
  acr:
    login_server: 
    simulator_repository: 
    simulator_version:

Azure Digital Twins

  adt:
    built_owner_id: 
    built_reader_id: 
    digital_twin_url:

Azure Explorer Database

  adx:
    built_contributor_id: 
    built_owner_id: 
    cluster_name: 
    cluster_principal_id: 
    cluster_uri: 
    database_name:

Cosmotech API

  api:
    connector.adt_id: 
    connector.adt_version: 
    connector.storage_id: 
    connector.storage_version: 
    connector.twin_id: 
    connector.twin_version: 
    dataset.adt_id: 
    dataset.storage_id: 
    dataset.twin_id: 
    organization_id: 
    organization_url: 
    run_templates: 
    scope: 
    send_scenario_metadata_to_event_hub: 
    solution_id: 
    url: 
    use_dedicated_event_hub_namespace: 
    workspace_id: 
    workspace_key:

Azure App Registration

  app:
    app_id: 
    name: 
    object_id: 
    principal_id:

Azure

    azure:
      cli_client_id:
      email: 
      eventhub_built_contributor_id:
      eventhub_built_data_receiver:
      eventhub_built_data_sender:
      function_artifact_url: 
      resource_group_name: 
      resource_location: 
      storage_account_name: 
      storage_blob_reader: 
      subscription_id:
      team_id:
      user_principal_id:

Azure Babylon App Registration

    babylon:
      client_id: 
      principal_id:

Github

    github:
      branch: 
      organization: 
      repository: 
      run_url: 
      workflow_path:

Cosmotech Platform

    platform:
      app_id: 
      principal_id: 
      scope_id:

PowerBI

    powerbi:
      dashboard_view: 
      scenario_view: 
      scope: 
      workspace.id:
      workspace.name:

WebApp

    webapp:
      deployment_name: 
      enable_insights: 
      hostname: 
      insights_instrumentation_key: 
      location: 
      static_domain: