Skip to content

load-csv-file

Help command

> csm-data custom load-csv-file --help

 Usage: csm-data custom load-csv-file [OPTIONS]                                 

 Stream one CSV file into an existing PostgreSQL table with COPY FROM STDIN     
 PostgreSQL parses the CSV and converts the values to the column types declared 
 on the table: the table must already exist and the CSV header must only        
 contain column names of the table (exact match, case-sensitive).               

 The file is loaded in a single transaction: if anything fails, nothing is      
 written.                                                                       

 Connection settings: either give all of --postgres-host, --postgres-db,        
 --postgres-schema and --postgres-user (with the password), or give none of     
 them to use the CoAL configuration file (CONFIG_FILE_PATH, or                  
 /mnt/coal/coal-config.toml in the cluster), then the POSTGRES_* environment    
 variables.                                                                     

 With --add-id, the transaction-local setting csm.run_id is set from --run-id,  
 so that a table column csm_run_id text NOT NULL DEFAULT                        
 current_setting('csm.run_id') is filled for every row.                         

 The postgresql user must have USAGE granted on the schema and INSERT on the    
 table.                                                                         

╭─ OPTIONS ────────────────────────────────────────────────────────────────────╮
│ *  --csv-file                        PATH     Path of the CSV file to load   │
│                                               [required]                     │
│    --table                           NAME     Target table name, used as is. │
│                                               Default is table prefix + file │
│                                               name without extension         │
│    --table-prefix                    PREFIX   Prefix added to the file name  │
│                                               to get the table name,         │
│                                               lowercased (default            │
│                                               Cosmotech_). Not usable with   │
│                                               --table                        │
│    --add-id                                   Fill the csm_run_id column of  │
│                                               the table with the run id      │
│    --run-id                          TEXT     Run id used with --add-id      │
│                                               ENV: CSM_RUN_ID                │
│    --postgres-host                   TEXT     PostgreSQL host URI            │
│                                               ENV: POSTGRES_HOST_URI         │
│    --postgres-port                   INTEGER  PostgreSQL database port       │
│                                               ENV: POSTGRES_HOST_PORT        │
│    --postgres-db                     TEXT     PostgreSQL database name       │
│                                               ENV: POSTGRES_DB_NAME          │
│    --postgres-schema                 TEXT     PostgreSQL schema name         │
│                                               ENV: POSTGRES_DB_SCHEMA        │
│    --postgres-user                   TEXT     PostgreSQL connection user     │
│                                               name                           │
│                                               ENV: POSTGRES_USER_NAME        │
│    --postgres-password               TEXT     PostgreSQL connection password │
│                                               ENV: POSTGRES_USER_PASSWORD    │
│    --encode-password/--no-encode…             Force encoding of password to  │
│                                               percent encoding               │
│                                               ENV:                           │
│                                               CSM_PSQL_FORCE_PASSWORD_ENCOD… │
│                                               DEFAULT: encode-password       │
│    --help                                     Show this message and exit.    │
╰──────────────────────────────────────────────────────────────────────────────╯

Optional dependency

This command needs psycopg: pip install "cosmotech-acceleration-library[custom]".