load-csv-file¶
Help command
> csm-data custom load-csv-file --help
Usage: csm-data custom load-csv-file [OPTIONS]
Stream one CSV file into an existing PostgreSQL table with COPY FROM STDIN
PostgreSQL parses the CSV and converts the values to the column types declared
on the table: the table must already exist and the CSV header must only
contain column names of the table (exact match, case-sensitive).
The file is loaded in a single transaction: if anything fails, nothing is
written.
Connection settings: either give all of --postgres-host, --postgres-db,
--postgres-schema and --postgres-user (with the password), or give none of
them to use the CoAL configuration file (CONFIG_FILE_PATH, or
/mnt/coal/coal-config.toml in the cluster), then the POSTGRES_* environment
variables.
With --add-id, the transaction-local setting csm.run_id is set from --run-id,
so that a table column csm_run_id text NOT NULL DEFAULT
current_setting('csm.run_id') is filled for every row.
The postgresql user must have USAGE granted on the schema and INSERT on the
table.
╭─ OPTIONS ────────────────────────────────────────────────────────────────────╮
│ * --csv-file PATH Path of the CSV file to load │
│ [required] │
│ --table NAME Target table name, used as is. │
│ Default is table prefix + file │
│ name without extension │
│ --table-prefix PREFIX Prefix added to the file name │
│ to get the table name, │
│ lowercased (default │
│ Cosmotech_). Not usable with │
│ --table │
│ --add-id Fill the csm_run_id column of │
│ the table with the run id │
│ --run-id TEXT Run id used with --add-id │
│ ENV: CSM_RUN_ID │
│ --postgres-host TEXT PostgreSQL host URI │
│ ENV: POSTGRES_HOST_URI │
│ --postgres-port INTEGER PostgreSQL database port │
│ ENV: POSTGRES_HOST_PORT │
│ --postgres-db TEXT PostgreSQL database name │
│ ENV: POSTGRES_DB_NAME │
│ --postgres-schema TEXT PostgreSQL schema name │
│ ENV: POSTGRES_DB_SCHEMA │
│ --postgres-user TEXT PostgreSQL connection user │
│ name │
│ ENV: POSTGRES_USER_NAME │
│ --postgres-password TEXT PostgreSQL connection password │
│ ENV: POSTGRES_USER_PASSWORD │
│ --encode-password/--no-encode… Force encoding of password to │
│ percent encoding │
│ ENV: │
│ CSM_PSQL_FORCE_PASSWORD_ENCOD… │
│ DEFAULT: encode-password │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
Optional dependency
This command needs psycopg: pip install "cosmotech-acceleration-library[custom]".