dump-to-s3¶
Help command
> csm-data store dump-to-s3 --help
Usage: csm-data store dump-to-s3 [OPTIONS]
Dump a datastore to a S3
Will upload everything from a given data store to a S3 bucket.
3 modes currently exists:
• sqlite: will dump the data store underlying database as is
• csv: will convert every table of the datastore to csv and send them as separate files
• parquet: will convert every table of the datastore to parquet and send them as separate files
Giving a prefix will add it to every upload (finishing the prefix with a "/" will allow to upload in a folder inside the bucket)
Make use of the boto3 library to access the bucket
More information is available on this page: [https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html]
╭─ OPTIONS ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * --store-folder PATH The folder containing the store files │
│ ENV: CSM_PARAMETERS_ABSOLUTE_PATH │
│ [required] │
│ --output-type [sqlite|csv|parquet] Choose the type of file output to use (sqlite, csv, parquet) │
│ * --bucket-name BUCKET The bucket on S3 to upload to │
│ ENV: CSM_DATA_BUCKET_NAME │
│ [required] │
│ --prefix PREFIX A prefix by which all uploaded files should start with in the bucket │
│ ENV: CSM_DATA_BUCKET_PREFIX │
│ --use-ssl/--no-ssl Use SSL to secure connection to S3 │
│ * --s3-url URL URL to connect to the S3 system │
│ ENV: AWS_ENDPOINT_URL │
│ [required] │
│ * --access-id ID Identity used to connect to the S3 system │
│ ENV: AWS_ACCESS_KEY_ID │
│ [required] │
│ * --secret-key ID Secret tied to the ID used to connect to the S3 system │
│ ENV: AWS_SECRET_ACCESS_KEY │
│ [required] │
│ --ssl-cert-bundle PATH Path to an alternate CA Bundle to validate SSL connections │
│ ENV: CSM_S3_CA_BUNDLE │
│ --web-help Open the web documentation │
│ --help Show this message and exit. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯