Skip to content

Superset

Embed and use Apache Superset dashboards in your web application to visualize and analyze simulation data stored in your database.

Apache Superset is an open-source data exploration and visualization platform that enables you to create interactive dashboards and charts from your data. When integrated with your database, Superset provides a flexible and powerful solution for understanding and representing large volumes of simulation data generated by the CosmoTech platform.

Why use Superset?

Superset offers several advantages for data visualization:

  • Open Source: free to use with an active community and regular updates.
  • Database Agnostic: connect to PostgreSQL, MySQL, SQLite, and many other databases.
  • Interactive Dashboards: create dynamic, interactive reports with filtering and drill-down capabilities.
  • SQL-based: write SQL queries directly to explore and transform your data.
  • Custom Visualizations: build tailored charts, graphs, and KPIs specific to your simulation scenarios.
  • Embedded Dashboards: seamlessly integrate Superset dashboards directly into your web application.

Common data tables

The CosmoTech platform typically stores the following data in your database:

  • ProbesMeasures: time-series data from simulation probes and measurements.
  • ScenarioMetadata: information about scenarios (name, description, parameters).
  • ScenarioRun: execution details and results of scenario runs.
  • ScenarioRunMetadata: metadata associated with each scenario run.
  • SimulationTotalFacts: aggregated facts and metrics from simulations.

Creating visualizations

Basic steps

  1. Connect to Database: configure your database connection in Superset.
  2. Create Dataset: register your database tables as Superset datasets.
  3. Build Charts: create visualizations using the chart builder or SQL Lab.
  4. Assemble Dashboard: combine multiple charts into a cohesive dashboard.
  5. Add Interactivity: configure filters, cross-filtering, and drill-down actions.
  6. Publish: save and share your dashboard for embedding.

Several different visualizations are available, for example:

  • Line Charts: display time-series trends from ProbesMeasures.
  • Bar Charts: compare metrics across different scenarios.
  • Big Number: show key performance indicators at a glance.
  • Tables: present detailed scenario parameters and results.
  • Pivot Tables: analyze multidimensional data with flexible grouping.
  • Area Charts: visualize cumulative trends over time.

Using SQL in Superset

You can write SQL queries directly in Superset's SQL Lab to:

  • Explore and filter data before creating visualizations.
  • Perform complex calculations and transformations.
  • Join multiple tables for comprehensive analysis.
  • Create virtual datasets from custom queries.

Example SQL query:

SELECT 
    DATE_TRUNC('hour', timestamp) AS hour,
    probe_name,
    AVG(value) AS avg_value
FROM probes_measures
WHERE simulation_run = 'run-123'
GROUP BY DATE_TRUNC('hour', timestamp), probe_name
ORDER BY hour ASC;

Embedding dashboards in the web application

Superset dashboards can be embedded directly into the CosmoTech web application:

  1. Create Dashboard: design your dashboard with relevant charts and filters.
  2. Enable Embedding: configure the dashboard for embedded use in Superset settings.
  3. Configure Workspace: set up the Superset connection in your CosmoTech configuration.
  4. Define Dashboard Settings: specify display options and filter configurations.
  5. Apply Dynamic Filters: link dashboard filters to scenario data for context-aware visualizations.

The workspace configuration includes settings for:

  • Dashboard ID and display name.
  • Filter visibility and default values.
  • Refresh intervals.
  • Dynamic filters based on scenario context.

Best practices

  • Optimize Queries: use aggregations and filters to reduce data volume.
  • Cache Results: configure caching for frequently accessed dashboards.
  • Limit Chart Complexity: keep dashboards focused with a reasonable number of visualizations.
  • Use Dashboard Filters: create global filters that apply across multiple charts.
  • Test Performance: ensure dashboards load quickly even with large datasets.
  • Document Charts: add descriptions to charts and datasets for clarity.

Dynamic filtering

Superset dashboards in the CosmoTech web application support dynamic filtering based on the current scenario context. This allows users to:

  • Automatically view data relevant to the selected scenario.
  • Compare results across different scenario runs.
  • Filter by specific parameters or time ranges.

Dynamic filters are configured in the workspace settings and can link Superset dashboard filters to scenario metadata stored in the database.

Troubleshooting

Common issues

  • Connection Errors: verify database host, port, and credentials are correct.
  • Authentication Failures: ensure proper database permissions are configured.
  • Slow Performance: optimize SQL queries and consider adding database indexes.
  • Empty Charts: check that data exists in the tables and queries return results.

Performance tips

  • Use database views or materialized views for complex queries.
  • Create indexes on frequently filtered columns.
  • Limit the date range of data being queried.
  • Enable Superset caching for frequently accessed dashboards.
  • Use asynchronous query execution for large datasets.

References

For more detailed information on Superset, please refer to: