Skip to main content

Introduction & setup

In this demo project, you build a weather data pipeline with dbt-polars. Along the way, you write Python and SQL models, split transformations into steps, add data tests, and finally ingest data incrementally from an API.

Prerequisites​

The demo project starts from the project created in the Quick start. Complete the quick start first, and run the commands of this tutorial from the quickstart folder.

Like the quick start, this tutorial uses the local catalog, which writes the tables to the data folder of the project.

Loading the data​

For this example, we use hourly weather data by Open-Meteo.com. Download open_meteo_raw.csv and put it in the seeds/ folder of the project:

quickstart/
├── dbt_project.yml
├── models/
│   ├── example/
│   │   ├── my_first_dbt_model.sql
│   │   ├── my_second_dbt_model.sql
│   │   └── schema.yml
│   └── my_python_model.py
├── seeds/
│   └── open_meteo_raw.csv
└── tests/

Then load it as a table:

dbt seed

This writes the open_meteo_raw table to data/main/, next to the example models.

Further reading​

  • See Seeds in the dbt documentation for loading CSV files as tables.
  • See Source data for reading raw files directly, without loading them first.
  • See Local storage for all options of the local catalog.