API Docs

📄️ asset-checks

Dagster allows you to define and execute checks on your software-defined assets. Each asset check verifies some property of a data asset, e.g. that is has no null values in a particular column.

An asset is an object in persistent storage, such as a table, file, or persisted machine learning model. An asset definition is a description, in code, of an asset that should exist and how to produce and update that asset.

📄️ cli

dagster asset

📄️ config

Pythonic config system

📄️ definitions

class dagster.Definitions

📄️ dynamic

These APIs provide the means for a simple kind of dynamic orchestration — where the work to be orchestrated is determined not at job definition time but at runtime, dependent on data that’s observed as part of job execution.

📄️ errors

Core Dagster error classes.

📄️ execution

Materializing Assets

📄️ external-assets

As Dagster doesn’t control scheduling or materializing external assets, it’s up to you to keep their metadata updated. The APIs in this reference can be used to keep external assets updated in Dagster.

📄️ graphs

The core of a job is a graph of ops - connected via data dependencies.

📄️ hooks

@dagster.success_hook

📄️ API Docs

📄️ internals

Note that APIs imported from Dagster submodules are not considered stable, and are potentially subject to change in the future.

📄️ io-managers

IO managers are user-provided objects that store op outputs and load them as inputs to downstream

📄️ jobs

A Job binds a Graph and the resources it needs to be executable.

🗃️ libraries

53 items

📄️ loggers

Built-in loggers

📄️ metadata

Dagster uses metadata to communicate arbitrary user-specified metadata about structured

📄️ ops

The foundational unit of computation in Dagster.

📄️ partitions

class dagster.PartitionsDefinition

📄️ pipes

Dagster Pipes is a toolkit for building integrations between Dagster and external execution environments. This reference outlines the APIs included with the dagster library, which should be used in the orchestration environment.

📄️ repositories

dagster.repository RepositoryDefinition

📄️ resources

Pythonic resource system

📄️ schedules-sensors

Dagster offers several ways to run data pipelines without manual intervation, including traditional scheduling and event-based triggers. Automating your Dagster pipelines can boost efficiency and ensure that data is produced consistently and reliably.

📄️ types

Dagster includes facilities for typing the input and output values of ops (“runtime” types).

📄️ utilities

dagster.filerelativepath