Sensors

Documentation

Sensors are a special type of Operator that are designed to do exactly one thing — wait for something to occur.

It can be time-based, or waiting for a file, or an external event, but all they do is wait until something happens, and then succeed so their downstream tasks can run.

Because they are primarily idle, Sensors have three different modes of running so you can be a bit more efficient about using them:

• poke (default): The Sensor takes up a worker slot for its entire runtime
• reschedule: The Sensor takes up a worker slot only when it is checking, and sleeps for a set duration between checks
• smart sensor: There is a single centralized version of this Sensor that batches all executions of it

Attention

Smart Sensors is a deprecated early-access feature that will be removed in Airflow 2.4.0. It is superseded by Deferrable Operators.

The poke and reschedule modes can be configured directly when you instantiate the sensor; generally, the trade-off between them is latency.

Note

Something that is checking every second should be in poke mode, while something that is checking every minute should be in reschedule mode.