Feature Request: support BigQuery reservations as Dataform config

[rsenseman]

Background

As far as I can tell, BigQuery reservations can only be applied by supplying the reservation syntax inline with the query, so reservations must be coded into every query, like this:

config {
  type: "view"
}
pre_operations { SET @@reservation='projects/<project_name>/locations/<region>/reservations/<reservation_name>'; }


SELECT
  *
FROM ${ ref("table_name") }

Ideally, this could be applied at a higher level so that reservations could be applied at the project level or at the very least supplied at the model config level

Desired behavior

At the model level:

config {
  type: "view",
  bigquery: {
    reservation: 'projects/<project_name>/locations/<region>/reservations/<reservation_name>'
  }
}

SELECT
  *
FROM ${ ref("table_name") }

At the workflow_settings.yaml level:

defaultReservation: 'projects/<project_name>/locations/<region>/reservations/<reservation_name>'

[max-ostapenko]

I've been using masthead-data/dataform-package for a while, but it relies on SET @@reservations which sometimes breaks the compiled SQL.

Being able to set into a dedicated job parameter will be super-straightforward and efficient.

@kolina I don't see any job-level params so far in user-facing API. Would it make sense to define as jobConfig attribute for tables, assertions and operations?

[kolina]

I think it totally makes sense to support configuration of run-time reservation parameters.

But I'm not sure if we should have this as part of a compiled graph action (because ideally it should be overridable when run without recompiling)

[max-ostapenko]

Oh, good point.
You mean when a user creates a workflow in the console he can define a reservation for it (uses compiled dag result).

So there are 2 different use cases:

1. default reservation per workflow,
2. and action-level - reservation overrides defined per action in the compiled DAG

I'd be happy to have the following:

action.runtime.reservation = action.compiled.reservation ?? workflow.reservation ?? null;

@kolina WDYT?

[max-ostapenko]

Did my first attempt in #2096.
@rsenseman @kolina please, looking forward to your feedback