Skip to content

Commit

Permalink
Merge branch 'current' into add-tags
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Dec 24, 2024
2 parents 9523740 + 6a1e40d commit 4557104
Show file tree
Hide file tree
Showing 18 changed files with 52 additions and 50 deletions.
1 change: 0 additions & 1 deletion website/docs/docs/build/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ dbt Cloud has a number of pre-defined variables built in. Variables are set auto
The following environment variable is set automatically for the dbt Cloud IDE:

- `DBT_CLOUD_GIT_BRANCH` — Provides the development Git branch name in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud).
- Available in dbt v1.6 and later.
- The variable changes when the branch is changed.
- Doesn't require restarting the IDE after a branch change.
- Currently not available in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation).
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/incremental-microbatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ from {{ source('sales', 'transactions') }}

### Full refresh

As a best practice, we recommend configuring `full_refresh: False` on microbatch models so that they ignore invocations with the `--full-refresh` flag. If you need to reprocess historical data, do so with a targeted backfill that specifies explicit start and end dates.
As a best practice, we recommend [configuring `full_refresh: false`](/reference/resource-configs/full_refresh) on microbatch models so that they ignore invocations with the `--full-refresh` flag. If you need to reprocess historical data, do so with a targeted backfill that specifies explicit start and end dates.

## Usage

Expand Down
5 changes: 2 additions & 3 deletions website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,10 @@ keywords:
- unit test, unit tests, unit testing, dag
---

:::note
<VersionCallout version="1.8" />


Unit testing functionality is available in [dbt Cloud Release Tracks](/docs/dbt-versions/cloud-release-tracks) or dbt Core v1.8+

:::

Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model.

Expand Down
8 changes: 4 additions & 4 deletions website/docs/docs/collaborate/explore-projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,21 +113,21 @@ Lenses are helpful to analyze a subset of the DAG if you're zoomed in, or to fin

A resource in your project is characterized by resource type, materialization type, or model layer, as well as its latest run or latest test status. Lenses are available for the following metadata:

- **Relationship**: Organizes resources by resource type, such as models, tests, seeds, and [more](/reference/node-selection/syntax). Resource type uses the `resource_type` selector.
- **Resource type**: Organizes resources by resource type, such as models, tests, seeds, saved query, and [more](/docs/build/projects). Resource type uses the `resource_type` selector.
- **Materialization type**: Identifies the strategy for building the dbt models in your data platform.
- **Latest status**: The status from the latest execution of the resource in the current environment. For example, diagnosing a failed DAG region.
- **Model layer**: The modeling layer that the model belongs to according to [best practices guide](https://docs.getdbt.com/best-practices/how-we-structure/1-guide-overview#guide-structure-overview). For example, discovering marts models to analyze.
- **Marts** &mdash; A model with the prefix `fct_` or `dim_` or a model that lives in the `/marts/` subdirectory.
- **Intermediate** &mdash; A model with the prefix `int_`. Or, a model that lives in the `/int/` or `/intermediate/` subdirectory.
- **Staging** &mdash; A model with the prefix `stg_`. Or, a model that lives in the `/staging/` subdirectory.
- **Test status**: The status from the latest execution of the tests that ran again this resource. In the case that a model has multiple tests with different results, the lens reflects the 'worst case' status.
- **Usage queries**: The number of queries against this resource over a given time period.
- **Consumption query history**: The number of queries against this resource over a given time period.

</Expandable>

### Example of lenses

Example of applying the **Materialization Type** _lens_ with the lineage graph zoomed out. In this view, each model name has a color according to the materialization type legend at the bottom, which specifies the materialization type. This color-coding helps to quickly identify the materialization types of different models.
Example of applying the **Materialization type** _lens_ with the lineage graph zoomed out. In this view, each model name has a color according to the materialization type legend at the bottom, which specifies the materialization type. This color-coding helps to quickly identify the materialization types of different models.

<Lightbox src="/img/docs/collaborate/dbt-explorer/example-materialization-type.jpg" width="100%" title="Example of the Materialization type lens" />

Expand All @@ -141,7 +141,7 @@ You can locate resources in your project by performing a keyword search in the s

<Expandable alt_header="Search features">

- **Partial keyword search** &mdash; This is also referred to as fuzzy search.
- **Partial keyword search** &mdash; Also referred to as fuzzy search. Explorer uses a "contains" logic to improve your search results. This means you can search for partial terms without knowing the exact root word of your search term.
- **Exclude keywords** &mdash; Prepend a minus sign (-) to the keyword you want to exclude from search results. For example, `-user` will exclude all matches of that keyword from search results.
- **Boolean operators** &mdash; Use Boolean operators to enhance your keyword search. For example, the search results for `users OR github` will include matches for either keyword.
- **Phrase search** &mdash; Surround a string of keywords with double quotation marks to search for that exact phrase (for example, `"stg users"`). To learn more, refer to [Phrase search](https://en.wikipedia.org/wiki/Phrase_search) on Wikipedia.
Expand Down
16 changes: 12 additions & 4 deletions website/docs/docs/deploy/model-notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,29 @@ groups:
- name: finance
owner:
# Email is required to receive model-level notifications, additional properties are also allowed.
name: "Finance Team"
description: "Models related to the finance department"
name: "Finance team"
email: [email protected]
favorite_food: donuts

- name: marketing
owner:
name: "Marketing Team"
description: "Models related to the marketing department"
name: "Marketing team"
email: [email protected]
favorite_food: jaffles

- name: docs
owner:
name: "Documentation team"
email: [email protected]
favorite_food: pizza
```
</File>
:::tip
The `owner` key is flexible and accepts arbitrary inputs in addition to the required `email` field. For example, you could include a custom field like `favorite_food` to add context about the team.
:::

## Attach groups to models

Attach groups to models as you would any other config, in either the `dbt_project.yml` or `whatever.yml` files. For example:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Running with dbt=1.0.0
Found 13 models, 2 tests, 1 archives, 0 analyses, 204 macros, 2 operations....
```

:::info dbt Cloud release tracks
:::note dbt Cloud release tracks
<Snippet path="_config-dbt-version-check" />

:::
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/batch_size.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "dbt uses `batch_size` to determine how large batches are when runn
datatype: hour | day | month | year
---

Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.
<VersionCallout version="1.9" />

## Definition

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/begin.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "dbt uses `begin` to determine when a microbatch incremental model
datatype: string
---

Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.
<VersionCallout version="1.9" />

## Definition

Expand Down
4 changes: 2 additions & 2 deletions website/docs/reference/resource-configs/databricks-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,8 +437,8 @@ Beginning with 1.9, `merge` behavior can be modified with the following addition
- `skip_not_matched_step`: If set to `true`, the 'not matched' clause will not be included.
- `matched_condition`: Condition to apply to the `WHEN MATCHED` clause. You should use the `target_alias` and `source_alias` to write a conditional expression, such as `DBT_INTERNAL_DEST.col1 = hash(DBT_INTERNAL_SOURCE.col2, DBT_INTERNAL_SOURCE.col3)`. This condition further restricts the matched set of rows.
- `not_matched_condition`: Condition to apply to the `WHEN NOT MATCHED [BY TARGET]` clause. This condition further restricts the set of rows in the target that do not match the source that will be inserted into the merged table.
- `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action: delete`.
- `not_matched_by_source_action`: If set to `delete`, a `DELETE` clause is added to the merge statement for `WHEN NOT MATCHED BY SOURCE`.
- `not_matched_by_source_condition`: Condition to apply to the further filter `WHEN NOT MATCHED BY SOURCE` clause. Only used in conjunction with `not_matched_by_source_action`.
- `not_matched_by_source_action`: The action to apply when the condition is met. Configure as an expression. For example: `not_matched_by_source_action: "update set t.attr1 = 'deleted', t.tech_change_ts = current_timestamp()"`.
- `merge_with_schema_evolution`: If set to `true`, the merge statement includes the `WITH SCHEMA EVOLUTION` clause.

For more details on the meaning of each merge clause, please see [the Databricks documentation](https://docs.databricks.com/en/sql/language-manual/delta-merge-into.html).
Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/delimiter.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ datatype: <string>
default_value: ","
---

Supported in v1.7 and higher.
<VersionCallout version="1.7" />

## Definition

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/resource-configs/event-time.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "dbt uses event_time to understand when an event occurred. When def
datatype: string
---

Available in [the dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.
<VersionCallout version="1.9" />

<Tabs>
<TabItem value="model" label="Models">
Expand Down
35 changes: 20 additions & 15 deletions website/docs/reference/resource-configs/full_refresh.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ description: "Setting the full_refresh config to false prevents a model or seed
datatype: boolean
---

The `full_refresh` config allows you to control whether a resource will always or never perform a full-refresh. This config overrides the `--full-refresh` command-line flag.

<Tabs
defaultValue="models"
values={[
Expand All @@ -19,8 +21,7 @@ datatype: boolean
```yml
models:
[<resource-path>](/reference/resource-configs/resource-path):
+full_refresh: false

+full_refresh: false | true
```
</File>
Expand All @@ -30,17 +31,14 @@ models:
```sql

{{ config(
full_refresh = false
full_refresh = false | true
) }}

select ...

```

</File>

The configured model(s) will not full-refresh when `dbt run --full-refresh` is invoked.

</TabItem>

<TabItem value="seeds">
Expand All @@ -50,27 +48,34 @@ The configured model(s) will not full-refresh when `dbt run --full-refresh` is i
```yml
seeds:
[<resource-path>](/reference/resource-configs/resource-path):
+full_refresh: false
+full_refresh: false | true

```

</File>

The configured seed(s) will not full-refresh when `dbt seed --full-refresh` is invoked.

</TabItem>

</Tabs>

- If `full_refresh:true` &mdash; the configured resources(s) will full-refresh when `dbt run --full-refresh` is invoked.
- If `full_refresh:false` &mdash; the configured resources(s) will _not_ full-refresh when `dbt run --full-refresh` is invoked.


## Description
Optionally set a resource to always or never full-refresh.
- If specified as `true` or `false`, the
`full_refresh` config will take precedence over the presence or absence of the `--full-refresh` flag.
- If the `full_refresh` config is `none` or omitted, the resource will use the value of the `--full-refresh` flag.

**Note:** The `--full-refresh` flag also supports a short name, `-f`.
The `full_refresh` config allows you to optionally configure whether a resource will always or never perform a full-refresh. This config is an override for the `--full-refresh` command line flag used when running dbt commands.


| `full_refresh` value | Behavior |
| ---------------------------- | -------- |
| `true` | The resource always full-refreshes, regardless of the presence or absence of the `--full-refresh` flag. |
| `false` | The resource never full-refreshes, even if the `--full-refresh` flag is provided. |
| `none` or omitted | The resource follows the behavior of the `--full-refresh` flag. If the flag is used, the resource will full-refresh; otherwise, it won't. |

This logic is encoded in the [`should_full_refresh()`](https://github.com/dbt-labs/dbt-adapters/blob/60005a0a2bd33b61cb65a591bc1604b1b3fd25d5/dbt/include/global_project/macros/materializations/configs.sql) macro.
#### Note
- The `--full-refresh` flag also supports a short name, `-f`.
- The [`should_full_refresh()`](https://github.com/dbt-labs/dbt-adapters/blob/60005a0a2bd33b61cb65a591bc1604b1b3fd25d5/dbt/include/global_project/macros/materializations/configs.sql) macro has logic encoded.

## Usage

Expand Down
3 changes: 1 addition & 2 deletions website/docs/reference/resource-configs/hard-deletes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ id: "hard-deletes"
sidebar_label: "hard_deletes"
---

Available from dbt v1.9 or with [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks).

<VersionCallout version="1.9" />

<File name='snapshots/schema.yml'>

Expand Down
3 changes: 1 addition & 2 deletions website/docs/reference/resource-configs/lookback.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ description: "dbt uses `lookback` to detrmine how many 'batches' of `batch_size`
datatype: int
---

Available in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9 and higher.

<VersionCallout version="1.9" />
## Definition

Set the `lookback` to an integer greater than or equal to zero. The default value is `1`. You can configure `lookback` for a [model](/docs/build/models) in your `dbt_project.yml` file, property YAML file, or config block.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ default_value: {"dbt_valid_from": "dbt_valid_from", "dbt_valid_to": "dbt_valid_t
id: "snapshot_meta_column_names"
---

Available in dbt Core v1.9+. Select v1.9 or newer from the version dropdown to view the configs. Try it now in the [dbt Cloud "Latest" release track](/docs/dbt-versions/cloud-release-tracks).
<VersionCallout version="1.9" />

<File name='snapshots/schema.yml'>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@ datatype: model_name
description: "Learn about concurrent_batches in dbt."
---

:::note

Available in dbt Core v1.9+ or the [dbt Cloud "Latest" release tracks](/docs/dbt-versions/cloud-release-tracks).

:::
<VersionCallout version="1.9" />

<Tabs>
<TabItem value="Project file">
Expand Down
5 changes: 1 addition & 4 deletions website/docs/reference/resource-properties/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,8 @@ resource_types: [models]
datatype: test
---

:::note
<VersionCallout version="1.8" />

This functionality is available in dbt Core v1.8+ and [dbt Cloud release tracks](/docs/dbt-versions/cloud-release-tracks).

:::

Unit tests validate your SQL modeling logic on a small set of static inputs before you materialize your full model in production. They support a test-driven development approach, improving both the efficiency of developers and reliability of code.

Expand Down
2 changes: 1 addition & 1 deletion website/src/components/versionCallout/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const VersionCallout = ({ version }) => {
Available from dbt v{version} or with the{' '}
<a href="/docs/dbt-versions/cloud-release-tracks">
dbt Cloud "Latest" release track
</a>{' '}.
</a>{''}.
</span>
</Admonition>
</div>
Expand Down

0 comments on commit 4557104

Please sign in to comment.