updated according to dbt-teradata 1.8.2 #6577
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@tallamohan is attempting to deploy a commit to the dbt-labs Team on Vercel. A member of the Team first needs to authorize it. |
github-actions
bot
added
content
mirnawong1
reviewed
Dec 9, 2024
mirnawong1
reviewed
Dec 9, 2024
Co-authored-by: Mirna Wong <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Co-authored-by: Mirna Wong <[email protected]>
|
looping in @amychen1776 as an fyi |
mirnawong1
reviewed
Dec 13, 2024
| @@ -210,7 +209,8 @@ For using cross-DB macros, teradata-utils as a macro namespace will not be used, | |||
|
|
|||
| ##### <a name="hash"></a>hash | |||
|
|
|||
| `Hash` macro needs an `md5` function implementation. Teradata doesn't support `md5` natively. You need to install a User Defined Function (UDF): | |||
| `Hash` macro needs an `md5` function implementation. Teradata doesn't support `md5` natively. You need to install a User Defined Function (UDF) and optionally specify `md5_udf` [variable](/docs/build/project-variables). <br> | |||
| If not specified the code defaults to using `GLOBAL_FUNCTIONS.hash_md5`. See below instructions on how to install the custom UDF: | |||
There was a problem hiding this comment.
Suggested change
| If not specified the code defaults to using `GLOBAL_FUNCTIONS.hash_md5`. See below instructions on how to install the custom UDF: | |
| If not specified the code defaults to using `GLOBAL_FUNCTIONS.hash_md5`. See the following instructions on how to install the custom UDF: |
There was a problem hiding this comment.
This information should be called out in the Teradata config page.
mirnawong1
reviewed
Dec 13, 2024
| @@ -241,6 +247,14 @@ dbt-teradata 1.8.0 and later versions support unit tests, enabling you to valida | |||
|
|
|||
| ## Limitations | |||
|
|
|||
| ### Browser Authentication | |||
There was a problem hiding this comment.
Suggested change
| ### Browser Authentication | |
| ### Browser authentication |
mirnawong1
reviewed
Dec 13, 2024
| @@ -241,6 +247,14 @@ dbt-teradata 1.8.0 and later versions support unit tests, enabling you to valida | |||
|
|
|||
| ## Limitations | |||
|
|
|||
| ### Browser Authentication | |||
| When running a dbt job with logmech set to "browser", the initial authentication opens a browser window where you must enter your username and password.<br> | |||
There was a problem hiding this comment.
@tallamohan can you turn these into bullets and remove the
. so:
- When running a dbt job with logmech set to "browser", the initial authentication opens a browser window where you must enter your username and password.
- After authentication, this window remains open, requiring you to manually switch back to the dbt console.
- For every subsequent connection, a new browser tab briefly opens, displaying the message "TERADATA BROWSER AUTHENTICATION COMPLETED," and silently reuses the existing session.
- However, the focus stays on the browser window, so you’ll need to manually switch back to the dbt console each time.
- This behavior is the default functionality of the teradatasql driver and cannot be avoided at this time.
- To prevent session expiration and the need to re-enter credentials, ensure the authentication browser window stays open until the job is complete.
mirnawong1
reviewed
Dec 13, 2024
| @@ -348,6 +348,18 @@ If a user sets some key-value pair with value as `'{model}'`, internally this `' | |||
| - For example, if the model the user is running is `stg_orders`, `{model}` will be replaced with `stg_orders` in runtime. | |||
| - If no `query_band` is set by the user, the default query_band used will be: ```org=teradata-internal-telem;appname=dbt;``` | |||
|
|
|||
| ## Unit testing | |||
| * Unit testing is supported in dbt-teradata, allowing users to write and execute unit tests using the dbt test command. | |||
| * For detailed guidance, refer to the dbt documentation. | |||
There was a problem hiding this comment.
Suggested change
| * For detailed guidance, refer to the dbt documentation. | |
| * For detailed guidance, refer to the [dbt unit tests documentation](/docs/build/documentation). |
mirnawong1
reviewed
Dec 13, 2024
| * `history_column_in_target` — Column in the target table of **period** datatype that tracks history. | ||
| * `unique_key`: The primary key of the model (excluding the valid time components), specified as a column name or list of column names. | ||
| * `valid_period`: Name of the model column indicating the period for which the record is considered to be valid. The datatype must be `PERIOD(DATE)` or `PERIOD(TIMESTAMP)`. | ||
| * `use_valid_to_time`: Wether the end bound value of the valid period in the input is considered by the strategy when building the valid timeline. Use 'no' if you consider your record to be valid until changed (and supply any value greater to the begin bound for the end bound of the period - a typical convention is `9999-12-31` of ``9999-12-31 23:59:59.999999`). Use 'yes' if you know until when the record is valid (typically this is a correction in the history timeline). |
There was a problem hiding this comment.
Suggested change
| * `use_valid_to_time`: Wether the end bound value of the valid period in the input is considered by the strategy when building the valid timeline. Use 'no' if you consider your record to be valid until changed (and supply any value greater to the begin bound for the end bound of the period - a typical convention is `9999-12-31` of ``9999-12-31 23:59:59.999999`). Use 'yes' if you know until when the record is valid (typically this is a correction in the history timeline). | |
| * `use_valid_to_time`: Whether the end bound value of the valid period in the input is considered by the strategy when building the valid timeline. Use `no` if you consider your record to be valid until changed (and supply any value greater to the begin bound for the end bound of the period. A typical convention is `9999-12-31` of ``9999-12-31 23:59:59.999999`). Use `yes` if you know until when the record is valid (typically this is a correction in the history timeline). |
mirnawong1
reviewed
Dec 13, 2024
| * Identify and adjust overlapping time slices: | ||
| * Overlapping time periods in the data are detected and corrected to maintain a consistent and non-overlapping timeline. | ||
| * Manage records needing to be overwritten or split based on the source and target data: | ||
| * The process of removing primary key duplicates (ie. two or more records with the same value for the `unique_key` and BEGIN() bond of the `valid_period` fields) in the dataset produced by the model. If such duplicates exist, the row with the lowest value is retained for all non-primary-key fields (in the order specified in the model) is retained. Full-row duplicates are always de-duplicated. |
There was a problem hiding this comment.
Suggested change
| * The process of removing primary key duplicates (ie. two or more records with the same value for the `unique_key` and BEGIN() bond of the `valid_period` fields) in the dataset produced by the model. If such duplicates exist, the row with the lowest value is retained for all non-primary-key fields (in the order specified in the model) is retained. Full-row duplicates are always de-duplicated. | |
| * The process of removing primary key duplicates (two or more records with the same value for the `unique_key` and BEGIN() bond of the `valid_period` fields) in the dataset produced by the model. If such duplicates exist, the row with the lowest value is retained for all non-primary-key fields (in the order specified in the model). Full-row duplicates are always de-duplicated. |
mirnawong1
reviewed
Dec 13, 2024
| * The process of removing primary key duplicates (ie. two or more records with the same value for the `unique_key` and BEGIN() bond of the `valid_period` fields) in the dataset produced by the model. If such duplicates exist, the row with the lowest value is retained for all non-primary-key fields (in the order specified in the model) is retained. Full-row duplicates are always de-duplicated. | ||
| * Identify and adjust overlapping time slices (if use_valid_to_time='yes): | ||
| * Overlapping time periods in the data are corrected to maintain a consistent and non-overlapping timeline. To do so, the valid period end bound of a record is adjusted to meet the begin bound of the next record with the same `unique_key` value and overlapping `valid_period` value if any. | ||
| * Manage records needing to be adjusted, deleted or split based on the source and target data: |
There was a problem hiding this comment.
Suggested change
| * Manage records needing to be adjusted, deleted or split based on the source and target data: | |
| * Manage records needing to be adjusted, deleted, or split based on the source and target data: |
mirnawong1
reviewed
Dec 13, 2024
| * Utilize the TD_NORMALIZE_MEET function to compact history: | ||
| * This function helps to normalize and compact the history by merging adjacent time periods, improving the efficiency and performance of the database. | ||
| * Compact history: | ||
| * Normalize and compact the history by merging records of adjacent time periods withe same value, optimizing database storage and performance. We use the function TD_NORMALIZE_MEET for this purpose. |
There was a problem hiding this comment.
Suggested change
| * Normalize and compact the history by merging records of adjacent time periods withe same value, optimizing database storage and performance. We use the function TD_NORMALIZE_MEET for this purpose. | |
| * Normalize and compact the history by merging records of adjacent time periods with the same value, optimizing database storage and performance. We use the function TD_NORMALIZE_MEET for this purpose. |
mirnawong1
reviewed
Dec 13, 2024
There was a problem hiding this comment.
hey @tallamohan , thanks for opening this pr up! i've made some suggestions adn couldn't commit them myself unfortunately. could you make those changes on your end and when it's ready, give me a tag and i'll look it over again?
amychen1776
reviewed
Dec 13, 2024
| For every subsequent connection, a new browser tab briefly opens, displaying the message "TERADATA BROWSER AUTHENTICATION COMPLETED," and silently reuses the existing session.<br> | ||
| However, the focus stays on the browser window, so you’ll need to manually switch back to the dbt console each time.<br> | ||
| This behavior is the default functionality of the teradatasql driver and cannot be avoided at this time.<br> | ||
| To prevent session expiration and the need to re-enter credentials, ensure the authentication browser window stays open until the job is complete. |
There was a problem hiding this comment.
This is not a great user experience. Do you have plans to provide caching and if so, can it be called out?
amychen1776
reviewed
Dec 13, 2024
| @@ -95,7 +95,6 @@ Parameter | Default | Type | Description | |||
| `browser_tab_timeout` | `"5"` | quoted integer | Specifies the number of seconds to wait before closing the browser tab after Browser Authentication is completed. The default is 5 seconds. The behavior is under the browser's control, and not all browsers support automatic closing of browser tabs. | |||
| `browser_timeout` | `"180"` | quoted integer | Specifies the number of seconds that the driver will wait for Browser Authentication to complete. The default is 180 seconds (3 minutes). | |||
| `column_name` | `"false"` | quoted boolean | Controls the behavior of cursor `.description` sequence `name` items. Equivalent to the Teradata JDBC Driver `COLUMN_NAME` connection parameter. False specifies that a cursor `.description` sequence `name` item provides the AS-clause name if available, or the column name if available, or the column title. True specifies that a cursor `.description` sequence `name` item provides the column name if available, but has no effect when StatementInfo parcel support is unavailable. | |||
| `connect_failure_ttl` | `"0"` | quoted integer | Specifies the time-to-live in seconds to remember the most recent connection failure for each IP address/port combination. The driver subsequently skips connection attempts to that IP address/port for the duration of the time-to-live. The default value of zero disables this feature. The recommended value is half the database restart time. Equivalent to the Teradata JDBC Driver `CONNECT_FAILURE_TTL` connection parameter. | |||
There was a problem hiding this comment.
Did you remove this functionality? If so it might be worth calling that out to avoid confusion
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What are you changing in this pull request and why?
Updated the setup and config pages of dbt-teradata according to the latest release dbt-teradata 1.8.2