[DRAFT] Build Enriched Traces & Transactions + Aggregation Tables #1161
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lithium323
reviewed
Dec 12, 2024
|
Added a rough diagram to the first text bubble in the PR |
MSilb7
commented
Dec 13, 2024
src/op_analytics/datapipeline/models/code/event_emitting_transactions.py
Outdated
Show resolved
Hide resolved
|
Remaining gap: How do we test in notebooks when there are input datasets we haven't built yet. |
MSilb7
commented
Dec 13, 2024
| @@ -15,6 +15,9 @@ def create_duckdb_macros(duckdb_client: duckdb.DuckDBPyConnection): | |||
|
|
|||
| CREATE OR REPLACE MACRO wei_to_gwei(a) | |||
| AS a::DECIMAL(28, 0) * 0.000000001::DECIMAL(10, 10); | |||
|
|
|||
| CREATE OR REPLACE MACRO gwei_to_eth(a) | |||
There was a problem hiding this comment.
modified decimal precision here, it wasn't allowing for a to be a decimal before
The definition of the L1 fee scalar changed in Fjord, we should not combine them together.
MSilb7
commented
Dec 14, 2024
| -- L2 Legacy Fee | ||
| if(l2_priority_fee = 0, l2_fee - l2_base_fee, 0) AS l2_legacy_extra_fee, | ||
| -- | ||
| -- L1 Base |
There was a problem hiding this comment.
We should build tests here that l1_base_fee + l1_blob_fee = l1_fee
This should hold through the history of time.
MSilb7
commented
Dec 14, 2024
| "refined_transactions_fees_v1": duckdb_client.view("refined_transactions_fees"), | ||
| "refined_trace_calls_v1": duckdb_client.view("refined_trace_calls"), | ||
| "event_emitting_transactions_v1": duckdb_client.view("event_emitting_transactions"), | ||
| "summary_v1": duckdb_client.view("daily_address_summary"), |
There was a problem hiding this comment.
should we rename this to daily_address_summary_v1? Maybe in a future PR to keep this clean-ish
MSilb7
commented
Dec 14, 2024
| refined_transactions_fees | ||
| WHERE | ||
| NOT is_system_transaction | ||
| AND gas_price > 0 |
There was a problem hiding this comment.
this used to be at the transactions_fees level, but I unfiltered there (so we preserve those transactions) and brought it here.
# Conflicts: # src/op_analytics/datapipeline/models/templates/daily_address_summary.sql.j2 # src/op_analytics/datapipeline/models/templates/transaction_fees.sql.j2
# Conflicts: # src/op_analytics/datapipeline/models/code/daily_address_summary.py # src/op_analytics/datapipeline/models/templates/daily_address_summary.sql.j2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Working PR - See overall project #1055
closes #1160 - Transaction Models Exploration
closes #1168 - Trace Models Exploration
The end results of these models are also expected to be used in conjunction with: #1151
TODO:
Enriched TransactionsExtend existing transaction_fees modelAggregation strategy:
I'm the worst at excalidraw, but this is what I envision. Progress moves left to right, this PR ends at "Aggregation Models"
