Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Eliminate Limit Rule #60

Merged
merged 15 commits into from
Feb 13, 2024
Merged

feat: Eliminate Limit Rule #60

merged 15 commits into from
Feb 13, 2024

Conversation

jurplel
Copy link
Member

@jurplel jurplel commented Feb 11, 2024

Depends on #58

@jurplel jurplel mentioned this pull request Feb 11, 2024
26 tasks
@jurplel jurplel marked this pull request as ready for review February 12, 2024 03:21
@jurplel
Copy link
Member Author

jurplel commented Feb 12, 2024

I am not sure how to construct a query that will test this case:

Limit with skip 0 and no fetch -> Eliminate from the tree

any ideas?

Copy link
Contributor

@AveryQi115 AveryQi115 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. I think for testing skip=0 and fetch is none datafusion repo creates fake logical plan with limit as the children node of the limit. If we want to do the same testing using sql, we might need to add support for subqueries or unions or other complicated nested queries to get this internal states...

@AveryQi115
Copy link
Contributor

I am not sure how to construct a query that will test this case:

Limit with skip 0 and no fetch -> Eliminate from the tree

any ideas?

Not sure if current optd can support things like select * from (select * from t1 limit 5) limit 6;

@jurplel
Copy link
Member Author

jurplel commented Feb 13, 2024

select * from (select * from t1 limit 5) limit 6;
This query does work, but it doesn't test the case that I was referring to. I don't know of any way to make a SQL query that will make skip: 0 and fetch: None.

| physical_plan after optd                         | PhysicalLimit { skip: 0, fetch: 6 }                                     |
|                                                  | └── PhysicalProjection { exprs: [ #0, #1 ] }                            |
|                                                  |     └── PhysicalLimit { skip: 0, fetch: 5 }                             |
|                                                  |         └── PhysicalProjection { exprs: [ #0, #1 ] }                    |
|                                                  |             └── PhysicalScan { table: t2 }                              |

I'll go ahead and merge this for now but let me know if we have a nice way to test with fabricated query plans.

@jurplel jurplel merged commit e846c58 into main Feb 13, 2024
1 check passed
@jurplel jurplel deleted the bowad/eliminate-limit branch February 13, 2024 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants