feature request : add an option to display a DAG

[djouallah]

it will be nice to have an easy to use option something like to_dag, that automatically show a diagram with all the tables dependencies.

it will something similar to duckdb.sql(" select 42").explain() that automatically generate query plan

users seems to like this kind of easy to use functionalities

[eakmanrq]

Thanks @djouallah for the request and I agree users would like that. Currently it feels a bit outside of the scope of SQLFrame and I would prefer if I could leverage some other library to generate it. Currently you can generate column-level lineage using the new lineage method but it relies on SQLGlot's implementation which isn't very user friendly. Context: #138

[eakmanrq]

One thing to clarify on this request, you want to see a DAG view of a given DataFrame correct? So if the DataFrame joined Table A and Table B then you would want to see a graph of Table A and Table B being joined together? Do you care about column-level lineage in this example?

[djouallah]

no, I don't care about column lineage

[tobymao]

you can leverage sqlglot's logical planner

https://github.com/tobymao/sqlglot/blob/main/sqlglot/planner.py