cli export doesn't support metric tables

[0neSe7en]

What type of bug is this?

Incorrect result

What subsystems are affected?

Other

Minimal reproduce step

• save metrics into tables with prometheus remote write.
• use greptime cli export ... to export the database.

What did you expect to see?

the exported data and sqls for creating database and tables.

What did you see instead?

create_database.sql and create_tables.sql are as expected.

the copy_from.sql has one line:

COPY DATABASE "greptime"."metrics" FROM './backup/greptime/metrics/' WITH (FORMAT='parquet')

but no other files.

And it seems like it had retrieved the data as it takes minutes to finish the job.

What operating system did you use?

macos and ubuntu

What version of GreptimeDB did you use?

0.10.2 and 0.11

Relevant log output and stack trace

2024-12-11T01:52:27.021343Z  INFO cmd: Starting app: greptime-cli
2024-12-11T01:52:27.581167Z  INFO cmd::cli::export: Success 1 jobs, cost: 559.657291ms
2024-12-11T01:53:07.672679Z  INFO cmd::cli::export: Finished exporting greptime.metrics with 138 table schemas to path: ./backup/greptime/metrics/
2024-12-11T01:53:07.673514Z  INFO cmd::cli::export: Success 1/1 jobs, cost: 40.092314405s
2024-12-11T01:53:07.944928Z  INFO cmd::cli::export: Executing sql: COPY DATABASE "greptime"."metrics" TO './backup/greptime/metrics/' WITH (FORMAT='parquet');
2024-12-11T01:55:19.730131Z  INFO cmd::cli::export: Finished exporting greptime.metrics data into path: ./backup/greptime/metrics/
2024-12-11T01:55:19.730476Z  INFO cmd::cli::export: Finished exporting greptime.metrics copy_from.sql
2024-12-11T01:55:19.730494Z  INFO cmd::cli::export: Success 1/1 jobs, costs: 132.056963048s
2024-12-11T01:55:19.730499Z  INFO cmd: Goodbye!

[waynexia]

Do you see those tables after copy from?

[0neSe7en]

no, only output three SQL files, and only one line in the copy_from.sql and the console output doesn't have any tables

[waynexia]

I don't see where the problem is. You should see those tables after executing the generated SQL files. Do you want to get the detailed table structures in SQL instead of copying databases?

[0neSe7en]

I was trying to backup the database. But no other files were saved but three SQL file

    └── metrics
        ├── copy_from.sql
        ├── create_database.sql
        └── create_tables.sql

no were saved.

<output-dir>/
└── greptime/
    └── <database>/
        ├── create_database.sql
        ├── create_tables.sql
        ├── copy_from.sql
        └── <data files>

we deploy the greptime cluster with 4 frontend, 4 datanode, 2 flownode and 2 meta.
I also wonder that where the parquet will be saved when I execute the COPY tbl TO '/xxx/xxx/output.parquet' WITH (FORMAT = 'parquet'); in the dashboard.

[0neSe7en]

the cli export doesn't save parquet files for normal tables as well. I guess it's related to the s3 and greptime cluster?

[WenyXu]

Hi @0neSe7en, in distributed mode, the data file will be stored on the node that executes the Copy database ... To statement.

[0neSe7en]

I see. even I run the greptime cli export locally. The node you mean frontend or the datanode?

[WenyXu]

I see. even I run the greptime cli export locally. The node you mean frontend or the datanode?

It's the frontend node. For now, you may need to connect one of the frontend nodes (to obtain all exported data) and execute the CLI cmd on that node. We will improve the user experience ASAP.

[0neSe7en]

yeah thanks. I wrote a script to copy from/to tables(databases) to S3.