Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](inverted index) Conduct More Checks on dict_compression #45805

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zzzxl1993
Copy link
Contributor

@zzzxl1993 zzzxl1993 commented Dec 23, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #44414

Problem Summary:

  1. dict_compression is only supported in v3 mode.
  2. dict_compression is only supported for string-type fields.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zzzxl1993
Copy link
Contributor Author

run buildall

starocean999
starocean999 previously approved these changes Dec 23, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 23, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@zzzxl1993
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 23, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 39873 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit df8afa03163956fcc3de9f576245ab58140eb226, data reload: false

------ Round 1 ----------------------------------
q1	17584	7496	7253	7253
q2	2045	186	201	186
q3	10506	1143	1194	1143
q4	10227	748	711	711
q5	7582	2732	2636	2636
q6	239	149	148	148
q7	985	620	616	616
q8	9255	1894	1959	1894
q9	6635	6441	6455	6441
q10	7050	2292	2282	2282
q11	469	258	263	258
q12	428	219	215	215
q13	17783	2865	2950	2865
q14	242	212	209	209
q15	549	506	511	506
q16	683	587	588	587
q17	987	525	510	510
q18	7388	6677	6764	6677
q19	1341	1105	1047	1047
q20	487	191	187	187
q21	4074	3196	3338	3196
q22	378	306	317	306
Total cold run time: 106917 ms
Total hot run time: 39873 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7239	7220	7192	7192
q2	330	224	224	224
q3	2919	2778	3030	2778
q4	2088	1868	1778	1778
q5	5534	5623	5652	5623
q6	228	141	141	141
q7	2249	1797	1814	1797
q8	3355	3500	3493	3493
q9	8976	8843	8950	8843
q10	3595	3541	3503	3503
q11	611	487	514	487
q12	804	602	615	602
q13	11879	3124	3105	3105
q14	296	269	275	269
q15	559	509	505	505
q16	750	619	649	619
q17	1814	1613	1712	1613
q18	7910	7597	7556	7556
q19	1696	1511	1443	1443
q20	2100	1803	1803	1803
q21	5393	5212	5176	5176
q22	631	611	585	585
Total cold run time: 70956 ms
Total hot run time: 59135 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190426 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit df8afa03163956fcc3de9f576245ab58140eb226, data reload: false

query1	1028	386	387	386
query2	6520	2405	2343	2343
query3	6708	220	207	207
query4	33912	23711	23386	23386
query5	4376	472	491	472
query6	305	205	184	184
query7	4631	313	312	312
query8	310	252	236	236
query9	9755	2774	2776	2774
query10	471	256	257	256
query11	18220	15160	15223	15160
query12	158	102	103	102
query13	1718	430	416	416
query14	9966	7268	6900	6900
query15	318	189	183	183
query16	8383	455	446	446
query17	1695	610	581	581
query18	2171	334	299	299
query19	368	157	154	154
query20	118	112	111	111
query21	215	100	135	100
query22	4505	4309	4231	4231
query23	34730	33564	34141	33564
query24	11412	2501	2441	2441
query25	671	376	369	369
query26	1843	158	156	156
query27	2904	326	335	326
query28	8225	2467	2503	2467
query29	1042	412	412	412
query30	301	148	147	147
query31	1044	812	801	801
query32	100	59	65	59
query33	776	285	289	285
query34	1017	528	509	509
query35	902	747	754	747
query36	1128	968	954	954
query37	297	74	76	74
query38	4163	4128	4266	4128
query39	1505	1599	1407	1407
query40	284	104	102	102
query41	70	46	47	46
query42	113	103	101	101
query43	566	511	501	501
query44	1252	833	809	809
query45	194	163	170	163
query46	1164	734	704	704
query47	1940	1828	1841	1828
query48	418	328	324	324
query49	1292	399	399	399
query50	823	389	379	379
query51	7252	7175	7037	7037
query52	101	90	88	88
query53	271	186	187	186
query54	1153	413	419	413
query55	83	77	79	77
query56	260	242	264	242
query57	1265	1078	1157	1078
query58	225	228	222	222
query59	3293	3123	3258	3123
query60	270	243	239	239
query61	109	104	141	104
query62	883	666	665	665
query63	219	193	193	193
query64	5081	669	660	660
query65	3232	3185	3281	3185
query66	1444	312	336	312
query67	15823	15695	15456	15456
query68	5726	577	611	577
query69	457	251	252	251
query70	1154	1080	1155	1080
query71	354	252	248	248
query72	6644	4086	4065	4065
query73	797	433	373	373
query74	10377	8833	8970	8833
query75	3412	2666	2654	2654
query76	3589	1048	1119	1048
query77	514	277	277	277
query78	10262	9314	9392	9314
query79	1661	607	611	607
query80	1149	436	438	436
query81	537	248	238	238
query82	972	126	169	126
query83	245	150	147	147
query84	236	79	73	73
query85	1318	297	292	292
query86	375	304	304	304
query87	4627	4485	4335	4335
query88	3280	2267	2211	2211
query89	421	303	297	297
query90	2163	192	190	190
query91	140	108	108	108
query92	62	52	53	52
query93	1235	551	558	551
query94	1174	290	288	288
query95	364	259	260	259
query96	629	285	287	285
query97	2831	2676	2710	2676
query98	228	192	198	192
query99	1546	1336	1318	1318
Total cold run time: 305470 ms
Total hot run time: 190426 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit df8afa03163956fcc3de9f576245ab58140eb226, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.08	0.07
query4	1.60	0.10	0.11
query5	0.44	0.42	0.40
query6	1.16	0.65	0.64
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.50	0.49
query10	0.55	0.58	0.56
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.74	2.76	2.71
query15	0.90	0.83	0.82
query16	0.38	0.38	0.39
query17	1.05	1.07	0.97
query18	0.23	0.21	0.20
query19	1.97	1.79	2.03
query20	0.01	0.01	0.01
query21	15.36	0.62	0.58
query22	2.90	2.39	1.68
query23	17.07	0.92	0.87
query24	3.07	0.60	0.95
query25	0.18	0.24	0.14
query26	0.28	0.13	0.14
query27	0.05	0.05	0.04
query28	11.09	1.11	1.08
query29	12.57	3.22	3.22
query30	0.25	0.07	0.06
query31	2.85	0.39	0.37
query32	3.25	0.47	0.46
query33	3.08	3.08	3.15
query34	17.05	4.46	4.48
query35	4.54	4.45	4.51
query36	0.69	0.48	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.03
query40	0.18	0.12	0.14
query41	0.09	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 107.73 s
Total hot run time: 32.06 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants