Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix](Replica) fix bug that replica remote data size set to data size #35097

Merged
merged 1 commit into from
May 23, 2024

Conversation

GoGoWen
Copy link
Contributor

@GoGoWen GoGoWen commented May 21, 2024

Proposed changes

when finish clone task, the replica's remote data size will set to data size, it's a bug.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

@caiconghui caiconghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 21, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@GoGoWen GoGoWen force-pushed the fix_replica_remote_size_error branch from 0b38843 to e3479bd Compare May 21, 2024 08:51
@GoGoWen
Copy link
Contributor Author

GoGoWen commented May 22, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50319 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e3479bdc641cc70e2b7adc4b90c0cdf71fda61b0, data reload: false

------ Round 1 ----------------------------------
q1	18255	4333	4330	4330
q2	2050	157	152	152
q3	10458	1901	1948	1901
q4	10622	1227	1345	1227
q5	8454	4242	3907	3907
q6	244	127	124	124
q7	2049	1580	1605	1580
q8	9612	2747	2709	2709
q9	13314	10358	10292	10292
q10	8647	3574	3541	3541
q11	415	243	253	243
q12	465	305	303	303
q13	19903	4021	4109	4021
q14	355	332	352	332
q15	522	485	473	473
q16	694	614	616	614
q17	1142	945	916	916
q18	7719	7347	7250	7250
q19	1687	1579	1549	1549
q20	568	316	305	305
q21	4419	4165	4157	4157
q22	505	410	393	393
Total cold run time: 122099 ms
Total hot run time: 50319 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4363	4364	4342	4342
q2	316	218	219	218
q3	4143	4140	4114	4114
q4	2750	2724	2745	2724
q5	7167	7110	7120	7110
q6	235	121	122	121
q7	3238	2855	2810	2810
q8	4390	4479	4472	4472
q9	16894	16810	16745	16745
q10	4207	4299	4295	4295
q11	745	669	698	669
q12	1018	830	868	830
q13	6943	3773	3728	3728
q14	452	423	434	423
q15	497	458	457	457
q16	735	687	683	683
q17	3813	3851	3862	3851
q18	8653	8717	8786	8717
q19	1713	1653	1705	1653
q20	2403	2180	2116	2116
q21	8514	8460	8448	8448
q22	1019	964	946	946
Total cold run time: 84208 ms
Total hot run time: 79472 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203987 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e3479bdc641cc70e2b7adc4b90c0cdf71fda61b0, data reload: false

query1	934	382	376	376
query2	6558	2750	2599	2599
query3	6920	216	206	206
query4	20118	17969	18114	17969
query5	19724	6517	6563	6517
query6	287	224	224	224
query7	4158	307	303	303
query8	273	255	246	246
query9	3123	2693	2614	2614
query10	426	327	303	303
query11	11389	10708	10757	10708
query12	119	75	74	74
query13	5584	678	678	678
query14	17628	13210	13233	13210
query15	395	228	223	223
query16	6447	278	257	257
query17	1719	1459	877	877
query18	2303	406	407	406
query19	212	142	150	142
query20	80	78	77	77
query21	187	96	94	94
query22	5238	5064	5055	5055
query23	32558	31932	31864	31864
query24	6976	6548	6513	6513
query25	516	431	437	431
query26	513	162	161	161
query27	1806	290	290	290
query28	6136	2343	2298	2298
query29	2924	2735	2723	2723
query30	244	173	166	166
query31	901	714	738	714
query32	72	62	60	60
query33	403	245	256	245
query34	851	467	491	467
query35	1108	903	906	903
query36	1455	1393	1173	1173
query37	90	59	58	58
query38	3099	2916	2928	2916
query39	1384	1319	1335	1319
query40	208	93	97	93
query41	38	41	37	37
query42	82	80	86	80
query43	708	712	631	631
query44	1168	716	714	714
query45	240	232	235	232
query46	1227	962	994	962
query47	1933	1753	1874	1753
query48	1019	718	703	703
query49	625	381	385	381
query50	860	628	637	628
query51	4690	4656	4664	4656
query52	98	82	85	82
query53	446	327	321	321
query54	2688	2488	2479	2479
query55	91	87	84	84
query56	238	206	212	206
query57	1409	1154	1051	1051
query58	214	204	193	193
query59	3935	4100	4229	4100
query60	207	214	202	202
query61	90	85	87	85
query62	833	461	464	461
query63	482	340	354	340
query64	2472	1491	1369	1369
query65	3652	3574	3547	3547
query66	758	389	370	370
query67	15780	15948	15959	15948
query68	8798	665	653	653
query69	561	334	345	334
query70	1439	1428	1503	1428
query71	413	296	314	296
query72	6528	3506	3387	3387
query73	735	324	331	324
query74	6269	5865	5800	5800
query75	5094	3758	3687	3687
query76	5363	1184	1187	1184
query77	909	276	258	258
query78	12729	11671	11671	11671
query79	7320	661	644	644
query80	1026	389	386	386
query81	498	233	237	233
query82	1490	97	100	97
query83	168	133	135	133
query84	263	69	69	69
query85	884	298	296	296
query86	338	337	291	291
query87	3225	3038	3026	3026
query88	4999	2360	2369	2360
query89	362	305	301	301
query90	1960	216	203	203
query91	169	133	148	133
query92	58	52	52	52
query93	3900	608	575	575
query94	716	208	201	201
query95	1091	1034	1048	1034
query96	650	340	341	340
query97	6462	6406	6420	6406
query98	184	183	177	177
query99	2950	841	842	841
Total cold run time: 308600 ms
Total hot run time: 203987 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e3479bdc641cc70e2b7adc4b90c0cdf71fda61b0, data reload: false

query1	0.02	0.02	0.02
query2	0.08	0.02	0.02
query3	0.25	0.04	0.04
query4	1.78	0.06	0.06
query5	0.52	0.53	0.52
query6	1.25	0.61	0.62
query7	0.01	0.02	0.02
query8	0.04	0.03	0.02
query9	0.53	0.48	0.50
query10	0.53	0.54	0.55
query11	0.11	0.08	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.61
query14	0.78	0.79	0.77
query15	0.78	0.76	0.76
query16	0.39	0.40	0.36
query17	0.98	1.00	0.99
query18	0.23	0.25	0.24
query19	1.92	1.86	1.76
query20	0.02	0.01	0.01
query21	15.46	0.57	0.55
query22	2.01	1.83	1.54
query23	17.59	1.04	1.01
query24	4.30	1.07	1.12
query25	0.37	0.09	0.06
query26	0.50	0.15	0.15
query27	0.05	0.03	0.03
query28	8.60	0.77	0.72
query29	12.75	2.40	2.36
query30	0.59	0.52	0.53
query31	2.81	0.38	0.38
query32	3.37	0.51	0.50
query33	3.05	3.09	3.03
query34	15.25	4.79	4.80
query35	4.86	4.86	4.86
query36	1.08	1.01	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.03	0.01	0.02
query43	0.03	0.02	0.01
Total cold run time: 103.99 s
Total hot run time: 30.82 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e3479bdc641cc70e2b7adc4b90c0cdf71fda61b0 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@GoGoWen GoGoWen merged commit 8a16dad into apache:branch-2.0 May 23, 2024
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants