[improve](metrics) Export file cache remote index byte metrics by hoshinojyunn · Pull Request #65398 · apache/doris

hoshinojyunn · 2026-07-08T17:06:56Z

Issue Number: None

Related PR: None

Problem Summary:

File cache statistics already track the remote-read bytes for inverted index and segment footer
index data, but BE Doris metrics only export the aggregate cache/local/remote/peer byte counters.
As a result, these two index-specific remote read costs are visible in query profile internals but
not exposed from the BE /metrics endpoint, which makes them harder to monitor and alert on in
cloud/file-cache scenarios.

This PR exposes the following two BE metrics:

doris_be_inverted_index_bytes_read_from_remote
doris_be_segment_footer_index_bytes_read_from_remote

The change keeps the existing file cache metrics aggregation path and extends it in three places:

Register the two counters in DorisMetrics
Extend FileCacheMetrics::AtomicStatistics and the aggregation/update path to accumulate them
Export them from the existing metrics hook and cover the behavior with a BE unit test

Release note

Add two new BE metrics for file cache remote index read bytes:

doris_be_inverted_index_bytes_read_from_remote
doris_be_segment_footer_index_bytes_read_from_remote

Check List (For Author)

Test
- Regression test
  - ./run-regression-test.sh --run -d cloud_p0 -s test_file_cache_remote_index_byte_metrics -runMode=cloud
- Unit Test
  - ./run-be-ut.sh --run --filter=DorisMetricsTest.*:FileCacheProfileReporterTest.*
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-07-08T17:10:05Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

hoshinojyunn · 2026-07-08T17:13:41Z

run buildall

hoshinojyunn · 2026-07-08T17:25:34Z

run buildall

hello-stephen · 2026-07-08T18:53:47Z

TPC-H: Total hot run time: 29891 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da72c65c497598542fd89c30fb6bab9510eb0e55, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17589	4150	4107	4107
q2	1995	371	212	212
q3	10284	1529	851	851
q4	4719	515	346	346
q5	7708	874	563	563
q6	225	182	144	144
q7	795	843	618	618
q8	10376	1480	1544	1480
q9	6266	4430	4398	4398
q10	6811	1831	1554	1554
q11	526	357	310	310
q12	711	587	443	443
q13	18164	3451	2719	2719
q14	279	276	241	241
q15	q16	793	789	703	703
q17	952	994	1089	994
q18	6988	5886	5684	5684
q19	1189	1294	1086	1086
q20	775	640	524	524
q21	5921	2682	2611	2611
q22	450	374	303	303
Total cold run time: 103516 ms
Total hot run time: 29891 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4509	4467	4448	4448
q2	306	350	226	226
q3	4741	5063	4466	4466
q4	2359	2418	1386	1386
q5	4687	4565	5057	4565
q6	286	223	151	151
q7	2143	1841	1624	1624
q8	2695	2361	2346	2346
q9	8187	8011	8135	8011
q10	5059	5135	4345	4345
q11	624	455	398	398
q12	819	803	567	567
q13	3266	3922	2968	2968
q14	321	327	283	283
q15	q16	795	751	659	659
q17	1428	1399	1356	1356
q18	8458	7578	6986	6986
q19	1079	1057	1043	1043
q20	2267	2242	1948	1948
q21	5703	4903	4834	4834
q22	566	498	417	417
Total cold run time: 60298 ms
Total hot run time: 53027 ms

hello-stephen · 2026-07-08T19:05:18Z

TPC-DS: Total hot run time: 181241 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da72c65c497598542fd89c30fb6bab9510eb0e55, data reload: false

query5	4355	632	479	479
query6	459	215	200	200
query7	4833	548	341	341
query8	334	188	174	174
query9	8787	4149	4134	4134
query10	478	359	314	314
query11	5927	2354	2160	2160
query12	152	105	100	100
query13	1268	630	446	446
query14	6554	5315	4973	4973
query14_1	4326	4343	4330	4330
query15	213	201	182	182
query16	1056	495	479	479
query17	1154	738	598	598
query18	2716	510	363	363
query19	213	189	155	155
query20	119	118	108	108
query21	232	159	138	138
query22	13592	13604	13425	13425
query23	17301	16488	16303	16303
query23_1	16259	16347	16378	16347
query24	7520	1801	1296	1296
query24_1	1333	1315	1325	1315
query25	584	470	392	392
query26	1189	345	207	207
query27	2589	625	395	395
query28	4441	2064	2045	2045
query29	1100	638	514	514
query30	342	261	229	229
query31	1114	1104	990	990
query32	111	74	71	71
query33	549	338	263	263
query34	1188	1134	675	675
query35	773	799	669	669
query36	1425	1401	1277	1277
query37	154	110	95	95
query38	1869	1705	1642	1642
query39	923	932	903	903
query39_1	886	871	910	871
query40	243	167	135	135
query41	66	61	65	61
query42	93	91	91	91
query43	324	328	287	287
query44	1444	784	781	781
query45	204	183	177	177
query46	1093	1230	774	774
query47	2364	2357	2311	2311
query48	408	379	298	298
query49	586	463	319	319
query50	998	429	331	331
query51	10869	10723	10844	10723
query52	84	90	77	77
query53	257	281	207	207
query54	274	232	222	222
query55	75	74	65	65
query56	291	276	282	276
query57	1425	1402	1348	1348
query58	279	277	260	260
query59	1563	1631	1453	1453
query60	327	269	251	251
query61	161	151	153	151
query62	714	655	592	592
query63	244	201	210	201
query64	2488	1032	877	877
query65	4859	4793	4820	4793
query66	1720	516	392	392
query67	29540	29406	29298	29298
query68	3347	1512	962	962
query69	452	309	273	273
query70	1083	1004	985	985
query71	356	334	310	310
query72	3089	2731	2307	2307
query73	848	803	440	440
query74	5142	4931	4750	4750
query75	2605	2584	2239	2239
query76	2315	1186	787	787
query77	360	385	293	293
query78	12431	12242	11923	11923
query79	1446	1232	778	778
query80	1273	531	474	474
query81	538	332	287	287
query82	596	156	126	126
query83	362	321	294	294
query84	278	157	129	129
query85	936	625	506	506
query86	443	287	290	287
query87	1828	1824	1767	1767
query88	3755	2794	2796	2794
query89	477	402	358	358
query90	1883	206	204	204
query91	204	189	163	163
query92	66	62	59	59
query93	1687	1640	1064	1064
query94	739	360	310	310
query95	772	496	485	485
query96	1063	759	342	342
query97	2697	2679	2577	2577
query98	228	207	201	201
query99	1134	1165	1034	1034
Total cold run time: 266311 ms
Total hot run time: 181241 ms

hello-stephen · 2026-07-08T19:10:19Z

ClickBench: Total hot run time: 25.24 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da72c65c497598542fd89c30fb6bab9510eb0e55, data reload: false

query1	0.00	0.00	0.00
query2	0.09	0.05	0.05
query3	0.26	0.14	0.13
query4	1.61	0.14	0.15
query5	0.24	0.22	0.23
query6	1.23	1.05	1.14
query7	0.04	0.01	0.00
query8	0.05	0.03	0.03
query9	0.37	0.33	0.32
query10	0.55	0.57	0.54
query11	0.19	0.14	0.14
query12	0.19	0.14	0.14
query13	0.47	0.48	0.47
query14	1.02	1.01	1.00
query15	0.63	0.60	0.60
query16	0.31	0.34	0.33
query17	1.13	1.17	1.14
query18	0.23	0.22	0.22
query19	2.03	1.99	1.94
query20	0.02	0.01	0.02
query21	15.43	0.19	0.14
query22	4.94	0.06	0.05
query23	16.14	0.30	0.12
query24	2.89	0.41	0.33
query25	0.11	0.06	0.04
query26	0.71	0.20	0.14
query27	0.05	0.04	0.03
query28	3.54	0.96	0.56
query29	12.45	4.08	3.28
query30	0.28	0.15	0.16
query31	2.77	0.60	0.31
query32	3.22	0.60	0.50
query33	3.19	3.26	3.29
query34	15.68	4.23	3.52
query35	3.58	3.55	3.59
query36	0.56	0.42	0.43
query37	0.09	0.06	0.07
query38	0.04	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.17	0.16
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.72 s
Total hot run time: 25.24 s

Issue Number: None Related PR: None Problem Summary: File cache profile statistics already track `inverted_index_bytes_read_from_remote` and `segment_footer_index_bytes_read_from_remote`, but BE Doris metrics only exported aggregate cache, remote, and peer read bytes. That made these index-specific remote read costs visible in query profile only, not in the BE metrics endpoint. This change registers two new Doris metrics, extends the existing `FileCacheMetrics` aggregation path to accumulate them, and exports them through the normal metrics hook. The update also extends the existing file cache profile unit test helpers so segment footer index statistics are covered and adds a test for the new metric export path. Expose `inverted_index_bytes_read_from_remote` and `segment_footer_index_bytes_read_from_remote` in BE Doris metrics. - Test: Manual compile verification - Started `./run-be-ut.sh --run --filter=FileCacheProfileReporterTest.*`, but it triggered a large `doris_be_test` rebuild and was interrupted after the changed objects compiled successfully - Recompiled changed objects with `ninja -C be/ut_build_ASAN src/common/CMakeFiles/Common.dir/metrics/doris_metrics.cpp.o src/io/CMakeFiles/IO.dir/cache/block_file_cache_profile.cpp.o test/CMakeFiles/doris_be_test.dir/io/cache/block_file_cache_profile_reporter_test.cpp.o` - Behavior changed: Yes (adds two new BE metrics for file cache remote index bytes) - Does this need documentation: No

hoshinojyunn · 2026-07-09T02:22:53Z

run buildall

hello-stephen · 2026-07-09T05:22:55Z

TPC-H: Total hot run time: 29644 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 09d5f0fcfb46cc31a31a4b40aac7f17f94a8b153, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17641	4165	4085	4085
q2	2054	331	208	208
q3	10252	1429	822	822
q4	4685	470	351	351
q5	7553	861	602	602
q6	181	171	135	135
q7	774	814	624	624
q8	9353	1593	1566	1566
q9	5591	4373	4356	4356
q10	6759	1808	1531	1531
q11	511	370	318	318
q12	714	568	427	427
q13	18111	3380	2737	2737
q14	274	267	246	246
q15	q16	782	778	721	721
q17	1007	1050	980	980
q18	7036	5790	5516	5516
q19	1296	1270	1063	1063
q20	739	659	573	573
q21	5981	2604	2489	2489
q22	453	357	294	294
Total cold run time: 101747 ms
Total hot run time: 29644 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4456	4370	4334	4334
q2	288	324	219	219
q3	4570	4939	4464	4464
q4	2107	2154	1374	1374
q5	4469	4309	4268	4268
q6	228	177	127	127
q7	1741	2065	1810	1810
q8	2525	2264	2185	2185
q9	8167	8126	7877	7877
q10	4762	4677	4313	4313
q11	559	406	380	380
q12	743	764	541	541
q13	3278	3641	2987	2987
q14	307	307	269	269
q15	q16	715	727	654	654
q17	1315	1336	1341	1336
q18	7888	7275	7272	7272
q19	1131	1147	1138	1138
q20	2197	2185	1915	1915
q21	5253	4565	4475	4475
q22	528	457	411	411
Total cold run time: 57227 ms
Total hot run time: 52349 ms

hello-stephen · 2026-07-09T05:34:15Z

TPC-DS: Total hot run time: 180696 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 09d5f0fcfb46cc31a31a4b40aac7f17f94a8b153, data reload: false

query5	4341	653	486	486
query6	460	234	202	202
query7	4881	595	334	334
query8	341	196	175	175
query9	8776	4046	4039	4039
query10	468	374	312	312
query11	5902	2364	2142	2142
query12	163	106	101	101
query13	1315	618	429	429
query14	6293	5334	5000	5000
query14_1	4316	4323	4290	4290
query15	218	213	185	185
query16	1082	476	475	475
query17	1152	745	597	597
query18	2739	485	368	368
query19	216	194	161	161
query20	115	113	106	106
query21	231	160	134	134
query22	13739	13602	13482	13482
query23	17457	16596	16221	16221
query23_1	16209	16298	16282	16282
query24	7673	1740	1309	1309
query24_1	1329	1334	1300	1300
query25	576	491	367	367
query26	1301	361	204	204
query27	2511	584	355	355
query28	4363	1982	1992	1982
query29	1053	605	480	480
query30	337	262	234	234
query31	1116	1092	982	982
query32	106	63	69	63
query33	527	322	257	257
query34	1160	1090	664	664
query35	762	782	668	668
query36	1377	1378	1231	1231
query37	156	101	92	92
query38	1883	1704	1614	1614
query39	919	906	903	903
query39_1	884	875	879	875
query40	244	163	143	143
query41	68	68	63	63
query42	91	91	89	89
query43	316	319	281	281
query44	1402	783	761	761
query45	217	209	192	192
query46	1060	1244	720	720
query47	2327	2366	2264	2264
query48	374	419	299	299
query49	580	436	307	307
query50	1060	403	316	316
query51	10646	10983	10721	10721
query52	85	88	74	74
query53	266	278	198	198
query54	270	233	218	218
query55	73	70	66	66
query56	299	296	292	292
query57	1440	1407	1293	1293
query58	277	254	267	254
query59	1579	1645	1490	1490
query60	302	262	255	255
query61	154	153	151	151
query62	691	656	587	587
query63	250	201	206	201
query64	2816	1048	958	958
query65	4883	4761	4761	4761
query66	1792	534	412	412
query67	29621	29576	29325	29325
query68	3286	1608	895	895
query69	423	328	270	270
query70	1083	974	967	967
query71	377	341	307	307
query72	3110	2736	2411	2411
query73	859	748	435	435
query74	5144	4937	4767	4767
query75	2626	2607	2223	2223
query76	2353	1200	778	778
query77	353	383	287	287
query78	12268	12448	11700	11700
query79	1386	1140	731	731
query80	1287	539	453	453
query81	561	333	285	285
query82	598	159	126	126
query83	364	314	292	292
query84	285	156	131	131
query85	999	605	515	515
query86	446	299	290	290
query87	1851	1830	1754	1754
query88	3725	2803	2808	2803
query89	457	396	351	351
query90	1994	203	197	197
query91	203	192	161	161
query92	65	62	60	60
query93	1535	1482	926	926
query94	728	345	313	313
query95	798	474	563	474
query96	1030	760	372	372
query97	2657	2678	2545	2545
query98	223	210	197	197
query99	1156	1191	1024	1024
Total cold run time: 266247 ms
Total hot run time: 180696 ms

hello-stephen · 2026-07-09T05:39:16Z

ClickBench: Total hot run time: 25.15 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 09d5f0fcfb46cc31a31a4b40aac7f17f94a8b153, data reload: false

query1	0.01	0.01	0.00
query2	0.08	0.05	0.06
query3	0.25	0.13	0.14
query4	1.61	0.14	0.14
query5	0.24	0.22	0.22
query6	1.24	1.05	1.03
query7	0.04	0.01	0.01
query8	0.07	0.04	0.04
query9	0.40	0.31	0.31
query10	0.57	0.60	0.55
query11	0.19	0.16	0.14
query12	0.18	0.15	0.14
query13	0.47	0.49	0.47
query14	1.04	1.01	1.01
query15	0.63	0.60	0.58
query16	0.32	0.34	0.32
query17	1.17	1.12	1.10
query18	0.24	0.21	0.21
query19	2.05	2.00	1.96
query20	0.02	0.02	0.01
query21	15.47	0.24	0.14
query22	4.80	0.06	0.05
query23	16.09	0.31	0.12
query24	3.00	0.42	0.34
query25	0.12	0.05	0.05
query26	0.74	0.20	0.16
query27	0.05	0.04	0.03
query28	3.50	0.92	0.55
query29	12.56	4.13	3.29
query30	0.28	0.16	0.16
query31	2.77	0.62	0.31
query32	3.22	0.59	0.49
query33	3.21	3.17	3.19
query34	15.64	4.20	3.55
query35	3.54	3.52	3.58
query36	0.56	0.45	0.43
query37	0.09	0.07	0.07
query38	0.05	0.03	0.04
query39	0.04	0.03	0.04
query40	0.18	0.17	0.15
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.9 s
Total hot run time: 25.15 s

hello-stephen · 2026-07-09T07:37:51Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (14/14) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	74.82% (29980/40071)
Line Coverage	58.87% (328789/558485)
Region Coverage	55.56% (275312/495547)
Branch Coverage	56.95% (121492/213340)

hoshinojyunn requested review from Gabriel39, gavinchou, liaoxin01 and morningman as code owners July 8, 2026 17:06

hoshinojyunn force-pushed the feat/profile branch from f128c74 to da72c65 Compare July 8, 2026 17:21

hoshinojyunn force-pushed the feat/profile branch from da72c65 to 09d5f0f Compare July 9, 2026 02:16

hoshinojyunn changed the title ~~[improvement](metrics) Export file cache remote index byte metrics~~ [improve](metrics) Export file cache remote index byte metrics Jul 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[improve](metrics) Export file cache remote index byte metrics#65398

[improve](metrics) Export file cache remote index byte metrics#65398
hoshinojyunn wants to merge 1 commit into
apache:masterfrom
hoshinojyunn:feat/profile

hoshinojyunn commented Jul 8, 2026 •

edited

Loading

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hoshinojyunn commented Jul 8, 2026

Uh oh!

hoshinojyunn commented Jul 8, 2026

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hoshinojyunn commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

hoshinojyunn commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hoshinojyunn commented Jul 8, 2026

Uh oh!

hoshinojyunn commented Jul 8, 2026

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hello-stephen commented Jul 8, 2026

Uh oh!

hoshinojyunn commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

Uh oh!

hello-stephen commented Jul 9, 2026

BE Regression && UT Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hoshinojyunn commented Jul 8, 2026 •

edited

Loading