Skip to content

YARN-12048. Fix keep-alive time units in Federation Router thread pools - #8885

Open
luoyuan3471 wants to merge 2 commits into
apache:trunkfrom
luoyuan3471:YARN-12048
Open

YARN-12048. Fix keep-alive time units in Federation Router thread pools#8885
luoyuan3471 wants to merge 2 commits into
apache:trunkfrom
luoyuan3471:YARN-12048

Conversation

@luoyuan3471

@luoyuan3471 luoyuan3471 commented Oct 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Problem

FederationClientInterceptor and FederationRMAdminInterceptor read the
thread-pool keep-alive duration in seconds but pass it to ThreadPoolExecutor
as milliseconds. A configured 60s becomes 60ms, while 500ms is truncated
to zero.

The Java fallback is also zero, whereas yarn-default.xml specifies 30s.

Solution

  • Read the duration in milliseconds while retaining seconds as the default unit for values without a suffix.
  • Apply the same correction to both interceptors.
  • Express the Java fallback in seconds and set it to 30 to match the existing XML default. This changes the fallback from zero to 30 seconds when the property is absent and default resources are not loaded; explicitly configured zero remains zero.
  • Add focused regression coverage for both interceptors, including missing configuration, the XML default, explicit zero, unit suffixes, and suffix-free values.
  • Clarify time-unit suffixes and suffix-free values in the configuration documentation, and align the Federation guide with the 30s XML default.

Validation

  • Local API and Router reactor compilation passed with JDK 17 and Maven 3.9.15.
  • TestFederationInterceptorThreadPool: 8 tests, 0 failures, 0 errors, 0 skipped.
  • Test results and git diff --check verified.
  • Squashing the commits preserved the tested file tree exactly.

Related Jira

YARN-12048

@luoyuan3471 luoyuan3471 mentioned this pull request Oct 29, 2026
Closed
@hadoop-yetus

hadoop-yetus commented Oct 29, 2026

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 24s #8885 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.
Subsystem Report/Notes
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/1/console
versions git=2.34.1
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@luoyuan3471
luoyuan3471 marked this pull request as ready for review October 29, 2026 09:27
@hadoop-yetus

hadoop-yetus commented Oct 29, 2026

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 22s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 1m 57s Maven dependency ordering for branch
+1 💚 mvninstall 28m 24s trunk passed
+1 💚 compile 3m 46s trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 compile 3m 30s trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 checkstyle 1m 19s trunk passed
+1 💚 mvnsite 2m 15s trunk passed
+1 💚 javadoc 2m 18s trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javadoc 2m 22s trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+0 🆗 spotbugs 0m 26s branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 17m 31s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 1m 13s the patch passed
+1 💚 compile 3m 22s the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javac 3m 22s the patch passed
+1 💚 compile 3m 41s the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 javac 3m 41s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 22s the patch passed
+1 💚 mvnsite 2m 11s the patch passed
-1 ❌ javadoc 0m 27s /results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-21.0.12.1+1-1-24.04.4-Ubuntu.txt hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-21.0.12.1+1-1-24.04.4-Ubuntu with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu generated 1 new + 695 unchanged - 0 fixed = 696 total (was 695)
-1 ❌ javadoc 0m 25s /results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-17.0.20.1+1-1-24.04-Ubuntu.txt hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-17.0.20.1+1-1-24.04-Ubuntu with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu generated 1 new + 674 unchanged - 0 fixed = 675 total (was 674)
+0 🆗 spotbugs 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs
+1 💚 shadedclient 17m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 45s hadoop-yarn-api in the patch passed.
+1 💚 unit 4m 30s hadoop-yarn-common in the patch passed.
+1 💚 unit 9m 34s hadoop-yarn-server-router in the patch passed.
+1 💚 unit 0m 18s hadoop-yarn-site in the patch passed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
125m 12s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/3/artifact/out/Dockerfile
GITHUB PR #8885
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint
uname Linux 7d0a6e079a2d 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1bc14fb
Default Java Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/3/testReport/
Max. process+thread count 1303 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/3/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

hadoop-yetus commented Oct 29, 2026

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 13m 25s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 1m 56s Maven dependency ordering for branch
+1 💚 mvninstall 43m 30s trunk passed
+1 💚 compile 6m 35s trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 compile 6m 36s trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 checkstyle 2m 27s trunk passed
+1 💚 mvnsite 3m 1s trunk passed
+1 💚 javadoc 3m 5s trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javadoc 3m 0s trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+0 🆗 spotbugs 0m 52s branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 29m 24s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 1m 24s the patch passed
+1 💚 compile 5m 50s the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javac 5m 50s the patch passed
+1 💚 compile 6m 28s the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 javac 6m 28s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 2m 20s the patch passed
+1 💚 mvnsite 2m 43s the patch passed
-1 ❌ javadoc 0m 51s /results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-21.0.12.1+1-1-24.04.4-Ubuntu.txt hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-21.0.12.1+1-1-24.04.4-Ubuntu with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu generated 1 new + 695 unchanged - 0 fixed = 696 total (was 695)
-1 ❌ javadoc 0m 51s /results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-17.0.20.1+1-1-24.04-Ubuntu.txt hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-17.0.20.1+1-1-24.04-Ubuntu with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu generated 1 new + 674 unchanged - 0 fixed = 675 total (was 674)
+0 🆗 spotbugs 0m 42s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs
+1 💚 shadedclient 28m 56s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 59s hadoop-yarn-common in the patch passed.
+1 💚 unit 11m 43s hadoop-yarn-server-router in the patch passed.
+1 💚 unit 0m 41s hadoop-yarn-site in the patch passed.
+1 💚 asflicense 1m 9s The patch does not generate ASF License warnings.
201m 29s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/2/artifact/out/Dockerfile
GITHUB PR #8885
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle markdownlint
uname Linux 3d291289bce7 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 97c7697
Default Java Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/2/testReport/
Max. process+thread count 1306 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8885/2/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@luoyuan-commits
YARN-12048. Fix keep-alive time units in Federation Router thread pools
1fcb643 
Follow-up: keep the new executor accessor package-private and move its regression test into the same package. This avoids a new Javadoc warning without exposing the executor publicly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

trunk YARN

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@luoyuan3471 @hadoop-yetus @luoyuan-commits