Skip to content

YARN-12049. Shut down the thread pool in FederationRMAdminInterceptor - #8887

Open
luoyuan3471 wants to merge 1 commit into
apache:trunkfrom
luoyuan3471:YARN-12049
Open

luoyuan3471 wants to merge 1 commit into
apache:trunkfrom
luoyuan3471:YARN-12049

Conversation

@luoyuan3471

@luoyuan3471 luoyuan3471 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem

FederationRMAdminInterceptor creates a per-user thread pool but inherits a
shutdown() method that only delegates to the next interceptor. Its own
executor is never shut down, so idle worker threads can remain alive when
core thread timeout is disabled.

Solution

  • Override shutdown() to shut down the executor before calling the parent method.
  • Guard against a null executor so shutdown is safe before initialization.
  • Add regression tests for worker termination, repeated shutdown, and shutdown before initialization.
  • Limit this change to the interceptor's thread-pool shutdown; service-stop, pipeline-wrapper, and finalization behavior are unchanged.

Validation

  • Local tests passed with JDK 17 and Maven 3.9.15.
  • TestFederationRMAdminInterceptor: 38 tests, 0 failures, 0 errors, 0 skipped.
  • The new thread-pool shutdown test fails without the production fix.
  • Surefire reports and git diff --check verified.

Related Jira

YARN-12049

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 21s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 28m 18s trunk passed
+1 💚 compile 0m 37s trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 compile 0m 37s trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 checkstyle 0m 35s trunk passed
+1 💚 mvnsite 0m 39s trunk passed
+1 💚 javadoc 0m 35s trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javadoc 0m 36s trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 spotbugs 0m 54s trunk passed
+1 💚 shadedclient 16m 29s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 17s the patch passed
+1 💚 compile 0m 18s the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javac 0m 18s the patch passed
+1 💚 compile 0m 18s the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 javac 0m 18s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 14s the patch passed
+1 💚 mvnsite 0m 19s the patch passed
+1 💚 javadoc 0m 16s the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚 javadoc 0m 17s the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚 spotbugs 0m 40s the patch passed
+1 💚 shadedclient 15m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 8m 53s hadoop-yarn-server-router in the patch passed.
+1 💚 asflicense 0m 24s The patch does not generate ASF License warnings.
79m 2s
Subsystem Report/Notes
Docker ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8887/1/artifact/out/Dockerfile
GITHUB PR #8887
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux ac2d58f507b4 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / dd3c530
Default Java Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8887/1/testReport/
Max. process+thread count 1305 (vs. ulimit of 10000)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8887/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants