HIVE-29708: HMS Memory leak: Ensure FileSystem.closeAllForUGI is invoked on proxy user failure

[Indhumathi27]

What changes were proposed in this pull request?

This patch fixes a FileSystem.CACHE leak in TxnUtils.findUserToRunAs().

When the Hive Metastore service user cannot access a table location, findUserToRunAs() creates a proxy user and attempts to access the path using the proxy user's FileSystem. If this access check throws an exception (for example, due to insufficient HDFS permissions), the cached FileSystem instance associated with the proxy user is never released because FileSystem.closeAllForUGI() is not invoked.

This patch ensures that FileSystem.closeAllForUGI(ugi) is always executed by moving the cleanup into a finally block, preventing leaked DistributedFileSystem instances regardless of whether the access check succeeds or fails.

A similar code path exists in the compactor Worker & MRCompactor, where proxy FileSystem instances could also be left cached on exception. This patch applies the same cleanup there as well to ensure consistent resource handling across both the Initiator and Worker.

Why are the changes needed?

When the proxy user access check fails, the cached DistributedFileSystem remains referenced by the static FileSystem.CACHE. During repeated compactor initiator scans, these cached instances accumulate, leading to increasing heap usage and eventually an OutOfMemoryError in the Hive Metastore.

Ensuring cleanup on all execution paths prevents the cache leak while preserving the existing behavior of determining the appropriate user to run compactions.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Manually tested. Unit test cases also added

[Aggarwal-Raghav]

@Indhumathi27 , The same fix can be applied on MRCompactor as well?

hive/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MRCompactor.java

Lines 127 to 140 in be59bd0

	UserGroupInformation ugi = UserGroupInformation.createProxyUser(context.getCompactionInfo().runAs,
	UserGroupInformation.getLoginUser());
	ugi.doAs((PrivilegedExceptionAction<Object>) () -> {
	run(context.getConf(), context.getTable(), context.getSd(),
	context.getValidWriteIdList(), context.getCompactionInfo(), context.getAcidDirectory());
	return null;
	});
	try {
	FileSystem.closeAllForUGI(ugi);
	} catch (IOException exception) {
	LOG.error("Could not clean up file-system handles for UGI: " + ugi + " for " + context.getCompactionInfo().getFullPartitionName(),
	exception);
	}
	}

[sonarqubecloud]

Quality Gate passed

Issues
163 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.3% Duplication on New Code

See analysis details on SonarQube Cloud