Fix IOException in MSSQL pipeline by disposing config file watchers#3243
Open
aaronburtle wants to merge 13 commits intomainfrom
Open
Fix IOException in MSSQL pipeline by disposing config file watchers#3243aaronburtle wants to merge 13 commits intomainfrom
aaronburtle wants to merge 13 commits intomainfrom
Conversation
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
aaronburtle
requested review from
Alekhya-Polavarapu,
Aniruddh25,
JerryNixon,
RubenCerna2079,
anushakolan,
rusamant,
sourabh1007,
souvikghosh04,
stuartpa and
vadeveka
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses intermittent MSSQL pipeline test failures caused by lingering config file watchers holding file handles during test cleanup, leading to IOException on File.Delete. It introduces explicit disposal of config file watcher resources and adds a retry-based safety net in test cleanup.
Changes:
- Implement
IDisposableonConfigFileWatcherto stop events, unsubscribe handlers, and dispose the underlying watcher. - Implement
IDisposableonFileSystemRuntimeConfigLoaderso DI can dispose the ownedConfigFileWatcherwhen the test server shuts down. - Add exponential backoff retries around test config file deletion in
CleanupAfterEachTest.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/Service.Tests/Configuration/ConfigurationTests.cs | Adds retry/backoff around deleting the custom config file during test cleanup to reduce flakiness from transient file locks. |
| src/Config/FileSystemRuntimeConfigLoader.cs | Makes the loader disposable so the DI container can clean up the hot-reload watcher on shutdown. |
| src/Config/ConfigFileWatcher.cs | Makes the watcher disposable to stop notifications and release OS resources/file handles. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
souvikghosh04
approved these changes
Mar 13, 2026
Contributor
souvikghosh04
left a comment
souvikghosh04
left a comment
There was a problem hiding this comment.
Approved with a follow-up comment
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
Author
|
/azp run |
… of github.com:Azure/data-api-builder into dev/aaronburtle/MSSQLIntegrationTestPipelineCleanupFix
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Aniruddh25
requested changes
Mar 13, 2026
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
… of github.com:Azure/data-api-builder into dev/aaronburtle/MSSQLIntegrationTestPipelineCleanupFix
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 6 pipeline(s). |
Aniruddh25
reviewed
Mar 18, 2026
| // arriving before that completes will fail with "Initialization of metadata incomplete." | ||
| JsonElement reloadGQLContents = default; | ||
| bool querySucceeded = false; | ||
| for (int attempt = 1; attempt <= 10; attempt++) |
Collaborator
There was a problem hiding this comment.
Please mention this retry attempt added for GraphQL too over here in the description.
Aniruddh25
approved these changes
Mar 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why make this change?
Closes #2992
Several configuration tests create a TestServer that sets up a
ConfigFileWatcherto support hot-reload. When the file watcher detects a change, its callback reads the file, holding a read handle. NeitherConfigFileWatchernorFileSystemRuntimeConfigLoaderimplementedIDisposable, so when the TestServer is disposed, the DI container could not clean up the file watcher. The watcher remained active, and if its callback was mid-read whenCleanupAfterEachTestcalledFile.Delete, the delete failed with anIOExceptionWhat is this change?
ConfigFileWatchernow implementsIDisposable. It stops raising events, unsubscribes the Changed handler, and disposes the underlyingIFileSystemWatcher.FileSystemRuntimeConfigLoadernow implementsIDisposable. It disposes the ownedConfigFileWatcher. As a DI singleton, it is now automatically disposed when the TestServer shuts down.CleanupAfterEachTestretries with exponential backoff. As a safety net,File.Deleteis retried up to 3 times onIOException(2s, 4s, 8s delays), matching the existing retry pattern used elsewhere.How was this tested?
Run against the pipeline. Several pipeline runs were initiated to add certainty that the fix is working, since this issue is intermittent.
https://dev.azure.com/sqldab/Data%20API%20builder%20Dependency%20Packages/_build/results?buildId=18957&view=results
https://dev.azure.com/sqldab/Data%20API%20builder%20Dependency%20Packages/_build/results?buildId=18958&view=results
https://dev.azure.com/sqldab/Data%20API%20builder%20Dependency%20Packages/_build/results?buildId=18952&view=results
https://dev.azure.com/sqldab/Data%20API%20builder%20Dependency%20Packages/_build/results?buildId=18959&view=results
https://dev.azure.com/sqldab/Data%20API%20builder%20Dependency%20Packages/_build/results?buildId=18960&view=results