Repository navigation
[action] [PR:28531] [system_health]: Adapt process-exit test wait to platform polling_interval - #1486
Merged
Conversation
…erval
### Description of PR
Summary:
`test_service_checker_with_process_exit` hard-coded a 180s wait (`WAIT_TIMEOUT`) for `healthd` to record a stopped critical process in STATE_DB `SYSTEM_HEALTH_INFO`. On BMC platforms (e.g. Nokia-7220-Th6p-2-bmc / `arm64-nokia_bmc_h6_128-r0`), `healthd` polls on a 300s interval and reloads config only on the next loop. Because 180s < the 300s poll interval, the record never appeared in time and the test failed with `AssertionError: <process> is not recorded`.
The fix is **BMC-specific**: only on BMC platforms (`duthost.is_bmc()`) is the wait derived from the real config via `get_system_health_config(duthost, 'polling_interval', FAST_INTERVAL)` and set to `max(WAIT_TIMEOUT, polling_interval * 2)` so a poll loop boundary is guaranteed to be crossed. All other platforms keep the original 180s `WAIT_TIMEOUT` behavior unchanged. This mirrors the `duthost.is_bmc()` conditional already used by `test_external_checker`, and `WAIT_TIMEOUT` is unchanged globally.
Fixes # (issue)
### Type of change
- [x] Bug fix
- [ ] Testbed and Framework(new/improvement)
- [ ] New Test case
- [ ] Skipped for non-supported platforms
- [ ] Test case improvement
### Back port request
- [ ] 202311
- [ ] 202405
- [ ] 202411
- [ ] 202505
- [ ] 202511
- [ ] 202512
- [ ] 202605
- [x] 202608
Tracking issue/work item for backport/cherry-pick request (GitHub issue or Microsoft ADO):
Failure type: regression
### Tested branch
- [x] master
- [ ] 202311
- [ ] 202405
- [ ] 202411
- [ ] 202505
- [ ] 202511
- [ ] 202512
- [ ] 202605
- [ ] N/A
### Test result
### Approach
#### What is the motivation for this PR?
`test_service_checker_with_process_exit` was failing on BMC platforms because its hard-coded 180s wait is shorter than the platform's 300s `healthd` polling_interval, so the stopped process was never recorded before the wait expired (`AssertionError: <process> is not recorded`).
#### How did you do it?
Gate the longer wait behind `duthost.is_bmc()`. On BMC, read the platform's configured poll interval with `get_system_health_config(duthost, 'polling_interval', FAST_INTERVAL)` and use `wait_timeout = max(WAIT_TIMEOUT, polling_interval * 2)` (two poll loops plus margin, never shorter than the previous default). Non-BMC platforms fall through to the original `WAIT_TIMEOUT`. This follows the same `duthost.is_bmc()` pattern already used by `test_external_checker`. `WAIT_TIMEOUT` is left unchanged for other tests.
#### How did you verify/test it?
`python -m py_compile` and `flake8 --max-line-length=120` both pass clean on the file. The logic mirrors the already-passing `test_external_checker` (BMC conditional) and `test_service_checker` (polling_interval-derived wait).
#### Any platform specific information?
Only BMC platforms where `healthd` polling_interval defaults to 300s (e.g. Nokia-7220-Th6p-2-bmc / `arm64-nokia_bmc_h6_128-r0`) are affected. Non-BMC platforms retain the exact previous 180s wait behavior.
#### Supported testbed topology if it's a new test case?
N/A — existing test.
### Documentation
N/A
Signed-off-by: Sonic Build Admin <sonicbld@microsoft.com>
Collaborator
Author
|
Original PR: sonic-net/sonic-mgmt#28531 |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Collaborator
Author
|
/azp run |
1 similar comment
Collaborator
Author
|
/azp run |
|
Azure Pipelines: 1 pipeline(s) were filtered out due to trigger conditions. |
3 of 22 tasks
|
Azure Pipelines: 1 pipeline(s) were filtered out due to trigger conditions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
test_service_checker_with_process_exithard-coded a 180s wait (WAIT_TIMEOUT) forhealthdto record a stopped critical process in STATE_DBSYSTEM_HEALTH_INFO. On BMC platforms (e.g. Nokia-7220-Th6p-2-bmc /arm64-nokia_bmc_h6_128-r0),healthdpolls on a 300s interval and reloads config only on the next loop. Because 180s < the 300s poll interval, the record never appeared in time and the test failed withAssertionError: <process> is not recorded.The fix is BMC-specific: only on BMC platforms (
duthost.is_bmc()) is the wait derived from the real config viaget_system_health_config(duthost, 'polling_interval', FAST_INTERVAL)and set tomax(WAIT_TIMEOUT, polling_interval * 2)so a poll loop boundary is guaranteed to be crossed. All other platforms keep the original 180sWAIT_TIMEOUTbehavior unchanged. This mirrors theduthost.is_bmc()conditional already used bytest_external_checker, andWAIT_TIMEOUTis unchanged globally.Fixes # (issue)
Type of change
Back port request
Tracking issue/work item for backport/cherry-pick request (GitHub issue or Microsoft ADO):
Failure type: regression
Tested branch
Test result
Approach
What is the motivation for this PR?
test_service_checker_with_process_exitwas failing on BMC platforms because its hard-coded 180s wait is shorter than the platform's 300shealthdpolling_interval, so the stopped process was never recorded before the wait expired (AssertionError: <process> is not recorded).How did you do it?
Gate the longer wait behind
duthost.is_bmc(). On BMC, read the platform's configured poll interval withget_system_health_config(duthost, 'polling_interval', FAST_INTERVAL)and usewait_timeout = max(WAIT_TIMEOUT, polling_interval * 2)(two poll loops plus margin, never shorter than the previous default). Non-BMC platforms fall through to the originalWAIT_TIMEOUT. This follows the sameduthost.is_bmc()pattern already used bytest_external_checker.WAIT_TIMEOUTis left unchanged for other tests.How did you verify/test it?
python -m py_compileandflake8 --max-line-length=120both pass clean on the file. The logic mirrors the already-passingtest_external_checker(BMC conditional) andtest_service_checker(polling_interval-derived wait).Any platform specific information?
Only BMC platforms where
healthdpolling_interval defaults to 300s (e.g. Nokia-7220-Th6p-2-bmc /arm64-nokia_bmc_h6_128-r0) are affected. Non-BMC platforms retain the exact previous 180s wait behavior.Supported testbed topology if it's a new test case?
N/A — existing test.
Documentation
N/A
Signed-off-by: Sonic Build Admin sonicbld@microsoft.com