Repository navigation
LTAR-420: report the failed bstackStep as the test failure; skip setSessionStatus executor on LTS - #4
Merged
MihirR-BS merged 2 commits intoOct 22, 2026
Conversation
…nStatus executor on LTS LCNC specs throw their own Error from the step helper, which Nightwatch keeps out of the command list. The first failed command was then an internal selector probe (executeAsyncScript "script timeout") whose result has no message or stack, so TestRunFinished shipped failure.backtrace ['', ''] and TestHub could not cluster unique errors. When no command failed at all, the test was reported as passed. On LTS runs the last failed step in global.__bstack_steps is now the authoritative failure. The setSessionStatus browserstack_executor call is skipped on LTS runs: the LTS grid does not intercept it, Chrome evaluates it as JS and it fails with "Unexpected token ':'" after every test.
LTS runs no longer send the setSessionStatus executor, so the test's pause on browser.execute never happened and the finish completed before the signal. afterEachExecution is the handler's last await before the stop check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Regression (rengg-lts) counterpart of nightwatchjs#66, for LTAR-420. Same two LTS-only changes, cherry-picked onto
lts-nightwatch-rengg-lts:bstackStep()as the test failure. LCNC specs throw their ownErrorfrom the step helper, which Nightwatch keeps out oftestcase.commands.sendTestRunEventthen took an internal selector probe (executeAsyncScript"script timeout", result{status: -1, value: null}) as the failure and uploadedfailure: [{backtrace: ['', '']}], so TestHub could not cluster unique errors; with no failed command the test was reportedpassed. On LTS runs the last failed step inglobal.__bstack_stepsis now the failure.setSessionStatusexecutor on LTS runs. The LTS grid does not interceptbrowserstack_executor; Chrome evaluates it as JS and fails withUnexpected token ':'after every test, which then lands in failure evidence as a script syntax error.Differences from nightwatchjs#66
src/utils/helper.js: resolved alongside LTAR-392'sshouldInstallLtsStopPath(both helpers kept).test/src/test-observability/ltsStopPath.js(LTAR-392, branch-only): "suppresses a TestRunFinished already past its entry check" held the finish in flight on thebrowser.executesession-status call, which LTS runs no longer make. It now holds atafterEachExecution, the handler's last await before theltsStoppingcheck. The guard itself is unchanged. The same adjustment will be needed upstream if LTS: flush and finish open runs as interrupted on SIGTERM nightwatchjs/nightwatch-plugin-browserstack#65 and LTS: report the failed bstackStep as the test failure; skip setSessionStatus executor on LTS nightwatchjs/nightwatch-plugin-browserstack#66 both land.Verification
npm test: 144 passing (136 on the base branch + 8 new). The one failure (getBuildName→ "returns empty for empty options") fails identically on the base branch; it asserts the checkout directory name.eslint nightwatch src test --quiet: clean.failedwith the step's "No selector matched…" message instead of['', '']; clean probes →failedinstead ofpassed; passing test → noUnexpected tokenerrors; non-LTS runs unchanged.