Skip to content

Changelog verification#2580

Open
anannya03 wants to merge 7 commits into
microsoft:mainfrom
anannya03:changelog_verify
Open

Changelog verification#2580
anannya03 wants to merge 7 commits into
microsoft:mainfrom
anannya03:changelog_verify

Conversation

@anannya03
Copy link
Copy Markdown
Contributor

@anannya03 anannya03 commented May 5, 2026

What does this PR do?

Adds changelog verification so PRs must either include a changelog entry or carry the skip-changelog label.

This change updates the Build pipeline to authenticate to GitHub for PR repos, invokes a new Test-ChangelogEntry.ps1 validation step from Analyze-Code.ps1, and adds the new script that checks changed files for changelog entries and falls back to validating the PR label through the GitHub REST API.

GitHub issue number?

#2495

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@anannya03 anannya03 marked this pull request as ready for review May 11, 2026 21:43
Copilot AI review requested due to automatic review settings May 11, 2026 21:43
@anannya03 anannya03 requested review from a team as code owners May 11, 2026 21:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds CI enforcement for changelog hygiene by requiring either a changelog entry in the PR diff or an explicit skip-changelog PR label, including GitHub authentication in PR pipelines to enable label checks.

Changes:

  • Introduces Test-ChangelogEntry.ps1 to detect changelog entry files and/or validate skip-changelog via GitHub REST API.
  • Hooks the new validation into Analyze-Code.ps1 and fails analysis when missing both entry and label.
  • Updates the analyze pipeline job to authenticate to GitHub for PR repos.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
eng/scripts/Test-ChangelogEntry.ps1 Adds the changelog/label validation script using git diff + GitHub API label lookup
eng/scripts/Analyze-Code.ps1 Executes changelog validation and turns failures into analysis errors
eng/pipelines/templates/jobs/analyze.yml Adds GitHub login step for PR repos so label checks can authenticate


Push-Location $RepoRoot
try {
$changedFiles = git diff --name-only --diff-filter=A "origin/main...HEAD" 2>&1
@@ -0,0 +1,56 @@
#!/bin/env pwsh
Write-Host "PR #$prNumber labels: $($prLabels -join ', ')"
}
catch {
Write-Warning "Failed to fetch PR labels from GitHub API: $_"

Push-Location $RepoRoot
try {
$changedFiles = git diff --name-only --diff-filter=A "origin/main...HEAD" 2>&1
exit 0
}

Write-Host "No changelog entry and no 'skip-changelog' label. If changelog entry is not required, add the 'skip-changelog' label to the PR to bypass this check. Or add a changelog entry TO the PR."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Untriaged

Development

Successfully merging this pull request may close these issues.

2 participants