Implement duplicate endpoint reservation and submission handling with…

[gibahjoe]

What type of PR is this? (check all applicable)

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update

Description

Prevents duplicate endpoint submissions while a request is being created in Jira or waiting to appear in Datasette.

• Adds an atomic Redis reservation keyed by endpoint URL, dataset and organisation.
• Sends duplicate submissions to the existing “already collecting data from this endpoint URL” confirmation screen.
• Releases the reservation immediately when Jira creation fails.
• Keeps successful submissions visible to duplicate checks for 24 hours.
• Uses a two-minute reservation TTL if the application process stops unexpectedly.

Related Tickets & Documents

• Ticket Link: N/A
• Related Issue In C&P, the duplicate endpoint check should still run for submitted endpoints, even if they have not yet been added. #1240
• Closes #: N/A

QA Instructions, Screenshots, Recordings

1. Use an environment with Redis and Jira configured.
2. Submit an endpoint for an organisation and dataset.
3. Before the endpoint appears in Datasette, attempt to submit the same endpoint, organisation and dataset again.
4. Confirm the user is redirected to /check/confirmation and sees the message that data is already being collected.
5. Simulate a Jira failure and confirm the user returns to the check-answers page and can retry.
6. Confirm a successful submission remains detected as a duplicate.

No UI screenshots included; this reuses the existing confirmation screen.

Added/updated tests?

• Yes
• No, and this is why:
• I need help with writing tests

Added/updated unit tests for Redis reservations, successful and failed submissions, and the duplicate redirect.

Test status: unit suite passed (770 passed, 3 skipped). Integration tests require Playwright browsers to be installed locally.

[optional] Are there any post deployment tasks we need to perform?

None.

[optional] Are there any dependencies on other PRs or Work?

None.

Summary by CodeRabbit

• New Features

  • Added duplicate-submission protection for endpoint requests.
  • Duplicate submissions now redirect to the confirmation page without creating another Jira request.
  • Recently submitted endpoints are recognised before checking existing datasets.
  • Submission protection automatically expires or is released when appropriate.

• Bug Fixes

  • Prevented repeated submissions during processing from creating duplicate requests.

• Tests

  • Added coverage for reservation, expiry, failure handling and duplicate-submission scenarios.

[coderabbitai]

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: duplicate endpoint reservation and submission handling.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 1240-in-cp-the-duplicate-endpoint-check-should-still-run-for-submitted-endpoints-even-if-they-have-not-yet-been-added

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	71.33%	2600 / 3645
🔵	Statements	70.43%	2745 / 3897
🔵	Functions	64.75%	518 / 800
🔵	Branches	64.56%	1301 / 2015

File Coverage

File	Stmts	Branches	Functions	Lines	Uncovered Lines
Changed Files
src/controllers/CheckAnswersController.js	84.26%	77.08%	72.72%	85.88%	18-20, 24-30, 67-70, 173-177, 204, 216-221, 227-228, 246-250
src/utils/redisLoader.js	66.42%	74.71%	90%	70.24%	25-27, 33-35, 56, 60, 64-65, 75, 79, 92-93, 103, 107, 117, 128, 145-146, 156, 162, 170, 175, 212, 225, 237, 246-250, 261, 269-296
src/utils/datasetteQueries/endpointAlreadyCollected.js	100%	100%	100%	100%

Generated in workflow #1571 for commit 3f59b50 by the Vitest Coverage Report Action

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/utils/redisLoader.js`:
- Around line 84-87: The 120-second reservation must remain valid while Jira
creation is pending. In src/utils/redisLoader.js lines 84-87, add a
token-checked lease renewal operation; in
src/controllers/CheckAnswersController.js lines 50-70, start renewing after
reservation acquisition and stop renewal in finally. Add a test that keeps Jira
creation pending beyond 120 seconds and verifies a duplicate POST remains
blocked.

🪄 Autofix

✅ Autofix completed

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2386ee6a-47a8-4621-ad0d-f68354d41080

📥 Commits

Reviewing files that changed from the base of the PR and between 3f6cc5a and dba1f91.

📒 Files selected for processing (6)

• src/controllers/CheckAnswersController.js
• src/utils/datasetteQueries/endpointAlreadyCollected.js
• src/utils/redisLoader.js
• test/unit/checkAnswersController.test.js
• test/unit/redisLoader.test.js
• test/unit/utils/endpointAlreadyCollected.test.js

[coderabbitai]

Fixes Applied Successfully

Fixed 4 file(s) based on 1 unresolved review comment.

Files modified:

• src/controllers/CheckAnswersController.js
• src/utils/redisLoader.js
• test/unit/checkAnswersController.test.js
• test/unit/redisLoader.test.js

Commit: 3f59b501137067633cf7b94035170c7b246f907e

The changes have been pushed to the 1240-in-cp-the-duplicate-endpoint-check-should-still-run-for-submitted-endpoints-even-if-they-have-not-yet-been-added branch.

Time taken: 8m 29s