From fe7442abace5938b040c86752aa383232759b62b Mon Sep 17 00:00:00 2001 From: David Festal Date: Tue, 18 Aug 2026 20:34:01 +0200 Subject: [PATCH 1/3] feat(prepare-sources): implement generate-manifests pipeline module Generate manifest.json (workspace package inventory) and, when the workspace contains backstage:^ dependencies, backstage-manifest.json with @backstage package metadata extracted from yarn.lock and validated against versions.backstage.io. Add a zero-dependency Yarn Berry lockfile parser/serializer for round-trip fidelity, shared manifest types, and fixture-based tests. Extend test utilities with a mockFetch helper and wire the module into the pipeline runner. Add @vitest/coverage-v8 and ignore the coverage directory. Assisted-by: Cursor Signed-off-by: David Festal --- .sonarcloud.properties | 2 + prepare-sources/lib/manifest-types.ts | 39 + prepare-sources/lib/modules.ts | 3 +- .../lib/modules/generate-manifests/README.md | 204 ++++ .../backstage-json-no-version-error/error | 1 + .../input/overlay/source.json | 6 + .../input/workspace/backstage.json | 3 + .../input/workspace/package.json | 8 + .../workspace/plugins/my-plugin/package.json | 7 + .../input/overlay/source.json | 6 + .../input/workspace/package.json | 8 + .../output/workspace/manifest.json | 15 + .../output/workspace/package.json | 8 + .../missing-yarn-lock-error/error | 1 + .../input/overlay/source.json | 6 + .../input/workspace/backstage.json | 3 + .../input/workspace/package.json | 8 + .../workspace/plugins/my-plugin/package.json | 7 + .../input/overlay/source.json | 6 + .../input/workspace/package.json | 8 + .../plugins/plugin-alpha/package.json | 26 + .../plugins/plugin-beta/package.json | 8 + .../output/workspace/manifest.json | 47 + .../output/workspace/package.json | 8 + .../plugins/plugin-alpha/package.json | 26 + .../plugins/plugin-beta/package.json | 8 + .../input/overlay/source.json | 6 + .../input/workspace/backstage.json | 3 + .../input/workspace/package.json | 8 + .../workspace/plugins/my-plugin/package.json | 8 + .../input/workspace/yarn.lock | 33 + .../output/workspace/backstage-manifest.json | 34 + .../output/workspace/backstage.json | 3 + .../output/workspace/manifest.json | 18 + .../output/workspace/package.json | 8 + .../workspace/plugins/my-plugin/package.json | 8 + .../output/workspace/yarn.lock | 33 + .../remote-backstage-manifest.json | 8 + .../no-backstage-json-error/error | 1 + .../input/overlay/source.json | 6 + .../input/workspace/package.json | 8 + .../workspace/plugins/my-plugin/package.json | 7 + .../__fixtures__/version-mismatch-error/error | 1 + .../input/overlay/source.json | 6 + .../input/workspace/backstage.json | 3 + .../input/workspace/package.json | 8 + .../workspace/plugins/my-plugin/package.json | 7 + .../input/workspace/yarn.lock | 8 + .../remote-backstage-manifest.json | 3 + .../modules/generate-manifests/index.test.ts | 364 +++++++ .../lib/modules/generate-manifests/index.ts | 365 +++++++ prepare-sources/lib/test-utils.ts | 71 +- prepare-sources/lib/yarn-lock-parser.test.ts | 953 ++++++++++++++++++ prepare-sources/lib/yarn-lock-parser.ts | 392 +++++++ 54 files changed, 2843 insertions(+), 2 deletions(-) create mode 100644 prepare-sources/lib/manifest-types.ts create mode 100644 prepare-sources/lib/modules/generate-manifests/README.md create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/error create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/backstage.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/plugins/my-plugin/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/manifest.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/error create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/backstage.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/plugins/my-plugin/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-alpha/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-beta/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/manifest.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-alpha/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-beta/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/backstage.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/plugins/my-plugin/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/yarn.lock create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage-manifest.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/manifest.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/plugins/my-plugin/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/yarn.lock create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/remote-backstage-manifest.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/error create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/plugins/my-plugin/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/error create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/overlay/source.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/backstage.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/plugins/my-plugin/package.json create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/yarn.lock create mode 100644 prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/remote-backstage-manifest.json create mode 100644 prepare-sources/lib/modules/generate-manifests/index.test.ts create mode 100644 prepare-sources/lib/modules/generate-manifests/index.ts create mode 100644 prepare-sources/lib/yarn-lock-parser.test.ts create mode 100644 prepare-sources/lib/yarn-lock-parser.ts diff --git a/.sonarcloud.properties b/.sonarcloud.properties index b832489..604b9ef 100644 --- a/.sonarcloud.properties +++ b/.sonarcloud.properties @@ -1,2 +1,4 @@ # Test fixtures are not production code; skip analysis. sonar.exclusions=prepare-sources/**/__fixtures__/**/* +# Code duplication detection usually doesn't make sense in test files. +sonar.cpd.exclusions=prepare-sources/lib/**/*.test.ts diff --git a/prepare-sources/lib/manifest-types.ts b/prepare-sources/lib/manifest-types.ts new file mode 100644 index 0000000..809288c --- /dev/null +++ b/prepare-sources/lib/manifest-types.ts @@ -0,0 +1,39 @@ +/** + * Shape of `manifest.json` — inventory of all workspace packages. + * Written by `generate-manifests`, consumed by `protocol-resolution`. + */ +export type WorkspaceManifest = { + packages: WorkspacePackageEntry[]; +}; + +export type WorkspacePackageEntry = { + name: string; + version: string; + path?: string; + dependencies?: Record; + peerDependencies?: Record; + peerDependenciesMeta?: Record>; + optionalDependencies?: Record; + devDependencies?: Record; + bin?: Record; +}; + +/** + * Shape of `backstage-manifest.json` — Backstage release packages with + * dependency metadata. Written by `generate-manifests` (only when the + * workspace has `backstage:^` deps), consumed by `protocol-resolution`. + */ +export type BackstageManifest = { + backstageVersion: string; + packages: BackstagePackageEntry[]; +}; + +export type BackstagePackageEntry = { + name: string; + version: string; + dependencies?: Record; + peerDependencies?: Record; + peerDependenciesMeta?: Record>; + optionalDependencies?: Record; + bin?: Record; +}; diff --git a/prepare-sources/lib/modules.ts b/prepare-sources/lib/modules.ts index 445011d..d3e627a 100644 --- a/prepare-sources/lib/modules.ts +++ b/prepare-sources/lib/modules.ts @@ -1,6 +1,7 @@ import type { ModuleContext, PipelineModule } from "./pipeline.ts"; import { run as hermeticPrep } from "./modules/hermetic-prep/index.ts"; import { run as makeSelfContained } from "./modules/make-self-contained/index.ts"; +import { run as generateManifests } from "./modules/generate-manifests/index.ts"; async function notImplemented(ctx: ModuleContext): Promise { ctx.log("not yet implemented"); @@ -10,7 +11,7 @@ async function notImplemented(ctx: ModuleContext): Promise { export const MODULES: readonly PipelineModule[] = [ { name: "seed-frontend-lockfiles", run: notImplemented }, { name: "make-self-contained", run: makeSelfContained }, - { name: "generate-manifests", run: notImplemented }, + { name: "generate-manifests", run: generateManifests }, { name: "plugin-removal", run: notImplemented }, { name: "file-cleanup", run: notImplemented }, { name: "protocol-resolution", run: notImplemented }, diff --git a/prepare-sources/lib/modules/generate-manifests/README.md b/prepare-sources/lib/modules/generate-manifests/README.md new file mode 100644 index 0000000..6f666be --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/README.md @@ -0,0 +1,204 @@ +# generate-manifests + +Generates package inventories consumed by `protocol-resolution` to resolve +`workspace:^` and `backstage:^` references to concrete npm semver ranges. + +**Pipeline position:** after `make-self-contained`, before `plugin-removal` — +see `modules.ts` for the current ordering. + +**Shared types:** `WorkspaceManifest`, `BackstageManifest`, and their entry +types live in `lib/manifest-types.ts`. Both this module and +`protocol-resolution` depend on those types. + +## Problem + +The `protocol-resolution` module needs to rewrite `workspace:^` and +`backstage:^` protocol references in `package.json` and `yarn.lock`. To do +this, it needs a lookup table mapping package names to their concrete versions +and dependency metadata. + +This information must be captured **before scrubbing** (`plugin-removal`), +because scrubbed packages are removed from disk but their versions are still +needed for protocol resolution — surviving packages may depend on scrubbed +ones via `workspace:^`. + +## What the module produces + +### `manifest.json` + +Inventory of all local workspace packages. Always generated. + +```json +{ + "packages": [ + { + "name": "@scope/plugin-foo", + "version": "1.2.3", + "path": "plugins/plugin-foo/package.json", + "dependencies": { "@scope/plugin-bar": "workspace:^" }, + "peerDependencies": { "react": "^18.0.0" }, + "peerDependenciesMeta": { "react": { "optional": true } }, + "optionalDependencies": { "@emotion/react": "^11.0.0" }, + "devDependencies": { "@types/react": "^18.0.0" }, + "bin": { "foo": "./dist/cli.js" } + } + ] +} +``` + +All dependency fields are optional — only included when non-empty. + +**Package discovery:** Packages are discovered via the `workspaces` field in +the root `package.json` (glob resolution), plus the root package itself. For +flat repos (no `workspaces` field), only the root package is included. + +**Filtering:** Glob matches whose path segments include any of `node_modules`, +`dist-dynamic`, `dist-scalprum`, `dist`, or `build` are excluded. This +prevents build output directories from polluting the manifest. + +**`backstage:^` detection:** The module checks `dependencies`, +`devDependencies`, `peerDependencies`, and `optionalDependencies` across all +packages for `backstage:^` values. If none are found, `backstage-manifest.json` +is skipped entirely. + +### `backstage-manifest.json` + +Backstage release packages with dependency metadata. Only generated when the +workspace has `backstage:^` dependencies. Requires `backstage.json` at the +workspace root (throws if missing). + +```json +{ + "backstageVersion": "1.42.5", + "packages": [ + { + "name": "@backstage/core-plugin-api", + "version": "1.10.9", + "dependencies": { "@backstage/types": "npm:^1.2.3" }, + "peerDependencies": { "react": "npm:^18.0.0" }, + "peerDependenciesMeta": { "react": { "optional": true } }, + "optionalDependencies": { "@emotion/react": "npm:^11.0.0" }, + "bin": { "backstage-core": "./dist/cli.js" } + } + ] +} +``` + +Note that dependency values in the backstage manifest use the Yarn-resolved +format (`npm:^X.Y.Z`), since they come from `yarn.lock` rather than +`package.json`. + +Neither file is included in the OCI artifact — they are build-time +intermediates. + +## Why backstage metadata comes from yarn.lock, not the npm registry + +The original `generate-backstage-manifest.js` in `sync-midstream.sh` fetched +the Backstage release manifest from `versions.backstage.io` (one HTTP call), +then enriched each of the ~600 packages with dependency metadata from +`registry.npmjs.org` (60+ batched HTTP calls). The result was cached to disk +by Backstage version to amortize the cost across workspaces. + +This module takes a different approach: it **extracts backstage package +metadata directly from the workspace's `yarn.lock`**, using the shared +`yarn-lock-parser` (`lib/yarn-lock-parser.ts`). + +### Rationale + +After `yarn install --immutable` (which runs before this module), the +`yarn.lock` already contains the resolved version and full dependency metadata +for every `@backstage/*` package. These entries were created by the Backstage +Yarn plugin, which resolves `backstage:^` by looking up the exact version from +`versions.backstage.io` at install time. So the lockfile is a local cache of +the canonical version mapping, enriched with the dependency metadata that npm +registry calls would have provided. + +**Benefits:** + +- **Zero npm registry calls.** The original script made hundreds of HTTP + requests. This module reads local files only (plus one validation fetch). +- **No caching needed.** The original script cached results to `/tmp/` to + avoid redundant fetches across workspaces in the same `sync-midstream.sh` + run. In the new per-workspace CI model (separate workflow run per + workspace), there's no shared filesystem to cache to. Since we read from the + lockfile, there's nothing to cache. +- **More accurate.** The lockfile reflects what was actually resolved for this + specific workspace, not a generic npm registry response. + +**Validation:** The module fetches the Backstage release manifest from +`versions.backstage.io` (a single HTTP call) and compares each extracted +version against the canonical manifest. A mismatch throws — indicating a stale +lockfile or unexpected inconsistency. This is a defensive check; by +construction, `yarn install --immutable` guarantees the lockfile is consistent +with the Backstage Yarn plugin's resolution. + +### Why the versions are guaranteed to match + +The `backstage:^` protocol is resolved by the Backstage Yarn plugin, which: + +1. Reads `backstage.json` to get the target release version +2. Fetches `versions.backstage.io/v1/releases//manifest.json` +3. Resolves each `backstage:^` to the exact version from that manifest +4. Writes the result to `yarn.lock` + +After `yarn install --immutable`, the lockfile entries for `backstage:^` are +by construction the same versions as the canonical manifest. The validation +fetch is belt-and-suspenders — it catches corruption, plugin bugs, or manual +lockfile edits that should never happen in CI. + +## Why upstream fetching is no longer needed + +The original `generate-workspace-manifest.js` fetched versions from GitHub for +packages that didn't exist locally — these were packages referenced by +`workspace:^` but deleted by scrubbing before the manifest was generated. The +manifest tagged these entries with `path: 'upstream'` and `source: 'upstream'` +so that downstream consumers (`createTypeShimsPackage` in `update-workspace.js`) +could distinguish them from packages that were once local. + +In the new pipeline, this entire mechanism is unnecessary because manifest +generation runs **before** `plugin-removal`. All workspace packages are still +on disk, so every entry in the manifest has a real `path`. When +`protocol-resolution` later needs to distinguish surviving from scrubbed +packages, it simply checks `fs.existsSync(dirname(pkg.path))` — exactly as the +old `update-workspace.js` already does (lines 1722, 1736, 1799). The +`path === 'upstream'` marker is never needed. + +## Differences from sync-midstream.sh + +| Aspect | Original (sync-midstream.sh) | New (this module) | +| ------------------------- | ----------------------------------------------------------- | ----------------------------------------------------------- | +| Upstream version fetching | Fetches missing `workspace:` deps from GitHub raw URLs | Not needed — all packages are local (runs before scrubbing) | +| Backstage metadata source | `versions.backstage.io` + npm registry (~600 HTTP calls) | `yarn.lock` extraction + one validation fetch | +| Caching | `/tmp/backstage-manifest-cache/` (shared across workspaces) | None needed | +| Error handling | Swallowed (`\|\| true`, stderr discarded) | Throws on error (pipeline aborts) | +| Package discovery | Recursive scan with skip-list heuristics | `workspaces` field glob resolution + filter | +| `bin` normalization | String → object (same) | String → object (same) | +| yarn.lock parsing | N/A (backstage manifest came from npm) | Shared `yarn-lock-parser` (`lib/yarn-lock-parser.ts`) | + +## Log messages + +Log messages are aligned with the original scripts for familiarity: + +| Log | When | +| ------------------------------------------------- | --------------------------------- | +| `Scanning for package.json files...` | Always (start of module) | +| `Found N local packages` | Always | +| ` - @scope/pkg@1.2.3` | Per package | +| `Manifest written to: manifest.json` | Always | +| `No backstage:^ dependencies found, skipping` | No `backstage:^` in any dep field | +| `Backstage version: X.Y.Z` | Backstage manifest path | +| `Extracted N @backstage/* entries from yarn.lock` | After lockfile parsing | +| `Validating against ` | Before validation fetch | +| `All N entries validated against Backstage X.Y.Z` | After successful validation | +| `Written: backstage-manifest.json (N packages)` | Backstage manifest written | + +## Error conditions + +| Condition | Behavior | +| ---------------------------------------------------------------- | ------------------------------------------- | +| No `backstage:^` deps in any `package.json` | Skip — no `backstage-manifest.json` written | +| Has `backstage:^` deps but no `backstage.json` | Throw | +| `backstage.json` exists but has no `version` field | Throw | +| `yarn.lock` missing (when backstage manifest needed) | Throw | +| Version mismatch between `yarn.lock` and `versions.backstage.io` | Throw with details | +| Failed to fetch `versions.backstage.io` | Throw with HTTP status | diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/error b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/error new file mode 100644 index 0000000..97f6ca6 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/error @@ -0,0 +1 @@ +/no 'version' field/ \ No newline at end of file diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/backstage.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/backstage.json new file mode 100644 index 0000000..4df5066 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/backstage.json @@ -0,0 +1,3 @@ +{ + "description": "backstage.json without a version field" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/plugins/my-plugin/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/plugins/my-plugin/package.json new file mode 100644 index 0000000..c47611b --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/backstage-json-no-version-error/input/workspace/plugins/my-plugin/package.json @@ -0,0 +1,7 @@ +{ + "name": "@test/my-plugin", + "version": "1.0.0", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/workspace/package.json new file mode 100644 index 0000000..6da0c9e --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/standalone-plugin", + "version": "3.0.1", + "bin": "./dist/cli.js", + "dependencies": { + "express": "^4.18.0" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/manifest.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/manifest.json new file mode 100644 index 0000000..e54c2d3 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/manifest.json @@ -0,0 +1,15 @@ +{ + "packages": [ + { + "name": "@test/standalone-plugin", + "version": "3.0.1", + "path": "package.json", + "dependencies": { + "express": "^4.18.0" + }, + "bin": { + "standalone-plugin": "./dist/cli.js" + } + } + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/package.json new file mode 100644 index 0000000..6da0c9e --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/flat-repo-single-package/output/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/standalone-plugin", + "version": "3.0.1", + "bin": "./dist/cli.js", + "dependencies": { + "express": "^4.18.0" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/error b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/error new file mode 100644 index 0000000..1eab240 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/error @@ -0,0 +1 @@ +/yarn\.lock not found/ \ No newline at end of file diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/backstage.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/backstage.json new file mode 100644 index 0000000..6c12971 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/backstage.json @@ -0,0 +1,3 @@ +{ + "version": "1.42.5" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/plugins/my-plugin/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/plugins/my-plugin/package.json new file mode 100644 index 0000000..c47611b --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/missing-yarn-lock-error/input/workspace/plugins/my-plugin/package.json @@ -0,0 +1,7 @@ +{ + "name": "@test/my-plugin", + "version": "1.0.0", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-alpha/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-alpha/package.json new file mode 100644 index 0000000..fb49c94 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-alpha/package.json @@ -0,0 +1,26 @@ +{ + "name": "@test/plugin-alpha", + "version": "2.1.0", + "bin": { + "alpha-tool": "./dist/cli.js" + }, + "dependencies": { + "@test/plugin-beta": "workspace:^", + "react": "^18.0.0" + }, + "devDependencies": { + "@types/react": "^18.0.0", + "typescript": "~5.3.0" + }, + "peerDependencies": { + "react-dom": "^18.0.0" + }, + "peerDependenciesMeta": { + "react-dom": { + "optional": true + } + }, + "optionalDependencies": { + "fsevents": "^2.3.0" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-beta/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-beta/package.json new file mode 100644 index 0000000..518a25d --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/input/workspace/plugins/plugin-beta/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/plugin-beta", + "version": "1.3.0", + "bin": "./dist/index.js", + "dependencies": { + "lodash": "^4.17.21" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/manifest.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/manifest.json new file mode 100644 index 0000000..7ec5cfe --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/manifest.json @@ -0,0 +1,47 @@ +{ + "packages": [ + { + "name": "@test/plugin-alpha", + "version": "2.1.0", + "path": "plugins/plugin-alpha/package.json", + "dependencies": { + "@test/plugin-beta": "workspace:^", + "react": "^18.0.0" + }, + "peerDependencies": { + "react-dom": "^18.0.0" + }, + "peerDependenciesMeta": { + "react-dom": { + "optional": true + } + }, + "optionalDependencies": { + "fsevents": "^2.3.0" + }, + "devDependencies": { + "@types/react": "^18.0.0", + "typescript": "~5.3.0" + }, + "bin": { + "alpha-tool": "./dist/cli.js" + } + }, + { + "name": "@test/plugin-beta", + "version": "1.3.0", + "path": "plugins/plugin-beta/package.json", + "dependencies": { + "lodash": "^4.17.21" + }, + "bin": { + "plugin-beta": "./dist/index.js" + } + }, + { + "name": "@test/workspace-root", + "version": "1.0.0", + "path": "package.json" + } + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-alpha/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-alpha/package.json new file mode 100644 index 0000000..fb49c94 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-alpha/package.json @@ -0,0 +1,26 @@ +{ + "name": "@test/plugin-alpha", + "version": "2.1.0", + "bin": { + "alpha-tool": "./dist/cli.js" + }, + "dependencies": { + "@test/plugin-beta": "workspace:^", + "react": "^18.0.0" + }, + "devDependencies": { + "@types/react": "^18.0.0", + "typescript": "~5.3.0" + }, + "peerDependencies": { + "react-dom": "^18.0.0" + }, + "peerDependenciesMeta": { + "react-dom": { + "optional": true + } + }, + "optionalDependencies": { + "fsevents": "^2.3.0" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-beta/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-beta/package.json new file mode 100644 index 0000000..518a25d --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-no-backstage/output/workspace/plugins/plugin-beta/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/plugin-beta", + "version": "1.3.0", + "bin": "./dist/index.js", + "dependencies": { + "lodash": "^4.17.21" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/backstage.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/backstage.json new file mode 100644 index 0000000..6c12971 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/backstage.json @@ -0,0 +1,3 @@ +{ + "version": "1.42.5" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/plugins/my-plugin/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/plugins/my-plugin/package.json new file mode 100644 index 0000000..a8becc3 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/plugins/my-plugin/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/my-plugin", + "version": "1.5.0", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^", + "@backstage/theme": "backstage:^" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/yarn.lock b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/yarn.lock new file mode 100644 index 0000000..ce2da80 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/yarn.lock @@ -0,0 +1,33 @@ +__metadata: + version: 8 + cacheKey: 10c0 + +"@backstage/core-plugin-api@backstage:^": + version: 1.10.9 + resolution: "@backstage/core-plugin-api@npm:1.10.9" + dependencies: + "@backstage/types": "npm:^1.2.3" + react: "npm:^18.0.0" + peerDependencies: + react: "npm:^18.0.0" + bin: + backstage-core: ./dist/cli.js + checksum: abc123 + +"@backstage/theme@backstage:^": + version: 0.6.3 + resolution: "@backstage/theme@npm:0.6.3" + optionalDependencies: + "@emotion/react": "npm:^11.0.0" + peerDependencies: + react: "npm:^18.0.0" + peerDependenciesMeta: + react: + optional: true + checksum: def456 + +"@test/workspace-root@workspace:.": + version: 0.0.0-use.local + resolution: "@test/workspace-root@workspace:." + languageName: unknown + linkType: soft diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage-manifest.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage-manifest.json new file mode 100644 index 0000000..ca63d43 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage-manifest.json @@ -0,0 +1,34 @@ +{ + "backstageVersion": "1.42.5", + "packages": [ + { + "name": "@backstage/core-plugin-api", + "version": "1.10.9", + "dependencies": { + "@backstage/types": "npm:^1.2.3", + "react": "npm:^18.0.0" + }, + "peerDependencies": { + "react": "npm:^18.0.0" + }, + "bin": { + "backstage-core": "./dist/cli.js" + } + }, + { + "name": "@backstage/theme", + "version": "0.6.3", + "peerDependencies": { + "react": "npm:^18.0.0" + }, + "peerDependenciesMeta": { + "react": { + "optional": true + } + }, + "optionalDependencies": { + "@emotion/react": "npm:^11.0.0" + } + } + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage.json new file mode 100644 index 0000000..6c12971 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/backstage.json @@ -0,0 +1,3 @@ +{ + "version": "1.42.5" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/manifest.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/manifest.json new file mode 100644 index 0000000..9f42332 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/manifest.json @@ -0,0 +1,18 @@ +{ + "packages": [ + { + "name": "@test/my-plugin", + "version": "1.5.0", + "path": "plugins/my-plugin/package.json", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^", + "@backstage/theme": "backstage:^" + } + }, + { + "name": "@test/workspace-root", + "version": "1.0.0", + "path": "package.json" + } + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/plugins/my-plugin/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/plugins/my-plugin/package.json new file mode 100644 index 0000000..a8becc3 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/plugins/my-plugin/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/my-plugin", + "version": "1.5.0", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^", + "@backstage/theme": "backstage:^" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/yarn.lock b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/yarn.lock new file mode 100644 index 0000000..ce2da80 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/output/workspace/yarn.lock @@ -0,0 +1,33 @@ +__metadata: + version: 8 + cacheKey: 10c0 + +"@backstage/core-plugin-api@backstage:^": + version: 1.10.9 + resolution: "@backstage/core-plugin-api@npm:1.10.9" + dependencies: + "@backstage/types": "npm:^1.2.3" + react: "npm:^18.0.0" + peerDependencies: + react: "npm:^18.0.0" + bin: + backstage-core: ./dist/cli.js + checksum: abc123 + +"@backstage/theme@backstage:^": + version: 0.6.3 + resolution: "@backstage/theme@npm:0.6.3" + optionalDependencies: + "@emotion/react": "npm:^11.0.0" + peerDependencies: + react: "npm:^18.0.0" + peerDependenciesMeta: + react: + optional: true + checksum: def456 + +"@test/workspace-root@workspace:.": + version: 0.0.0-use.local + resolution: "@test/workspace-root@workspace:." + languageName: unknown + linkType: soft diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/remote-backstage-manifest.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/remote-backstage-manifest.json new file mode 100644 index 0000000..47d329b --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/monorepo-with-backstage/remote-backstage-manifest.json @@ -0,0 +1,8 @@ +{ + "packages": [ + { "name": "@backstage/core-plugin-api", "version": "1.10.9" }, + { "name": "@backstage/theme", "version": "0.6.3" }, + { "name": "@backstage/types", "version": "1.2.3" }, + { "name": "@backstage/catalog-model", "version": "1.8.1" } + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/error b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/error new file mode 100644 index 0000000..50fb61e --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/error @@ -0,0 +1 @@ +/backstage:.*dependencies but no backstage\.json/ \ No newline at end of file diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/plugins/my-plugin/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/plugins/my-plugin/package.json new file mode 100644 index 0000000..c47611b --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/no-backstage-json-error/input/workspace/plugins/my-plugin/package.json @@ -0,0 +1,7 @@ +{ + "name": "@test/my-plugin", + "version": "1.0.0", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/error b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/error new file mode 100644 index 0000000..4bd5b61 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/error @@ -0,0 +1 @@ +/backstage version mismatch/ \ No newline at end of file diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/overlay/source.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/overlay/source.json new file mode 100644 index 0000000..2d01d75 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/overlay/source.json @@ -0,0 +1,6 @@ +{ + "repo": "https://github.com/example/repo", + "repo-ref": "abc123", + "repo-flat": false, + "repo-backstage-version": "1.45.1" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/backstage.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/backstage.json new file mode 100644 index 0000000..6c12971 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/backstage.json @@ -0,0 +1,3 @@ +{ + "version": "1.42.5" +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/package.json new file mode 100644 index 0000000..0136fd8 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/package.json @@ -0,0 +1,8 @@ +{ + "name": "@test/workspace-root", + "version": "1.0.0", + "private": true, + "workspaces": [ + "plugins/*" + ] +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/plugins/my-plugin/package.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/plugins/my-plugin/package.json new file mode 100644 index 0000000..c47611b --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/plugins/my-plugin/package.json @@ -0,0 +1,7 @@ +{ + "name": "@test/my-plugin", + "version": "1.0.0", + "dependencies": { + "@backstage/core-plugin-api": "backstage:^" + } +} diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/yarn.lock b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/yarn.lock new file mode 100644 index 0000000..01f9cf3 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/input/workspace/yarn.lock @@ -0,0 +1,8 @@ +__metadata: + version: 8 + cacheKey: 10c0 + +"@backstage/core-plugin-api@backstage:^": + version: 1.10.8 + resolution: "@backstage/core-plugin-api@npm:1.10.8" + checksum: stale123 diff --git a/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/remote-backstage-manifest.json b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/remote-backstage-manifest.json new file mode 100644 index 0000000..cf77e4e --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/__fixtures__/version-mismatch-error/remote-backstage-manifest.json @@ -0,0 +1,3 @@ +{ + "packages": [{ "name": "@backstage/core-plugin-api", "version": "1.10.9" }] +} diff --git a/prepare-sources/lib/modules/generate-manifests/index.test.ts b/prepare-sources/lib/modules/generate-manifests/index.test.ts new file mode 100644 index 0000000..6d7b5d9 --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/index.test.ts @@ -0,0 +1,364 @@ +import fs from "node:fs"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vite-plus/test"; + +import { + loadFixture, + mockFetch, + type MockFetchResponse, + testInputOutputExpectations, +} from "../../test-utils.ts"; +import { + run, + extractBackstageEntries, + extractPackageInfo, + findWorkspacePackageJsonFiles, + getBackstageVersion, + normalizeBin, +} from "./index.ts"; + +afterEach(() => { + vi.restoreAllMocks(); +}); + +describe("generate-manifests", () => { + testInputOutputExpectations(import.meta.dirname, run, { + setup: (fixture, fixtureDir) => { + const releaseFile = path.join(fixtureDir, "remote-backstage-manifest.json"); + if (!fs.existsSync(releaseFile)) return; + + const backstageJsonPath = path.join(fixture.ctx.workspacePath, "backstage.json"); + if (!fs.existsSync(backstageJsonPath)) return; + + const version = JSON.parse(fs.readFileSync(backstageJsonPath, "utf8")).version; + const responseBody = JSON.parse(fs.readFileSync(releaseFile, "utf8")); + mockFetch({ + [`https://versions.backstage.io/v1/releases/${version}/manifest.json`]: responseBody, + }); + }, + }); + + it("logs package counts for workspace with backstage deps", async () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-with-backstage"); + const releaseFile = path.join( + import.meta.dirname, + "__fixtures__/monorepo-with-backstage/remote-backstage-manifest.json", + ); + const version = "1.42.5"; + mockFetch({ + [`https://versions.backstage.io/v1/releases/${version}/manifest.json`]: JSON.parse( + fs.readFileSync(releaseFile, "utf8"), + ), + }); + await run(fixture.ctx); + expect(fixture.ctx.log).toHaveBeenCalledWith(expect.stringContaining("Found")); + expect(fixture.ctx.log).toHaveBeenCalledWith(expect.stringContaining("Manifest written to:")); + expect(fixture.ctx.log).toHaveBeenCalledWith( + expect.stringContaining("Written: backstage-manifest.json"), + ); + }); + + it("throws when versions.backstage.io returns HTTP error", async () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-with-backstage"); + const version = "1.42.5"; + mockFetch({ + [`https://versions.backstage.io/v1/releases/${version}/manifest.json`]: { + body: null, + status: 503, + ok: false, + } satisfies MockFetchResponse, + }); + await expect(run(fixture.ctx)).rejects.toThrow(/HTTP 503/); + }); + + it("throws when manifest response has no packages array", async () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-with-backstage"); + const version = "1.42.5"; + mockFetch({ + [`https://versions.backstage.io/v1/releases/${version}/manifest.json`]: { + releaseVersion: version, + }, + }); + await expect(run(fixture.ctx)).rejects.toThrow(/invalid Backstage manifest response/); + }); + + it("skips malformed entries in manifest packages array", async () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-with-backstage"); + const version = "1.42.5"; + mockFetch({ + [`https://versions.backstage.io/v1/releases/${version}/manifest.json`]: { + packages: [ + null, + 42, + { name: "@backstage/core-plugin-api", version: "1.10.9" }, + { name: "@backstage/theme", version: "0.6.3" }, + { name: "@backstage/types", version: "1.2.3" }, + { name: "@backstage/catalog-model", version: "1.8.1" }, + { noVersion: true }, + { name: 123, version: "1.0.0" }, + ], + }, + }); + await run(fixture.ctx); + expect(fixture.ctx.log).toHaveBeenCalledWith(expect.stringContaining("Found")); + }); +}); + +// --------------------------------------------------------------------------- +// Unit tests for extractBackstageEntries +// --------------------------------------------------------------------------- + +describe("extractBackstageEntries", () => { + it("extracts all field types from a complete lockfile", () => { + const lockContent = ` +"react@npm:^18.0.0": + version: 18.3.1 + resolution: "react@npm:18.3.1" + checksum: def + +"@backstage/core-plugin-api@backstage:^": + version: 1.10.9 + resolution: "@backstage/core-plugin-api@npm:1.10.9" + dependencies: + "@backstage/types": "npm:^1.2.3" + bin: + backstage-cli: ./bin/backstage-cli + checksum: abc123 + +"@backstage/theme@backstage:^": + version: 0.6.3 + resolution: "@backstage/theme@npm:0.6.3" + peerDependencies: + react: "npm:^18.0.0" + peerDependenciesMeta: + react: + optional: true + @emotion/react: + reason: styling + optional: false + optionalDependencies: + "@emotion/react": "npm:^11.0.0" + checksum: xyz789 +`; + const entries = extractBackstageEntries(lockContent); + expect(entries).toHaveLength(2); + + expect(entries[0]).toEqual({ + name: "@backstage/core-plugin-api", + version: "1.10.9", + dependencies: { "@backstage/types": "npm:^1.2.3" }, + bin: { "backstage-cli": "./bin/backstage-cli" }, + }); + + expect(entries[1]).toEqual({ + name: "@backstage/theme", + version: "0.6.3", + peerDependencies: { react: "npm:^18.0.0" }, + peerDependenciesMeta: { + react: { optional: true }, + "@emotion/react": { reason: "styling", optional: false }, + }, + optionalDependencies: { "@emotion/react": "npm:^11.0.0" }, + }); + }); + + it("skips entries with malformed descriptor, missing version, or no backstage deps", () => { + const lockContent = ` +"react@npm:^18.0.0": + version: 18.3.1 + checksum: def + +"backstage:^": + version: 1.0.0 + resolution: "something@npm:1.0.0" + checksum: abc + +"@backstage/core-plugin-api@backstage:^": + resolution: "@backstage/core-plugin-api@npm:1.10.9" + checksum: abc +`; + expect(extractBackstageEntries(lockContent)).toEqual([]); + }); + + it("omits empty dependency and peerDependenciesMeta sections", () => { + const lockContent = ` +"@backstage/core-plugin-api@backstage:^": + version: 1.10.9 + resolution: "@backstage/core-plugin-api@npm:1.10.9" + dependencies: + peerDependenciesMeta: + checksum: abc +`; + const entries = extractBackstageEntries(lockContent); + expect(entries).toHaveLength(1); + const entry = entries[0]; + expect(entry).toBeDefined(); + expect(entry?.dependencies).toBeUndefined(); + expect(entry?.peerDependenciesMeta).toBeUndefined(); + }); + + it("omits dependency fields when all map entries are malformed", () => { + const lockContent = [ + '"@backstage/core-plugin-api@backstage:^":', + " version: 1.10.9", + ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', + " dependencies:", + " no-colon-garbage", + " also-no-colon-garbage", + " checksum: abc", + ].join("\n"); + const entries = extractBackstageEntries(lockContent); + expect(entries).toHaveLength(1); + const entry = entries[0]; + expect(entry).toBeDefined(); + expect(entry?.dependencies).toBeUndefined(); + }); + + it("omits peerDependenciesMeta when all nested-map keys are malformed", () => { + const lockContent = [ + '"@backstage/core-plugin-api@backstage:^":', + " version: 1.10.9", + ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', + " peerDependenciesMeta:", + " not-a-key-header 1.0.0", + " orphan-prop: true", + " checksum: abc", + ].join("\n"); + const entries = extractBackstageEntries(lockContent); + expect(entries).toHaveLength(1); + const entry = entries[0]; + expect(entry).toBeDefined(); + expect(entry?.peerDependenciesMeta).toBeUndefined(); + }); + + it("handles malformed lockfile content gracefully", () => { + const lockContent = [ + '"@backstage/core-plugin-api@backstage:^":', + " version: 1.10.9", + ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', + " dependencies:", + " !!!not-a-valid-line!!!", + ' "@backstage/types": "npm:^1.2.3"', + " peerDependenciesMeta:", + " orphan-field: true", + " react:", + " optional: true", + " ", + " checksum: abc", + ].join("\n"); + const entries = extractBackstageEntries(lockContent); + expect(entries).toHaveLength(1); + const entry = entries[0]; + expect(entry).toBeDefined(); + expect(entry?.dependencies).toEqual({ "@backstage/types": "npm:^1.2.3" }); + expect(entry?.peerDependenciesMeta).toEqual({ react: { optional: true } }); + }); +}); + +// --------------------------------------------------------------------------- +// Unit tests for normalizeBin +// --------------------------------------------------------------------------- + +describe("normalizeBin", () => { + const cases: Array< + [string, string, string | Record, Record | undefined] + > = [ + ["scoped string → object", "@scope/my-cli", "./dist/cli.js", { "my-cli": "./dist/cli.js" }], + ["unscoped string → object", "my-tool", "./cli.js", { "my-tool": "./cli.js" }], + [ + "object passthrough", + "@scope/pkg", + { "tool-a": "./a.js", "tool-b": "./b.js" }, + { "tool-a": "./a.js", "tool-b": "./b.js" }, + ], + ["empty object → undefined", "@scope/pkg", {}, undefined], + ]; + + it.each(cases)("%s", (_label, name, bin, expected) => { + expect(normalizeBin(name, bin)).toEqual(expected); + }); +}); + +// --------------------------------------------------------------------------- +// Unit tests for findWorkspacePackageJsonFiles +// --------------------------------------------------------------------------- + +describe("findWorkspacePackageJsonFiles", () => { + it("finds root package.json for flat repo (no workspaces field)", () => { + using fixture = loadFixture(import.meta.dirname, "flat-repo-single-package"); + const files = findWorkspacePackageJsonFiles(fixture.ctx.workspacePath); + expect(files).toHaveLength(1); + expect(files[0]).toContain("package.json"); + }); + + it("finds root + workspace members for monorepo", () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-no-backstage"); + const files = findWorkspacePackageJsonFiles(fixture.ctx.workspacePath); + expect(files).toHaveLength(3); + }); + + it("handles edge cases: missing root, dist-dynamic exclusion, empty glob match", () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-no-backstage"); + const wp = fixture.ctx.workspacePath; + + fs.mkdirSync(path.join(wp, "plugins", "empty-dir"), { recursive: true }); + expect(findWorkspacePackageJsonFiles(wp)).toHaveLength(3); + + const distDir = path.join(wp, "plugins", "dist-dynamic"); + fs.mkdirSync(distDir, { recursive: true }); + fs.writeFileSync( + path.join(distDir, "package.json"), + JSON.stringify({ name: "should-be-excluded", version: "0.0.0" }), + ); + expect(findWorkspacePackageJsonFiles(wp)).toHaveLength(3); + expect(findWorkspacePackageJsonFiles(wp).every((f) => !f.includes("dist-dynamic"))).toBe(true); + + fs.unlinkSync(path.join(wp, "package.json")); + expect(findWorkspacePackageJsonFiles(wp)).toEqual([]); + }); +}); + +// --------------------------------------------------------------------------- +// Unit tests for getBackstageVersion +// --------------------------------------------------------------------------- + +describe("getBackstageVersion", () => { + it("reads version from backstage.json", () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-with-backstage"); + expect(getBackstageVersion(fixture.ctx.workspacePath)).toBe("1.42.5"); + }); + + it("throws when backstage.json is missing", () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-no-backstage"); + expect(() => getBackstageVersion(fixture.ctx.workspacePath)).toThrow(/no backstage\.json/); + }); +}); + +// --------------------------------------------------------------------------- +// Unit tests for extractPackageInfo +// --------------------------------------------------------------------------- + +describe("extractPackageInfo", () => { + it("returns null for missing name or version, filtering them from manifest", async () => { + using fixture = loadFixture(import.meta.dirname, "monorepo-no-backstage"); + const rootPkgPath = path.join(fixture.ctx.workspacePath, "package.json"); + const content = JSON.parse(fs.readFileSync(rootPkgPath, "utf8")); + + fs.writeFileSync(rootPkgPath, JSON.stringify({ ...content, name: undefined })); + expect(extractPackageInfo(rootPkgPath, fixture.ctx.workspacePath)).toBeNull(); + + fs.writeFileSync(rootPkgPath, JSON.stringify({ ...content, version: undefined })); + expect(extractPackageInfo(rootPkgPath, fixture.ctx.workspacePath)).toBeNull(); + + fs.writeFileSync(rootPkgPath, JSON.stringify(content)); + const alphaPath = path.join( + fixture.ctx.workspacePath, + "plugins", + "plugin-alpha", + "package.json", + ); + fs.writeFileSync(alphaPath, JSON.stringify({ version: "1.0.0" })); + await run(fixture.ctx); + expect(fixture.ctx.log).toHaveBeenCalledWith(expect.stringMatching(/Found 2 local packages/)); + }); +}); diff --git a/prepare-sources/lib/modules/generate-manifests/index.ts b/prepare-sources/lib/modules/generate-manifests/index.ts new file mode 100644 index 0000000..aeda39e --- /dev/null +++ b/prepare-sources/lib/modules/generate-manifests/index.ts @@ -0,0 +1,365 @@ +import fs from "node:fs"; +import path from "node:path"; +import type { ModuleContext } from "../../pipeline.ts"; +import type { + BackstageManifest, + BackstagePackageEntry, + WorkspaceManifest, + WorkspacePackageEntry, +} from "../../manifest-types.ts"; +import { + type LockfileBlock, + getMap, + getNestedMap, + getScalar, + parseLockfile, +} from "../../yarn-lock-parser.ts"; + +const VERSIONS_BACKSTAGE_IO = "https://versions.backstage.io/v1/releases"; + +const EXCLUDE_DIRS = new Set(["node_modules", "dist-dynamic", "dist-scalprum", "dist", "build"]); + +/** + * Generates `manifest.json` (workspace package inventory) and optionally + * `backstage-manifest.json` (Backstage release packages with dependency + * metadata, extracted from `yarn.lock` and validated against + * `versions.backstage.io`). + * + * Must run before scrubbing so that versions of to-be-scrubbed packages are + * captured. Not included in the OCI artifact. + */ +export async function run(ctx: ModuleContext): Promise { + ctx.log("Scanning for package.json files..."); + const workspaceManifest = generateWorkspaceManifest(ctx); + + ctx.log(`Found ${workspaceManifest.packages.length} local packages`); + for (const pkg of workspaceManifest.packages) { + ctx.log(` - ${pkg.name}@${pkg.version}`); + } + + const manifestPath = path.join(ctx.workspacePath, "manifest.json"); + writeJson(manifestPath, workspaceManifest); + ctx.log(`Manifest written to: manifest.json`); + + if (!manifestHasBackstageDeps(workspaceManifest)) { + ctx.log("No backstage:^ dependencies found, skipping"); + return; + } + + const backstageManifest = await generateBackstageManifest(ctx); + const backstagePath = path.join(ctx.workspacePath, "backstage-manifest.json"); + writeJson(backstagePath, backstageManifest); + ctx.log(`Written: backstage-manifest.json (${backstageManifest.packages.length} packages)`); +} + +function manifestHasBackstageDeps(manifest: WorkspaceManifest): boolean { + return manifest.packages.some((pkg) => { + const allDeps = [ + pkg.dependencies, + pkg.devDependencies, + pkg.peerDependencies, + pkg.optionalDependencies, + ]; + return allDeps.some((deps) => deps && Object.values(deps).includes("backstage:^")); + }); +} + +// --------------------------------------------------------------------------- +// Workspace manifest +// --------------------------------------------------------------------------- + +function generateWorkspaceManifest(ctx: ModuleContext): WorkspaceManifest { + const packageJsonPaths = findWorkspacePackageJsonFiles(ctx.workspacePath); + const packages: WorkspacePackageEntry[] = []; + + for (const pkgPath of packageJsonPaths) { + const entry = extractPackageInfo(pkgPath, ctx.workspacePath); + if (entry) packages.push(entry); + } + + packages.sort((a, b) => a.name.localeCompare(b.name)); + return { packages }; +} + +/** + * Discover workspace package.json files using the `workspaces` field from + * the root package.json, filtered to exclude build output directories. + * For flat repos (no `workspaces` field), returns just the root. + */ +export function findWorkspacePackageJsonFiles(workspacePath: string): string[] { + const rootPkgPath = path.join(workspacePath, "package.json"); + if (!fs.existsSync(rootPkgPath)) return []; + + const rootPkg = JSON.parse(fs.readFileSync(rootPkgPath, "utf8")); + const patterns: string[] = rootPkg.workspaces ?? []; + const results = [rootPkgPath]; + + for (const pattern of patterns) { + const matches = fs.globSync(pattern, { cwd: workspacePath }); + for (const match of matches) { + if (match.split(path.sep).some((seg) => EXCLUDE_DIRS.has(seg))) continue; + const pkgPath = path.join(workspacePath, match, "package.json"); + if (fs.existsSync(pkgPath)) results.push(pkgPath); + } + } + + return results; +} + +const DEP_FIELDS = [ + "dependencies", + "peerDependencies", + "peerDependenciesMeta", + "optionalDependencies", + "devDependencies", +] as const; + +export function extractPackageInfo( + packageJsonPath: string, + baseDir: string, +): WorkspacePackageEntry | null { + const content = JSON.parse(fs.readFileSync(packageJsonPath, "utf8")); + if (!content.name || !content.version) return null; + + const entry: WorkspacePackageEntry = { + name: content.name, + version: content.version, + path: path.relative(path.resolve(baseDir), packageJsonPath), + }; + + for (const field of DEP_FIELDS) { + if (content[field] && Object.keys(content[field]).length > 0) { + (entry as Record)[field] = content[field]; + } + } + + if (content.bin) { + entry.bin = normalizeBin(content.name, content.bin); + } + + return entry; +} + +/** + * Normalize the `bin` field to always be an object. + * String form `"./cli.js"` becomes `{ "": "./cli.js" }`. + */ +export function normalizeBin( + packageName: string, + bin: string | Record, +): Record | undefined { + if (typeof bin === "string") { + const slashPos = packageName.lastIndexOf("/"); + const unscopedName = slashPos === -1 ? packageName : packageName.substring(slashPos + 1); + return { [unscopedName]: bin }; + } + if (Object.keys(bin).length > 0) return bin; + return undefined; +} + +// --------------------------------------------------------------------------- +// Backstage manifest +// --------------------------------------------------------------------------- + +async function generateBackstageManifest(ctx: ModuleContext): Promise { + const backstageVersion = getBackstageVersion(ctx.workspacePath); + ctx.log(`Backstage version: ${backstageVersion}`); + + const yarnLockPath = path.join(ctx.workspacePath, "yarn.lock"); + if (!fs.existsSync(yarnLockPath)) { + throw new Error("yarn.lock not found — cannot extract backstage package metadata"); + } + + const lockContent = fs.readFileSync(yarnLockPath, "utf8"); + const entries = extractBackstageEntries(lockContent); + ctx.log(`Extracted ${entries.length} @backstage/* entries from yarn.lock`); + + await validateBackstageVersions(entries, backstageVersion, ctx); + + return { + backstageVersion, + packages: entries.toSorted((a, b) => a.name.localeCompare(b.name)), + }; +} + +/** + * Read the Backstage version from `backstage.json` in the workspace. + * Throws if `backstage:^` deps exist but no `backstage.json` is found. + */ +export function getBackstageVersion(workspacePath: string): string { + const backstageJsonPath = path.join(workspacePath, "backstage.json"); + if (!fs.existsSync(backstageJsonPath)) { + throw new Error( + "workspace has backstage:^ dependencies but no backstage.json — " + + "cannot determine Backstage version", + ); + } + const content = JSON.parse(fs.readFileSync(backstageJsonPath, "utf8")); + if (!content.version) { + throw new Error("backstage.json exists but has no 'version' field"); + } + return content.version; +} + +// --------------------------------------------------------------------------- +// yarn.lock extraction (using shared parser) +// --------------------------------------------------------------------------- + +/** + * Extract `@backstage/*` package entries resolved via `backstage:^` from a + * yarn.lock string. Returns structured entries with version and dependency + * metadata. + * + * Uses the shared `yarn-lock-parser` for structural parsing, then maps the + * parsed blocks to `BackstagePackageEntry` objects. + */ +export function extractBackstageEntries(lockContent: string): BackstagePackageEntry[] { + const lockfile = parseLockfile(lockContent); + return lockfile.blocks + .map(parseBackstageBlock) + .filter((entry): entry is BackstagePackageEntry => entry !== undefined); +} + +function parseBackstageBlock(block: LockfileBlock): BackstagePackageEntry | undefined { + const descriptor = block.descriptors.find((d) => d.includes("backstage:^")); + if (!descriptor) return undefined; + + const nameMatch = /(@backstage\/[^@]+)/.exec(descriptor); + const name = nameMatch?.[1]; + if (!name) return undefined; + + const version = getScalar(block, "version"); + if (!version) return undefined; + + const entry: BackstagePackageEntry = { name, version }; + + const deps = getStrippedMap(block, "dependencies"); + if (deps) entry.dependencies = deps; + + const peers = getStrippedMap(block, "peerDependencies"); + if (peers) entry.peerDependencies = peers; + + const peersMeta = getNestedMap(block, "peerDependenciesMeta"); + if (peersMeta) { + entry.peerDependenciesMeta = convertNestedMapValues(peersMeta); + } + + const optional = getStrippedMap(block, "optionalDependencies"); + if (optional) entry.optionalDependencies = optional; + + const bin = getStrippedMap(block, "bin"); + if (bin) entry.bin = bin; + + return entry; +} + +function stripMapQuotes(raw: { [key: string]: string }): Record { + const result: Record = {}; + for (const [key, value] of Object.entries(raw)) { + result[key.replace(/^"|"$/g, "")] = value.replace(/^"|"$/g, ""); + } + return result; +} + +function getStrippedMap( + block: LockfileBlock, + fieldName: string, +): Record | undefined { + const raw = getMap(block, fieldName); + if (!raw) return undefined; + const stripped = stripMapQuotes(raw); + return Object.keys(stripped).length > 0 ? stripped : undefined; +} + +function convertNestedMapValues(peersMeta: { + [key: string]: { [prop: string]: string }; +}): Record> | undefined { + const entries = Object.entries(peersMeta); + if (entries.length === 0) return undefined; + const converted: Record> = {}; + for (const [pkg, props] of entries) { + const record: Record = {}; + converted[pkg] = record; + for (const [key, val] of Object.entries(props)) { + const unquoted = val.replace(/^"|"$/g, ""); + let value: unknown; + if (unquoted === "true") { + value = true; + } else if (unquoted === "false") { + value = false; + } else { + value = unquoted; + } + record[key] = value; + } + } + return converted; +} + +// --------------------------------------------------------------------------- +// Backstage version validation +// --------------------------------------------------------------------------- + +async function validateBackstageVersions( + entries: BackstagePackageEntry[], + backstageVersion: string, + ctx: ModuleContext, +): Promise { + const url = `${VERSIONS_BACKSTAGE_IO}/${backstageVersion}/manifest.json`; + ctx.log(`Validating against ${url}`); + + const response = await fetch(url); + if (!response.ok) { + throw new Error( + `failed to fetch Backstage manifest for ${backstageVersion}: HTTP ${response.status}`, + ); + } + + const manifest: unknown = await response.json(); + if ( + typeof manifest !== "object" || + manifest === null || + !("packages" in manifest) || + !Array.isArray(manifest.packages) + ) { + throw new Error(`invalid Backstage manifest response from ${url}`); + } + const versionMap = new Map(); + for (const pkg of manifest.packages) { + if ( + typeof pkg === "object" && + pkg !== null && + "name" in pkg && + "version" in pkg && + typeof pkg.name === "string" && + typeof pkg.version === "string" + ) { + versionMap.set(pkg.name, pkg.version); + } + } + + const mismatches: string[] = []; + for (const entry of entries) { + const expected = versionMap.get(entry.name); + if (expected && expected !== entry.version) { + mismatches.push(`${entry.name}: yarn.lock has ${entry.version}, manifest has ${expected}`); + } + } + + if (mismatches.length > 0) { + throw new Error( + `backstage version mismatch between yarn.lock and versions.backstage.io:\n` + + mismatches.join("\n"), + ); + } + + ctx.log(`All ${entries.length} entries validated against Backstage ${backstageVersion}`); +} + +// --------------------------------------------------------------------------- +// Utilities +// --------------------------------------------------------------------------- + +function writeJson(filePath: string, data: unknown): void { + fs.writeFileSync(filePath, JSON.stringify(data, null, 2) + "\n"); +} diff --git a/prepare-sources/lib/test-utils.ts b/prepare-sources/lib/test-utils.ts index fcce6fe..99d94b6 100644 --- a/prepare-sources/lib/test-utils.ts +++ b/prepare-sources/lib/test-utils.ts @@ -5,6 +5,68 @@ import { describe, expect, it, vi } from "vite-plus/test"; import { readSourceFile } from "./source.ts"; import type { ModuleContext } from "./pipeline.ts"; +// --------------------------------------------------------------------------- +// Fetch mocking +// --------------------------------------------------------------------------- + +export type MockFetchResponse = { + body: unknown; + status?: number; + ok?: boolean; +}; + +function isMockFetchResponse(value: object): value is MockFetchResponse { + return "body" in value; +} + +function jsonResponse(body: unknown, status = 200): Response { + return new Response(JSON.stringify(body), { + status, + headers: { "Content-Type": "application/json" }, + }); +} + +/** + * Stub global `fetch` with a URL → response mapping. + * + * Values can be plain objects (treated as 200 OK JSON bodies) or + * `MockFetchResponse` objects for control over status/ok. + * + * Unmocked URLs cause the returned `fetch` to throw, making tests fail fast + * if unexpected network calls happen. Call `vi.restoreAllMocks()` (or rely on + * the `afterEach` below) to restore the real `fetch`. + */ +export function mockFetch( + mappings: Record, + options?: { unmockedBehavior?: "throw" | "empty" }, +): void { + const behavior = options?.unmockedBehavior ?? "throw"; + vi.stubGlobal("fetch", (input: string | URL | Request): Promise => { + let url: string; + if (typeof input === "string") { + url = input; + } else if (input instanceof URL) { + url = input.href; + } else { + url = input.url; + } + if (url in mappings) { + const raw = mappings[url]; + if (raw === undefined) { + return Promise.reject(new Error(`unmocked fetch: ${url}`)); + } + if (isMockFetchResponse(raw)) { + return Promise.resolve(jsonResponse(raw.body, raw.status ?? 200)); + } + return Promise.resolve(jsonResponse(raw)); + } + if (behavior === "empty") { + return Promise.resolve(jsonResponse(null, 404)); + } + return Promise.reject(new Error(`unmocked fetch: ${url}`)); + }); +} + /** Assertion helpers scoped to a single directory. */ export interface DirAssertions { readFile(relativePath: string): string; @@ -209,10 +271,15 @@ export function loadFixture( * * @param testDir Pass `import.meta.dirname` from your test file. * @param run The module's `run` function. + * @param options Optional hooks: + * - `setup(fixture, fixtureDir)` — called before each fixture's module run. + * Use for per-fixture mocking (e.g. `mockFetch` from network response + * fixture files). Cleanup is handled by `afterEach` / `vi.restoreAllMocks`. */ export function testInputOutputExpectations( testDir: string, run: (ctx: ModuleContext) => Promise, + options?: { setup?: (fixture: ModuleFixture, fixtureDir: string) => void }, ): void { const fixturesDir = path.join(testDir, "__fixtures__"); if (!fs.existsSync(fixturesDir)) { @@ -228,8 +295,10 @@ export function testInputOutputExpectations( describe("fixtures", () => { for (const name of fixtures) { it(name, async () => { + const fixtureDir = path.join(fixturesDir, name); using fixture = loadFixture(testDir, name); - const errorFile = path.join(fixturesDir, name, "error"); + options?.setup?.(fixture, fixtureDir); + const errorFile = path.join(fixtureDir, "error"); if (fs.existsSync(errorFile)) { const raw = fs.readFileSync(errorFile, "utf8").trim(); diff --git a/prepare-sources/lib/yarn-lock-parser.test.ts b/prepare-sources/lib/yarn-lock-parser.test.ts new file mode 100644 index 0000000..94689e9 --- /dev/null +++ b/prepare-sources/lib/yarn-lock-parser.test.ts @@ -0,0 +1,953 @@ +import { describe, expect, it } from "vitest"; +import { + type Lockfile, + type LockfileBlock, + getMap, + getNestedMap, + getScalar, + parseLockfile, + serializeBlock, + serializeLockfile, +} from "./yarn-lock-parser.ts"; + +function roundTrip(lockfile: Lockfile): string { + return serializeLockfile(lockfile); +} + +function expectFirstBlock(lockfile: Lockfile): LockfileBlock { + const block = lockfile.blocks[0]; + expect(block).toBeDefined(); + if (!block) { + throw new Error("expected block"); + } + return block; +} + +// --------------------------------------------------------------------------- +// Round-trip fidelity +// --------------------------------------------------------------------------- + +describe("round-trip fidelity", () => { + const cases: [string, string][] = [ + [ + "simple lockfile", + [ + '# This file is generated by running "yarn install" inside your project.', + "# Manual changes might be lost - proceed with caution!", + "", + "__metadata:", + " version: 8", + " cacheKey: 10c0", + "", + '"@backstage/core-plugin-api@backstage:^":', + " version: 1.10.9", + ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', + " dependencies:", + ' "@backstage/types": "npm:^1.2.3"', + ' react: "npm:^18.0.0"', + " checksum: abc123", + " languageName: node", + " linkType: hard", + "", + ].join("\n"), + ], + [ + "combined keys", + [ + "__metadata:", + " version: 8", + "", + '"@aws-sdk/types@npm:^3.222.0, @aws-sdk/types@npm:^3.347.0, @aws-sdk/types@npm:^3.973.1":', + " version: 3.973.1", + ' resolution: "@aws-sdk/types@npm:3.973.1"', + " checksum: abc", + "", + ].join("\n"), + ], + [ + "nested map (peerDependenciesMeta)", + [ + "__metadata:", + " version: 8", + "", + '"@backstage/theme@backstage:^":', + " version: 0.6.3", + " peerDependencies:", + ' react: "npm:^18.0.0"', + " peerDependenciesMeta:", + " react:", + " optional: true", + " linkType: hard", + "", + ].join("\n"), + ], + [ + "unquoted simple keys in deps", + [ + "__metadata:", + " version: 8", + "", + '"ajv@npm:^8.17.1":', + " version: 8.17.1", + " dependencies:", + ' fast-deep-equal: "npm:^3.1.3"', + ' json-schema-traverse: "npm:^1.0.0"', + ' require-from-string: "npm:^2.0.2"', + ' uri-js: "npm:^4.4.1"', + " linkType: hard", + "", + ].join("\n"), + ], + [ + "workspace block", + [ + "__metadata:", + " version: 8", + "", + '"@test/root@workspace:.":', + " version: 0.0.0-use.local", + ' resolution: "@test/root@workspace:."', + " languageName: unknown", + " linkType: soft", + "", + ].join("\n"), + ], + [ + "block with bin", + [ + "__metadata:", + " version: 8", + "", + '"@backstage/core-plugin-api@backstage:^":', + " version: 1.10.9", + " bin:", + " backstage-core: ./dist/cli.js", + " checksum: abc123", + "", + ].join("\n"), + ], + [ + "multiple blocks", + [ + "__metadata:", + " version: 8", + "", + '"pkg-a@npm:^1.0.0":', + " version: 1.0.0", + " linkType: hard", + "", + '"pkg-b@npm:^2.0.0":', + " version: 2.0.0", + " dependencies:", + ' pkg-a: "npm:^1.0.0"', + " linkType: hard", + "", + ].join("\n"), + ], + ]; + + it.each(cases)("%s", (_name, input) => { + const lockfile = parseLockfile(input); + const output = serializeLockfile(lockfile); + expect(output).toBe(input); + }); +}); + +// --------------------------------------------------------------------------- +// Parsing +// --------------------------------------------------------------------------- + +describe("parseLockfile", () => { + it("captures the preamble including __metadata", () => { + const input = [ + "# comment", + "", + "__metadata:", + " version: 8", + " cacheKey: 10c0", + "", + '"pkg@npm:^1.0.0":', + " version: 1.0.0", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.preamble).toBe("# comment\n\n__metadata:\n version: 8\n cacheKey: 10c0"); + expect(lockfile.blocks).toHaveLength(1); + }); + + it("handles lockfile without __metadata", () => { + const input = ['"pkg@npm:^1.0.0":', " version: 1.0.0", ""].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.preamble).toBe(""); + expect(lockfile.blocks).toHaveLength(1); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toEqual({ + kind: "scalar", + raw: "1.0.0", + }); + }); + + it("parses combined descriptor keys", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"@aws/types@npm:^3.0.0, @aws/types@npm:^3.1.0":', + " version: 3.1.0", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.descriptors).toEqual(["@aws/types@npm:^3.0.0", "@aws/types@npm:^3.1.0"]); + }); + + it("parses scalar, map, and nested-map fields", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"@backstage/theme@backstage:^":', + " version: 0.6.3", + ' resolution: "@backstage/theme@npm:0.6.3"', + " dependencies:", + ' "@emotion/react": "npm:^11.0.0"', + " peerDependencies:", + ' react: "npm:^18.0.0"', + " peerDependenciesMeta:", + " react:", + " optional: true", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + + expect(block.fields.version).toEqual({ kind: "scalar", raw: "0.6.3" }); + expect(block.fields.resolution).toEqual({ + kind: "scalar", + raw: '"@backstage/theme@npm:0.6.3"', + }); + expect(block.fields.dependencies).toEqual({ + kind: "map", + entries: { '"@emotion/react"': '"npm:^11.0.0"' }, + }); + expect(block.fields.peerDependencies).toEqual({ + kind: "map", + entries: { react: '"npm:^18.0.0"' }, + }); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "true" } }, + }); + expect(block.fields.linkType).toEqual({ kind: "scalar", raw: "hard" }); + }); + + it("skips blank lines between blocks", () => { + const input = [ + "__metadata:", + " version: 8", + "", + "", + '"a@npm:1":', + " version: 1", + "", + "", + "", + '"b@npm:2":', + " version: 2", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.blocks).toHaveLength(2); + }); + + it("skips stray non-descriptor lines", () => { + const input = [ + "__metadata:", + " version: 8", + "", + "this is a stray line without trailing colon", + "", + '"pkg@npm:1":', + " version: 1", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.blocks).toHaveLength(1); + const block = expectFirstBlock(lockfile); + expect(block.descriptors).toEqual(["pkg@npm:1"]); + }); + + it("handles unquoted descriptor (non-standard)", () => { + const input = ["my-pkg@npm:1.0.0:", " version: 1.0.0", ""].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.blocks).toHaveLength(1); + const block = expectFirstBlock(lockfile); + expect(block.descriptors).toEqual(["my-pkg@npm:1.0.0"]); + }); + + it("skips mid-file comment lines", () => { + const input = [ + "__metadata:", + " version: 8", + "", + "# Inline comment between blocks", + '"pkg@npm:1":', + " version: 1", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.blocks).toHaveLength(1); + }); + + it("handles nested-map with orphan property before key", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"pkg@npm:1":', + " version: 1", + " peerDependenciesMeta:", + " orphan: true", + " react:", + " optional: true", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "true" } }, + }); + expect(block.fields.linkType).toEqual({ kind: "scalar", raw: "hard" }); + }); +}); + +// --------------------------------------------------------------------------- +// Serialization +// --------------------------------------------------------------------------- + +describe("serializeBlock", () => { + it("quotes all descriptors", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:^1.0.0"], + fields: { version: { kind: "scalar", raw: "1.0.0" } }, + }; + const output = serializeBlock(block); + expect(output).toBe('"pkg@npm:^1.0.0":\n version: 1.0.0'); + }); + + it("serializes combined descriptors within one pair of quotes", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:^1.0.0", "pkg@npm:^1.1.0"], + fields: { version: { kind: "scalar", raw: "1.1.0" } }, + }; + const output = serializeBlock(block); + expect(output.startsWith('"pkg@npm:^1.0.0, pkg@npm:^1.1.0":')).toBe(true); + }); +}); + +// --------------------------------------------------------------------------- +// Convenience accessors +// --------------------------------------------------------------------------- + +describe("getScalar", () => { + it("returns unquoted value", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:1"], + fields: { + version: { kind: "scalar", raw: "1.0.0" }, + resolution: { kind: "scalar", raw: '"pkg@npm:1.0.0"' }, + }, + }; + expect(getScalar(block, "version")).toBe("1.0.0"); + expect(getScalar(block, "resolution")).toBe("pkg@npm:1.0.0"); + }); + + it("returns undefined for missing or non-scalar field", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:1"], + fields: { + deps: { kind: "map", entries: { a: "1" } }, + }, + }; + expect(getScalar(block, "missing")).toBeUndefined(); + expect(getScalar(block, "deps")).toBeUndefined(); + }); +}); + +describe("getMap", () => { + it("returns entries", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:1"], + fields: { + dependencies: { + kind: "map", + entries: { react: '"npm:^18.0.0"' }, + }, + }, + }; + expect(getMap(block, "dependencies")).toEqual({ + react: '"npm:^18.0.0"', + }); + }); + + it("returns undefined for missing or non-map field", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:1"], + fields: { + version: { kind: "scalar", raw: "1" }, + }, + }; + expect(getMap(block, "missing")).toBeUndefined(); + expect(getMap(block, "version")).toBeUndefined(); + }); +}); + +describe("getNestedMap", () => { + it("returns entries", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:1"], + fields: { + peerDependenciesMeta: { + kind: "nested-map", + entries: { react: { optional: "true" } }, + }, + }, + }; + expect(getNestedMap(block, "peerDependenciesMeta")).toEqual({ + react: { optional: "true" }, + }); + }); + + it("returns undefined for missing or non-nested-map field", () => { + const block: LockfileBlock = { + descriptors: ["pkg@npm:1"], + fields: { + version: { kind: "scalar", raw: "1" }, + }, + }; + expect(getNestedMap(block, "missing")).toBeUndefined(); + expect(getNestedMap(block, "version")).toBeUndefined(); + }); +}); + +// --------------------------------------------------------------------------- +// Mutation scenarios (for protocol-resolution) +// --------------------------------------------------------------------------- + +describe("mutation", () => { + it("allows updating a scalar value while preserving order", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"pkg@npm:^1.0.0":', + " version: 1.0.0", + ' resolution: "pkg@npm:1.0.0"', + " checksum: abc", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + block.fields.version = { kind: "scalar", raw: "2.0.0" }; + + expect(roundTrip(lockfile)).toBe(input.replace("version: 1.0.0", "version: 2.0.0")); + }); + + it("allows replacing a dependency value in a map", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"pkg@npm:^1.0.0":', + " version: 1.0.0", + " dependencies:", + ' react: "npm:^17.0.0"', + ' lodash: "npm:^4.0.0"', + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + const deps = block.fields.dependencies; + if (deps?.kind === "map") { + deps.entries.react = '"npm:^18.0.0"'; + } + + const output = roundTrip(lockfile); + expect(output).toContain('react: "npm:^18.0.0"'); + expect(output).toContain('lodash: "npm:^4.0.0"'); + expect(output.indexOf("react")).toBeLessThan(output.indexOf("lodash")); + }); + + it("allows adding a new block", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"pkg-a@npm:^1.0.0":', + " version: 1.0.0", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + lockfile.blocks.push({ + descriptors: ["pkg-b@npm:^2.0.0"], + fields: { + version: { kind: "scalar", raw: "2.0.0" }, + linkType: { kind: "scalar", raw: "hard" }, + }, + }); + + const output = roundTrip(lockfile); + expect(output).toContain('"pkg-b@npm:^2.0.0":'); + expect(output).toContain("version: 2.0.0"); + }); + + it("allows removing a block", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"pkg-a@npm:^1.0.0":', + " version: 1.0.0", + "", + '"pkg-b@npm:^2.0.0":', + " version: 2.0.0", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + lockfile.blocks = lockfile.blocks.filter((b) => { + const firstDescriptor = b.descriptors[0]; + return firstDescriptor === undefined || !firstDescriptor.includes("pkg-b"); + }); + + const output = roundTrip(lockfile); + expect(output).toContain("pkg-a"); + expect(output).not.toContain("pkg-b"); + }); + + it("allows enriching descriptors (adding a specifier)", () => { + const input = [ + "__metadata:", + " version: 8", + "", + '"@backstage/core@backstage:^":', + " version: 1.0.0", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + block.descriptors.push("@backstage/core@npm:1.0.0"); + + const output = roundTrip(lockfile); + expect(output).toContain('"@backstage/core@backstage:^, @backstage/core@npm:1.0.0":'); + }); +}); + +// --------------------------------------------------------------------------- +// Edge cases: defensive branches +// --------------------------------------------------------------------------- + +describe("parser edge cases", () => { + it("stops parsing block body on an empty line", () => { + const input = ['"pkg@npm:1":', " version: 1.0.0", "", " linkType: hard", ""].join("\n"); + + const lockfile = parseLockfile(input); + expect(lockfile.blocks).toHaveLength(1); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toEqual({ + kind: "scalar", + raw: "1.0.0", + }); + expect(block.fields.linkType).toBeUndefined(); + }); + + it("stops parsing block body on a 2-space line that is not a valid field", () => { + const input = [ + '"pkg@npm:1":', + " version: 1.0.0", + " no-colon-here", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toBeDefined(); + expect(block.fields.linkType).toBeUndefined(); + }); + + it("handles a section header with no sub-content", () => { + const input = [ + '"pkg@npm:1":', + " version: 1.0.0", + " dependencies:", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.dependencies).toBeUndefined(); + expect(block.fields.linkType).toEqual({ kind: "scalar", raw: "hard" }); + }); + + it("skips malformed 4-space map entries", () => { + const input = [ + '"pkg@npm:1":', + " version: 1.0.0", + " dependencies:", + " valid-dep: 1.0.0", + " no-colon-garbage", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.dependencies).toEqual({ + kind: "map", + entries: { "valid-dep": "1.0.0" }, + }); + }); + + it("skips malformed 6-space nested-map property lines", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " react:", + " optional: true", + " garbage-no-colon", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "true" } }, + }); + }); + + it("breaks nested-map on a 4-space line that is not a key header", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " react:", + " optional: true", + " not-a-key-header 1.0.0", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "true" } }, + }); + }); + + it("stops parsing on degenerate field lines with empty name", () => { + const input = [ + '"pkg@npm:1":', + " version: 1.0.0", + " : orphan-value", + " :", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toBeDefined(); + expect(block.fields[""]).toBeUndefined(); + expect(block.fields.linkType).toBeUndefined(); + }); + + it("stops parsing on field lines with colon not followed by whitespace", () => { + const input = [ + '"pkg@npm:1":', + " version: 1.0.0", + " name:value-no-space", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toBeDefined(); + expect(block.fields.name).toBeUndefined(); + expect(block.fields.linkType).toBeUndefined(); + }); + + it("handles map keys containing colons", () => { + const input = [ + '"pkg@npm:1":', + " dependencies:", + ' "key:with:colons": "npm:^1.0.0"', + " simple-key: value", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.dependencies).toEqual({ + kind: "map", + entries: { + '"key:with:colons"': '"npm:^1.0.0"', + "simple-key": "value", + }, + }); + }); + + it("handles nested-map keys containing colons", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " key:sub:", + " optional: true", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { "key:sub": { optional: "true" } }, + }); + }); + + it("handles nested-map property names containing colons", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " react:", + " prop:name: true", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { "prop:name": "true" } }, + }); + }); + + it("drops orphan 4-space line that appears before any field header", () => { + const input = ['"pkg@npm:1":', " orphan-deep-line: value", " version: 1.0.0", ""].join( + "\n", + ); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toEqual({ kind: "scalar", raw: "1.0.0" }); + }); + + it("stops block parsing on stray non-indented line before first field", () => { + const input = ['"pkg@npm:1":', "stray-line-at-top", " version: 1.0.0", ""].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.version).toBeUndefined(); + }); + + it("stops block parsing on stray non-indented line inside a map section", () => { + const input = [ + '"pkg@npm:1":', + " dependencies:", + " valid: 1.0.0", + "stray-line", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.dependencies).toEqual({ + kind: "map", + entries: { valid: "1.0.0" }, + }); + expect(block.fields.linkType).toBeUndefined(); + }); + + it("stops block parsing on stray non-indented line inside a nested-map section", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " react:", + " optional: true", + "stray-line", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "true" } }, + }); + expect(block.fields.linkType).toBeUndefined(); + }); + + it("does not misidentify 5-space lines as nested-map headers", () => { + const input = [ + '"pkg@npm:1":', + " section:", + " five-space-key: value", + " normal-key: value", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + // 5-space line is NOT a nested-map header → section is parsed as flat map + expect(block.fields.section).toEqual({ + kind: "map", + entries: { " five-space-key": "value", "normal-key": "value" }, + }); + }); + + it("skips map entries with empty key (colon at indent boundary)", () => { + const input = [ + '"pkg@npm:1":', + " dependencies:", + " : empty-key-value", + " valid: 1.0.0", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.dependencies).toEqual({ + kind: "map", + entries: { valid: "1.0.0" }, + }); + }); + + it("skips nested-map key headers with empty key", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " :", + " optional: true", + " react:", + " optional: false", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "false" } }, + }); + }); + + it("skips nested-map properties with empty prop name", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " react:", + " : empty-prop-value", + " optional: true", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: { react: { optional: "true" } }, + }); + }); + + it("produces empty map when all entries are malformed", () => { + const input = [ + '"pkg@npm:1":', + " dependencies:", + " no-colon-here", + " also-no-colon", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.dependencies).toEqual({ + kind: "map", + entries: {}, + }); + }); + + it("produces empty nested-map when all keys are malformed", () => { + const input = [ + '"pkg@npm:1":', + " peerDependenciesMeta:", + " not-a-key-header 1.0.0", + " orphan: true", + " linkType: hard", + "", + ].join("\n"); + + const lockfile = parseLockfile(input); + const block = expectFirstBlock(lockfile); + expect(block.fields.peerDependenciesMeta).toEqual({ + kind: "nested-map", + entries: {}, + }); + }); + + it("serializes lockfile with empty preamble", () => { + const lockfile: Lockfile = { + preamble: "", + blocks: [ + { + descriptors: ["pkg@npm:1"], + fields: { version: { kind: "scalar", raw: "1.0.0" } }, + }, + ], + }; + const output = serializeLockfile(lockfile); + expect(output).toBe('\n"pkg@npm:1":\n version: 1.0.0\n'); + }); +}); + +// --------------------------------------------------------------------------- +// Real-world lockfile round-trip +// --------------------------------------------------------------------------- + +describe("real-world fixture round-trip", () => { + it("round-trips the monorepo-with-backstage fixture", async () => { + const fs = await import("node:fs"); + const path = await import("node:path"); + const fixturePath = path.join( + import.meta.dirname, + "modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/yarn.lock", + ); + const content = fs.readFileSync(fixturePath, "utf8"); + const lockfile = parseLockfile(content); + const output = serializeLockfile(lockfile); + expect(output).toBe(content); + }); +}); diff --git a/prepare-sources/lib/yarn-lock-parser.ts b/prepare-sources/lib/yarn-lock-parser.ts new file mode 100644 index 0000000..482d2ec --- /dev/null +++ b/prepare-sources/lib/yarn-lock-parser.ts @@ -0,0 +1,392 @@ +/** + * Zero-dependency Yarn Berry lockfile (SYML) parser and serializer. + * + * Designed for round-trip fidelity: parse → serialize produces output + * identical to the input for unmodified content. Field and entry ordering + * is preserved via JavaScript object insertion-order guarantees (ES2015+). + * + * Used by `generate-manifests` (read-only extraction), `protocol-resolution` + * (block replacement, descriptor enrichment, value rewriting), `package-cleanup` + * (block removal), and `validate` (structural comparison). + */ + +// --------------------------------------------------------------------------- +// Types +// --------------------------------------------------------------------------- + +export type Lockfile = { + preamble: string; + blocks: LockfileBlock[]; +}; + +export type LockfileBlock = { + descriptors: string[]; + fields: { [name: string]: LockfileField }; +}; + +export type LockfileField = ScalarField | MapField | NestedMapField; + +export type ScalarField = { kind: "scalar"; raw: string }; +export type MapField = { kind: "map"; entries: { [key: string]: string } }; +export type NestedMapField = { + kind: "nested-map"; + entries: { [key: string]: { [prop: string]: string } }; +}; + +// --------------------------------------------------------------------------- +// String-based matchers — no regex captures, no `string | undefined` +// +// Each function uses indexOf + substring + trimStart/trim to parse SYML +// field lines. All return types are fully narrowed (`string`, not +// `string | undefined`), so no v8-ignore guards are needed. +// --------------------------------------------------------------------------- + +/** + * Match a 2-space scalar field: ` name: value` + * Equivalent to `/^ ([^:]+):\s+(.+)$/` + * Name cannot contain colons → first colon is the separator. + */ +function matchScalar(line: string): { name: string; raw: string } | null { + const colonPos = line.indexOf(":", 2); + if (colonPos === -1) return null; + const name = line.substring(2, colonPos); + if (name.length === 0) return null; + const rest = line.substring(colonPos + 1); + const raw = rest.trimStart(); + if (raw.length === 0 || rest.length === raw.length) return null; + return { name, raw }; +} + +/** + * Match a 2-space section header: ` name:` (with optional trailing whitespace) + * Equivalent to `/^ ([^:]+):\s*$/` + * Name cannot contain colons → first colon is the separator. + */ +function matchSectionHeader(line: string): string | null { + const colonPos = line.indexOf(":", 2); + if (colonPos === -1) return null; + const name = line.substring(2, colonPos); + if (name.length === 0) return null; + const rest = line.substring(colonPos + 1); + if (rest.trim().length !== 0) return null; + return name; +} + +/** + * Match a 4-space map entry: ` key: value` + * Equivalent to `/^\s{4}(.+?):\s+(.+)$/` + * Key may contain colons (non-greedy) → try each colon left-to-right. + */ +function matchMapEntry(line: string): [key: string, value: string] | null { + let colonPos = line.indexOf(":", 4); + while (colonPos !== -1) { + const rest = line.substring(colonPos + 1); + const value = rest.trimStart(); + if (rest.length !== value.length && value.length > 0) { + const key = line.substring(4, colonPos); + if (key.length > 0) return [key, value]; + } + colonPos = line.indexOf(":", colonPos + 1); + } + return null; +} + +/** + * Match a 4-space nested-map key header: ` key:` (with optional trailing ws) + * Equivalent to `/^\s{4}(.+?):\s*$/` + * Key may contain colons → try each colon left-to-right. + */ +function matchNestedKey(line: string): string | null { + let colonPos = line.indexOf(":", 4); + while (colonPos !== -1) { + const rest = line.substring(colonPos + 1); + if (rest.trim().length === 0) { + const key = line.substring(4, colonPos); + if (key.length > 0) return key; + } + colonPos = line.indexOf(":", colonPos + 1); + } + return null; +} + +/** + * Match a 6-space nested-map property: ` prop: value` + * Equivalent to `/^\s{6}(.+?):\s+(.+)$/` + * Prop may contain colons → try each colon left-to-right. + */ +function matchNestedProp(line: string): [prop: string, value: string] | null { + let colonPos = line.indexOf(":", 6); + while (colonPos !== -1) { + const rest = line.substring(colonPos + 1); + const value = rest.trimStart(); + if (rest.length !== value.length && value.length > 0) { + const prop = line.substring(6, colonPos); + if (prop.length > 0) return [prop, value]; + } + colonPos = line.indexOf(":", colonPos + 1); + } + return null; +} + +/** + * Detect whether a 4-space line is a nested-map key header (e.g. ` react:`) + * rather than a flat map entry (e.g. ` react: "npm:^18"`). + * Equivalent to `/^\s{4}\S.*:\s*$/` — 4-space indent, non-space start, colon + * at end with optional trailing whitespace. + */ +function isNestedMapHeader(line: string): boolean { + if (!line.startsWith(" ") || line.charAt(4) === " ") return false; + const colonPos = line.indexOf(":", 4); + if (colonPos === -1) return false; + return line.substring(colonPos + 1).trim().length === 0; +} + +// --------------------------------------------------------------------------- +// Structured grouping types — typed fields instead of raw string[] +// --------------------------------------------------------------------------- + +type BlockGroup = { descriptor: string; body: string[] }; +type FieldGroup = { header: string; subLines: string[] }; + +// --------------------------------------------------------------------------- +// Parser +// --------------------------------------------------------------------------- + +export function parseLockfile(content: string): Lockfile { + const lines = content.split("\n"); + + const preambleEnd = findPreambleEnd(lines); + const preamble = lines.slice(0, preambleEnd).join("\n"); + + const blockLines = lines.slice(preambleEnd); + const blocks = splitIntoBlockGroups(blockLines).map(parseBlockGroup); + + return { preamble, blocks }; +} + +/** + * Find where the preamble ends (comment lines, blank lines, __metadata block). + * Returns the index of the first non-preamble line. + */ +function findPreambleEnd(lines: string[]): number { + let i = 0; + for (const line of lines) { + if (line.startsWith("#") || line === "") { + i++; + } else if (line.startsWith("__metadata:")) { + i++; // the __metadata: line itself + for (const metaLine of lines.slice(i)) { + if (!metaLine.startsWith(" ")) break; + i++; + } + break; + } else { + break; + } + } + return i; +} + +/** + * Split lines into block groups, where each group starts with a descriptor + * line (starts at column 0, ends with ":") and includes all following + * indented (2+-space) body lines. Blank lines and 0-space non-descriptor + * lines stop body collection — matching the original parser's strict + * block-termination behavior. + */ +function splitIntoBlockGroups(lines: string[]): BlockGroup[] { + const groups: BlockGroup[] = []; + let current: BlockGroup | null = null; + for (const line of lines) { + if (!line.startsWith(" ") && !line.startsWith("#") && line.endsWith(":")) { + current = { descriptor: line, body: [] }; + groups.push(current); + } else if (current !== null && line.startsWith(" ")) { + current.body.push(line); + } else { + current = null; + } + } + return groups; +} + +function parseBlockGroup(group: BlockGroup): LockfileBlock { + const descriptors = parseDescriptors(group.descriptor.slice(0, -1)); + const fields = parseFields(group.body); + return { descriptors, fields }; +} + +function parseDescriptors(raw: string): string[] { + const trimmed = raw.trim(); + if (trimmed.startsWith('"') && trimmed.endsWith('"')) { + return trimmed.slice(1, -1).split(", "); + } + return [trimmed]; +} + +/** + * Group body lines by top-level field (2-space indent), then parse each + * group into a LockfileField. + */ +function parseFields(bodyLines: string[]): { [name: string]: LockfileField } { + const fields: { [name: string]: LockfileField } = {}; + + for (const { header, subLines } of groupByField(bodyLines)) { + const scalar = matchScalar(header); + if (scalar) { + fields[scalar.name] = { kind: "scalar", raw: scalar.raw }; + continue; + } + + const sectionName = matchSectionHeader(header); + if (sectionName && subLines.length > 0) { + const isNested = subLines.some((l) => l.startsWith(" ") || isNestedMapHeader(l)); + if (isNested) { + fields[sectionName] = parseNestedMapLines(subLines); + } else { + fields[sectionName] = parseMapLines(subLines); + } + continue; + } + if (sectionName) continue; // valid section header with no sub-content + break; // unrecognized line — stop parsing (matches original behavior) + } + + return fields; +} + +/** + * Group body lines so that each 2-space line starts a new group, and + * deeper-indented lines (4+) are appended to the current group. + * Returns structured objects with typed fields. + */ +function groupByField(lines: string[]): FieldGroup[] { + const groups: FieldGroup[] = []; + let current: FieldGroup | null = null; + for (const line of lines) { + if (line.startsWith(" ") && !line.startsWith(" ")) { + current = { header: line, subLines: [] }; + groups.push(current); + } else if (current !== null) { + current.subLines.push(line); + } + } + return groups; +} + +function parseMapLines(lines: string[]): MapField { + const entries: { [key: string]: string } = {}; + for (const line of lines) { + const kv = matchMapEntry(line); + if (kv) { + entries[kv[0]] = kv[1]; + } + } + return { kind: "map", entries }; +} + +function parseNestedMapLines(lines: string[]): NestedMapField { + const entries: { [key: string]: { [prop: string]: string } } = {}; + let currentEntry: { [prop: string]: string } | null = null; + + for (const line of lines) { + if (line.startsWith(" ")) { + if (currentEntry) { + const pv = matchNestedProp(line); + if (pv) { + currentEntry[pv[0]] = pv[1]; + } + } + continue; + } + + const key = matchNestedKey(line); + if (key) { + currentEntry = {}; + entries[key] = currentEntry; + } + } + + return { kind: "nested-map", entries }; +} + +// --------------------------------------------------------------------------- +// Serializer +// --------------------------------------------------------------------------- + +export function serializeLockfile(lockfile: Lockfile): string { + const parts: string[] = []; + + if (lockfile.preamble) { + parts.push(lockfile.preamble); + } + + for (const block of lockfile.blocks) { + parts.push("", serializeBlock(block)); + } + + return parts.join("\n") + "\n"; +} + +export function serializeBlock(block: LockfileBlock): string { + const descriptorLine = `"${block.descriptors.join(", ")}":`; + const lines = [descriptorLine]; + + for (const [name, field] of Object.entries(block.fields)) { + switch (field.kind) { + case "scalar": + lines.push(` ${name}: ${field.raw}`); + break; + case "map": + lines.push(` ${name}:`); + for (const [key, value] of Object.entries(field.entries)) { + lines.push(` ${key}: ${value}`); + } + break; + case "nested-map": + lines.push(` ${name}:`); + for (const [key, props] of Object.entries(field.entries)) { + lines.push(` ${key}:`); + for (const [prop, value] of Object.entries(props)) { + lines.push(` ${prop}: ${value}`); + } + } + break; + /* v8 ignore next 4 -- exhaustive switch; compile-time guarantee */ + default: { + const exhaustiveCheck: never = field; + return exhaustiveCheck; + } + } + } + + return lines.join("\n"); +} + +// --------------------------------------------------------------------------- +// Convenience accessors +// --------------------------------------------------------------------------- + +export function getScalar(block: LockfileBlock, fieldName: string): string | undefined { + const field = block.fields[fieldName]; + if (field?.kind !== "scalar") return undefined; + return field.raw.replace(/^"|"$/g, ""); +} + +export function getMap( + block: LockfileBlock, + fieldName: string, +): { [key: string]: string } | undefined { + const field = block.fields[fieldName]; + if (field?.kind !== "map") return undefined; + return field.entries; +} + +export function getNestedMap( + block: LockfileBlock, + fieldName: string, +): { [key: string]: { [prop: string]: string } } | undefined { + const field = block.fields[fieldName]; + if (field?.kind !== "nested-map") return undefined; + return field.entries; +} From 1639b76a537694cb23351f9e760bdea820c3ebc5 Mon Sep 17 00:00:00 2001 From: Jon Koops Date: Sun, 4 Oct 2026 18:12:09 +0200 Subject: [PATCH 2/3] refactor(prepare-sources): use @yarnpkg utilities for yarn.lock parsing Remove yarn-lock-parser in favor of @yarnpkg/parsers and @yarnpkg/core. Pin got under @yarnpkg/core until https://github.com/yarnpkg/berry/pull/7282 is on npm. Signed-off-by: Jon Koops --- prepare-sources/README.md | 4 + .../lib/modules/generate-manifests/README.md | 6 +- .../modules/generate-manifests/index.test.ts | 78 +- .../lib/modules/generate-manifests/index.ts | 152 +-- prepare-sources/lib/yarn-lock-parser.test.ts | 953 --------------- prepare-sources/lib/yarn-lock-parser.ts | 392 ------ prepare-sources/package-lock.json | 1049 ++++++++++++++++- prepare-sources/package.json | 5 + 8 files changed, 1177 insertions(+), 1462 deletions(-) delete mode 100644 prepare-sources/lib/yarn-lock-parser.test.ts delete mode 100644 prepare-sources/lib/yarn-lock-parser.ts diff --git a/prepare-sources/README.md b/prepare-sources/README.md index 5ad3a77..2b8c57e 100644 --- a/prepare-sources/README.md +++ b/prepare-sources/README.md @@ -94,3 +94,7 @@ Run tests: ```bash vp test ``` + +## TODO + +Remove `overrides.@yarnpkg/core.got` when [berry#7282](https://github.com/yarnpkg/berry/pull/7282) lands on npm. diff --git a/prepare-sources/lib/modules/generate-manifests/README.md b/prepare-sources/lib/modules/generate-manifests/README.md index 6f666be..c6174a6 100644 --- a/prepare-sources/lib/modules/generate-manifests/README.md +++ b/prepare-sources/lib/modules/generate-manifests/README.md @@ -100,8 +100,8 @@ then enriched each of the ~600 packages with dependency metadata from by Backstage version to amortize the cost across workspaces. This module takes a different approach: it **extracts backstage package -metadata directly from the workspace's `yarn.lock`**, using the shared -`yarn-lock-parser` (`lib/yarn-lock-parser.ts`). +metadata directly from the workspace's `yarn.lock`**, using Yarn Berry's +official packages. ### Rationale @@ -173,7 +173,7 @@ old `update-workspace.js` already does (lines 1722, 1736, 1799). The | Error handling | Swallowed (`\|\| true`, stderr discarded) | Throws on error (pipeline aborts) | | Package discovery | Recursive scan with skip-list heuristics | `workspaces` field glob resolution + filter | | `bin` normalization | String → object (same) | String → object (same) | -| yarn.lock parsing | N/A (backstage manifest came from npm) | Shared `yarn-lock-parser` (`lib/yarn-lock-parser.ts`) | +| yarn.lock parsing | N/A (backstage manifest came from npm) | Yarn Berry packages | ## Log messages diff --git a/prepare-sources/lib/modules/generate-manifests/index.test.ts b/prepare-sources/lib/modules/generate-manifests/index.test.ts index 6d7b5d9..91e39c8 100644 --- a/prepare-sources/lib/modules/generate-manifests/index.test.ts +++ b/prepare-sources/lib/modules/generate-manifests/index.test.ts @@ -111,11 +111,13 @@ describe("generate-manifests", () => { describe("extractBackstageEntries", () => { it("extracts all field types from a complete lockfile", () => { const lockContent = ` +# Unrelated npm package — should be ignored. "react@npm:^18.0.0": version: 18.3.1 resolution: "react@npm:18.3.1" checksum: def +# Valid @backstage row with dependencies and bin. "@backstage/core-plugin-api@backstage:^": version: 1.10.9 resolution: "@backstage/core-plugin-api@npm:1.10.9" @@ -125,6 +127,7 @@ describe("extractBackstageEntries", () => { backstage-cli: ./bin/backstage-cli checksum: abc123 +# Valid @backstage row with peers, peerDependenciesMeta, optionalDependencies. "@backstage/theme@backstage:^": version: 0.6.3 resolution: "@backstage/theme@npm:0.6.3" @@ -133,7 +136,8 @@ describe("extractBackstageEntries", () => { peerDependenciesMeta: react: optional: true - @emotion/react: + # Scoped key must be quoted for parseSyml. + "@emotion/react": reason: styling optional: false optionalDependencies: @@ -162,20 +166,60 @@ describe("extractBackstageEntries", () => { }); }); + it("extracts from compound lockfile keys (comma-separated aliases)", () => { + const body = ` + version: 1.10.9 + resolution: "@backstage/core-plugin-api@npm:1.10.9" + dependencies: + "@backstage/types": "npm:^1.2.3" + checksum: abc +`; + const npmFirst = ` +"@backstage/core-plugin-api@npm:1.10.9, @backstage/core-plugin-api@backstage:^":${body}`; + const backstageFirst = ` +"@backstage/core-plugin-api@backstage:^, @backstage/core-plugin-api@npm:1.10.9":${body}`; + + const expected = { + name: "@backstage/core-plugin-api", + version: "1.10.9", + dependencies: { "@backstage/types": "npm:^1.2.3" }, + }; + + expect(extractBackstageEntries(npmFirst)).toEqual([expected]); + expect(extractBackstageEntries(backstageFirst)).toEqual([expected]); + }); + it("skips entries with malformed descriptor, missing version, or no backstage deps", () => { const lockContent = ` +# npm-only key — not @backstage / backstage:. "react@npm:^18.0.0": version: 18.3.1 checksum: def +# Descriptor without a package name. "backstage:^": version: 1.0.0 resolution: "something@npm:1.0.0" checksum: abc +# @backstage key but no string version field. "@backstage/core-plugin-api@backstage:^": resolution: "@backstage/core-plugin-api@npm:1.10.9" checksum: abc + +# backstage: protocol on a non-@backstage package name. +"other@backstage:^": + version: 1.2.3 + resolution: "other@npm:1.2.3" + checksum: ghi + +# @backstage scope but lock key has only an npm: alias (no backstage:). +"@backstage/only-npm@npm:1.0.0": + version: 1.0.0 + checksum: npm-only + +# Top-level scalar — entry body is not a mapping. +not-an-entry: scalar `; expect(extractBackstageEntries(lockContent)).toEqual([]); }); @@ -185,6 +229,7 @@ describe("extractBackstageEntries", () => { "@backstage/core-plugin-api@backstage:^": version: 1.10.9 resolution: "@backstage/core-plugin-api@npm:1.10.9" + # Empty maps — parseSyml yields null; fields should be omitted from output. dependencies: peerDependenciesMeta: checksum: abc @@ -203,8 +248,8 @@ describe("extractBackstageEntries", () => { " version: 1.10.9", ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', " dependencies:", - " no-colon-garbage", - " also-no-colon-garbage", + " # Key with null value — not a string semver range.", + " not-a-version:", " checksum: abc", ].join("\n"); const entries = extractBackstageEntries(lockContent); @@ -220,8 +265,8 @@ describe("extractBackstageEntries", () => { " version: 1.10.9", ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', " peerDependenciesMeta:", - " not-a-key-header 1.0.0", - " orphan-prop: true", + " # Package entry must be a mapping, not a string.", + ' react: "nope"', " checksum: abc", ].join("\n"); const entries = extractBackstageEntries(lockContent); @@ -230,29 +275,6 @@ describe("extractBackstageEntries", () => { expect(entry).toBeDefined(); expect(entry?.peerDependenciesMeta).toBeUndefined(); }); - - it("handles malformed lockfile content gracefully", () => { - const lockContent = [ - '"@backstage/core-plugin-api@backstage:^":', - " version: 1.10.9", - ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', - " dependencies:", - " !!!not-a-valid-line!!!", - ' "@backstage/types": "npm:^1.2.3"', - " peerDependenciesMeta:", - " orphan-field: true", - " react:", - " optional: true", - " ", - " checksum: abc", - ].join("\n"); - const entries = extractBackstageEntries(lockContent); - expect(entries).toHaveLength(1); - const entry = entries[0]; - expect(entry).toBeDefined(); - expect(entry?.dependencies).toEqual({ "@backstage/types": "npm:^1.2.3" }); - expect(entry?.peerDependenciesMeta).toEqual({ react: { optional: true } }); - }); }); // --------------------------------------------------------------------------- diff --git a/prepare-sources/lib/modules/generate-manifests/index.ts b/prepare-sources/lib/modules/generate-manifests/index.ts index aeda39e..736af05 100644 --- a/prepare-sources/lib/modules/generate-manifests/index.ts +++ b/prepare-sources/lib/modules/generate-manifests/index.ts @@ -7,13 +7,8 @@ import type { WorkspaceManifest, WorkspacePackageEntry, } from "../../manifest-types.ts"; -import { - type LockfileBlock, - getMap, - getNestedMap, - getScalar, - parseLockfile, -} from "../../yarn-lock-parser.ts"; +import { parseDescriptor, parseRange, stringifyIdent } from "@yarnpkg/core/structUtils"; +import { parseSyml } from "@yarnpkg/parsers"; const VERSIONS_BACKSTAGE_IO = "https://versions.backstage.io/v1/releases"; @@ -202,98 +197,109 @@ export function getBackstageVersion(workspacePath: string): string { } // --------------------------------------------------------------------------- -// yarn.lock extraction (using shared parser) +// yarn.lock extraction // --------------------------------------------------------------------------- +const BACKSTAGE_PROTOCOL = "backstage:"; + +/** + * Splits a compound lockfile descriptor key into individual descriptor strings. + */ +function splitDescriptorKey(lockfileKey: string): string[] { + return lockfileKey.split(/ *, */); +} + /** * Extract `@backstage/*` package entries resolved via `backstage:^` from a * yarn.lock string. Returns structured entries with version and dependency * metadata. - * - * Uses the shared `yarn-lock-parser` for structural parsing, then maps the - * parsed blocks to `BackstagePackageEntry` objects. */ export function extractBackstageEntries(lockContent: string): BackstagePackageEntry[] { - const lockfile = parseLockfile(lockContent); - return lockfile.blocks - .map(parseBackstageBlock) - .filter((entry): entry is BackstagePackageEntry => entry !== undefined); -} + const parsed = parseSyml(lockContent); + const entries: BackstagePackageEntry[] = []; -function parseBackstageBlock(block: LockfileBlock): BackstagePackageEntry | undefined { - const descriptor = block.descriptors.find((d) => d.includes("backstage:^")); - if (!descriptor) return undefined; + for (const [key, raw] of Object.entries(parsed)) { + if (key === "__metadata") continue; + const entry = toBackstageEntry(key, raw); + if (entry) entries.push(entry); + } - const nameMatch = /(@backstage\/[^@]+)/.exec(descriptor); - const name = nameMatch?.[1]; - if (!name) return undefined; + return entries; +} - const version = getScalar(block, "version"); - if (!version) return undefined; +/** + * Maps a yarn.lock entry to a {@link BackstagePackageEntry}. + * + * @param key - Compound descriptor key for a single top-level yarn.lock entry. + * @param raw - Parsed YAML body (`version`, `dependencies`, etc.). + * @returns Metadata when this row resolves an `@backstage/*` package from the + * workspace `backstage:^` range; `undefined` otherwise. + */ +function toBackstageEntry(key: string, raw: unknown): BackstagePackageEntry | undefined { + if (!isRecord(raw)) return undefined; - const entry: BackstagePackageEntry = { name, version }; + const name = backstagePackageName(key); + if (!name) return undefined; - const deps = getStrippedMap(block, "dependencies"); - if (deps) entry.dependencies = deps; + const version = raw.version; + if (typeof version !== "string") return undefined; - const peers = getStrippedMap(block, "peerDependencies"); - if (peers) entry.peerDependencies = peers; + return { + name, + version, + dependencies: stringRecord(raw.dependencies), + peerDependencies: stringRecord(raw.peerDependencies), + peerDependenciesMeta: peerMetaRecord(raw.peerDependenciesMeta), + optionalDependencies: stringRecord(raw.optionalDependencies), + bin: stringRecord(raw.bin), + }; +} - const peersMeta = getNestedMap(block, "peerDependenciesMeta"); - if (peersMeta) { - entry.peerDependenciesMeta = convertNestedMapValues(peersMeta); +/** + * Resolves the npm package name for a yarn.lock entry from its compound descriptor key. + * + * @param key - Compound descriptor key for a single top-level yarn.lock entry. + * @returns Package name from the `backstage:` alias in `key`, or `undefined` + * when `key` does not denote an `@backstage/*` `backstage:^` dependency. + */ +function backstagePackageName(key: string): string | undefined { + for (const descriptorString of splitDescriptorKey(key)) { + const descriptor = parseDescriptor(descriptorString); + // Only `@backstage/*` packages (aliases in a key share the same scope and name). + if (descriptor.scope !== "backstage") continue; + // Compound keys can list several aliases; only the `backstage:` one counts here. + if (parseRange(descriptor.range).protocol !== BACKSTAGE_PROTOCOL) continue; + return stringifyIdent(descriptor); } - - const optional = getStrippedMap(block, "optionalDependencies"); - if (optional) entry.optionalDependencies = optional; - - const bin = getStrippedMap(block, "bin"); - if (bin) entry.bin = bin; - - return entry; + return undefined; } -function stripMapQuotes(raw: { [key: string]: string }): Record { +function stringRecord(value: unknown): Record | undefined { + if (!isRecord(value)) return undefined; const result: Record = {}; - for (const [key, value] of Object.entries(raw)) { - result[key.replace(/^"|"$/g, "")] = value.replace(/^"|"$/g, ""); + for (const [key, raw] of Object.entries(value)) { + if (typeof raw !== "string") continue; + result[key] = raw; } - return result; -} - -function getStrippedMap( - block: LockfileBlock, - fieldName: string, -): Record | undefined { - const raw = getMap(block, fieldName); - if (!raw) return undefined; - const stripped = stripMapQuotes(raw); - return Object.keys(stripped).length > 0 ? stripped : undefined; + return Object.keys(result).length > 0 ? result : undefined; } -function convertNestedMapValues(peersMeta: { - [key: string]: { [prop: string]: string }; -}): Record> | undefined { - const entries = Object.entries(peersMeta); - if (entries.length === 0) return undefined; +function peerMetaRecord(value: unknown): Record> | undefined { + if (!isRecord(value)) return undefined; const converted: Record> = {}; - for (const [pkg, props] of entries) { + for (const [pkg, props] of Object.entries(value)) { + if (!isRecord(props)) continue; const record: Record = {}; - converted[pkg] = record; - for (const [key, val] of Object.entries(props)) { - const unquoted = val.replace(/^"|"$/g, ""); - let value: unknown; - if (unquoted === "true") { - value = true; - } else if (unquoted === "false") { - value = false; - } else { - value = unquoted; - } - record[key] = value; + for (const [prop, raw] of Object.entries(props)) { + record[prop] = raw === "true" ? true : raw === "false" ? false : raw; } + converted[pkg] = record; } - return converted; + return Object.keys(converted).length > 0 ? converted : undefined; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null; } // --------------------------------------------------------------------------- diff --git a/prepare-sources/lib/yarn-lock-parser.test.ts b/prepare-sources/lib/yarn-lock-parser.test.ts deleted file mode 100644 index 94689e9..0000000 --- a/prepare-sources/lib/yarn-lock-parser.test.ts +++ /dev/null @@ -1,953 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { - type Lockfile, - type LockfileBlock, - getMap, - getNestedMap, - getScalar, - parseLockfile, - serializeBlock, - serializeLockfile, -} from "./yarn-lock-parser.ts"; - -function roundTrip(lockfile: Lockfile): string { - return serializeLockfile(lockfile); -} - -function expectFirstBlock(lockfile: Lockfile): LockfileBlock { - const block = lockfile.blocks[0]; - expect(block).toBeDefined(); - if (!block) { - throw new Error("expected block"); - } - return block; -} - -// --------------------------------------------------------------------------- -// Round-trip fidelity -// --------------------------------------------------------------------------- - -describe("round-trip fidelity", () => { - const cases: [string, string][] = [ - [ - "simple lockfile", - [ - '# This file is generated by running "yarn install" inside your project.', - "# Manual changes might be lost - proceed with caution!", - "", - "__metadata:", - " version: 8", - " cacheKey: 10c0", - "", - '"@backstage/core-plugin-api@backstage:^":', - " version: 1.10.9", - ' resolution: "@backstage/core-plugin-api@npm:1.10.9"', - " dependencies:", - ' "@backstage/types": "npm:^1.2.3"', - ' react: "npm:^18.0.0"', - " checksum: abc123", - " languageName: node", - " linkType: hard", - "", - ].join("\n"), - ], - [ - "combined keys", - [ - "__metadata:", - " version: 8", - "", - '"@aws-sdk/types@npm:^3.222.0, @aws-sdk/types@npm:^3.347.0, @aws-sdk/types@npm:^3.973.1":', - " version: 3.973.1", - ' resolution: "@aws-sdk/types@npm:3.973.1"', - " checksum: abc", - "", - ].join("\n"), - ], - [ - "nested map (peerDependenciesMeta)", - [ - "__metadata:", - " version: 8", - "", - '"@backstage/theme@backstage:^":', - " version: 0.6.3", - " peerDependencies:", - ' react: "npm:^18.0.0"', - " peerDependenciesMeta:", - " react:", - " optional: true", - " linkType: hard", - "", - ].join("\n"), - ], - [ - "unquoted simple keys in deps", - [ - "__metadata:", - " version: 8", - "", - '"ajv@npm:^8.17.1":', - " version: 8.17.1", - " dependencies:", - ' fast-deep-equal: "npm:^3.1.3"', - ' json-schema-traverse: "npm:^1.0.0"', - ' require-from-string: "npm:^2.0.2"', - ' uri-js: "npm:^4.4.1"', - " linkType: hard", - "", - ].join("\n"), - ], - [ - "workspace block", - [ - "__metadata:", - " version: 8", - "", - '"@test/root@workspace:.":', - " version: 0.0.0-use.local", - ' resolution: "@test/root@workspace:."', - " languageName: unknown", - " linkType: soft", - "", - ].join("\n"), - ], - [ - "block with bin", - [ - "__metadata:", - " version: 8", - "", - '"@backstage/core-plugin-api@backstage:^":', - " version: 1.10.9", - " bin:", - " backstage-core: ./dist/cli.js", - " checksum: abc123", - "", - ].join("\n"), - ], - [ - "multiple blocks", - [ - "__metadata:", - " version: 8", - "", - '"pkg-a@npm:^1.0.0":', - " version: 1.0.0", - " linkType: hard", - "", - '"pkg-b@npm:^2.0.0":', - " version: 2.0.0", - " dependencies:", - ' pkg-a: "npm:^1.0.0"', - " linkType: hard", - "", - ].join("\n"), - ], - ]; - - it.each(cases)("%s", (_name, input) => { - const lockfile = parseLockfile(input); - const output = serializeLockfile(lockfile); - expect(output).toBe(input); - }); -}); - -// --------------------------------------------------------------------------- -// Parsing -// --------------------------------------------------------------------------- - -describe("parseLockfile", () => { - it("captures the preamble including __metadata", () => { - const input = [ - "# comment", - "", - "__metadata:", - " version: 8", - " cacheKey: 10c0", - "", - '"pkg@npm:^1.0.0":', - " version: 1.0.0", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.preamble).toBe("# comment\n\n__metadata:\n version: 8\n cacheKey: 10c0"); - expect(lockfile.blocks).toHaveLength(1); - }); - - it("handles lockfile without __metadata", () => { - const input = ['"pkg@npm:^1.0.0":', " version: 1.0.0", ""].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.preamble).toBe(""); - expect(lockfile.blocks).toHaveLength(1); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toEqual({ - kind: "scalar", - raw: "1.0.0", - }); - }); - - it("parses combined descriptor keys", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"@aws/types@npm:^3.0.0, @aws/types@npm:^3.1.0":', - " version: 3.1.0", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.descriptors).toEqual(["@aws/types@npm:^3.0.0", "@aws/types@npm:^3.1.0"]); - }); - - it("parses scalar, map, and nested-map fields", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"@backstage/theme@backstage:^":', - " version: 0.6.3", - ' resolution: "@backstage/theme@npm:0.6.3"', - " dependencies:", - ' "@emotion/react": "npm:^11.0.0"', - " peerDependencies:", - ' react: "npm:^18.0.0"', - " peerDependenciesMeta:", - " react:", - " optional: true", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - - expect(block.fields.version).toEqual({ kind: "scalar", raw: "0.6.3" }); - expect(block.fields.resolution).toEqual({ - kind: "scalar", - raw: '"@backstage/theme@npm:0.6.3"', - }); - expect(block.fields.dependencies).toEqual({ - kind: "map", - entries: { '"@emotion/react"': '"npm:^11.0.0"' }, - }); - expect(block.fields.peerDependencies).toEqual({ - kind: "map", - entries: { react: '"npm:^18.0.0"' }, - }); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "true" } }, - }); - expect(block.fields.linkType).toEqual({ kind: "scalar", raw: "hard" }); - }); - - it("skips blank lines between blocks", () => { - const input = [ - "__metadata:", - " version: 8", - "", - "", - '"a@npm:1":', - " version: 1", - "", - "", - "", - '"b@npm:2":', - " version: 2", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.blocks).toHaveLength(2); - }); - - it("skips stray non-descriptor lines", () => { - const input = [ - "__metadata:", - " version: 8", - "", - "this is a stray line without trailing colon", - "", - '"pkg@npm:1":', - " version: 1", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.blocks).toHaveLength(1); - const block = expectFirstBlock(lockfile); - expect(block.descriptors).toEqual(["pkg@npm:1"]); - }); - - it("handles unquoted descriptor (non-standard)", () => { - const input = ["my-pkg@npm:1.0.0:", " version: 1.0.0", ""].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.blocks).toHaveLength(1); - const block = expectFirstBlock(lockfile); - expect(block.descriptors).toEqual(["my-pkg@npm:1.0.0"]); - }); - - it("skips mid-file comment lines", () => { - const input = [ - "__metadata:", - " version: 8", - "", - "# Inline comment between blocks", - '"pkg@npm:1":', - " version: 1", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.blocks).toHaveLength(1); - }); - - it("handles nested-map with orphan property before key", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"pkg@npm:1":', - " version: 1", - " peerDependenciesMeta:", - " orphan: true", - " react:", - " optional: true", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "true" } }, - }); - expect(block.fields.linkType).toEqual({ kind: "scalar", raw: "hard" }); - }); -}); - -// --------------------------------------------------------------------------- -// Serialization -// --------------------------------------------------------------------------- - -describe("serializeBlock", () => { - it("quotes all descriptors", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:^1.0.0"], - fields: { version: { kind: "scalar", raw: "1.0.0" } }, - }; - const output = serializeBlock(block); - expect(output).toBe('"pkg@npm:^1.0.0":\n version: 1.0.0'); - }); - - it("serializes combined descriptors within one pair of quotes", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:^1.0.0", "pkg@npm:^1.1.0"], - fields: { version: { kind: "scalar", raw: "1.1.0" } }, - }; - const output = serializeBlock(block); - expect(output.startsWith('"pkg@npm:^1.0.0, pkg@npm:^1.1.0":')).toBe(true); - }); -}); - -// --------------------------------------------------------------------------- -// Convenience accessors -// --------------------------------------------------------------------------- - -describe("getScalar", () => { - it("returns unquoted value", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:1"], - fields: { - version: { kind: "scalar", raw: "1.0.0" }, - resolution: { kind: "scalar", raw: '"pkg@npm:1.0.0"' }, - }, - }; - expect(getScalar(block, "version")).toBe("1.0.0"); - expect(getScalar(block, "resolution")).toBe("pkg@npm:1.0.0"); - }); - - it("returns undefined for missing or non-scalar field", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:1"], - fields: { - deps: { kind: "map", entries: { a: "1" } }, - }, - }; - expect(getScalar(block, "missing")).toBeUndefined(); - expect(getScalar(block, "deps")).toBeUndefined(); - }); -}); - -describe("getMap", () => { - it("returns entries", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:1"], - fields: { - dependencies: { - kind: "map", - entries: { react: '"npm:^18.0.0"' }, - }, - }, - }; - expect(getMap(block, "dependencies")).toEqual({ - react: '"npm:^18.0.0"', - }); - }); - - it("returns undefined for missing or non-map field", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:1"], - fields: { - version: { kind: "scalar", raw: "1" }, - }, - }; - expect(getMap(block, "missing")).toBeUndefined(); - expect(getMap(block, "version")).toBeUndefined(); - }); -}); - -describe("getNestedMap", () => { - it("returns entries", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:1"], - fields: { - peerDependenciesMeta: { - kind: "nested-map", - entries: { react: { optional: "true" } }, - }, - }, - }; - expect(getNestedMap(block, "peerDependenciesMeta")).toEqual({ - react: { optional: "true" }, - }); - }); - - it("returns undefined for missing or non-nested-map field", () => { - const block: LockfileBlock = { - descriptors: ["pkg@npm:1"], - fields: { - version: { kind: "scalar", raw: "1" }, - }, - }; - expect(getNestedMap(block, "missing")).toBeUndefined(); - expect(getNestedMap(block, "version")).toBeUndefined(); - }); -}); - -// --------------------------------------------------------------------------- -// Mutation scenarios (for protocol-resolution) -// --------------------------------------------------------------------------- - -describe("mutation", () => { - it("allows updating a scalar value while preserving order", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"pkg@npm:^1.0.0":', - " version: 1.0.0", - ' resolution: "pkg@npm:1.0.0"', - " checksum: abc", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - block.fields.version = { kind: "scalar", raw: "2.0.0" }; - - expect(roundTrip(lockfile)).toBe(input.replace("version: 1.0.0", "version: 2.0.0")); - }); - - it("allows replacing a dependency value in a map", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"pkg@npm:^1.0.0":', - " version: 1.0.0", - " dependencies:", - ' react: "npm:^17.0.0"', - ' lodash: "npm:^4.0.0"', - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - const deps = block.fields.dependencies; - if (deps?.kind === "map") { - deps.entries.react = '"npm:^18.0.0"'; - } - - const output = roundTrip(lockfile); - expect(output).toContain('react: "npm:^18.0.0"'); - expect(output).toContain('lodash: "npm:^4.0.0"'); - expect(output.indexOf("react")).toBeLessThan(output.indexOf("lodash")); - }); - - it("allows adding a new block", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"pkg-a@npm:^1.0.0":', - " version: 1.0.0", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - lockfile.blocks.push({ - descriptors: ["pkg-b@npm:^2.0.0"], - fields: { - version: { kind: "scalar", raw: "2.0.0" }, - linkType: { kind: "scalar", raw: "hard" }, - }, - }); - - const output = roundTrip(lockfile); - expect(output).toContain('"pkg-b@npm:^2.0.0":'); - expect(output).toContain("version: 2.0.0"); - }); - - it("allows removing a block", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"pkg-a@npm:^1.0.0":', - " version: 1.0.0", - "", - '"pkg-b@npm:^2.0.0":', - " version: 2.0.0", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - lockfile.blocks = lockfile.blocks.filter((b) => { - const firstDescriptor = b.descriptors[0]; - return firstDescriptor === undefined || !firstDescriptor.includes("pkg-b"); - }); - - const output = roundTrip(lockfile); - expect(output).toContain("pkg-a"); - expect(output).not.toContain("pkg-b"); - }); - - it("allows enriching descriptors (adding a specifier)", () => { - const input = [ - "__metadata:", - " version: 8", - "", - '"@backstage/core@backstage:^":', - " version: 1.0.0", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - block.descriptors.push("@backstage/core@npm:1.0.0"); - - const output = roundTrip(lockfile); - expect(output).toContain('"@backstage/core@backstage:^, @backstage/core@npm:1.0.0":'); - }); -}); - -// --------------------------------------------------------------------------- -// Edge cases: defensive branches -// --------------------------------------------------------------------------- - -describe("parser edge cases", () => { - it("stops parsing block body on an empty line", () => { - const input = ['"pkg@npm:1":', " version: 1.0.0", "", " linkType: hard", ""].join("\n"); - - const lockfile = parseLockfile(input); - expect(lockfile.blocks).toHaveLength(1); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toEqual({ - kind: "scalar", - raw: "1.0.0", - }); - expect(block.fields.linkType).toBeUndefined(); - }); - - it("stops parsing block body on a 2-space line that is not a valid field", () => { - const input = [ - '"pkg@npm:1":', - " version: 1.0.0", - " no-colon-here", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toBeDefined(); - expect(block.fields.linkType).toBeUndefined(); - }); - - it("handles a section header with no sub-content", () => { - const input = [ - '"pkg@npm:1":', - " version: 1.0.0", - " dependencies:", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.dependencies).toBeUndefined(); - expect(block.fields.linkType).toEqual({ kind: "scalar", raw: "hard" }); - }); - - it("skips malformed 4-space map entries", () => { - const input = [ - '"pkg@npm:1":', - " version: 1.0.0", - " dependencies:", - " valid-dep: 1.0.0", - " no-colon-garbage", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.dependencies).toEqual({ - kind: "map", - entries: { "valid-dep": "1.0.0" }, - }); - }); - - it("skips malformed 6-space nested-map property lines", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " react:", - " optional: true", - " garbage-no-colon", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "true" } }, - }); - }); - - it("breaks nested-map on a 4-space line that is not a key header", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " react:", - " optional: true", - " not-a-key-header 1.0.0", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "true" } }, - }); - }); - - it("stops parsing on degenerate field lines with empty name", () => { - const input = [ - '"pkg@npm:1":', - " version: 1.0.0", - " : orphan-value", - " :", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toBeDefined(); - expect(block.fields[""]).toBeUndefined(); - expect(block.fields.linkType).toBeUndefined(); - }); - - it("stops parsing on field lines with colon not followed by whitespace", () => { - const input = [ - '"pkg@npm:1":', - " version: 1.0.0", - " name:value-no-space", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toBeDefined(); - expect(block.fields.name).toBeUndefined(); - expect(block.fields.linkType).toBeUndefined(); - }); - - it("handles map keys containing colons", () => { - const input = [ - '"pkg@npm:1":', - " dependencies:", - ' "key:with:colons": "npm:^1.0.0"', - " simple-key: value", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.dependencies).toEqual({ - kind: "map", - entries: { - '"key:with:colons"': '"npm:^1.0.0"', - "simple-key": "value", - }, - }); - }); - - it("handles nested-map keys containing colons", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " key:sub:", - " optional: true", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { "key:sub": { optional: "true" } }, - }); - }); - - it("handles nested-map property names containing colons", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " react:", - " prop:name: true", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { "prop:name": "true" } }, - }); - }); - - it("drops orphan 4-space line that appears before any field header", () => { - const input = ['"pkg@npm:1":', " orphan-deep-line: value", " version: 1.0.0", ""].join( - "\n", - ); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toEqual({ kind: "scalar", raw: "1.0.0" }); - }); - - it("stops block parsing on stray non-indented line before first field", () => { - const input = ['"pkg@npm:1":', "stray-line-at-top", " version: 1.0.0", ""].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.version).toBeUndefined(); - }); - - it("stops block parsing on stray non-indented line inside a map section", () => { - const input = [ - '"pkg@npm:1":', - " dependencies:", - " valid: 1.0.0", - "stray-line", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.dependencies).toEqual({ - kind: "map", - entries: { valid: "1.0.0" }, - }); - expect(block.fields.linkType).toBeUndefined(); - }); - - it("stops block parsing on stray non-indented line inside a nested-map section", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " react:", - " optional: true", - "stray-line", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "true" } }, - }); - expect(block.fields.linkType).toBeUndefined(); - }); - - it("does not misidentify 5-space lines as nested-map headers", () => { - const input = [ - '"pkg@npm:1":', - " section:", - " five-space-key: value", - " normal-key: value", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - // 5-space line is NOT a nested-map header → section is parsed as flat map - expect(block.fields.section).toEqual({ - kind: "map", - entries: { " five-space-key": "value", "normal-key": "value" }, - }); - }); - - it("skips map entries with empty key (colon at indent boundary)", () => { - const input = [ - '"pkg@npm:1":', - " dependencies:", - " : empty-key-value", - " valid: 1.0.0", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.dependencies).toEqual({ - kind: "map", - entries: { valid: "1.0.0" }, - }); - }); - - it("skips nested-map key headers with empty key", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " :", - " optional: true", - " react:", - " optional: false", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "false" } }, - }); - }); - - it("skips nested-map properties with empty prop name", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " react:", - " : empty-prop-value", - " optional: true", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: { react: { optional: "true" } }, - }); - }); - - it("produces empty map when all entries are malformed", () => { - const input = [ - '"pkg@npm:1":', - " dependencies:", - " no-colon-here", - " also-no-colon", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.dependencies).toEqual({ - kind: "map", - entries: {}, - }); - }); - - it("produces empty nested-map when all keys are malformed", () => { - const input = [ - '"pkg@npm:1":', - " peerDependenciesMeta:", - " not-a-key-header 1.0.0", - " orphan: true", - " linkType: hard", - "", - ].join("\n"); - - const lockfile = parseLockfile(input); - const block = expectFirstBlock(lockfile); - expect(block.fields.peerDependenciesMeta).toEqual({ - kind: "nested-map", - entries: {}, - }); - }); - - it("serializes lockfile with empty preamble", () => { - const lockfile: Lockfile = { - preamble: "", - blocks: [ - { - descriptors: ["pkg@npm:1"], - fields: { version: { kind: "scalar", raw: "1.0.0" } }, - }, - ], - }; - const output = serializeLockfile(lockfile); - expect(output).toBe('\n"pkg@npm:1":\n version: 1.0.0\n'); - }); -}); - -// --------------------------------------------------------------------------- -// Real-world lockfile round-trip -// --------------------------------------------------------------------------- - -describe("real-world fixture round-trip", () => { - it("round-trips the monorepo-with-backstage fixture", async () => { - const fs = await import("node:fs"); - const path = await import("node:path"); - const fixturePath = path.join( - import.meta.dirname, - "modules/generate-manifests/__fixtures__/monorepo-with-backstage/input/workspace/yarn.lock", - ); - const content = fs.readFileSync(fixturePath, "utf8"); - const lockfile = parseLockfile(content); - const output = serializeLockfile(lockfile); - expect(output).toBe(content); - }); -}); diff --git a/prepare-sources/lib/yarn-lock-parser.ts b/prepare-sources/lib/yarn-lock-parser.ts deleted file mode 100644 index 482d2ec..0000000 --- a/prepare-sources/lib/yarn-lock-parser.ts +++ /dev/null @@ -1,392 +0,0 @@ -/** - * Zero-dependency Yarn Berry lockfile (SYML) parser and serializer. - * - * Designed for round-trip fidelity: parse → serialize produces output - * identical to the input for unmodified content. Field and entry ordering - * is preserved via JavaScript object insertion-order guarantees (ES2015+). - * - * Used by `generate-manifests` (read-only extraction), `protocol-resolution` - * (block replacement, descriptor enrichment, value rewriting), `package-cleanup` - * (block removal), and `validate` (structural comparison). - */ - -// --------------------------------------------------------------------------- -// Types -// --------------------------------------------------------------------------- - -export type Lockfile = { - preamble: string; - blocks: LockfileBlock[]; -}; - -export type LockfileBlock = { - descriptors: string[]; - fields: { [name: string]: LockfileField }; -}; - -export type LockfileField = ScalarField | MapField | NestedMapField; - -export type ScalarField = { kind: "scalar"; raw: string }; -export type MapField = { kind: "map"; entries: { [key: string]: string } }; -export type NestedMapField = { - kind: "nested-map"; - entries: { [key: string]: { [prop: string]: string } }; -}; - -// --------------------------------------------------------------------------- -// String-based matchers — no regex captures, no `string | undefined` -// -// Each function uses indexOf + substring + trimStart/trim to parse SYML -// field lines. All return types are fully narrowed (`string`, not -// `string | undefined`), so no v8-ignore guards are needed. -// --------------------------------------------------------------------------- - -/** - * Match a 2-space scalar field: ` name: value` - * Equivalent to `/^ ([^:]+):\s+(.+)$/` - * Name cannot contain colons → first colon is the separator. - */ -function matchScalar(line: string): { name: string; raw: string } | null { - const colonPos = line.indexOf(":", 2); - if (colonPos === -1) return null; - const name = line.substring(2, colonPos); - if (name.length === 0) return null; - const rest = line.substring(colonPos + 1); - const raw = rest.trimStart(); - if (raw.length === 0 || rest.length === raw.length) return null; - return { name, raw }; -} - -/** - * Match a 2-space section header: ` name:` (with optional trailing whitespace) - * Equivalent to `/^ ([^:]+):\s*$/` - * Name cannot contain colons → first colon is the separator. - */ -function matchSectionHeader(line: string): string | null { - const colonPos = line.indexOf(":", 2); - if (colonPos === -1) return null; - const name = line.substring(2, colonPos); - if (name.length === 0) return null; - const rest = line.substring(colonPos + 1); - if (rest.trim().length !== 0) return null; - return name; -} - -/** - * Match a 4-space map entry: ` key: value` - * Equivalent to `/^\s{4}(.+?):\s+(.+)$/` - * Key may contain colons (non-greedy) → try each colon left-to-right. - */ -function matchMapEntry(line: string): [key: string, value: string] | null { - let colonPos = line.indexOf(":", 4); - while (colonPos !== -1) { - const rest = line.substring(colonPos + 1); - const value = rest.trimStart(); - if (rest.length !== value.length && value.length > 0) { - const key = line.substring(4, colonPos); - if (key.length > 0) return [key, value]; - } - colonPos = line.indexOf(":", colonPos + 1); - } - return null; -} - -/** - * Match a 4-space nested-map key header: ` key:` (with optional trailing ws) - * Equivalent to `/^\s{4}(.+?):\s*$/` - * Key may contain colons → try each colon left-to-right. - */ -function matchNestedKey(line: string): string | null { - let colonPos = line.indexOf(":", 4); - while (colonPos !== -1) { - const rest = line.substring(colonPos + 1); - if (rest.trim().length === 0) { - const key = line.substring(4, colonPos); - if (key.length > 0) return key; - } - colonPos = line.indexOf(":", colonPos + 1); - } - return null; -} - -/** - * Match a 6-space nested-map property: ` prop: value` - * Equivalent to `/^\s{6}(.+?):\s+(.+)$/` - * Prop may contain colons → try each colon left-to-right. - */ -function matchNestedProp(line: string): [prop: string, value: string] | null { - let colonPos = line.indexOf(":", 6); - while (colonPos !== -1) { - const rest = line.substring(colonPos + 1); - const value = rest.trimStart(); - if (rest.length !== value.length && value.length > 0) { - const prop = line.substring(6, colonPos); - if (prop.length > 0) return [prop, value]; - } - colonPos = line.indexOf(":", colonPos + 1); - } - return null; -} - -/** - * Detect whether a 4-space line is a nested-map key header (e.g. ` react:`) - * rather than a flat map entry (e.g. ` react: "npm:^18"`). - * Equivalent to `/^\s{4}\S.*:\s*$/` — 4-space indent, non-space start, colon - * at end with optional trailing whitespace. - */ -function isNestedMapHeader(line: string): boolean { - if (!line.startsWith(" ") || line.charAt(4) === " ") return false; - const colonPos = line.indexOf(":", 4); - if (colonPos === -1) return false; - return line.substring(colonPos + 1).trim().length === 0; -} - -// --------------------------------------------------------------------------- -// Structured grouping types — typed fields instead of raw string[] -// --------------------------------------------------------------------------- - -type BlockGroup = { descriptor: string; body: string[] }; -type FieldGroup = { header: string; subLines: string[] }; - -// --------------------------------------------------------------------------- -// Parser -// --------------------------------------------------------------------------- - -export function parseLockfile(content: string): Lockfile { - const lines = content.split("\n"); - - const preambleEnd = findPreambleEnd(lines); - const preamble = lines.slice(0, preambleEnd).join("\n"); - - const blockLines = lines.slice(preambleEnd); - const blocks = splitIntoBlockGroups(blockLines).map(parseBlockGroup); - - return { preamble, blocks }; -} - -/** - * Find where the preamble ends (comment lines, blank lines, __metadata block). - * Returns the index of the first non-preamble line. - */ -function findPreambleEnd(lines: string[]): number { - let i = 0; - for (const line of lines) { - if (line.startsWith("#") || line === "") { - i++; - } else if (line.startsWith("__metadata:")) { - i++; // the __metadata: line itself - for (const metaLine of lines.slice(i)) { - if (!metaLine.startsWith(" ")) break; - i++; - } - break; - } else { - break; - } - } - return i; -} - -/** - * Split lines into block groups, where each group starts with a descriptor - * line (starts at column 0, ends with ":") and includes all following - * indented (2+-space) body lines. Blank lines and 0-space non-descriptor - * lines stop body collection — matching the original parser's strict - * block-termination behavior. - */ -function splitIntoBlockGroups(lines: string[]): BlockGroup[] { - const groups: BlockGroup[] = []; - let current: BlockGroup | null = null; - for (const line of lines) { - if (!line.startsWith(" ") && !line.startsWith("#") && line.endsWith(":")) { - current = { descriptor: line, body: [] }; - groups.push(current); - } else if (current !== null && line.startsWith(" ")) { - current.body.push(line); - } else { - current = null; - } - } - return groups; -} - -function parseBlockGroup(group: BlockGroup): LockfileBlock { - const descriptors = parseDescriptors(group.descriptor.slice(0, -1)); - const fields = parseFields(group.body); - return { descriptors, fields }; -} - -function parseDescriptors(raw: string): string[] { - const trimmed = raw.trim(); - if (trimmed.startsWith('"') && trimmed.endsWith('"')) { - return trimmed.slice(1, -1).split(", "); - } - return [trimmed]; -} - -/** - * Group body lines by top-level field (2-space indent), then parse each - * group into a LockfileField. - */ -function parseFields(bodyLines: string[]): { [name: string]: LockfileField } { - const fields: { [name: string]: LockfileField } = {}; - - for (const { header, subLines } of groupByField(bodyLines)) { - const scalar = matchScalar(header); - if (scalar) { - fields[scalar.name] = { kind: "scalar", raw: scalar.raw }; - continue; - } - - const sectionName = matchSectionHeader(header); - if (sectionName && subLines.length > 0) { - const isNested = subLines.some((l) => l.startsWith(" ") || isNestedMapHeader(l)); - if (isNested) { - fields[sectionName] = parseNestedMapLines(subLines); - } else { - fields[sectionName] = parseMapLines(subLines); - } - continue; - } - if (sectionName) continue; // valid section header with no sub-content - break; // unrecognized line — stop parsing (matches original behavior) - } - - return fields; -} - -/** - * Group body lines so that each 2-space line starts a new group, and - * deeper-indented lines (4+) are appended to the current group. - * Returns structured objects with typed fields. - */ -function groupByField(lines: string[]): FieldGroup[] { - const groups: FieldGroup[] = []; - let current: FieldGroup | null = null; - for (const line of lines) { - if (line.startsWith(" ") && !line.startsWith(" ")) { - current = { header: line, subLines: [] }; - groups.push(current); - } else if (current !== null) { - current.subLines.push(line); - } - } - return groups; -} - -function parseMapLines(lines: string[]): MapField { - const entries: { [key: string]: string } = {}; - for (const line of lines) { - const kv = matchMapEntry(line); - if (kv) { - entries[kv[0]] = kv[1]; - } - } - return { kind: "map", entries }; -} - -function parseNestedMapLines(lines: string[]): NestedMapField { - const entries: { [key: string]: { [prop: string]: string } } = {}; - let currentEntry: { [prop: string]: string } | null = null; - - for (const line of lines) { - if (line.startsWith(" ")) { - if (currentEntry) { - const pv = matchNestedProp(line); - if (pv) { - currentEntry[pv[0]] = pv[1]; - } - } - continue; - } - - const key = matchNestedKey(line); - if (key) { - currentEntry = {}; - entries[key] = currentEntry; - } - } - - return { kind: "nested-map", entries }; -} - -// --------------------------------------------------------------------------- -// Serializer -// --------------------------------------------------------------------------- - -export function serializeLockfile(lockfile: Lockfile): string { - const parts: string[] = []; - - if (lockfile.preamble) { - parts.push(lockfile.preamble); - } - - for (const block of lockfile.blocks) { - parts.push("", serializeBlock(block)); - } - - return parts.join("\n") + "\n"; -} - -export function serializeBlock(block: LockfileBlock): string { - const descriptorLine = `"${block.descriptors.join(", ")}":`; - const lines = [descriptorLine]; - - for (const [name, field] of Object.entries(block.fields)) { - switch (field.kind) { - case "scalar": - lines.push(` ${name}: ${field.raw}`); - break; - case "map": - lines.push(` ${name}:`); - for (const [key, value] of Object.entries(field.entries)) { - lines.push(` ${key}: ${value}`); - } - break; - case "nested-map": - lines.push(` ${name}:`); - for (const [key, props] of Object.entries(field.entries)) { - lines.push(` ${key}:`); - for (const [prop, value] of Object.entries(props)) { - lines.push(` ${prop}: ${value}`); - } - } - break; - /* v8 ignore next 4 -- exhaustive switch; compile-time guarantee */ - default: { - const exhaustiveCheck: never = field; - return exhaustiveCheck; - } - } - } - - return lines.join("\n"); -} - -// --------------------------------------------------------------------------- -// Convenience accessors -// --------------------------------------------------------------------------- - -export function getScalar(block: LockfileBlock, fieldName: string): string | undefined { - const field = block.fields[fieldName]; - if (field?.kind !== "scalar") return undefined; - return field.raw.replace(/^"|"$/g, ""); -} - -export function getMap( - block: LockfileBlock, - fieldName: string, -): { [key: string]: string } | undefined { - const field = block.fields[fieldName]; - if (field?.kind !== "map") return undefined; - return field.entries; -} - -export function getNestedMap( - block: LockfileBlock, - fieldName: string, -): { [key: string]: { [prop: string]: string } } | undefined { - const field = block.fields[fieldName]; - if (field?.kind !== "nested-map") return undefined; - return field.entries; -} diff --git a/prepare-sources/package-lock.json b/prepare-sources/package-lock.json index 3185040..a9c38c8 100644 --- a/prepare-sources/package-lock.json +++ b/prepare-sources/package-lock.json @@ -8,6 +8,8 @@ "name": "prepare-sources", "version": "0.0.0", "dependencies": { + "@yarnpkg/core": "^4.9.2", + "@yarnpkg/parsers": "^3.1.0", "yaml": "^2.9.1" }, "devDependencies": { @@ -17,6 +19,15 @@ "vite-plus": "^0.2.9" } }, + "node_modules/@arcanis/slice-ansi": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@arcanis/slice-ansi/-/slice-ansi-1.1.1.tgz", + "integrity": "sha512-xguP2WR2Dv0gQ7Ykbdb7BNCnPnIPB94uTi0Z2NvkRBEnhbwjOQ7QyQKJXrVQg4qDpiD9hA5l5cCwy/z2OXgc3w==", + "license": "MIT", + "dependencies": { + "grapheme-splitter": "^1.0.4" + } + }, "node_modules/@babel/code-frame": { "version": "7.29.7", "resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz", @@ -109,6 +120,18 @@ "dev": true, "license": "MIT" }, + "node_modules/@isaacs/fs-minipass": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz", + "integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==", + "license": "ISC", + "dependencies": { + "minipass": "^7.0.4" + }, + "engines": { + "node": ">=18.0.0" + } + }, "node_modules/@jridgewell/resolve-uri": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", @@ -137,6 +160,41 @@ "@jridgewell/sourcemap-codec": "^1.4.14" } }, + "node_modules/@nodelib/fs.scandir": { + "version": "2.1.5", + "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", + "integrity": "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g==", + "license": "MIT", + "dependencies": { + "@nodelib/fs.stat": "2.0.5", + "run-parallel": "^1.1.9" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/@nodelib/fs.stat": { + "version": "2.0.5", + "resolved": "https://registry.npmjs.org/@nodelib/fs.stat/-/fs.stat-2.0.5.tgz", + "integrity": "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/@nodelib/fs.walk": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/@nodelib/fs.walk/-/fs.walk-1.2.8.tgz", + "integrity": "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg==", + "license": "MIT", + "dependencies": { + "@nodelib/fs.scandir": "2.1.5", + "fastq": "^1.6.0" + }, + "engines": { + "node": ">= 8" + } + }, "node_modules/@oxc-project/runtime": { "version": "0.143.0", "resolved": "https://registry.npmjs.org/@oxc-project/runtime/-/runtime-0.143.0.tgz", @@ -955,6 +1013,18 @@ "dev": true, "license": "MIT" }, + "node_modules/@sindresorhus/is": { + "version": "4.6.0", + "resolved": "https://registry.npmjs.org/@sindresorhus/is/-/is-4.6.0.tgz", + "integrity": "sha512-t09vSN3MdfsyCHoFcTRCH/iUtG7OJ0CsjzB8cjAmKc/va/kIgeDI/TxsigdncE/4be734m0cvIYwNaV4i2XqAw==", + "license": "MIT", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sindresorhus/is?sponsor=1" + } + }, "node_modules/@standard-schema/spec": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz", @@ -962,6 +1032,18 @@ "dev": true, "license": "MIT" }, + "node_modules/@szmarczak/http-timer": { + "version": "4.0.6", + "resolved": "https://registry.npmjs.org/@szmarczak/http-timer/-/http-timer-4.0.6.tgz", + "integrity": "sha512-4BAffykYOgO+5nzBWYwE3W90sBgLJoUPRWWcL8wlyiM8IB8ipJz3UMJ9KXQd1RKQXpKp8Tutn80HZtWsu2u76w==", + "license": "MIT", + "dependencies": { + "defer-to-connect": "^2.0.0" + }, + "engines": { + "node": ">=10" + } + }, "node_modules/@testing-library/dom": { "version": "10.4.1", "resolved": "https://registry.npmjs.org/@testing-library/dom/-/dom-10.4.1.tgz", @@ -1003,6 +1085,18 @@ "dev": true, "license": "MIT" }, + "node_modules/@types/cacheable-request": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/@types/cacheable-request/-/cacheable-request-6.0.3.tgz", + "integrity": "sha512-IQ3EbTzGxIigb1I3qPZc1rWJnH0BmSKv5QYTalEwweFvyBDLSAe24zP0le/hyi7ecGfZVlIVAg4BZqb8WBwKqw==", + "license": "MIT", + "dependencies": { + "@types/http-cache-semantics": "*", + "@types/keyv": "^3.1.4", + "@types/node": "*", + "@types/responselike": "^1.0.0" + } + }, "node_modules/@types/chai": { "version": "5.2.3", "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", @@ -1021,6 +1115,12 @@ "dev": true, "license": "MIT" }, + "node_modules/@types/emscripten": { + "version": "1.41.6", + "resolved": "https://registry.npmjs.org/@types/emscripten/-/emscripten-1.41.6.tgz", + "integrity": "sha512-uN+9i8bFT5CUcZfyIEYDrSueACEyKGbUs5kC/72DGlZZoinh84sJfVV0i8UOJD1asdzkvLPBRrKs41kZ8MdEXg==", + "license": "MIT" + }, "node_modules/@types/estree": { "version": "1.0.9", "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz", @@ -1028,16 +1128,51 @@ "dev": true, "license": "MIT" }, + "node_modules/@types/http-cache-semantics": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/@types/http-cache-semantics/-/http-cache-semantics-4.2.0.tgz", + "integrity": "sha512-L3LgimLHXtGkWikKnsPg0/VFx9OGZaC+eN1u4r+OB1XRqH3meBIAVC2zr1WdMH+RHmnRkqliQAOHNJ/E0j/e0Q==", + "license": "MIT" + }, + "node_modules/@types/keyv": { + "version": "3.1.4", + "resolved": "https://registry.npmjs.org/@types/keyv/-/keyv-3.1.4.tgz", + "integrity": "sha512-BQ5aZNSCpj7D6K2ksrRCTmKRLEpnPvWDiLPfoGyhZ++8YtiK9d/3DBKPJgry359X/P1PfruyYwvnvwFjuEiEIg==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, "node_modules/@types/node": { "version": "24.13.3", "resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.3.tgz", "integrity": "sha512-Dh8vAsV36ig5wa9OX4pXvMc9D3Veibfw2wix0CUwYODLD8nkj9UsLjASr49nPg+2eKzxhBV+v7L8pXvT4e639Q==", - "dev": true, "license": "MIT", "dependencies": { "undici-types": "~7.18.0" } }, + "node_modules/@types/responselike": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/@types/responselike/-/responselike-1.0.3.tgz", + "integrity": "sha512-H/+L+UkTV33uf49PH5pCAUBVPNj2nDBXTN+qS1dOwyyg24l3CcicicCA7ca+HMvJBZcFgl5r8e+RR6elsb4Lyw==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@types/semver": { + "version": "7.8.0", + "resolved": "https://registry.npmjs.org/@types/semver/-/semver-7.8.0.tgz", + "integrity": "sha512-1mAINjtQCXXeLkJ9ehXkwOcBpqtLxiVtKhpUf83DdRNdQKV0iXZpaHYqRr7nj+wvxuJzoAmAwXI+sCNMv1CzLQ==", + "license": "MIT" + }, + "node_modules/@types/treeify": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/@types/treeify/-/treeify-1.0.3.tgz", + "integrity": "sha512-hx0o7zWEUU4R2Amn+pjCBQQt23Khy/Dk56gQU5xi5jtPL1h83ACJCeFaB2M/+WO1AntvWrSoVnnCAfI1AQH4Cg==", + "license": "MIT" + }, "node_modules/@typescript/typescript-aix-ppc64": { "version": "7.0.2", "resolved": "https://registry.npmjs.org/@typescript/typescript-aix-ppc64/-/typescript-aix-ppc64-7.0.2.tgz", @@ -1812,6 +1947,107 @@ "node": ">=20.0.0" } }, + "node_modules/@yarnpkg/core": { + "version": "4.9.2", + "resolved": "https://registry.npmjs.org/@yarnpkg/core/-/core-4.9.2.tgz", + "integrity": "sha512-O3wZzM97Pp0ZEhr1CF13UIpAmaJ7u+nWSzNoZpaUnP5YDYC9EzGiza7L+6FN1f7c+WzkYHp6alyRcOGD2NmJCw==", + "license": "BSD-2-Clause", + "dependencies": { + "@arcanis/slice-ansi": "^1.1.1", + "@types/semver": "^7.1.0", + "@types/treeify": "^1.0.0", + "@yarnpkg/fslib": "^3.1.5", + "@yarnpkg/libzip": "^3.2.2", + "@yarnpkg/parsers": "^3.1.0", + "@yarnpkg/shell": "^4.1.3", + "camelcase": "^5.3.1", + "chalk": "^4.1.2", + "ci-info": "^4.0.0", + "clipanion": "^4.0.0-rc.2", + "cross-spawn": "^7.0.3", + "diff": "^5.1.0", + "dotenv": "^16.3.1", + "es-toolkit": "^1.39.7", + "fast-glob": "^3.2.2", + "got": "patch:got@npm%3A11.8.2#~/.yarn/patches/got-npm-11.8.2-c1eb105458.patch", + "hpagent": "^1.2.0", + "micromatch": "^4.0.2", + "p-limit": "^2.2.0", + "semver": "^7.8.5", + "strip-ansi": "^6.0.0", + "tar": "^7.5.3", + "tinylogic": "^2.0.0", + "treeify": "^1.1.0", + "tslib": "^2.4.0" + }, + "engines": { + "node": ">=18.12.0" + } + }, + "node_modules/@yarnpkg/fslib": { + "version": "3.1.5", + "resolved": "https://registry.npmjs.org/@yarnpkg/fslib/-/fslib-3.1.5.tgz", + "integrity": "sha512-hXaPIWl5GZA+rXcx+yaKWUuePJruZuD+3A5A2X6paEBfFsyCD7oEp88lSMj1ym1ehBWUmhNH/YGOp+SrbmSBPg==", + "license": "BSD-2-Clause", + "dependencies": { + "tslib": "^2.4.0" + }, + "engines": { + "node": ">=18.12.0" + } + }, + "node_modules/@yarnpkg/libzip": { + "version": "3.2.2", + "resolved": "https://registry.npmjs.org/@yarnpkg/libzip/-/libzip-3.2.2.tgz", + "integrity": "sha512-Kqxgjfy6SwwC4tTGQYToIWtUhIORTpkowqgd9kkMiBixor0eourHZZAggt/7N4WQKt9iCyPSkO3Xvr44vXUBAw==", + "license": "BSD-2-Clause", + "dependencies": { + "@types/emscripten": "^1.39.6", + "@yarnpkg/fslib": "^3.1.3", + "tslib": "^2.4.0" + }, + "engines": { + "node": ">=18.12.0" + }, + "peerDependencies": { + "@yarnpkg/fslib": "^3.1.3" + } + }, + "node_modules/@yarnpkg/parsers": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@yarnpkg/parsers/-/parsers-3.1.0.tgz", + "integrity": "sha512-WoxUM/Be4hfsX06FxsvpGgfYqwgivMV7/Ol7aFuSfSmY6rRaiju4QxOEe9RUS0iYcSHWl5i9AhB1cMoE0p+XiA==", + "license": "BSD-2-Clause", + "dependencies": { + "js-yaml": "^4.3.0", + "tslib": "^2.4.0" + }, + "engines": { + "node": ">=18.12.0" + } + }, + "node_modules/@yarnpkg/shell": { + "version": "4.1.3", + "resolved": "https://registry.npmjs.org/@yarnpkg/shell/-/shell-4.1.3.tgz", + "integrity": "sha512-5igwsHbPtSAlLdmMdKqU3atXjwhtLFQXsYAG0sn1XcPb3yF8WxxtWxN6fycBoUvFyIHFz1G0KeRefnAy8n6gdw==", + "license": "BSD-2-Clause", + "dependencies": { + "@yarnpkg/fslib": "^3.1.2", + "@yarnpkg/parsers": "^3.0.3", + "chalk": "^4.1.2", + "clipanion": "^4.0.0-rc.2", + "cross-spawn": "^7.0.3", + "fast-glob": "^3.2.2", + "micromatch": "^4.0.2", + "tslib": "^2.4.0" + }, + "bin": { + "shell": "lib/cli.js" + }, + "engines": { + "node": ">=18.12.0" + } + }, "node_modules/@yuku-codegen/binding-darwin-arm64": { "version": "0.5.48", "resolved": "https://registry.npmjs.org/@yuku-codegen/binding-darwin-arm64/-/binding-darwin-arm64-0.5.48.tgz", @@ -2145,7 +2381,6 @@ "version": "5.0.1", "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz", "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==", - "dev": true, "license": "MIT", "engines": { "node": ">=8" @@ -2164,6 +2399,12 @@ "url": "https://github.com/chalk/ansi-styles?sponsor=1" } }, + "node_modules/argparse": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz", + "integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==", + "license": "Python-2.0" + }, "node_modules/aria-query": { "version": "5.3.0", "resolved": "https://registry.npmjs.org/aria-query/-/aria-query-5.3.0.tgz", @@ -2203,6 +2444,54 @@ "dev": true, "license": "MIT" }, + "node_modules/braces": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/braces/-/braces-3.0.3.tgz", + "integrity": "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==", + "license": "MIT", + "dependencies": { + "fill-range": "^7.1.1" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/cacheable-lookup": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/cacheable-lookup/-/cacheable-lookup-5.0.4.tgz", + "integrity": "sha512-2/kNscPhpcxrOigMZzbiWF7dz8ilhb/nIHU3EyZiXWXpeq/au8qJ8VhdftMkty3n7Gj6HIGalQG8oiBNB3AJgA==", + "license": "MIT", + "engines": { + "node": ">=10.6.0" + } + }, + "node_modules/cacheable-request": { + "version": "7.0.4", + "resolved": "https://registry.npmjs.org/cacheable-request/-/cacheable-request-7.0.4.tgz", + "integrity": "sha512-v+p6ongsrp0yTGbJXjgxPow2+DL93DASP4kXCDKb8/bwRtt9OEF3whggkkDkGNzgcWy2XaF4a8nZglC7uElscg==", + "license": "MIT", + "dependencies": { + "clone-response": "^1.0.2", + "get-stream": "^5.1.0", + "http-cache-semantics": "^4.0.0", + "keyv": "^4.0.0", + "lowercase-keys": "^2.0.0", + "normalize-url": "^6.0.1", + "responselike": "^2.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/camelcase": { + "version": "5.3.1", + "resolved": "https://registry.npmjs.org/camelcase/-/camelcase-5.3.1.tgz", + "integrity": "sha512-L28STB170nwWS63UjtlEOE3dldQApaJXZkOI1uMFfzf3rRuPegHaHesyee+YxQ+W6SvRDQV6UrdOdRiR153wJg==", + "license": "MIT", + "engines": { + "node": ">=6" + } + }, "node_modules/chai": { "version": "6.2.2", "resolved": "https://registry.npmjs.org/chai/-/chai-6.2.2.tgz", @@ -2213,6 +2502,106 @@ "node": ">=18" } }, + "node_modules/chalk": { + "version": "4.1.2", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz", + "integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==", + "license": "MIT", + "dependencies": { + "ansi-styles": "^4.1.0", + "supports-color": "^7.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/chalk/chalk?sponsor=1" + } + }, + "node_modules/chalk/node_modules/ansi-styles": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz", + "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==", + "license": "MIT", + "dependencies": { + "color-convert": "^2.0.1" + }, + "engines": { + "node": ">=8" + }, + "funding": { + "url": "https://github.com/chalk/ansi-styles?sponsor=1" + } + }, + "node_modules/chownr": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz", + "integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==", + "license": "BlueOak-1.0.0", + "engines": { + "node": ">=18" + } + }, + "node_modules/ci-info": { + "version": "4.4.0", + "resolved": "https://registry.npmjs.org/ci-info/-/ci-info-4.4.0.tgz", + "integrity": "sha512-77PSwercCZU2Fc4sX94eF8k8Pxte6JAwL4/ICZLFjJLqegs7kCuAsqqj/70NQF6TvDpgFjkubQB2FW2ZZddvQg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/sibiraj-s" + } + ], + "license": "MIT", + "engines": { + "node": ">=8" + } + }, + "node_modules/clipanion": { + "version": "4.0.0-rc.4", + "resolved": "https://registry.npmjs.org/clipanion/-/clipanion-4.0.0-rc.4.tgz", + "integrity": "sha512-CXkMQxU6s9GklO/1f714dkKBMu1lopS1WFF0B8o4AxPykR1hpozxSiUZ5ZUeBjfPgCWqbcNOtZVFhB8Lkfp1+Q==", + "license": "MIT", + "workspaces": [ + "website" + ], + "dependencies": { + "typanion": "^3.8.0" + }, + "peerDependencies": { + "typanion": "*" + } + }, + "node_modules/clone-response": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/clone-response/-/clone-response-1.0.3.tgz", + "integrity": "sha512-ROoL94jJH2dUVML2Y/5PEDNaSHgeOdSDicUyS7izcF63G6sTc/FTjLub4b8Il9S8S0beOfYt0TaA5qvFK+w0wA==", + "license": "MIT", + "dependencies": { + "mimic-response": "^1.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/color-convert": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz", + "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==", + "license": "MIT", + "dependencies": { + "color-name": "~1.1.4" + }, + "engines": { + "node": ">=7.0.0" + } + }, + "node_modules/color-name": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz", + "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==", + "license": "MIT" + }, "node_modules/convert-source-map": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz", @@ -2220,6 +2609,56 @@ "dev": true, "license": "MIT" }, + "node_modules/cross-spawn": { + "version": "7.0.6", + "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz", + "integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==", + "license": "MIT", + "dependencies": { + "path-key": "^3.1.0", + "shebang-command": "^2.0.0", + "which": "^2.0.1" + }, + "engines": { + "node": ">= 8" + } + }, + "node_modules/decompress-response": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz", + "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==", + "license": "MIT", + "dependencies": { + "mimic-response": "^3.1.0" + }, + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/decompress-response/node_modules/mimic-response": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz", + "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==", + "license": "MIT", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/defer-to-connect": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/defer-to-connect/-/defer-to-connect-2.0.1.tgz", + "integrity": "sha512-4tvttepXG1VaYGrRibk5EwJd1t4udunSOVMdLSAL6mId1ix438oPwPZMALY41FCijukO1L0twNcGsdzS7dHgDg==", + "license": "MIT", + "engines": { + "node": ">=10" + } + }, "node_modules/dequal": { "version": "2.0.3", "resolved": "https://registry.npmjs.org/dequal/-/dequal-2.0.3.tgz", @@ -2240,6 +2679,15 @@ "node": ">=8" } }, + "node_modules/diff": { + "version": "5.2.2", + "resolved": "https://registry.npmjs.org/diff/-/diff-5.2.2.tgz", + "integrity": "sha512-vtcDfH3TOjP8UekytvnHH1o1P4FcUdt4eQ1Y+Abap1tk/OB2MWQvcwS2ClCd1zuIhc3JKOx6p3kod8Vfys3E+A==", + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.3.1" + } + }, "node_modules/dom-accessibility-api": { "version": "0.5.16", "resolved": "https://registry.npmjs.org/dom-accessibility-api/-/dom-accessibility-api-0.5.16.tgz", @@ -2247,6 +2695,27 @@ "dev": true, "license": "MIT" }, + "node_modules/dotenv": { + "version": "16.6.1", + "resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz", + "integrity": "sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==", + "license": "BSD-2-Clause", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://dotenvx.com" + } + }, + "node_modules/end-of-stream": { + "version": "1.4.5", + "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz", + "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==", + "license": "MIT", + "dependencies": { + "once": "^1.4.0" + } + }, "node_modules/es-module-lexer": { "version": "2.3.2", "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.3.2.tgz", @@ -2254,6 +2723,18 @@ "dev": true, "license": "MIT" }, + "node_modules/es-toolkit": { + "version": "1.52.0", + "resolved": "https://registry.npmjs.org/es-toolkit/-/es-toolkit-1.52.0.tgz", + "integrity": "sha512-XTNEJQh1tY1ZJVcf6ayP/2n4ZPyaHlW2FWs7xvw5ddPuhUVjLD3olQVQS7kf58JbAB48iL0uL/jerTrjtV3lDA==", + "license": "MIT", + "workspaces": [ + "docs", + "benchmarks", + "tests/types", + "tests/browser-compat" + ] + }, "node_modules/estree-walker": { "version": "3.0.3", "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", @@ -2274,6 +2755,31 @@ "node": ">=12.0.0" } }, + "node_modules/fast-glob": { + "version": "3.3.3", + "resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.3.3.tgz", + "integrity": "sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg==", + "license": "MIT", + "dependencies": { + "@nodelib/fs.stat": "^2.0.2", + "@nodelib/fs.walk": "^1.2.3", + "glob-parent": "^5.1.2", + "merge2": "^1.3.0", + "micromatch": "^4.0.8" + }, + "engines": { + "node": ">=8.6.0" + } + }, + "node_modules/fastq": { + "version": "1.20.3", + "resolved": "https://registry.npmjs.org/fastq/-/fastq-1.20.3.tgz", + "integrity": "sha512-XKv5nnLs6nLF71NgiKJLIZFLkPyIEuOselLG7ujZnGrRfQK8HpvY+WqKhAJUAdLomwVHErVS4LfxFlPq0/FTAw==", + "license": "ISC", + "dependencies": { + "reusify": "^1.0.4" + } + }, "node_modules/fdir": { "version": "6.5.0", "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz", @@ -2292,6 +2798,18 @@ } } }, + "node_modules/fill-range": { + "version": "7.1.1", + "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz", + "integrity": "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==", + "license": "MIT", + "dependencies": { + "to-regex-range": "^5.0.1" + }, + "engines": { + "node": ">=8" + } + }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", @@ -2307,23 +2825,144 @@ "node": "^8.16.0 || ^10.6.0 || >=11.0.0" } }, - "node_modules/has-flag": { - "version": "4.0.0", - "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz", - "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==", - "dev": true, + "node_modules/get-stream": { + "version": "5.2.0", + "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz", + "integrity": "sha512-nBF+F1rAZVCu/p7rjzgA+Yb4lfYXrpl7a6VmJrU8wF9I1CKvP/QwPNZHnOlwbTkY6dvtFIzFMSyQXbLoTQPRpA==", "license": "MIT", + "dependencies": { + "pump": "^3.0.0" + }, "engines": { "node": ">=8" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" } }, - "node_modules/html-escaper": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-2.0.2.tgz", + "node_modules/glob-parent": { + "version": "5.1.2", + "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", + "integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==", + "license": "ISC", + "dependencies": { + "is-glob": "^4.0.1" + }, + "engines": { + "node": ">= 6" + } + }, + "node_modules/got": { + "version": "11.8.2", + "resolved": "https://registry.npmjs.org/got/-/got-11.8.2.tgz", + "integrity": "sha512-D0QywKgIe30ODs+fm8wMZiAcZjypcCodPNuMz5H9Mny7RJ+IjJ10BdmGW7OM7fHXP+O7r6ZwapQ/YQmMSvB0UQ==", + "license": "MIT", + "dependencies": { + "@sindresorhus/is": "^4.0.0", + "@szmarczak/http-timer": "^4.0.5", + "@types/cacheable-request": "^6.0.1", + "@types/responselike": "^1.0.0", + "cacheable-lookup": "^5.0.3", + "cacheable-request": "^7.0.1", + "decompress-response": "^6.0.0", + "http2-wrapper": "^1.0.0-beta.5.2", + "lowercase-keys": "^2.0.0", + "p-cancelable": "^2.0.0", + "responselike": "^2.0.0" + }, + "engines": { + "node": ">=10.19.0" + }, + "funding": { + "url": "https://github.com/sindresorhus/got?sponsor=1" + } + }, + "node_modules/grapheme-splitter": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/grapheme-splitter/-/grapheme-splitter-1.0.4.tgz", + "integrity": "sha512-bzh50DW9kTPM00T8y4o8vQg89Di9oLJVLW/KaOGIXJWP/iqCN6WKYkbNOF04vFLJhwcpYUh9ydh/+5vpOqV4YQ==", + "license": "MIT" + }, + "node_modules/has-flag": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz", + "integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==", + "license": "MIT", + "engines": { + "node": ">=8" + } + }, + "node_modules/hpagent": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/hpagent/-/hpagent-1.2.0.tgz", + "integrity": "sha512-A91dYTeIB6NoXG+PxTQpCCDDnfHsW9kc06Lvpu1TEe9gnd6ZFeiBoRO9JvzEv6xK7EX97/dUE8g/vBMTqTS3CA==", + "license": "MIT", + "engines": { + "node": ">=14" + } + }, + "node_modules/html-escaper": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-2.0.2.tgz", "integrity": "sha512-H2iMtd0I4Mt5eYiapRdIDjp+XzelXQ0tFE4JS7YFwFevXXMmOp9myNrUvCg0D6ws8iqkRPBfKHgbwig1SmlLfg==", "dev": true, "license": "MIT" }, + "node_modules/http-cache-semantics": { + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/http-cache-semantics/-/http-cache-semantics-4.3.0.tgz", + "integrity": "sha512-M5t5LlJpS1UHMjvwRQVdFHvPISGeLAxNcrWuJkeGh0KxsqCHZ1O3NXZU/8x7cD0BDcGW8kapxMKTvwlqrNkHkA==", + "license": "BSD-2-Clause" + }, + "node_modules/http2-wrapper": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/http2-wrapper/-/http2-wrapper-1.0.3.tgz", + "integrity": "sha512-V+23sDMr12Wnz7iTcDeJr3O6AIxlnvT/bmaAAAP/Xda35C90p9599p0F1eHR/N1KILWSoWVAiOMFjBBXaXSMxg==", + "license": "MIT", + "dependencies": { + "quick-lru": "^5.1.1", + "resolve-alpn": "^1.0.0" + }, + "engines": { + "node": ">=10.19.0" + } + }, + "node_modules/is-extglob": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz", + "integrity": "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-glob": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.3.tgz", + "integrity": "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg==", + "license": "MIT", + "dependencies": { + "is-extglob": "^2.1.1" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/is-number": { + "version": "7.0.0", + "resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz", + "integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==", + "license": "MIT", + "engines": { + "node": ">=0.12.0" + } + }, + "node_modules/isexe": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz", + "integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==", + "license": "ISC" + }, "node_modules/istanbul-lib-coverage": { "version": "3.2.2", "resolved": "https://registry.npmjs.org/istanbul-lib-coverage/-/istanbul-lib-coverage-3.2.2.tgz", @@ -2370,6 +3009,43 @@ "dev": true, "license": "MIT" }, + "node_modules/js-yaml": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.2.tgz", + "integrity": "sha512-SFNOvSJ+Dgf/9An904Yx+CgSlIPCkIpao4qo51lpee25TIRejdH3rhR4EZMGoNx3/TP3O+wzWuiTFl4sqbltzA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/puzrin" + }, + { + "type": "github", + "url": "https://github.com/sponsors/nodeca" + } + ], + "license": "MIT", + "dependencies": { + "argparse": "^2.0.1" + }, + "bin": { + "js-yaml": "bin/js-yaml.js" + } + }, + "node_modules/json-buffer": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/json-buffer/-/json-buffer-3.0.1.tgz", + "integrity": "sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==", + "license": "MIT" + }, + "node_modules/keyv": { + "version": "4.5.4", + "resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.4.tgz", + "integrity": "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==", + "license": "MIT", + "dependencies": { + "json-buffer": "3.0.1" + } + }, "node_modules/lightningcss": { "version": "1.33.0", "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.33.0.tgz", @@ -2643,6 +3319,15 @@ "url": "https://opencollective.com/parcel" } }, + "node_modules/lowercase-keys": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/lowercase-keys/-/lowercase-keys-2.0.0.tgz", + "integrity": "sha512-tqNXrS78oMOE73NMxK4EMLQsQowWf8jKooH9g7xPavRT706R6bkQJ6DY2Te7QukaZsulxa30wQ7bk0pm4XiHmA==", + "license": "MIT", + "engines": { + "node": ">=8" + } + }, "node_modules/lz-string": { "version": "1.5.0", "resolved": "https://registry.npmjs.org/lz-string/-/lz-string-1.5.0.tgz", @@ -2691,6 +3376,70 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/merge2": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/merge2/-/merge2-1.4.1.tgz", + "integrity": "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg==", + "license": "MIT", + "engines": { + "node": ">= 8" + } + }, + "node_modules/micromatch": { + "version": "4.0.8", + "resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.8.tgz", + "integrity": "sha512-PXwfBhYu0hBCPw8Dn0E+WDYb7af3dSLVWKi3HGv84IdF4TyFoC0ysxFd0Goxw7nSv4T/PzEJQxsYsEiFCKo2BA==", + "license": "MIT", + "dependencies": { + "braces": "^3.0.3", + "picomatch": "^2.3.1" + }, + "engines": { + "node": ">=8.6" + } + }, + "node_modules/micromatch/node_modules/picomatch": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.2.tgz", + "integrity": "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==", + "license": "MIT", + "engines": { + "node": ">=8.6" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } + }, + "node_modules/mimic-response": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-1.0.1.tgz", + "integrity": "sha512-j5EctnkH7amfV/q5Hgmoal1g2QHFJRraOtmx0JpIqkxhBhI/lJSl1nMpQ45hVarwNETOoWEimndZ4QK0RHxuxQ==", + "license": "MIT", + "engines": { + "node": ">=4" + } + }, + "node_modules/minipass": { + "version": "7.1.3", + "resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz", + "integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==", + "license": "BlueOak-1.0.0", + "engines": { + "node": ">=16 || 14 >=14.17" + } + }, + "node_modules/minizlib": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz", + "integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==", + "license": "MIT", + "dependencies": { + "minipass": "^7.1.2" + }, + "engines": { + "node": ">= 18" + } + }, "node_modules/mrmime": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/mrmime/-/mrmime-2.0.1.tgz", @@ -2720,6 +3469,18 @@ "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" } }, + "node_modules/normalize-url": { + "version": "6.1.0", + "resolved": "https://registry.npmjs.org/normalize-url/-/normalize-url-6.1.0.tgz", + "integrity": "sha512-DlL+XwOy3NxAQ8xuC0okPgK46iuVNAK01YN7RueYBqqFeGsBjV9XmCAzAdgt+667bCl5kPh9EqKKDwnaPG1I7A==", + "license": "MIT", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/obug": { "version": "2.1.4", "resolved": "https://registry.npmjs.org/obug/-/obug-2.1.4.tgz", @@ -2734,6 +3495,15 @@ "node": ">=12.20.0" } }, + "node_modules/once": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz", + "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==", + "license": "ISC", + "dependencies": { + "wrappy": "1" + } + }, "node_modules/oxfmt": { "version": "0.62.0", "resolved": "https://registry.npmjs.org/oxfmt/-/oxfmt-0.62.0.tgz", @@ -2853,6 +3623,48 @@ "@oxlint-tsgolint/win32-x64": "7.0.2001" } }, + "node_modules/p-cancelable": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/p-cancelable/-/p-cancelable-2.1.1.tgz", + "integrity": "sha512-BZOr3nRQHOntUjTrH8+Lh54smKHoHyur8We1V8DSMVrl5A2malOOwuJRnKRDjSnkoeBh4at6BwEnb5I7Jl31wg==", + "license": "MIT", + "engines": { + "node": ">=8" + } + }, + "node_modules/p-limit": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz", + "integrity": "sha512-//88mFWSJx8lxCzwdAABTJL2MyWB12+eIY7MDL2SqLmAkeKU9qxRvWuSyTjm3FUmpBEMuFfckAIqEaVGUDxb6w==", + "license": "MIT", + "dependencies": { + "p-try": "^2.0.0" + }, + "engines": { + "node": ">=6" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/p-try": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/p-try/-/p-try-2.2.0.tgz", + "integrity": "sha512-R4nPAVTAU0B9D35/Gk3uJf/7XYbQcyohSKdvAxIRSNghFl4e71hVoGnBNQz9cWaXxO2I10KTC+3jMdvvoKw6dQ==", + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/path-key": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz", + "integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==", + "license": "MIT", + "engines": { + "node": ">=8" + } + }, "node_modules/pathe": { "version": "2.0.3", "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz", @@ -2934,6 +3746,48 @@ "node": "^10.13.0 || ^12.13.0 || ^14.15.0 || >=15.0.0" } }, + "node_modules/pump": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz", + "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==", + "license": "MIT", + "dependencies": { + "end-of-stream": "^1.1.0", + "once": "^1.3.1" + } + }, + "node_modules/queue-microtask": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/queue-microtask/-/queue-microtask-1.2.3.tgz", + "integrity": "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, + "node_modules/quick-lru": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/quick-lru/-/quick-lru-5.1.1.tgz", + "integrity": "sha512-WuyALRjWPDGtt/wzJiadO5AXY+8hZ80hVpe6MyivgraREW751X3SbhRvG3eLKOYN+8VEvqLcf3wdnt44Z4S4SA==", + "license": "MIT", + "engines": { + "node": ">=10" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/react-is": { "version": "17.0.2", "resolved": "https://registry.npmjs.org/react-is/-/react-is-17.0.2.tgz", @@ -2941,11 +3795,61 @@ "dev": true, "license": "MIT" }, + "node_modules/resolve-alpn": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/resolve-alpn/-/resolve-alpn-1.2.1.tgz", + "integrity": "sha512-0a1F4l73/ZFZOakJnQ3FvkJ2+gSTQWz/r2KE5OdDY0TxPm5h4GkqkWWfM47T7HsbnOtcJVEF4epCVy6u7Q3K+g==", + "license": "MIT" + }, + "node_modules/responselike": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/responselike/-/responselike-2.0.1.tgz", + "integrity": "sha512-4gl03wn3hj1HP3yzgdI7d3lCkF95F21Pz4BPGvKHinyQzALR5CapwC8yIi0Rh58DEMQ/SguC03wFj2k0M/mHhw==", + "license": "MIT", + "dependencies": { + "lowercase-keys": "^2.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, + "node_modules/reusify": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/reusify/-/reusify-1.1.0.tgz", + "integrity": "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==", + "license": "MIT", + "engines": { + "iojs": ">=1.0.0", + "node": ">=0.10.0" + } + }, + "node_modules/run-parallel": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/run-parallel/-/run-parallel-1.2.0.tgz", + "integrity": "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", + "dependencies": { + "queue-microtask": "^1.2.2" + } + }, "node_modules/semver": { "version": "7.8.5", "resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz", "integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==", - "dev": true, "license": "ISC", "bin": { "semver": "bin/semver.js" @@ -2954,6 +3858,27 @@ "node": ">=10" } }, + "node_modules/shebang-command": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz", + "integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==", + "license": "MIT", + "dependencies": { + "shebang-regex": "^3.0.0" + }, + "engines": { + "node": ">=8" + } + }, + "node_modules/shebang-regex": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz", + "integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==", + "license": "MIT", + "engines": { + "node": ">=8" + } + }, "node_modules/siginfo": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", @@ -3000,11 +3925,22 @@ "dev": true, "license": "MIT" }, + "node_modules/strip-ansi": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz", + "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==", + "license": "MIT", + "dependencies": { + "ansi-regex": "^5.0.1" + }, + "engines": { + "node": ">=8" + } + }, "node_modules/supports-color": { "version": "7.2.0", "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz", "integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==", - "dev": true, "license": "MIT", "dependencies": { "has-flag": "^4.0.0" @@ -3013,6 +3949,22 @@ "node": ">=8" } }, + "node_modules/tar": { + "version": "7.5.22", + "resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz", + "integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==", + "license": "BlueOak-1.0.0", + "dependencies": { + "@isaacs/fs-minipass": "^4.0.0", + "chownr": "^3.0.0", + "minipass": "^7.1.2", + "minizlib": "^3.1.0", + "yallist": "^5.0.0" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/tinybench": { "version": "2.9.0", "resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz", @@ -3047,6 +3999,12 @@ "url": "https://github.com/sponsors/SuperchupuDev" } }, + "node_modules/tinylogic": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/tinylogic/-/tinylogic-2.0.0.tgz", + "integrity": "sha512-dljTkiLLITtsjqBvTA1MRZQK/sGP4kI3UJKc3yA9fMzYbMF2RhcN04SeROVqJBIYYOoJMM8u0WDnhFwMSFQotw==", + "license": "MIT" + }, "node_modules/tinypool": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/tinypool/-/tinypool-2.1.0.tgz", @@ -3067,6 +4025,18 @@ "node": ">=14.0.0" } }, + "node_modules/to-regex-range": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz", + "integrity": "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==", + "license": "MIT", + "dependencies": { + "is-number": "^7.0.0" + }, + "engines": { + "node": ">=8.0" + } + }, "node_modules/totalist": { "version": "3.0.1", "resolved": "https://registry.npmjs.org/totalist/-/totalist-3.0.1.tgz", @@ -3077,6 +4047,30 @@ "node": ">=6" } }, + "node_modules/treeify": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/treeify/-/treeify-1.1.0.tgz", + "integrity": "sha512-1m4RA7xVAJrSGrrXGs0L3YTwyvBs2S8PbRHaLZAkFw7JR8oIFwYtysxlBZhYIa7xSyiYJKZ3iGrrk55cGA3i9A==", + "license": "MIT", + "engines": { + "node": ">=0.6" + } + }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "license": "0BSD" + }, + "node_modules/typanion": { + "version": "3.14.0", + "resolved": "https://registry.npmjs.org/typanion/-/typanion-3.14.0.tgz", + "integrity": "sha512-ZW/lVMRabETuYCd9O9ZvMhAh8GslSqaUjxmK/JLPCh6l73CvLBiuXswj/+7LdnWOgYsQ130FqLzFz5aGT4I3Ug==", + "license": "MIT", + "workspaces": [ + "website" + ] + }, "node_modules/typescript": { "version": "7.0.2", "resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz", @@ -3116,7 +4110,6 @@ "version": "7.18.2", "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz", "integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==", - "dev": true, "license": "MIT" }, "node_modules/vite": { @@ -3367,6 +4360,21 @@ } } }, + "node_modules/which": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz", + "integrity": "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==", + "license": "ISC", + "dependencies": { + "isexe": "^2.0.0" + }, + "bin": { + "node-which": "bin/node-which" + }, + "engines": { + "node": ">= 8" + } + }, "node_modules/why-is-node-running": { "version": "2.3.0", "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz", @@ -3384,6 +4392,12 @@ "node": ">=8" } }, + "node_modules/wrappy": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", + "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==", + "license": "ISC" + }, "node_modules/ws": { "version": "8.21.3", "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.3.tgz", @@ -3406,6 +4420,15 @@ } } }, + "node_modules/yallist": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz", + "integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==", + "license": "BlueOak-1.0.0", + "engines": { + "node": ">=18" + } + }, "node_modules/yaml": { "version": "2.9.1", "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.1.tgz", diff --git a/prepare-sources/package.json b/prepare-sources/package.json index 06b0dd9..35273b7 100644 --- a/prepare-sources/package.json +++ b/prepare-sources/package.json @@ -9,6 +9,8 @@ "check:fix": "vp check --fix" }, "dependencies": { + "@yarnpkg/core": "^4.9.2", + "@yarnpkg/parsers": "^3.1.0", "yaml": "^2.9.1" }, "devDependencies": { @@ -18,6 +20,9 @@ "vite-plus": "^0.2.9" }, "overrides": { + "@yarnpkg/core": { + "got": "11.8.2" + }, "vite": "npm:@voidzero-dev/vite-plus-core@0.2.9" }, "devEngines": { From 9b3aa1f00f1fd2405838e9018fd04d158bae0df0 Mon Sep 17 00:00:00 2001 From: Jon Koops Date: Tue, 6 Oct 2026 17:44:22 +0200 Subject: [PATCH 3/3] docs(prepare-sources): trim generate-manifests README Align module docs with other pipeline modules and mark generate-manifests implemented in the prepare-sources overview. Signed-off-by: Jon Koops --- prepare-sources/README.md | 2 +- .../lib/modules/generate-manifests/README.md | 158 ++++-------------- 2 files changed, 33 insertions(+), 127 deletions(-) diff --git a/prepare-sources/README.md b/prepare-sources/README.md index 2b8c57e..9c34565 100644 --- a/prepare-sources/README.md +++ b/prepare-sources/README.md @@ -27,7 +27,7 @@ node lib/cli.ts \ | --- | ------------------------- | ------------------------------------------------------------------------------ | --------------- | | 1 | `seed-frontend-lockfiles` | Seed `dist-dynamic/yarn.lock` for frontend plugins after the initial export | Not implemented | | 2 | `make-self-contained` | Merge repo-root `.yarn/` and `.yarnrc.yml` into the workspace (non-flat repos) | Implemented | -| 3 | `generate-manifests` | Produce `manifest.json` and `backstage-manifest.json` for protocol resolution | Not implemented | +| 3 | `generate-manifests` | Produce `manifest.json` and `backstage-manifest.json` for protocol resolution | Implemented | | 4 | `plugin-removal` | Remove unsupported/community plugins and update `plugins-list.yaml` | Not implemented | | 5 | `file-cleanup` | Strip test files, mocks, stories, and dev-only artifacts | Not implemented | | 6 | `protocol-resolution` | Resolve `workspace:^` and `backstage:^` protocols; generate `type-shims` | Not implemented | diff --git a/prepare-sources/lib/modules/generate-manifests/README.md b/prepare-sources/lib/modules/generate-manifests/README.md index c6174a6..7bb29e2 100644 --- a/prepare-sources/lib/modules/generate-manifests/README.md +++ b/prepare-sources/lib/modules/generate-manifests/README.md @@ -1,28 +1,18 @@ # generate-manifests -Generates package inventories consumed by `protocol-resolution` to resolve -`workspace:^` and `backstage:^` references to concrete npm semver ranges. - -**Pipeline position:** after `make-self-contained`, before `plugin-removal` — -see `modules.ts` for the current ordering. - -**Shared types:** `WorkspaceManifest`, `BackstageManifest`, and their entry -types live in `lib/manifest-types.ts`. Both this module and -`protocol-resolution` depend on those types. +Builds `manifest.json` and (when needed) `backstage-manifest.json`: package +inventories used to resolve `workspace:^` and `backstage:^` to concrete semver +ranges. ## Problem -The `protocol-resolution` module needs to rewrite `workspace:^` and -`backstage:^` protocol references in `package.json` and `yarn.lock`. To do -this, it needs a lookup table mapping package names to their concrete versions -and dependency metadata. - -This information must be captured **before scrubbing** (`plugin-removal`), -because scrubbed packages are removed from disk but their versions are still -needed for protocol resolution — surviving packages may depend on scrubbed -ones via `workspace:^`. +Downstream steps need a name → version (and dependency metadata) lookup for +every workspace package and for Backstage release packages. That snapshot must +be captured while all workspace packages are still on disk, because entries in +`package.json` and `yarn.lock` can still reference packages that are removed +later in the pipeline. -## What the module produces +## What the module does ### `manifest.json` @@ -91,114 +81,30 @@ format (`npm:^X.Y.Z`), since they come from `yarn.lock` rather than Neither file is included in the OCI artifact — they are build-time intermediates. -## Why backstage metadata comes from yarn.lock, not the npm registry - -The original `generate-backstage-manifest.js` in `sync-midstream.sh` fetched -the Backstage release manifest from `versions.backstage.io` (one HTTP call), -then enriched each of the ~600 packages with dependency metadata from -`registry.npmjs.org` (60+ batched HTTP calls). The result was cached to disk -by Backstage version to amortize the cost across workspaces. - -This module takes a different approach: it **extracts backstage package -metadata directly from the workspace's `yarn.lock`**, using Yarn Berry's -official packages. - -### Rationale +## Why Backstage metadata from `yarn.lock` -After `yarn install --immutable` (which runs before this module), the -`yarn.lock` already contains the resolved version and full dependency metadata -for every `@backstage/*` package. These entries were created by the Backstage -Yarn plugin, which resolves `backstage:^` by looking up the exact version from -`versions.backstage.io` at install time. So the lockfile is a local cache of -the canonical version mapping, enriched with the dependency metadata that npm -registry calls would have provided. +`yarn.lock` already holds resolved `@backstage/*` versions and dependency +metadata after install. The [Backstage Yarn plugin](https://github.com/backstage/backstage/tree/master/packages/yarn-plugin) maps each `backstage:^` +dependency to a concrete npm version (from `backstage.json` and the release +manifest on `versions.backstage.io`); Yarn records the result in the lockfile. +This module extracts that metadata from the lockfile for +`backstage-manifest.json`. **Benefits:** -- **Zero npm registry calls.** The original script made hundreds of HTTP - requests. This module reads local files only (plus one validation fetch). -- **No caching needed.** The original script cached results to `/tmp/` to - avoid redundant fetches across workspaces in the same `sync-midstream.sh` - run. In the new per-workspace CI model (separate workflow run per - workspace), there's no shared filesystem to cache to. Since we read from the - lockfile, there's nothing to cache. -- **More accurate.** The lockfile reflects what was actually resolved for this - specific workspace, not a generic npm registry response. - -**Validation:** The module fetches the Backstage release manifest from -`versions.backstage.io` (a single HTTP call) and compares each extracted -version against the canonical manifest. A mismatch throws — indicating a stale -lockfile or unexpected inconsistency. This is a defensive check; by -construction, `yarn install --immutable` guarantees the lockfile is consistent -with the Backstage Yarn plugin's resolution. - -### Why the versions are guaranteed to match - -The `backstage:^` protocol is resolved by the Backstage Yarn plugin, which: - -1. Reads `backstage.json` to get the target release version -2. Fetches `versions.backstage.io/v1/releases//manifest.json` -3. Resolves each `backstage:^` to the exact version from that manifest -4. Writes the result to `yarn.lock` - -After `yarn install --immutable`, the lockfile entries for `backstage:^` are -by construction the same versions as the canonical manifest. The validation -fetch is belt-and-suspenders — it catches corruption, plugin bugs, or manual -lockfile edits that should never happen in CI. - -## Why upstream fetching is no longer needed - -The original `generate-workspace-manifest.js` fetched versions from GitHub for -packages that didn't exist locally — these were packages referenced by -`workspace:^` but deleted by scrubbing before the manifest was generated. The -manifest tagged these entries with `path: 'upstream'` and `source: 'upstream'` -so that downstream consumers (`createTypeShimsPackage` in `update-workspace.js`) -could distinguish them from packages that were once local. - -In the new pipeline, this entire mechanism is unnecessary because manifest -generation runs **before** `plugin-removal`. All workspace packages are still -on disk, so every entry in the manifest has a real `path`. When -`protocol-resolution` later needs to distinguish surviving from scrubbed -packages, it simply checks `fs.existsSync(dirname(pkg.path))` — exactly as the -old `update-workspace.js` already does (lines 1722, 1736, 1799). The -`path === 'upstream'` marker is never needed. - -## Differences from sync-midstream.sh - -| Aspect | Original (sync-midstream.sh) | New (this module) | -| ------------------------- | ----------------------------------------------------------- | ----------------------------------------------------------- | -| Upstream version fetching | Fetches missing `workspace:` deps from GitHub raw URLs | Not needed — all packages are local (runs before scrubbing) | -| Backstage metadata source | `versions.backstage.io` + npm registry (~600 HTTP calls) | `yarn.lock` extraction + one validation fetch | -| Caching | `/tmp/backstage-manifest-cache/` (shared across workspaces) | None needed | -| Error handling | Swallowed (`\|\| true`, stderr discarded) | Throws on error (pipeline aborts) | -| Package discovery | Recursive scan with skip-list heuristics | `workspaces` field glob resolution + filter | -| `bin` normalization | String → object (same) | String → object (same) | -| yarn.lock parsing | N/A (backstage manifest came from npm) | Yarn Berry packages | - -## Log messages - -Log messages are aligned with the original scripts for familiarity: - -| Log | When | -| ------------------------------------------------- | --------------------------------- | -| `Scanning for package.json files...` | Always (start of module) | -| `Found N local packages` | Always | -| ` - @scope/pkg@1.2.3` | Per package | -| `Manifest written to: manifest.json` | Always | -| `No backstage:^ dependencies found, skipping` | No `backstage:^` in any dep field | -| `Backstage version: X.Y.Z` | Backstage manifest path | -| `Extracted N @backstage/* entries from yarn.lock` | After lockfile parsing | -| `Validating against ` | Before validation fetch | -| `All N entries validated against Backstage X.Y.Z` | After successful validation | -| `Written: backstage-manifest.json (N packages)` | Backstage manifest written | - -## Error conditions - -| Condition | Behavior | -| ---------------------------------------------------------------- | ------------------------------------------- | -| No `backstage:^` deps in any `package.json` | Skip — no `backstage-manifest.json` written | -| Has `backstage:^` deps but no `backstage.json` | Throw | -| `backstage.json` exists but has no `version` field | Throw | -| `yarn.lock` missing (when backstage manifest needed) | Throw | -| Version mismatch between `yarn.lock` and `versions.backstage.io` | Throw with details | -| Failed to fetch `versions.backstage.io` | Throw with HTTP status | +- **No npm registry fan-out.** Dependency metadata for hundreds of + `@backstage/*` packages comes from the lockfile, not per-package registry + requests. +- **Workspace-accurate.** The manifest reflects versions and dependency trees + Yarn actually resolved for this repo, not a generic registry view. + +**Release-line validation:** `backstage.json` names a single Backstage release +(for example `1.42.5`). The manifest at +`versions.backstage.io/v1/releases//manifest.json` lists every +`@backstage/*` package version on that line. After extraction, the module +fetches that manifest once and checks each lockfile package against it. A +mismatch means the lockfile is not a coherent set for the claimed release — +for example `backstage.json` was bumped without `yarn install`, a bad merge in +`yarn.lock`, or a `resolutions` entry pinning a package off the release line. +The build fails with per-package details instead of writing a +`backstage-manifest.json` that downstream resolution would trust.