breaking: implement the new sysand index read protocol

[consideRatio]

• fixes Change client's index reading behavior (breaking) #286
• part of More transparent directory structure in local environments and index #5

---

The client now reads remote indexes through the static-file protocol
documented in docs/src/index-protocol.md, with the publish-side API
scope split into docs/src/index-api-protocol.md.

This breaks compatibility with the old HTTP registry layout and with
0.3 registry-shaped lockfiles. Users must regenerate lockfiles, and
index servers must serve the new discovery, index.json, versions.json,
and per-version file layout.

Wire-format changes (reads):

• <discovery-root>/sysand-index-config.json is fetched lazily on first
  index use to resolve index_root and api_root. A 404 defaults both
  roots to the discovery root; other non-2xx responses are hard errors.
• index_root / api_root must be absolute HTTP(S) base URLs without
  userinfo. Relative URLs, credentials in URLs, and non-base paths are
  rejected.
• <index_root>/index.json is now project enumeration. Each projects
  entry carries iri and optional status; omitted status is
  equivalent to available.
• index.json project status supports available and removed.
  List-all operations filter removed projects, while direct resolution of
  a specific IRI still goes through that project's versions.json.
• Per-project versions.json replaces the old versions.txt list. Each
  entry carries version, usage, project_digest, kpar_size,
  kpar_digest, and optional status.
• versions.json entries are validated at ingest: SemVer only, no build
  metadata, descending precedence order, no duplicates, and lowercase
  sha256:<64-hex> digests.
• versions.json status supports available, yanked, and removed.
  New resolution filters to available versions; direct reads of removed
  versions fail distinctly; yanked versions remain readable for pinned
  lockfiles.
• Per-version layout becomes <project>/<version>/.project.json,
  .meta.json, and project.kpar.
• IRI path mapping is pkg:sysand/<publisher>/<name> →
  <publisher>/<name>/; all other IRIs use
  _iri/<sha256(normalized_iri)>/.
• Non-canonical pkg:sysand/... IRIs are hard-rejected through the new
  purl validation path, including
  InterchangeProjectUsageRaw::validate.
• project_digest is verified before .project.json / .meta.json are
  exposed. kpar_digest and kpar_size are enforced during streamed
  archive download.

Wire-format changes (publish):

• sysand publish --index now treats the configured URL as the discovery
  root, performs mandatory discovery with the configured
  HTTPAuthentication policy, and posts to <api_root>/v1/upload.
• Publish still supports direct API-root URLs because a missing discovery
  document defaults api_root to the configured URL.
• Publish credentials are matched after discovery against the actual
  upload URL. Discovery can therefore be auth-gated while upload remains
  bearer-only.
• Publish payload preparation now runs before network access, rejects
  SemVer build metadata, enforces the KPAR size limit, and sends
  archive-integrity metadata including the KPAR SHA-256 digest.

Lock/sync format changes:

• Lockfile format bumps from 0.3 to 0.4; old registry-shaped
  lockfiles are rejected with a regenerate message.
• Source::Registry is renamed to Source::Index.
• New Source::IndexKpar records index_kpar, index_kpar_size, and
  index_kpar_digest, preserving the raw-archive integrity tripwire from
  lock to sync.
• sync can now install index-backed KPARs directly from the lockfile and
  verifies the recorded size/digest without re-reading versions.json.
• Lockfile validation now distinguishes canonical project digest format
  from index KPAR digest format and rejects uppercase index KPAR digests
  before canonicalization.

Public Rust API:

• New modules: env::discovery, env::index, project::index_entry,
  and purl. Removed: env::reqwest_http::HTTPEnvironmentAsync.
• IndexEnvironmentAsync replaces the old HTTP registry environment and
  lazily resolves discovery endpoints.
• CombinedResolver / CombinedProjectStorage terminology changes from
  Registry to Index.
• standard_index_resolver and standard_resolver now return Result
  because discovery-root shape validation can fail.
• ProjectRead / ProjectReadAsync wrappers must explicitly forward
  get_info, get_meta, version, usage, and
  checksum_canonical_hex; the derive macro now forwards these methods.
• CanonicalizationError::map_project_read is added for wrapper error
  forwarding.
• ReqwestKparDownloadedProject now accepts optional expected KPAR
  digest/size in new and new_guess_root, verifies them during
  download, and exposes ensure_downloaded_verified /
  is_downloaded_and_verified. Builder setters like
  with_expected_sha256_hex / with_expected_size were removed.
• ReqwestKparDownloadedError gains DigestMismatch and SizeMismatch.
• InterchangeProjectValidationError::MalformedSysandPurl is added.
• core/Cargo.toml adds optional idna under networking and enables
  tokio/sync.

Other updates:

• The solver now uses ProjectRead::version and ProjectRead::usage
  summaries, allowing index candidates to be solved from versions.json
  without downloading archives.
• The protocol docs now distinguish project persistence from project
  retirement, and version persistence from version retirement.
• .meta.json created serialization is centralized and now emits
  second precision with a Z suffix.
• Java/Python bindings handle the resolver's new fallible construction;
  Java enables Tokio drivers; Python test setup isolates uv from outer
  conda environments.
• Tests cover discovery, IRI normalization, index read validation,
  digest/size drift, index project status filtering, yanked/removed
  behavior, publish discovery, and lock/sync round-trips for index-backed
  projects.

[andrius-puksta-sensmetry]

Impressive...

[andrius-puksta-sensmetry]

Just a start...

[andrius-puksta-sensmetry]

Looking good.