feat(people): add organizations and employment records - #593
Conversation
roborev: Combined Review (
|
538ed05 to
7c31adf
Compare
roborev: Combined Review (
|
wesm
left a comment
There was a problem hiding this comment.
Thanks — the core engineering here is solid (dual-backend concurrency, merge lock ordering, partial dates are all handled carefully, and the tests exercise real DB/HTTP paths). But there are two data-loss-class bugs to fix, and the PR is carrying roughly a third more code than the feature needs. Requesting changes.
Blocking bugs
employment setbuilds the PATCH body from changed flags only, but the server PATCH is full-replace (cmd/msgvault/cmd/employment.go:63). Fixing a title on a past employment nulls role, department, and dates, and flips it back tois_current=true.organization setalready does the correct read-merge-write;employment setshould too. There's no test forset.PATCH /organizations/{id}un-retires on any field-only update:Retiredis a non-pointerboolwith full-replace semantics (internal/api/organizations.go:40). Renaming a retired org silently reactivates it. The CLI works around this internally, which shows the footgun is live for other clients. Make it*bool.- Employment writes have no SQLite busy-retry, unlike attribute writes (which this PR generalized
retryAttributeWritefor). Concurrent writes for one person surface rawSQLITE_BUSYas an untyped 500, and both concurrency regression tests skip on SQLite, so the default backend has no coverage here (internal/store/employments.go:423).
Behavior surprises to resolve
- An update with
is_primaryomitted can silently auto-promote that employment to primary, changing the person's derived company as a side effect of a typo fix (employments.go:453). is_current=truecan coexist with a pastend_date; nothing validates consistency (employments.go:397).- Profile GET opens a write transaction with
FOR UPDATEwhere the person analog uses a read snapshot; on Postgres, reads serialize behind merges (organization_profile.go:161). - An invalid
primary_domainis silently dropped instead of erroring; the profile-identifier path errors for the same condition (organizations.go:523).
Scope: please trim
organization_namescopies 13 person-name columns the code never reads (organizations now havehonorific_prefixes), andorganization_schema_parity_test.goenforces the copy. Strip the unused columns and the test that freezes them in.- The duplicate-suggestions + merge workflow (~1,000 lines across store/API/CLI) goes beyond "organizations and temporal employment associations" — the roadmap lists same-name suggestions as bonus research. Suggest a follow-up PR.
- Dead surface:
UpdateOrganizationContext/RetireOrganizationContext/UnretireOrganizationContexthave no non-test callers;PrimaryCurrentEmploymentsContextis uncalled;SupersedeOrganizationAttributeValueContextis wired in the serve adapter but no route calls it (so org attributes can't be cleared over HTTP);PUT /organizations/{id}/profilehas no consumer;resolved_byis never written and tests assert it's always nil. organization_attributes.gois a ~90% copy ofperson_attributes.go, and the copies have already drifted (different error sentinels and locking for the same operation). TheretryAttributeWriteextraction in this PR shows the parameterization works — please share the rest.- CLI: about 10 of the 22 subcommands aren't needed for the advertised workflow (org attributes, the duplicates trio, merge, and
organization employments, which duplicatesemployment list --organization).
Smaller items
organization duplicates resolvederefsresp.JSON200.Statuswithout the nil guard every other subcommand has (organization.go:476).getCLIOrganizationhas a fallback for "older daemons" that can't exist — this PR introduces the endpoint — and the only test for the delete/retire path exercises that dead fallback.- Org endpoints reuse
decodePersonRequest, so malformed org bodies say "Invalid person request". - Flag inconsistency:
--endon add/set vs--end-dateon end. - Merge leaves open duplicate suggestions pointing at the merged org, unconditionally nulls
employments.address_id, and allows merging a live org into a retired one.
With the bugs fixed and the trims above, this drops from ~10.7k to roughly 6-7k hand-written lines without losing anything the description promises.
|
looking |
|
Pushed ec139bd addressing the review (note: the PR head had moved to 7c31adf, which already added the org-attribute DELETE route and profile history endpoint — the fixes are rebased on that). Fixed
Trimmed
Kept, deliberately
Net effect: −2,153 lines against the previous head. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed 85f2271 addressing all four roborev findings — each verified against the code before fixing, and each has a regression test:
OpenAPI specs, Go client, and web schema regenerated; store/api/cmd suites pass locally including the artifact-staleness checks. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed d741dfa addressing both findings — verified against the code first:
OpenAPI specs, Go client, and web schema regenerated; store/api/cmd suites pass including artifact-staleness checks. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed 28cdbff addressing both findings plus the CI failure:
🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed 67cd031 addressing the media-retention finding — verified first: the reconciliation hashed only the request's inline bytes, and since reads expose media metadata without the bytes, any GET-derived PUT computed an empty hash, mismatched the stored row, and superseded it with a URI-only copy. The fix is the retention-identifier approach: The HTTP regression test writes URI-plus-inline media, re-sends the GET-shaped row alongside an unrelated category addition, and proves the row is retained (same envelope ID, OpenAPI specs, Go client, and web schema regenerated; store/api/cmd suites pass including artifact-staleness checks. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed 7188c61. Confirmed: the media validation required Store-side validation only — no API schema change, so no regeneration was needed. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed 8deee64 (+ a gofmt follow-up). Confirmed the limitation: the retention hash only survived an exact match — any edit to media_type, uri, or an envelope field superseded the active row and the hash-only replacement then failed at insert for lacking data, so metadata edits required re-uploading the bytes. As suggested, retention hashes are now resolved against the active media rows before reconciliation: the stored bytes are loaded into the input, so when an edit supersedes and reinserts the row, the replacement carries the content. A hash with no active match is still rejected as The HTTP regression test edits an inline media row's type and URI sending only the retention hash, and proves the replacement row keeps Store-side only again — no API schema change, no regeneration needed. Store/api suites pass. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
Pushed 8c54234. Confirmed: the The reference-edge CTE now adds a second edge: employments of included people → their organizations' person-valued attribute values → the target's participants. It's gated on the source schema having the employment tables (older sources skip it) and on both The regression test builds an off-message person referenced by an included person's employer: with all three opt-ins the target person and the referencing attribute value survive the copy; without 🤖 Generated with Claude Code |
roborev: Combined Review (
|
Add first-class organizations with versioned profiles, typed custom attributes, lifecycle and merge operations, plus current and historical employment records with primary-employment constraints and read-time person and vCard projections, exposed through daemon APIs, generated clients, and organization/employment CLI commands. Squashed follow-ups from review: - fix(people): address review findings on organizations and employment - fix(people): address roborev findings on ec139bd - fix(people): round-trip envelope metadata and retire merged-org values - fix(people): copy employment data into subsets, guard survivor deletion - fix(people): retain inline media through GET-derived profile writes - fix(people): accept inline-only media retention via content_hash - fix(people): carry retained media bytes through metadata edits - fix(store): follow organization-attribute references in identity closure - fix(cli): demote a primary employment when set makes it historical Co-Authored-By: Wes McKinney <wesmckinn+git@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
8c54234 to
d14e7aa
Compare
|
Force-pushed d14e7aa: all review follow-ups squashed into the single feature commit and rebased onto current main (15c8f2c), authored as the original PR author with co-author trailers. Also fixes the last roborev finding: Rebase notes — main's typed person relationships (#592) landed on the same seams, resolved by keeping both features:
OpenAPI specs, Go client, and web schema regenerated against the merged surface. On the rebased tree: 🤖 Generated with Claude Code |
roborev: Combined Review (
|
- Profile replacement no longer rejects addresses, contact points, or media that share a business value under distinct PROP-IDs, TYPE labels, or ordinals: the duplicate check now includes an envelope discriminator, and reconciliation matches desired rows against multiple current rows sharing a value, preferring the one whose full envelope matches. Imported profiles with legitimate value duplicates round-trip with stable row identity - MergeOrganizationsContext retries on SQLite snapshot-upgrade busy errors via the shared bounded retry; each attempt re-reads both roots, so a concurrent change surfaces as a typed revision conflict instead of a raw SQLITE_BUSY failure - Replace bare "ETag" and "int64" literals in internal/api with package constants; the merge with the relationships feature pushed both over the goconst threshold that failed CI lint Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 4731c7c addressing both findings and the CI lint failure:
Store and api suites pass, vet clean. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
An organization-scoped listing rendered the organization ID on every row — the one value the caller already knows — and omitted the person, making employees unidentifiable. The counterpart column now follows the scope: person listings show ORGANIZATION, organization listings show PERSON. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 7dec131. Confirmed: On the CI failure: 🤖 Generated with Claude Code |
roborev: Combined Review (
|
employment list never sent limit or offset, silently capping output at the server's default page of 200 with no way to reach later records. Add --limit and --offset, forwarded to both the person- and organization-scoped requests, matching organization list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed b055a8a. Confirmed: both employment list endpoints accept 🤖 Generated with Claude Code |
roborev: Combined Review (
|
- SetOrganizationAttributeBody now carries active_from and active_until, forwarded to the store, so the HTTP API can record backdated and historical attribute values the way the store already supports. The HTTP test covers a backdated write and a fully historical write that lands as history without becoming the current value - organization attribute set gains --ordinal, forwarded through the body, so multi-valued attributes can be updated in place instead of only appended, and --expected-value-id targets the intended ordinal Regenerates OpenAPI specs, Go client, and web schema types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 456f75f addressing both findings — verified first:
OpenAPI specs, Go client, and web schema regenerated; api/cmd suites and the artifact-staleness checks pass. 🤖 Generated with Claude Code |
roborev: Combined Review (
|
employment set could not remove a date — an empty --start or --end reached the server and was rejected — and had no inverse of --not-current, so an ended employment could never be marked current again. Add --clear-start and --clear-end, which omit the date from the full-replace body, and --current, which reactivates an employment. An empty --start/--end now fails fast pointing at the clearing flag, --current conflicts with --not-current, and --current with a retained end date asks for --clear-end rather than sending a state the store rejects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed b7f06be for the CLI finding, and the CI failure is a repo-wide issue with a separate fix: CLI dates and reactivation: confirmed — an empty CI: this 🤖 Generated with Claude Code |
roborev: Combined Review (
|
|
I'm dropping the SQL enum CHECKs in favor of validating in Go (SQLite can't alter constraints like these without rebuilding the table), would have liked to have caught these earlier and prevented them from getting into the codebase at all |
The seven-value source IN (...) CHECK appeared on twenty tables per backend. A schema CHECK is a compatibility ceiling on SQLite: CREATE TABLE IF NOT EXISTS never updates constraints on existing archives and ALTER cannot modify them, so extending the provenance vocabulary would require rebuilding every table. The kinds vocabularies and the communication-service catalog already avoid database enums for exactly this reason, and the store validates provenance at the Go boundary (ParseProvenance / Provenance.Valid plus API enum tags). Drop the enum CHECK everywhere, keeping NOT NULL and the cross-field invariants (confidence range and declared-source scope, date shape, active windows). Tests that pinned the frozen vocabulary now prove the opposite contract: the database accepts future provenance values while the store boundary still rejects them. One legacy note: person_attribute_values shipped its CHECK in v0.19.1, so archives created before this change keep the frozen vocabulary on that one table until a rebuild migration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 3a679fb: dropped the seven-value Rationale: a schema CHECK is a compatibility ceiling on SQLite — The four tests that pinned the frozen vocabulary now prove the intended contract instead: the database accepts future provenance values while the store boundary still rejects them. One legacy caveat: 🤖 Generated with Claude Code |
roborev: Combined Review (
|
The create-subset help and runtime warnings still described only person data, understating what the profiles and attributes opt-ins now export. Both now name employment history, referenced organizations with their profiles, contacts, and media, and organization attribute values. The copy result reports exported organization and employment counts, and the CLI prints them under --include-profiles for auditing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed 91e7010. Confirmed: the
🤖 Generated with Claude Code |
roborev: Combined Review (
|
What changed
organizationandemploymentCLI commands.Why
People can have several current or historical roles, but copied company and title fields cannot preserve that history or represent one organization consistently across people. First-class records make employment changes queryable without losing source evidence.
Usage
Refs #534