Skip to content

feat(sentdm_broadcast): SentDM v3 SMS/WhatsApp broadcast action with auto-registered webhooks and graph audit trail#10

Merged
eldonm merged 10 commits into
dev-cockpitfrom
jb/sentdm-action
May 14, 2026
Merged

feat(sentdm_broadcast): SentDM v3 SMS/WhatsApp broadcast action with auto-registered webhooks and graph audit trail#10
eldonm merged 10 commits into
dev-cockpitfrom
jb/sentdm-action

Conversation

@barnwell

Copy link
Copy Markdown

Summary

Adds a new SentDMBroadcastAction that wraps the
SentDM v3 REST API for sending
template-based SMS / WhatsApp broadcasts, reading message status, listing
templates, and receiving delivery-status callbacks. Persists each sent
message as a graph node so webhook events can be folded back into the agent
graph as an audit trail and the action becomes composable with other
actions (await agent.get_action_by_type("SentDMBroadcastAction")).

What's new

jvagent/action/sentdm_broadcast/ (new core action package)

  • SentDMBroadcastAction(Action) — public async methods callable from
    Python or HTTP: send_broadcast, get_message_status,
    get_message_activities, list_templates, get_account,
    healthcheck, plus webhook lifecycle (reconcile_webhook_endpoint,
    get_webhook_url).
  • Configurable defaults via agent.yaml: api_base,
    default_channels, default_template_id / default_template_name,
    profile_id, timeout, sandbox, plus webhook tuning
    (webhook_display_name, webhook_event_types, webhook_retry_count,
    webhook_timeout_seconds).
  • SENTDM_API_KEY is the only required env var. Webhook
    auto-registration additionally needs JVAGENT_PUBLIC_BASE_URL and
    JVSPATIAL_JWT_SECRET_KEY.
  • Auto-registered webhook on on_register / on_reload: same
    pattern as WhatsAppAction — generates a signed callback URL
    ({JVAGENT_PUBLIC_BASE_URL}/api/sentdm/webhook/{action_id}?api_key=...),
    reconciles SentDM's webhook list via GET /v3/webhooks, deletes stale
    entries, creates a fresh one with POST /v3/webhooks, and persists the
    signing secret returned by SentDM.
  • HMAC-SHA256 signature verification on every inbound webhook
    (X-Webhook-Signature) using hmac.compare_digest. Idempotency via an
    in-memory LRU keyed on X-Webhook-ID.
  • Graph persistence (new SentDMBroadcastRecord(Node)): each
    successful POST /v3/messages writes one record per
    (message_id, recipient, channel) with indexed action_id,
    agent_id, sentdm_message_id, plus send-time metadata
    (to, channel, template_id/name, parameters,
    idempotency_key, profile_id, sandbox) and mutable status fields
    (status, last_event_field, last_event_payload, last_status_at,
    events[], error, updated_at). Connected back to the action node.
    Persistence is gated by persist_records / persist_sandbox_sends /
    record_event_history_limit attributes; failures are logged but never
    fail the send.
  • Webhook handler updates the record: parses the SentDM message id
    out of payload.payload.{id,message_id,messageId} (also checks nested
    message.* / data.*), normalizes the new status (known
    payload.status token, then event.<suffix> fallback), appends to
    the bounded events log, and saves. Unknown ids still return 200 so
    SentDM stops retrying.
  • Recovery via refresh_record: re-fetches via
    GET /v3/messages/{id} and folds the result in as a "refresh" event
    (covers missed webhooks / dropped events).

HTTP endpoints (all tagged ["SentDM"] so they appear under the
SentDM section in /docs, /redoc, and /openapi.json):

Method Path Auth Purpose
POST /api/actions/{id}/sentdm/broadcast admin Send a broadcast
GET /api/actions/{id}/sentdm/messages/{message_id} admin Message status (SentDM-truth)
GET /api/actions/{id}/sentdm/messages/{message_id}/activities admin Delivery activity log
GET /api/actions/{id}/sentdm/templates admin List templates
GET /api/actions/{id}/sentdm/status admin Healthcheck (/v3/me)
POST /api/actions/{id}/sentdm/webhook/register admin Force webhook reconcile
GET /api/actions/{id}/sentdm/webhook admin Read-only view of registered URL + signing-secret status
GET /api/actions/{id}/sentdm/broadcasts admin List persisted records (filters: status, to, sentdm_message_id, page, page_size)
GET /api/actions/{id}/sentdm/broadcasts/{record_id} admin Single record with full events[]
POST /api/actions/{id}/sentdm/broadcasts/{record_id}/refresh admin Re-fetch status from SentDM and fold into the record
POST /api/sentdm/webhook/{action_id} api_key (system) Inbound delivery-event receiver

Standalone test CLI (jvagent/action/sentdm_broadcast/test_cli.py)

  • Talks only over HTTP — no jvagent imports at runtime.
  • Loads .env (defaults to examples/jvagent_app/.env; override with
    --env-file; bypass entirely with --no-env).
  • Auto-login mode by default when the env supplies credentials
    (JVAGENT_API_KEY or JVAGENT_ADMIN_EMAIL + JVAGENT_ADMIN_PASSWORD),
    including auto-selecting the agent that hosts a SentDM action. Use
    --prompt to force the interactive flow.
  • Auth uses POST /api/auth/login with the correct {email, password}
    shape (jvspatial's UserLogin requires an EmailStr).
  • Menu items: healthcheck, send broadcast (sandbox-on by default), list
    templates, get message status, get message activities, reconcile
    webhook, show registered webhook URL, list / show / refresh persisted
    broadcast records, switch agent, quit.

Example app wiring

  • examples/jvagent_app/agents/jvagent/example_agent/agent.yaml
    registers jvagent/sentdm_broadcast_action with safe defaults
    (sandbox: true, SMS-only, persist on / persist-sandbox off,
    event-history cap of 25).
  • examples/jvagent_app/agents/jvagent/example_agent/README.md
    documents the new action, required env vars, and all endpoints.

Docs

  • jvagent/action/sentdm_broadcast/README.md — covers env vars,
    configurable attributes, the graph data model, every HTTP endpoint,
    the auto-registered webhook flow, status derivation, and the test CLI.
  • SentDM upstream API references checked into
    jvagent/action/sentdm_broadcast/docs/ (llms.txt, openapi.json,
    openapi.yaml, postman.json) for offline reference.

Notable design choices

  • Not wired into ResponseBus / ChannelAdapter. SentDM has no
    inbound-message webhook (only delivery status), so the push-broadcast
    shape doesn't map to the reply-to-interaction model that adapters were
    built for. The action is invoked by callers directly
    (await sentdm.send_broadcast(...)) instead.
  • Record creation is best-effort. A persistence failure after a
    successful upstream send is logged but doesn't bubble out, so the
    caller still sees the SentDM response.
  • Webhook signature verification happens before the DB hit. Bad
    signatures get a 401 and dedupes return 200 short of any record
    lookup — no extra DB load from spoofed traffic.

Files

  • jvagent/action/sentdm_broadcast/__init__.py
  • jvagent/action/sentdm_broadcast/info.yaml
  • jvagent/action/sentdm_broadcast/sentdm_broadcast_action.py
  • jvagent/action/sentdm_broadcast/endpoints.py
  • jvagent/action/sentdm_broadcast/models.py
  • jvagent/action/sentdm_broadcast/webhook_auth.py
  • jvagent/action/sentdm_broadcast/test_cli.py
  • jvagent/action/sentdm_broadcast/README.md
  • jvagent/action/sentdm_broadcast/docs/llms.txt
  • jvagent/action/sentdm_broadcast/docs/openapi.json
  • jvagent/action/sentdm_broadcast/docs/openapi.yaml
  • jvagent/action/sentdm_broadcast/docs/postman.json
  • examples/jvagent_app/agents/jvagent/example_agent/agent.yaml
  • examples/jvagent_app/agents/jvagent/example_agent/README.md

Test plan

  • Set SENTDM_API_KEY in examples/jvagent_app/.env and start
    jvagent against the example app. Confirm
    GET /api/actions/{id}/sentdm/status returns healthy: true with
    configured: true and the account's channels.
  • With JVAGENT_PUBLIC_BASE_URL unset, confirm the action still
    loads, sentdm_status reports webhook_registered: false, and
    the logs include a webhook reconcile skipped: JVAGENT_PUBLIC_BASE_URL is not set debug line.
  • Set JVAGENT_PUBLIC_BASE_URL (e.g. an ngrok URL) and
    JVSPATIAL_JWT_SECRET_KEY, restart, and confirm
    GET /api/actions/{id}/sentdm/webhook returns a non-null
    webhook_url, sentdm_webhook_id, and has_signing_secret: true.
    Verify in SentDM's dashboard that a webhook entry with our URL
    exists and stale entries from earlier runs were removed.
  • Send a sandbox broadcast via
    POST /api/actions/{id}/sentdm/broadcast with sandbox: true;
    confirm 200 response, no SentDMBroadcastRecord node is
    written (default), and SentDM does not actually send.
  • Send a real broadcast (sandbox: false and a real template);
    confirm one n.SentDMBroadcastRecord.* node per
    (recipient, channel) appears in the DB with status: accepted
    and events[0].field == "send".
  • Trigger a synthetic webhook (or wait for a real status change)
    and confirm GET /api/actions/{id}/sentdm/broadcasts/{record_id}
    shows the new event in events[], an updated status /
    last_status_at, and the latest payload under
    last_event_payload. Verify a tampered X-Webhook-Signature
    gets a 401 and that re-sending the same X-Webhook-ID is
    reported as {"status": "duplicate"}.
  • Call
    POST /api/actions/{id}/sentdm/broadcasts/{record_id}/refresh
    and confirm a new "refresh" entry is appended and the latest
    server-truth status is folded in.
  • Confirm the new endpoints show up grouped under "SentDM" in
    /docs (Swagger UI) and /redoc, with all path/query params and
    request bodies typed correctly.
  • Run the test CLI with python jvagent/action/sentdm_broadcast/test_cli.py
    against a local jvagent, verify auto-login picks up
    examples/jvagent_app/.env, the SentDM action is auto-selected,
    and the new menu items (8 – 11) round-trip through the new
    endpoints.
  • pre-commit run --all-files (or
    python -m black --check jvagent/action/sentdm_broadcast/ +
    python -m flake8 jvagent/action/sentdm_broadcast/) is clean.

barnwell added 4 commits May 12, 2026 14:20
…st action

This commit introduces the `SentDMBroadcastAction`, which wraps the SentDM v3 REST API to facilitate sending template-based SMS and WhatsApp broadcasts. It includes a README with setup instructions, configuration options, and usage examples, as well as webhook authentication utilities and an interactive test CLI for endpoint testing.

fix: improve error handling in SentDM API documentation
This commit updates the README for the SentDM broadcast action, clarifying CLI usage, environment variable requirements, and the auto-mode functionality. It also adds detailed documentation for the SentDM broadcast action in the example agent's README and YAML configuration, including required environment variables and webhook setup instructions.
…hooks into the graph

Previously POST /v3/messages was fire-and-forget at the graph level: the
returned message id was dropped, and the webhook handler verified the
signature, logged the event, and returned 200. Delivery status changes
never made it into the graph, so callers had no audit trail and missed
webhooks could only be recovered by re-querying SentDM by hand.

Add a SentDMBroadcastRecord(Node) and wire it through both ends:

- New SentDMBroadcastRecord in models.py with indexed action_id /
  agent_id / sentdm_message_id, captured send metadata (to, channel,
  template, parameters, idempotency_key, profile_id, sandbox), and
  mutable status fields (status, last_event_field, last_event_payload,
  last_status_at, events[], error, updated_at).

- send_broadcast now parses the SentDM response (robust to several
  shapes: messages[], data{}, data.messages[], top-level dict) and
  creates one record per (message_id, recipient, channel), connected
  back to the action node. New attributes persist_records (default on),
  persist_sandbox_sends (off), record_event_history_limit (25) gate the
  behavior. Persistence failures are logged but never fail the send.

- Webhook handler now extracts the SentDM message id from the event
  payload (id / message_id / messageId at top level, then under
  message.* / data.*), looks up the local record by indexed lookup,
  normalizes the new status (payload.status first, falling back to
  event.<suffix> parsing), appends to the bounded events[] audit log,
  and updates status / last_status_at / error. Unknown message ids
  still return 200 so SentDM stops retrying.

- New admin endpoints (all tags=["SentDM"], shown in /docs):
    GET  /actions/{id}/sentdm/webhook                       (read-only)
    GET  /actions/{id}/sentdm/broadcasts                    (filterable)
    GET  /actions/{id}/sentdm/broadcasts/{record_id}        (full events)
    POST /actions/{id}/sentdm/broadcasts/{record_id}/refresh

- New action method refresh_record(record_id) re-fetches via
  GET /v3/messages/{id} and folds the result into the record (recovers
  from missed webhooks).

- Test CLI grows four menu items: show webhook URL, list broadcasts,
  show one record, refresh a record.

- Docs: action README adds a "Graph data model" section, the new
  attributes, and the new endpoints; example_agent agent.yaml exposes
  the persist_* attributes at safe defaults; example_agent README lists
  the new endpoints.

No new dependencies. ResponseBus / ChannelAdapter wiring intentionally
not added — SentDM has no inbound-message webhook, only delivery events,
so the push-broadcast shape doesn't map to the reply-to-interaction
model that adapters are built for.
@barnwell barnwell self-assigned this May 13, 2026
barnwell added 5 commits May 14, 2026 09:54
…ion functions

This commit introduces several improvements to the SentDM webhook handling:

- Added `_normalize_sentdm_webhook_envelope` function to standardize the incoming webhook payload structure, accommodating both direct and nested formats.
- Updated the `sentdm_webhook_receive` function to utilize the new normalization method, improving payload handling and logging.
- Introduced new tests for status derivation and envelope normalization to ensure correctness.
- Enhanced the README documentation to clarify webhook event types and filters, including default behaviors for event filters.
- Updated the `SentDMBroadcastAction` class to support new event filter configurations and improve status derivation logic.

These changes aim to streamline the integration with SentDM and enhance the reliability of message status tracking.
This commit enhances the error handling within the SentDM webhook processing. It introduces more robust logging for webhook events, ensuring that errors are captured and reported accurately. Additionally, it refines the normalization process for incoming webhook payloads, improving the overall reliability of message status updates. These changes aim to provide better visibility into webhook interactions and facilitate easier debugging.

refactor(sentdm_broadcast): update endpoint paths and improve documentation

This commit refactors the SentDM broadcast action by updating endpoint paths to remove the "sentdm" prefix, simplifying the API structure. The changes include:

- Renaming endpoints from `/api/actions/{action_id}/sentdm/...` to `/api/actions/{action_id}/...` for broadcasting and webhook registration.
- Adjusting the corresponding methods and tests to reflect the new endpoint structure.
- Enhancing the README documentation to clarify the updated paths and their purposes.

These modifications aim to streamline the API and improve usability for developers interacting with the SentDM broadcast functionality.
…e sandbox sends

This commit modifies the example_agent's YAML configuration by commenting out the existing profile_id for clarity and enabling the persist_sandbox_sends option. These changes aim to improve the configuration's usability and ensure that sandbox sends are recorded during testing.

refactor(sentdm_broadcast): update webhook tags and enhance path handling

This commit refines the SentDM broadcast action by updating webhook tags for clarity and improving path handling for legacy webhook URLs. Key changes include:

- Changed webhook tags from "Webhooks" to "SentDM Broadcast" for better categorization.
- Enhanced path handling to accommodate both current and legacy webhook paths, ensuring compatibility and improved reconciliation logic.
- Updated tests to verify the handling of legacy webhook paths.

These modifications aim to streamline the webhook processing and improve the overall clarity of the API documentation.

refactor(sentdm_broadcast): enhance CLI prompts and logging for broadcast action

This commit improves the user experience for the SentDMBroadcastAction by refining the CLI prompts and adding informative logging. Key changes include:

- Updated prompts to allow optional environment variable defaults for recipient phone numbers, template IDs, and parameters JSON, reducing user input effort.
- Enhanced logging to provide clearer information on the broadcast process, especially regarding sandbox sends and record creation.
- Adjusted the README documentation to reflect these changes and clarify the usage of new environment variables.

These modifications aim to streamline the broadcast sending process and improve overall usability for developers.
…ook processing

This commit introduces several improvements to the SentDMBroadcastAction and its associated webhook handling:

- Added new static methods to extract message descriptors from various envelope structures, accommodating both lists and nested formats.
- Enhanced the `_extract_sent_message_descriptors` method to handle new API response shapes, ensuring accurate retrieval of message data.
- Introduced tests to validate the extraction logic for different envelope scenarios, including single messages and nested results.
- Updated the webhook processing to resolve message IDs more effectively, improving the reliability of status updates and record handling.

These changes aim to streamline the integration with SentDM and enhance the accuracy of message tracking and processing.
@barnwell barnwell marked this pull request as ready for review May 14, 2026 19:12
@barnwell barnwell requested a review from eldonm May 14, 2026 19:12
@eldonm eldonm merged commit 6a62dba into dev-cockpit May 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants