Payment Processing Service

Asynchronous payment processing microservice: the API accepts requests, processing runs in the background through payment gateway emulation, and the result is delivered to the client via webhook notification.

Stack: Python 3.13, FastAPI, Pydantic v2, SQLAlchemy 2.0 (async), PostgreSQL, RabbitMQ (FastStream), Alembic, dishka (DI), loguru. Package manager: uv. Type checking: mypy strict, linting: ruff.

Architecture

Hexagonal (ports & adapters). Dependencies flow strictly inward: presentation → application → domain, infrastructure connects via ports.

• app/domain — clean core: Payment entity with lifecycle rules, Money value object, enumerations, domain exceptions. Standard library only.
• app/application — ports (typing.Protocol), DTOs, retry policy, and use cases: CreatePayment, GetPayment, ProcessPayment, HandleProcessingFailure, RelayOutboxBatch. Knows about the domain, not about infrastructure.
• app/infrastructure — port adapters: SQLAlchemy repositories with mappers ORM model ↔ domain entity, unit of work, RabbitMQ publisher and failure router, gateway emulator, HTTP webhook delivery with circuit breaker, system clock and ID generator, settings, logging, correlation ID.
• app/presentation — inbound adapters: HTTP (routers, schemas, middleware, error handlers), AMQP subscriber, outbox relay loop. Thin: parse the transport and call use cases.
• app/setup — dishka DI providers by layer (scopes APP/REQUEST) and container assembly: each process has its own set of providers.
• app/entrypoints — composition root per process: api, consumer, relay.

Three processes from one codebase:

• api — accepts payments and returns their status. Does not interact with the broker at all: writes both the payment and an event to the outbox table in a single transaction (transactional outbox).
• relay — fetches unpublished events from outbox in batches under FOR UPDATE SKIP LOCKED and publishes them to RabbitMQ. If it crashes after publishing but before marking published_at, the event will be sent again, and the consumer can handle it.
• consumer — the sole handler: emulates the gateway (2-5 s, 90% success rate), updates status, sends webhook. Idempotent to repeated delivery.

Retry and DLQ

Retries are built on TTL + dead-letter exchange, with no plugins or timers in code. On an infrastructure failure, the consumer requeues the event to a retry queue with TTL (2 s, then 4 s) and an x-attempt counter in the header. When TTL expires, the broker itself returns the message to the work queue. After the third failed attempt, the event goes to payments.dlq with an x-last-error header. The work queue also has a configured DLX: a message that cannot even be parsed goes to the DLQ instead of being lost. Routing decisions are made by a pure RetryPolicy in the application layer — it is covered by unit tests without the broker.

Idempotency

• API: unique index on idempotency_key. Retrying a request with the same key returns the original payment with the same 202. Request body matching is not checked; the key determines it.
• Consumer: status is committed before webhook delivery. On retry, the payment is already in a terminal status, so the gateway is not called again, only webhook delivery is retried. The webhook_delivered_at marker protects against duplicate sends. Delivery is "at-least-once": the webhook receiver needs to deduplicate by payment_id.

Webhook Resilience

Delivery goes through a circuit breaker with per-host state: after WEBHOOK_CB_FAILURE_THRESHOLD consecutive failures, the circuit opens and attempts to that host fail immediately without waiting for a timeout; after WEBHOOK_CB_RECOVERY_SECONDS a single probe request is allowed through. Delivery failure remains an ordinary infrastructure failure and follows the broker's common retry topology.

End-to-End Correlation ID

X-Correlation-Id is accepted from the client (or generated by middleware), travels in the outbox event payload, in the AMQP x-correlation-id header, in the webhook header, and in every log line of all three processes — the payment's path is traced by a single value.

Health

• GET /health — liveness, used by compose health check.
• GET /health/ready — readiness: checks the database (SELECT 1), returns 503 if unavailable.

Design Decisions from Spec

• 10% gateway "errors" are a business outcome (status=failed + webhook), not a failure. Retry triggers only on infrastructure problems: unreachable webhook_url, database errors, unexpected exceptions.
• "3 attempts" means three complete processing passes: first + two retries with 2 and 4 second delays.
• Relay is separated into its own process: it must survive API crashes and not interfere with its horizontal scaling.

Running

docker compose up -d --build

This brings up postgres, rabbitmq, runs migrations once, api (port 8000), consumer, and relay. .env is optional: the stack starts with default dev values.

• Swagger: http://localhost:8000/docs
• RabbitMQ: http://localhost:15672 (guest/guest)

Locally (requires running postgres and rabbitmq):

uv run uvicorn app.entrypoints.api:app --reload
uv run faststream run app.entrypoints.consumer:app
uv run python -m app.entrypoints.relay

In dev environment, POST /internal/webhook-sink is available — a webhook receiver for testing the full cycle without an external service (not mounted in prod).

Configuration

Variable	Default	Purpose
APP_ENV	dev	Environment dev/prod; docs and dev key are disabled in prod
DATABASE_URL	localhost	PostgreSQL DSN (asyncpg)
RABBITMQ_URL	localhost	RabbitMQ DSN
API_KEY	dev-api-key	API key; prod will not start with the default value
LOG_LEVEL	empty	Log level; empty = DEBUG in dev, INFO in prod
WEBHOOK_TIMEOUT	10.0	Webhook delivery timeout, seconds
OUTBOX_POLL_INTERVAL	1.0	Relay tick interval, seconds
OUTBOX_BATCH_SIZE	100	Relay batch size
GATEWAY_SUCCESS_RATE	0.9	Gateway emulator success rate
GATEWAY_DELAY_MIN / MAX	2.0 / 5.0	Gateway emulator delay, seconds
WEBHOOK_CB_FAILURE_THRESHOLD	5	Consecutive failures before circuit opens
WEBHOOK_CB_RECOVERY_SECONDS	30.0	Delay before probe attempt

Request Examples

Create a payment (X-API-Key and Idempotency-Key headers are required):

curl -X POST http://localhost:8000/api/v1/payments \
  -H "X-API-Key: dev-api-key" \
  -H "Idempotency-Key: order-42" \
  -H "Content-Type: application/json" \
  -d '{
    "amount": "199.90",
    "currency": "RUB",
    "description": "Order 42",
    "metadata": {"order_id": 42},
    "webhook_url": "https://example.com/hook"
  }'

Response 202 Accepted:

{"payment_id": "f76d6692-...", "status": "pending", "created_at": "2026-08-27T05:04:56Z"}

Retrying with the same Idempotency-Key returns the same payment_id. After 2-5 seconds the payment transitions to succeeded or failed, and a POST is sent to webhook_url:

{"payment_id": "...", "status": "succeeded", "amount": "199.90", "currency": "RUB",
 "processed_at": "...", "metadata": {"order_id": 42}}

Get payment status:

curl http://localhost:8000/api/v1/payments/<payment_id> -H "X-API-Key: dev-api-key"

401 without the key, 422 without Idempotency-Key, 404 for an unknown id.

Tests

Test pyramid: tests/unit (domain, use cases with fake ports, circuit breaker), tests/integration (repositories and UoW on SQLite, broker adapters with fake), tests/e2e (ASGI client of the real app with a test DI container).

uv run pytest --cov=app
uv run mypy app tests
uv run ruff check .

Future Development

• /metrics (Prometheus) is intentionally not added: there is no scraper in the stack, a dead dependency is worse than its absence.
• Idempotency based on request body (hash comparison) if key collision becomes an issue.
• Retry delay tiers are fixed by TTL queues; arbitrary delays with jitter would require the delayed-exchange plugin, not timers in code.