Describe the feature
Problem
The Azure Table outbox persister serializes all transport operations for a single incoming message into one JSON string and writes it to the TransportOperations property of a single OutboxRecord row. Azure Table Storage enforces two hard limits that this design collides with (see Scalability and performance targets for Table storage and Understanding the Table service data model):
- 64 KiB per string property (UTF-16, ~32K characters)
- 1 MiB per entity in total (sum of all property values)
When the serialized payload exceeds 64 KiB the commit of the outbox transaction fails with PropertyValueTooLarge, the incoming message is not acknowledged, and the endpoint enters a poison-message loop that cannot be resolved by retries:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: xxxxx
---> Azure.Data.Tables.TableTransactionFailedException: The property value exceeds the maximum allowed size (64KB). If the property value is a string, it is UTF-16 encoded and the maximum number of characters should be 32K or less.
RequestId:xxxxx
Time:xxxxxxx
Status: 400 (Bad Request)
ErrorCode: PropertyValueTooLarge
Why existing workarounds are insufficient
The Azure Blob Storage Data Bus is opt-in per message property and requires application-level changes to message contracts, yet it only addresses message-body size, not the outbox's own envelope. Since the persistence is meant to abstract Azure Table Storage, users should not have to redesign messaging contracts to avoid a storage-engine limit the persistence could handle transparently.
Recommended solution
Have the Azure Table outbox transparently offload oversized transport-operation payloads to Azure Blob Storage.
Considered alternative — splitting operations across rows
The outbox metadata and individual transport operations could each become separate rows, batch-loaded back together. This is the Microsoft-documented Wide entities pattern for working around the 1 MiB entity cap. On its own, however, it does not solve the problem: a single transport operation (large body + headers) can still exceed 64 KiB.
Additional Context
Original issue body
Describe the feature.
When outbox needs to persist transportoperations that serialized exceed 64KB Azure Table storage rejects the write operation:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: fdr-nb-allocatie-servicebus
---> Azure.Data.Tables.TableTransactionFailedException: The property value exceeds the maximum allowed size (64KB). If the property value is a string, it is UTF-16 encoded and the maximum number of characters should be 32K or less.
RequestId:93d0e9ac-c002-004c-3886-cfd6ef000000
Time:2023-08-15T14:40:19.7363326Z
Status: 400 (Bad Request)
ErrorCode: PropertyValueTooLarge
The user can configure databus with Azure BLOB storage when only sending a few messages but better would be Azure storage outbox would detect that the transport operations exceed 64KB and store the byte array in BLOB storage instead.
This way the user does not need to use databus and Azure persistence outbox can also work when sending a larger amount of messages that would exceed 64KB even when using databus for certain properties.
Additional Context
No response
Describe the feature
Problem
The Azure Table outbox persister serializes all transport operations for a single incoming message into one JSON string and writes it to the
TransportOperationsproperty of a singleOutboxRecordrow. Azure Table Storage enforces two hard limits that this design collides with (see Scalability and performance targets for Table storage and Understanding the Table service data model):When the serialized payload exceeds 64 KiB the commit of the outbox transaction fails with
PropertyValueTooLarge, the incoming message is not acknowledged, and the endpoint enters a poison-message loop that cannot be resolved by retries:Why existing workarounds are insufficient
The Azure Blob Storage Data Bus is opt-in per message property and requires application-level changes to message contracts, yet it only addresses message-body size, not the outbox's own envelope. Since the persistence is meant to abstract Azure Table Storage, users should not have to redesign messaging contracts to avoid a storage-engine limit the persistence could handle transparently.
Recommended solution
Have the Azure Table outbox transparently offload oversized transport-operation payloads to Azure Blob Storage.
Considered alternative — splitting operations across rows
The outbox metadata and individual transport operations could each become separate rows, batch-loaded back together. This is the Microsoft-documented Wide entities pattern for working around the 1 MiB entity cap. On its own, however, it does not solve the problem: a single transport operation (large body + headers) can still exceed 64 KiB.
Additional Context
Original issue body
Describe the feature.
When outbox needs to persist transportoperations that serialized exceed 64KB Azure Table storage rejects the write operation:
The user can configure databus with Azure BLOB storage when only sending a few messages but better would be Azure storage outbox would detect that the transport operations exceed 64KB and store the byte array in BLOB storage instead.
This way the user does not need to use databus and Azure persistence outbox can also work when sending a larger amount of messages that would exceed 64KB even when using databus for certain properties.
Additional Context
No response