docs: add arrow to the well-known encodings registry

[amacneil]

Summary

Adds Apache Arrow as a well-known message encoding and schema encoding in the format registry.

• Message encoding arrow: each message is a single Arrow IPC encapsulated RecordBatch conforming to the channel's schema. The schema lives in the Schema record (not repeated per message), columns must not be dictionary-encoded so every RecordBatch decodes independently, and bodies must not use Arrow body compression (use Chunk compression instead) — mirroring how protobuf/flatbuffer keep the descriptor in the Schema record and a bare payload in each message.
• Schema encoding arrow: the schema data is a serialized Arrow IPC Schema message.

The registry text is kept terse and format-only, consistent with the existing entries; motivation and rationale are below rather than in the spec.

Motivation

Arrow is a stable, language-neutral columnar format with a well-specified IPC representation, and it's a natural fit for MCAP's "arbitrary serialization format" model. Standardizing a single arrow encoding lets us build one decode path that serves several increasingly common conversion sources:

• Arrow files (IPC stream/file) — decode RecordBatches directly.
• Parquet files — read natively into Arrow RecordBatches (Parquet isn't shardable into self-contained per-message units, so storing as Arrow is the clean target), then store with the same arrow encoding.
• Rerun .rrd files — Rerun stores Arrow record batches internally, so they decode into the same arrow representation.

Establishing the registry convention is the prerequisite for a future mcap convert path from these formats (with the source's chosen timestamp column/timeline mapped to each message's log_time) and for mcap cat decoding support. This PR is docs-only and intentionally scoped to the registry entries.

Alternatives considered

The central decision is what Message.data holds. We chose exactly one bare encapsulated RecordBatch, with the schema stored once in the Schema record. Alternatives we weighed:

• Self-contained IPC stream per message (schema + optional dictionaries + one RecordBatch). This is how Rerun's ArrowMsg is shaped, and it has the appeal of being self-describing and usable via the high-level StreamWriter/StreamReader APIs. Rejected because it repeats the schema in every message — measured at ~130 B for a tiny schema up to a few KB for wide/nested ones, which roughly doubles a single-row message (largely recovered by Chunk compression since the bytes are identical, but still a per-message CPU cost) — and it makes the Schema record redundant/optional, degrading mcap list schemas/mcap info. Keeping the schema in the Schema record mirrors protobuf/flatbuffer.
• "Any valid IPC stream containing at least a RecordBatch." Rejected as under-specified: it leaves open whether multiple RecordBatches, DictionaryBatches, or an embedded schema may appear, and what wins if an embedded schema disagrees with the Schema record. Those are exactly the interop forks a registry entry should remove. If a self-describing form is ever wanted, it should be pinned precisely ("a complete stream with a schema and exactly one RecordBatch") rather than left open-ended.
• Allowing dictionary-encoded columns. Rejected. A bare RecordBatch can't carry the preceding DictionaryBatch it would need, and dictionary IDs are stream-ordered state that is fragile under MCAP's random access, merge, filter, and sort. Converters hydrate dictionary columns to their value type on the way in (values preserved, only the encoding dropped). The rule covers nested fields (structs, lists, maps, unions), not just top-level columns.
• Allowing Arrow body compression (BodyCompression). Disallowed. It would double-compress on top of MCAP Chunk compression and force every reader to implement Arrow per-buffer LZ4/ZSTD; Chunk compression already covers size.
• Multiple Arrow message types / Tensor / SparseTensor. Out of scope. Standalone Tensor/SparseTensor aren't part of the IPC file/stream format; tensor data that appears as extension-typed columns inside a RecordBatch (arrow.fixed_shape_tensor, etc.) is already supported since we store the schema and batch verbatim.
• Row granularity (one row vs. many rows per message). This is a converter concern, not an encoding concern. The encoding allows a RecordBatch to contain one or more rows; MCAP treats the whole batch as occurring at the message's log_time. A converter can default to one row per message (most faithful to MCAP-native, per-message time semantics) and offer batching as an opt-in size/throughput trade-off.

Net: the single-RecordBatch form is the leanest, has the simplest decode path (read_record_batch(data, schema)), keeps crisp one-message/one-log_time semantics, stays robust under the CLI's reorder/merge/filter/sort operations, and matches the existing registry entries.

Notes

• Added IPC, pyarrow, and structs to the cspell word list so the spell-check CI passes.
• No library/code changes; existing implementations treat Schema.data and Message.data as opaque bytes, so this entry is purely a documented convention.

  

[claude]

LGTM