Support differentiating between DTS-HD and DTS Express in TS, MP4 and Matroska extractors

[nift4]

The TS extractor already supports DTS Express which uses the same headers as DTS-HD, but it had two bugs that made it impratical to use with DTS-HD streams:

• The mime type was hardcoded to DTS Express instead of reading from header. (Also, the DTS-HD sample actually is DTS Express.)
• The core format was emitted before the extension substream is read in a typical DTS-HD stream, which contains interleaved core and extension samples. Meanwhile, a DTS Express stream consists of extension samples only. This interleaved stream should still be output with the format of the extension samples, that is, DTS-HD, as opposed to core format (DTS only). To test this, a DTS-HD MA sample borrowed from the Matroska samples in ExoPlayer and remuxed with ffmpeg is added.

Meanwhile, Matroska had the wrong mime type assignment for DTS Express codec id. Also, FFmpeg outputs DTS Express files with "DTS" codec ID instead of "DTS Express", so we need to add it to the detection logic anyway. That however is free because we must do the detection anyway for DTS-HD, and we can even reuse all the code needed for TS logic.

The MP4 sample description box doesn't contain information about whether a stream is DTS Express, so use the same detection approach for MP4.

Issue: #2487

[FongMi]

Not enough. Please follow the ffmpeg implementation.

[nift4]

@FongMi please can you elaborate a bit?

[FongMi]

https://github.com/FongMi/media/blob/release-1.10.0-fongmi/libraries/extractor/src/main/java/androidx/media3/extractor/ts/DtsReader.java
https://github.com/FongMi/media/blob/release-1.10.0-fongmi/libraries/extractor/src/main/java/androidx/media3/extractor/DtsUtil.java

This is what I wrote using copilot + sonnet 4.6

[nift4]

@FongMi Well, I see that you did a lot of changes and most of them seem to address the same issue that we shouldn't emit core format but extension format instead (I solved it with the PatientTrackOutput wrapper and you added a new state and coreFormatPendingEmit instead). My fix is a bit subtle because it intercepts the format() call and waits until extension header is parsed before sending, allowing to overwrite the format which is cached. But it works and it's readable.

I also see another issue which I didn't solve but you solved it, that we should wait for core frame after seeking instead of sending extension substream frame first - it makes sense and I'll add it to my patch.

Is there any other problem I need to address?

[FongMi]

That should be all. I'll test the ISO file again after you finish.

[nift4]

@FongMi It should be good now, let me know if it works. Thanks :)

[FongMi]

Users have reported that the playback quality of DTS-HD is not as good as my version, and that stuttering occurs. This may be because your Core+EXTSS is sent in segments, while mine is sent all at once.

[FongMi]

FongMi@6755c3e
Please also implement DTS-MA and DTS-X recognition.
I've switched DtsUtil to your version to reduce merge conflicts.

[nift4]

@FongMi

Users have reported that the playback quality of DTS-HD is not as good as my version, and that stuttering occurs. This may be because your Core+EXTSS is sent in segments, while mine is sent all at once.

Should be fixed, please check again. Thanks for testing!

Please also implement DTS-MA and DTS-X recognition.

It's on my TODO list, but I will do it in a seperate PR when this is merged. This one is already quite big.

[FongMi]

@nift4 Users say it's still very laggy.

Core Architectural Difference

nick — PatientTrackOutput double-buffer

TS input → PatientTrackOutput internal buffer → flush() → TrackOutput

Every frame goes through two copies:

// Copy 1: into internal ParsableByteArray
public void sampleData(ParsableByteArray data, int length) {
    this.data.ensureCapacity(this.data.limit() + length);
    ByteBuffer tmp = ByteBuffer.wrap(this.data.getData()); // new object every call
    tmp.put(data.getData(), data.getPosition(), length);
}

// Copy 2: out to the real TrackOutput
public void flush() {
    output.sampleData(data, data.limit());
}

fongmi — zero-copy passthrough

TS input → TrackOutput (direct)

Instead of buffering, it adds a STATE_CHECKING_FOR_EXTSS_AFTER_CORE state: after reading a core frame, it peeks 4 bytes to check whether an EXSS follows, then decides to combine or emit immediately — no intermediate buffer needed.

---

Performance Comparison

Criterion	fongmi	nick
Data copies per frame	1 (direct write)	2 (buffered)
Per-frame heap allocation	None	ByteBuffer.wrap() each call
ensureCapacity reallocation risk	None	Yes (large frames)
DTS:X (XLL-X) auto-detection	Yes	No
Post-seek resync	skipExtssUntilCore (precise)	waitingForResyncAfterSeek (basic)
GC pressure	Low	Higher

---

Verdict

fongmi is faster, for three reasons:

1. Zero-copy: data is written directly to TrackOutput, eliminating the PatientTrackOutput intermediate buffer and second memcpy. For large DTS-HD MA frames (potentially hundreds of KB each), this is significant.

2. No per-frame heap allocation: nick allocates a new ByteBuffer wrapper on every sampleData() call, increasing GC pressure.

3. Richer functionality: fongmi adds XLL-X scanning to auto-detect DTS:X object audio and properly upgrades the track format to AUDIO_DTS_X, which nick lacks entirely.

nick's PatientTrackOutput is a cleaner abstraction conceptually, but the double-copy cost makes it a poor trade-off for a hot path that processes audio frames continuously.

[nift4]

Hi @FongMi, I ran my sample file through both versions of extractors and only difference I saw in result was average bitrate was set on mine. That's set from core header, I imagine it might cause buffer allocation to be too small, so I removed it. Can you re-test?

No per-frame heap allocation: nick allocates a new ByteBuffer wrapper on every sampleData() call, increasing GC pressure.

Also, I don't think that's the problem causing audible lags, but I changed it to only do that on the first frame (after seek).