Skip to content

Integration tests flake against dufs: connection reset on back-to-back requests with a body #3

Description

@lollipopkit

Observed while preparing 2.0.0. test/webdav_client_test.dart fails roughly 2 runs in 8 with HttpException: Connection reset by peer. The failing operation varies (Basic read, Basic conditional put, the exists following rename), so it is not tied to one test.

The library is not at fault. Recording the evidence here so it is not re-investigated from scratch.

Attribution

Same client, same PROPFIND request, different servers:

Server Failures
Dart HttpServer, request body drained 0 / 200
Dart HttpServer, request body ignored 0 / 200
dufs 0.46.0 3 / 200

Trigger is a request body

Five alternating rounds:

Request Failures
PROPFIND with body 9 / 500
PROPFIND without body 0 / 500

GET (no body) and PUT (body, which dufs reads in full) were both 0 / 100. Only PROPFIND is affected.

The request dio emits is well formed, so this is not a malformed-request problem:

PROPFIND /present.txt HTTP/1.1
content-length: 90
depth: 0
authorization: Basic ...
content-type: application/xml

<?xml version="1.0" encoding="utf-8"?><d:propfind xmlns:d="DAV:"><d:allprop/></d:propfind>

Rate dependent, not an idle timeout

Gap between requests Failures
0 ms 3 / 150
2 ms 1 / 150
10 ms 0 / 150
50 ms 0 / 150

Failures appear only when requests are sent back to back and disappear once the requests are spaced out, which is the opposite of what an idle keep-alive timeout would produce. dufs does not send Connection: close on these responses, yet the connection goes away. A raw socket pipelining 200 PROPFINDs with bodies over a single connection completed all 200, so dufs handles bodies correctly in general; the race is specific to a request arriving immediately after the previous response was written. dufs exposes no keep-alive configuration.

The exact location in dufs or hyper was not identified.

Options

  1. Leave the tests as they are and accept the flake.
  2. Space out or retry at the connection level inside the integration test only.
  3. Retry once on connection-level errors for idempotent methods (GET, HEAD, OPTIONS, PROPFIND) in the client. Dart HttpClient does not do this, and real servers do drop pooled connections, so this has value beyond dufs. It also risks retrying something non-idempotent if the method list is drawn too widely, so it needs a deliberate decision rather than being folded into a release.

Unrelated observation in the same file

_baseTestDir = ".test" is resolved against the test process working directory, while dufs serves from a directory under systemTemp. The delete/recreate in setUp therefore never touches the data under test, and isolation between tests currently depends on mkdir returning 405 for an existing collection.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinglib

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions