diff --git a/docs/todo.md b/docs/todo.md index 9e11cae10e..ab797b282f 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -36,12 +36,22 @@ - Investigate FTP support with Swoole's built-in coroutine FTP implementation. It supplies `ftp_*` functions but is not discoverable as `ext-ftp`, so Composer rejects `league/flysystem-ftp` and `RequiresPhpExtension('ftp')` skips the driver test. Resolve normal development and production installation without bypassing dependency checks, then add the adapter to root `require-dev`, use a test requirement that accepts either FTP implementation, and update the installation guidance. Basic transfers through Hypervel and Flysystem have been verified inside a Swoole coroutine. +## AI and MCP + +- Port the MCP package in its separately assigned session, retaining the AI SDK's optional MCP adapters in the AI port. Audit client/session lifetimes and safe bounded reuse, HTTP/stdio cancellation and cleanup, coroutine-local request resolution through server/tool/resource paths, and lazy response iteration. Resolve the current MCP Request from CoroutineContext keyed by Request::class, shared with the AI adapter's MCP_REQUEST constant. Run the AI adapter's real MCP-dependent tests once the package is available and remove analysis annotations made obsolete by that dependency. Finalize the AI docs' MCP section and links when the MCP package and its documentation page land. AI's ordinary tools and providers must remain usable without MCP. Do not ship placeholder MCP classes or expand the AI port into the full MCP package. + ## HTTP Client +- Recheck streaming performance once a build containing the Swoole native `curl_multi_select()` fix is available. The affected hook can consume ready response data internally and then wait for another event or its timeout before returning. Hypervel therefore sometimes delays an SSE event or completion by up to one second, even though the data has arrived. This requires no Hypervel workaround. + - Run `Hypervel\Tests\Http\Client\CurlStreamingHandlerTest::testNativeWaitReportsDataArrivingBetweenPerformAndSelect` against the patched extension with its temporary `SWOOLE_VERSION_ID <= 60203` skip removed locally. Verify both mid-stream readiness and completion, plus the existing quiet-provider timeout/cancellation and cleanup tests. Update the committed version gate only after identifying the actual fixed release; the current gate does not establish that a later release is fixed. + - In an idle window, use `tests/Benchmarks/HttpTransport` and its README to repeat alternating paired measurements before/after high-water buffering on the same patched runtime. Keep dependencies, TLS verification, idle-retention bound, PHP/OPcache/JIT/GC settings and request concurrency identical. Cover paced events, unpaced bulk responses and slow consumers, then repeat sustained concurrent traffic. Check first-event and completion latency (including outliers), throughput, client CPU, memory, connection reuse and descriptor cleanup. Confirm the one-second stalls disappear without changing Hypervel's select interval or buffering strategy. + - Record the native PR/build, settings and raw reports before publishing buffering performance claims. If a verified problem remains, investigate its owning layer rather than adding polling, shorter waits or a producer coroutine to hide it. +- Resolve the Guzzle-dependent test skips when the upstream changes are available: adopt configurable idle-handle retention for named buffered requests and unskip `HttpConnectionTest::testNamedBufferedRequestsReuseConnectionsAcrossConcurrentBursts`; once shared StreamHandler request deadlines are fixed, remove the `ClientInterface::MAJOR_VERSION >= 8` skip from `HttpClientStreamingTest::testConcurrentFallbackRequestsKeepIndependentDeadlines`. Verify both supported Guzzle branches when adopting these changes. - Once the HTTP client can optionally use Swoole's coroutine HTTP client as its transport instead of curl, explore and benchmark that transport for Inertia SSR requests. Each SSR render posts the page JSON to a local SSR server through the `inertia-ssr` connection (`HttpGateway::CONNECTION`), which makes it a good candidate for the alternative transport. Compare the curl and Swoole transports on that connection against a local SSR server under concurrent load, measuring latency, throughput, client CPU, memory and connection reuse, and use the Swoole transport for the SSR connection if it is a clear improvement. ## HTTP Server +- Integrate a server-level HTTP/2 stream-cancel event once Swoole exposes it in a supported release, and raise the `ext-swoole` constraint to that release when integrating it. The current `RST_STREAM` path removes the stream without notifying PHP, so an AI response producer waiting on a silent provider cannot stop immediately when its caller cancels that stream. Use the event's connection and stream identities with Hypervel's active-response map to cancel only the affected opted-in producer. Preserve sibling streams, application close callbacks and terminable work. Whole-connection closes continue using existing onClose handling; no native per-response subscription is needed. Keep the AI package usable on existing supported Swoole releases with connection-close handling where available, failed-write cleanup and provider timeouts; document the remaining HTTP/2 limitation. This enhancement is not an AI-port release prerequisite. Use the actual released event name and arguments rather than polling or anticipating an unreleased API. - Require a Swoole release that resets signal-listener state in forked server workers before releasing Hypervel 0.4. In Swoole 6.2.3 and earlier, a worker forked after the manager calls `Process::signal()` inherits the listener count, so `Coroutine\System::waitSignal()` fails in it. Hypervel's SIGINT shutdown handling registers a manager callback in both server modes, so after a reload, `max_request` recycling or a crash restart, replacement workers stop receiving configured signal handlers and Artisan traps. Once a fixed release is verified, raise the `ext-swoole` constraint and remove the version skip from `ShutdownOnInterruptListenerTest::testReplacementWorkersKeepTheirSignalHandlers()`. - Remove trailer-stream one-chunk lookahead once the minimum supported Swoole release includes [swoole-src#6124](https://github.com/swoole/swoole-src/pull/6124). Current releases send an empty `END_STREAM` DATA frame before trailer HEADERS when `end()` receives no body after `write()`, so `ResponseBridge` retains the final chunk for `end($chunk)` and delays delivery by one chunk. Once fixed, raise the `ext-swoole` constraint, write every chunk immediately, emit trailers, call bare `end()`, invert the deterministic bridge ordering tests, and add real gRPC incremental-delivery coverage. diff --git a/src/auth/src/AuthManager.php b/src/auth/src/AuthManager.php index 7c8714c76f..fd4f6dcd6c 100755 --- a/src/auth/src/AuthManager.php +++ b/src/auth/src/AuthManager.php @@ -181,11 +181,8 @@ protected function getConfig(string $name): ?array */ public function getDefaultDriver(): string { - if (CoroutineContext::has(self::DEFAULT_GUARD_CONTEXT_KEY)) { - return CoroutineContext::get(self::DEFAULT_GUARD_CONTEXT_KEY); - } - - return $this->app->make('config')->string('auth.defaults.guard'); + return CoroutineContext::get(self::DEFAULT_GUARD_CONTEXT_KEY) + ?? $this->app->make('config')->string('auth.defaults.guard'); } /** diff --git a/src/auth/src/Passwords/PasswordBrokerManager.php b/src/auth/src/Passwords/PasswordBrokerManager.php index 2d23d304b9..1e34ab5443 100644 --- a/src/auth/src/Passwords/PasswordBrokerManager.php +++ b/src/auth/src/Passwords/PasswordBrokerManager.php @@ -162,8 +162,10 @@ public function resolveBrokerNameForGuard(UnitEnum|string $guard): ?string */ public function getDefaultDriver(): string { - if (CoroutineContext::has(self::DEFAULT_BROKER_CONTEXT_KEY)) { - return CoroutineContext::get(self::DEFAULT_BROKER_CONTEXT_KEY); + $name = CoroutineContext::get(self::DEFAULT_BROKER_CONTEXT_KEY); + + if ($name !== null) { + return $name; } $guard = $this->app->make(AuthFactoryContract::class)->getDefaultDriver(); diff --git a/src/container/src/Container.php b/src/container/src/Container.php index 332240ddf2..d59d95873d 100755 --- a/src/container/src/Container.php +++ b/src/container/src/Container.php @@ -732,13 +732,15 @@ public function extend(string $abstract, Closure $closure): void $this->autoSingletons[$abstract] = $closure($this->autoSingletons[$abstract], $this); $this->rebound($abstract); - } elseif ($this->isScoped($abstract) && CoroutineContext::has(self::SCOPED_CONTEXT_PREFIX . $abstract)) { + } elseif ($this->isScoped($abstract) + && ($instance = CoroutineContext::get(self::SCOPED_CONTEXT_PREFIX . $abstract)) !== null + ) { // Apply the extender to the current cached scoped instance immediately. // Also queue it in $this->extenders for future resolutions — unlike singletons // (which persist for the worker lifetime), scoped instances are destroyed each // request by forgetScopedInstances() and must re-apply extenders on rebuild. $contextKey = self::SCOPED_CONTEXT_PREFIX . $abstract; - CoroutineContext::set($contextKey, $closure(CoroutineContext::get($contextKey), $this)); + CoroutineContext::set($contextKey, $closure($instance, $this)); $this->extenders[$abstract][] = $closure; @@ -1191,8 +1193,10 @@ protected function resolve( && ! $needsContextualBuild ) { $contextKey = self::SCOPED_CONTEXT_PREFIX . $abstract; - if (CoroutineContext::has($contextKey)) { - return CoroutineContext::get($contextKey); + $instance = CoroutineContext::get($contextKey); + + if ($instance !== null) { + return $instance; } } @@ -1338,7 +1342,7 @@ protected function resolve( if ($publishedCache === 'scoped') { $contextKey = self::SCOPED_CONTEXT_PREFIX . $abstract; - if (CoroutineContext::has($contextKey) + if ($publishedValue !== null && CoroutineContext::get($contextKey) === $publishedValue ) { CoroutineContext::forget($contextKey); diff --git a/src/context/src/CoroutineContext.php b/src/context/src/CoroutineContext.php index dc7199a15d..6fcdf339b1 100644 --- a/src/context/src/CoroutineContext.php +++ b/src/context/src/CoroutineContext.php @@ -270,6 +270,21 @@ private static function prepareForCopy(array $values): array return $values; } + /** + * Store a value in non-coroutine context only. + * + * Tests only. This writes worker-global storage even from a coroutine, + * so request-time use can leak state between concurrent requests. + * + * @param TKey $id + * @param TValue $value + * @return TValue + */ + public static function setNonCoroutine(UnitEnum|string $id, mixed $value): mixed + { + return static::$nonCoroutineContext[is_string($id) ? $id : enum_value($id)] = $value; + } + /** * Get a value from non-coroutine context only. * diff --git a/src/contracts/src/Filesystem/Filesystem.php b/src/contracts/src/Filesystem/Filesystem.php index 0e9db947af..28bd6378be 100644 --- a/src/contracts/src/Filesystem/Filesystem.php +++ b/src/contracts/src/Filesystem/Filesystem.php @@ -27,10 +27,20 @@ interface Filesystem public function path(string $path): string; /** - * Determine if a file exists. + * Determine if a file or directory exists. */ public function exists(string $path): bool; + /** + * Determine if a file exists. + */ + public function fileExists(string $path): bool; + + /** + * Determine if a directory exists. + */ + public function directoryExists(string $path): bool; + /** * Get the contents of a file. */ diff --git a/src/contracts/src/JsonSchema/JsonSchema.php b/src/contracts/src/JsonSchema/JsonSchema.php index 79749476ea..6eed31fb49 100644 --- a/src/contracts/src/JsonSchema/JsonSchema.php +++ b/src/contracts/src/JsonSchema/JsonSchema.php @@ -20,7 +20,7 @@ interface JsonSchema /** * Create a new object schema instance. * - * @param array|(Closure(JsonSchema): array) $properties + * @param array|(Closure(JsonSchema): array) $properties */ public function object(Closure|array $properties = []): ObjectType; diff --git a/src/contracts/src/Routing/ResponseFactory.php b/src/contracts/src/Routing/ResponseFactory.php index d86f87a9d8..78253f4f74 100644 --- a/src/contracts/src/Routing/ResponseFactory.php +++ b/src/contracts/src/Routing/ResponseFactory.php @@ -6,6 +6,7 @@ use BackedEnum; use Closure; +use Hypervel\Http\IterableStreamedResponse; use Hypervel\Http\JsonResponse; use Hypervel\Http\RedirectResponse; use Hypervel\Http\Response; @@ -44,6 +45,8 @@ public function jsonp(string $callback, mixed $data = [], int $status = 200, arr /** * Create a new event stream response. + * + * @return IterableStreamedResponse */ public function eventStream(Closure $callback, array $headers = [], StreamedEvent|string|null $endStreamWith = ''): StreamedResponse; diff --git a/src/database/README.md b/src/database/README.md index fb0ec1430b..043e087b9d 100644 --- a/src/database/README.md +++ b/src/database/README.md @@ -7,6 +7,7 @@ Documentation: https://hypervel.org/docs/database ## Differences From Laravel +- Outgoing framework HTTP requests return idle database sessions to their pools. Code using temporary tables, session locks or retained raw PDOs across these calls must use `withPinnedSession()`; ordinary queries and active transactions keep their behavior. See [releasing and pinning connections](https://hypervel.org/docs/database#releasing-and-pinning-connections). - MySQL and MariaDB connections require `strict` or `modes` in their configuration instead of implicitly inheriting the server's SQL mode, so SQL literals can be escaped correctly. - Laravel's external database pooler support uses a `::direct` connection suffix. Hypervel instead uses normal named connections for each endpoint and `migrations_connection` for schema and migration paths. This keeps direct and pooled endpoints as normal configured connections with their own pool settings, so Hypervel does not support Laravel's `::direct` suffix. - Laravel exposes PDO methods on its base connection class. Hypervel's base `Connection` is driver-neutral, while PDO-backed drivers extend `PdoConnection`. Code that requires direct PDO access should accept or narrow to `PdoConnection`. When bringing in Laravel database updates, keep driver-neutral connection behavior on `Connection`, PDO mechanics on `PdoConnection`, and driver-specific behavior on the matching connection, grammar, schema builder, or processor. diff --git a/src/database/src/Concerns/ManagesTransactions.php b/src/database/src/Concerns/ManagesTransactions.php index 5890ec206f..ad7318627b 100644 --- a/src/database/src/Concerns/ManagesTransactions.php +++ b/src/database/src/Concerns/ManagesTransactions.php @@ -408,9 +408,10 @@ protected function handleRollBackException(Throwable $e): void protected function forgetLostConnection(): void { try { - $this->resetTransactionState(); - } finally { $this->forgetDriverResources(); + } finally { + // Rollback callbacks may reconnect; only forget the failed session. + $this->resetTransactionState(); } } diff --git a/src/database/src/Connection.php b/src/database/src/Connection.php index 969c14c932..be13df0f2c 100755 --- a/src/database/src/Connection.php +++ b/src/database/src/Connection.php @@ -112,6 +112,11 @@ abstract class Connection implements ConnectionInterface, NonCopyableContext */ protected int $foreignKeyConstraintSuppressionDepth = 0; + /** + * The number of operations retaining the current physical session. + */ + protected int $sessionPinDepth = 0; + /** * The transaction manager instance. */ @@ -549,57 +554,63 @@ public function prepareBindings(array $bindings): array */ protected function run(string $query, array $bindings, Closure $callback): mixed { - foreach ($this->beforeExecutingCallbacks as $beforeExecutingCallback) { - $beforeExecutingCallback($query, $bindings, $this); - } + ++$this->sessionPinDepth; - $this->reconnectIfMissingConnection(); + try { + foreach ($this->beforeExecutingCallbacks as $beforeExecutingCallback) { + $beforeExecutingCallback($query, $bindings, $this); + } - $start = hrtime(true) / 1e9; + $this->reconnectIfMissingConnection(); - // Here we will run this query. If an exception occurs we'll determine if it was - // caused by a connection that has been lost. If that is the cause, we'll try - // to re-establish connection and re-run the query with a fresh connection. - try { + $start = hrtime(true) / 1e9; + + // Here we will run this query. If an exception occurs we'll determine if it was + // caused by a connection that has been lost. If that is the cause, we'll try + // to re-establish connection and re-run the query with a fresh connection. try { - $result = $this->runQueryCallback($query, $bindings, $callback); - } catch (QueryException $e) { - $result = $this->handleQueryException( - $e, - $query, - $bindings, - $callback - ); - } - } catch (CanceledException $exception) { - throw $exception; - } catch (Throwable $exception) { - $events = $this->events; - - if ($events?->hasListeners(QueryFailed::class)) { - $events->dispatch(new QueryFailed( - $query, - $bindings, - $this->getElapsedTime($start), - $this, - $exception, - $this->latestReadWriteTypeUsed(), - )); + try { + $result = $this->runQueryCallback($query, $bindings, $callback); + } catch (QueryException $e) { + $result = $this->handleQueryException( + $e, + $query, + $bindings, + $callback + ); + } + } catch (CanceledException $exception) { + throw $exception; + } catch (Throwable $exception) { + $events = $this->events; + + if ($events?->hasListeners(QueryFailed::class)) { + $events->dispatch(new QueryFailed( + $query, + $bindings, + $this->getElapsedTime($start), + $this, + $exception, + $this->latestReadWriteTypeUsed(), + )); + } + + throw $exception; } - throw $exception; - } + // Once we have run the query we will calculate the time that it took to run and + // then log the query, bindings, and execution time so we will report them on + // the event that the developer needs them. We'll log time in milliseconds. + $this->logQuery( + $query, + $bindings, + $this->getElapsedTime($start) + ); - // Once we have run the query we will calculate the time that it took to run and - // then log the query, bindings, and execution time so we will report them on - // the event that the developer needs them. We'll log time in milliseconds. - $this->logQuery( - $query, - $bindings, - $this->getElapsedTime($start) - ); - - return $result; + return $result; + } finally { + --$this->sessionPinDepth; + } } /** @@ -617,67 +628,73 @@ protected function run(string $query, array $bindings, Closure $callback): mixed */ protected function runStreaming(string $query, array $bindings, Closure $callback): Generator { - foreach ($this->beforeExecutingCallbacks as $beforeExecutingCallback) { - $beforeExecutingCallback($query, $bindings, $this); - } + ++$this->sessionPinDepth; + + try { + foreach ($this->beforeExecutingCallbacks as $beforeExecutingCallback) { + $beforeExecutingCallback($query, $bindings, $this); + } - $this->reconnectIfMissingConnection(); + $this->reconnectIfMissingConnection(); - $start = hrtime(true) / 1e9; - $hasYielded = false; + $start = hrtime(true) / 1e9; + $hasYielded = false; + + $execute = function (string $query, array $bindings) use ($callback, &$hasYielded): Generator { + try { + foreach ($callback($query, $bindings) as $key => $value) { + $readWriteType = $this->latestReadWriteTypeRetrieved; + $hasYielded = true; + + try { + yield $key => $value; + } finally { + // A consumer may run another query while this operation is suspended. + $this->latestReadWriteTypeRetrieved = $readWriteType; + } + } + } catch (CanceledException|StreamClosedException $exception) { + throw $exception; + } catch (Exception $exception) { + ++$this->errorCount; + + throw $this->newQueryException($query, $bindings, $exception); + } + }; - $execute = function (string $query, array $bindings) use ($callback, &$hasYielded): Generator { try { - foreach ($callback($query, $bindings) as $key => $value) { - $readWriteType = $this->latestReadWriteTypeRetrieved; - $hasYielded = true; - - try { - yield $key => $value; - } finally { - // A consumer may run another query while this operation is suspended. - $this->latestReadWriteTypeRetrieved = $readWriteType; + try { + yield from $execute($query, $bindings); + } catch (QueryException $exception) { + if ($hasYielded) { + throw $exception; } + + yield from $this->handleQueryException($exception, $query, $bindings, $execute); } } catch (CanceledException|StreamClosedException $exception) { throw $exception; - } catch (Exception $exception) { - ++$this->errorCount; - - throw $this->newQueryException($query, $bindings, $exception); - } - }; - - try { - try { - yield from $execute($query, $bindings); - } catch (QueryException $exception) { - if ($hasYielded) { - throw $exception; + } catch (Throwable $exception) { + $events = $this->events; + + if ($events?->hasListeners(QueryFailed::class)) { + $events->dispatch(new QueryFailed( + $query, + $bindings, + $this->getElapsedTime($start), + $this, + $exception, + $this->latestReadWriteTypeUsed(), + )); } - yield from $this->handleQueryException($exception, $query, $bindings, $execute); - } - } catch (CanceledException|StreamClosedException $exception) { - throw $exception; - } catch (Throwable $exception) { - $events = $this->events; - - if ($events?->hasListeners(QueryFailed::class)) { - $events->dispatch(new QueryFailed( - $query, - $bindings, - $this->getElapsedTime($start), - $this, - $exception, - $this->latestReadWriteTypeUsed(), - )); + throw $exception; } - throw $exception; + $this->logQuery($query, $bindings, $this->getElapsedTime($start)); + } finally { + --$this->sessionPinDepth; } - - $this->logQuery($query, $bindings, $this->getElapsedTime($start)); } /** @@ -1056,6 +1073,37 @@ public function clearBeforeExecutingCallbacks(): void $this->beforeExecutingCallbacks = []; } + /** + * Retain the physical database session for the duration of the callback. + * + * @template TReturn + * + * @param Closure(): TReturn $callback + * @return TReturn + */ + public function withPinnedSession(Closure $callback): mixed + { + ++$this->sessionPinDepth; + + try { + return $callback(); + } finally { + --$this->sessionPinDepth; + } + } + + /** + * Determine whether an operation requires the current physical session. + * + * @internal + */ + public function hasPinnedSession(): bool + { + return $this->sessionPinDepth > 0 + || $this->transactions > 0 + || $this->foreignKeyConstraintSuppressionDepth > 0; + } + /** * Begin a foreign key constraint suppression scope. * diff --git a/src/database/src/ConnectionResolver.php b/src/database/src/ConnectionResolver.php index fe57466464..24145bcb33 100755 --- a/src/database/src/ConnectionResolver.php +++ b/src/database/src/ConnectionResolver.php @@ -8,6 +8,7 @@ use Hypervel\Context\CoroutineContext; use Hypervel\Contracts\Container\Container; use Hypervel\Coroutine\Coroutine; +use Hypervel\Database\Pool\ConnectionLease; use Hypervel\Database\Pool\PooledConnection; use Hypervel\Database\Pool\PoolManager; use Swoole\Coroutine\CanceledException; @@ -19,7 +20,7 @@ /** * Resolves database connections from a connection pool. * - * Retains pooled wrappers until their owning coroutine or task ends. + * Retain logical connections while allowing idle physical sessions to be returned early. */ class ConnectionResolver implements ConnectionResolverInterface { @@ -45,7 +46,7 @@ class ConnectionResolver implements ConnectionResolverInterface /** * Pooled wrappers retained by non-coroutine task execution. * - * @var array + * @var array */ protected array $nonCoroutineConnections = []; @@ -62,9 +63,8 @@ public function __construct( /** * Get a database connection instance. * - * The connection is retrieved from a pool and stored in the current - * coroutine's context. When the coroutine ends, the connection is - * automatically released back to the pool. + * Store a stable logical connection in the current coroutine's context. + * The current physical lease is settled when the coroutine ends. */ public function connection(UnitEnum|string|null $name = null): ConnectionInterface { @@ -80,12 +80,10 @@ public function connection(UnitEnum|string|null $name = null): ConnectionInterfa $connectionOwnerName = $connectionName->requested; $contextKey = $this->getContextKey($connectionOwnerName); - // Check if this coroutine already has a connection - if (CoroutineContext::has($contextKey)) { - $connection = CoroutineContext::get($contextKey); - if ($connection instanceof ConnectionInterface) { - return $connection; - } + $connection = CoroutineContext::get($contextKey); + + if ($connection instanceof ConnectionInterface) { + return $connection; } $pool = $this->poolManager->pool($connectionName->requested); @@ -97,24 +95,33 @@ public function connection(UnitEnum|string|null $name = null): ConnectionInterfa $connectionOwnerName = $pool->getName(); $contextKey = $this->getContextKey($connectionOwnerName); - if (CoroutineContext::has($contextKey)) { - $connection = CoroutineContext::get($contextKey); - - if ($connection instanceof ConnectionInterface) { - if ($connectionName->isWrite() && $connection instanceof Connection) { - $connection->useWriteConnectionWhenReading(); - } + $connection = CoroutineContext::get($contextKey); - return $connection; + if ($connection instanceof ConnectionInterface) { + if ($connectionName->isWrite() && $connection instanceof Connection) { + $connection->useWriteConnectionWhenReading(); } + + return $connection; } } + if (! Coroutine::inCoroutine() && isset($this->nonCoroutineConnections[$connectionOwnerName])) { + // A purge can remove the context entry before the task ends. + $previous = $this->nonCoroutineConnections[$connectionOwnerName]; + unset($this->nonCoroutineConnections[$connectionOwnerName]); + $previous->discard(); + } + /** @var PooledConnection $pooledConnection */ $pooledConnection = $pool->borrow(); + $leaseContextKey = $this->getLeaseContextKey($connectionOwnerName); try { - $connection = $pooledConnection->getConnection(); + $owner = $pool->usesSessionLeases() ? $pooledConnection->newLease() : $pooledConnection; + $connection = $owner instanceof ConnectionLease + ? $owner->connection + : $pooledConnection->getConnection(); // Keep the borrowed alias so migrations and error rendering reuse this connection. if ($connectionName->role !== null && ! $sharedInMemorySqlite) { @@ -127,20 +134,29 @@ public function connection(UnitEnum|string|null $name = null): ConnectionInterfa CoroutineContext::set($contextKey, $connection); + if ($owner instanceof ConnectionLease) { + CoroutineContext::set($leaseContextKey, $owner); + } + // Listeners can resolve this connection. Notify before registering a // deferred release, since a listener failure discards the wrapper. $pooledConnection->dispatchConnectionEstablishedEvent(); if (Coroutine::inCoroutine()) { - Coroutine::defer(function () use ($pooledConnection, $contextKey): void { - CoroutineContext::forget($contextKey); - $pooledConnection->release(); + Coroutine::defer(function () use ($owner, $contextKey, $leaseContextKey): void { + try { + $owner->release(); + } finally { + CoroutineContext::forget($contextKey); + CoroutineContext::forget($leaseContextKey); + } }); } else { - $this->nonCoroutineConnections[$connectionOwnerName] = $pooledConnection; + $this->nonCoroutineConnections[$connectionOwnerName] = $owner; } } catch (Throwable $exception) { CoroutineContext::forget($contextKey); + CoroutineContext::forget($leaseContextKey); unset($this->nonCoroutineConnections[$connectionOwnerName]); $this->discardFailedConnection($pooledConnection, $exception); @@ -151,6 +167,18 @@ public function connection(UnitEnum|string|null $name = null): ConnectionInterfa return $connection; } + /** + * Return idle physical sessions owned by the current execution. + */ + public static function releaseIdleConnections(): void + { + foreach (CoroutineContext::getContainer() ?? [] as $value) { + if ($value instanceof ConnectionLease) { + $value->releaseIfIdle(); + } + } + } + /** * Release connections retained by non-coroutine task execution. * @@ -159,7 +187,7 @@ public function connection(UnitEnum|string|null $name = null): ConnectionInterfa public function releaseConnections(): void { $this->terminateConnections( - static function (PooledConnection $connection): void { + static function (ConnectionLease|PooledConnection $connection): void { $connection->release(); }, ); @@ -173,7 +201,7 @@ static function (PooledConnection $connection): void { public function discardConnections(): void { $this->terminateConnections( - static function (PooledConnection $connection): void { + static function (ConnectionLease|PooledConnection $connection): void { $connection->discard(); }, ); @@ -215,6 +243,14 @@ protected function getContextKey(string $name): string return sprintf('__database.connection.%s', $name); } + /** + * Get the context key for the current logical connection's lease. + */ + protected function getLeaseContextKey(string $name): string + { + return sprintf('__database.lease.%s', $name); + } + /** * Discard a failed connection while preserving cancellation precedence. */ @@ -239,24 +275,25 @@ protected function terminateConnections(Closure $terminate): void $connections = $this->nonCoroutineConnections; $this->nonCoroutineConnections = []; - foreach (array_keys($connections) as $name) { - CoroutineContext::forget($this->getContextKey($name)); - } - - CoroutineContext::forget(self::DEFAULT_CONNECTION_CONTEXT_KEY); - $exception = null; - foreach ($connections as $connection) { - try { - $terminate($connection); - } catch (Throwable $throwable) { - if ($exception === null - || ($throwable instanceof CanceledException && ! $exception instanceof CanceledException) - ) { - $exception = $throwable; + try { + foreach ($connections as $name => $connection) { + try { + $terminate($connection); + } catch (Throwable $throwable) { + if ($exception === null + || ($throwable instanceof CanceledException && ! $exception instanceof CanceledException) + ) { + $exception = $throwable; + } + } finally { + CoroutineContext::forget($this->getContextKey($name)); + CoroutineContext::forget($this->getLeaseContextKey($name)); } } + } finally { + CoroutineContext::forget(self::DEFAULT_CONNECTION_CONTEXT_KEY); } if ($exception !== null) { diff --git a/src/database/src/Connectors/ConnectionFactory.php b/src/database/src/Connectors/ConnectionFactory.php index 34c6be73cf..15f6a3ada4 100755 --- a/src/database/src/Connectors/ConnectionFactory.php +++ b/src/database/src/Connectors/ConnectionFactory.php @@ -65,6 +65,25 @@ public function make(array $config, ?string $name = null): Connection return $this->createPdoConnectionFromConfig($config); } + /** + * Create a logical connection from selected endpoint configuration and PDO resolvers. + * + * @param Closure(): PDO $pdo + * @param null|Closure(): PDO $readPdo + * + * @internal + */ + public function makeWithPdoResolvers(array $config, Closure $pdo, ?Closure $readPdo = null): PdoConnection + { + return $this->createConnection( + $config['driver'], + $pdo, + $config['database'], + $config['prefix'], + $config, + )->setReadPdo($readPdo); + } + /** * Get the extension resolver for a connection configuration. */ @@ -178,6 +197,16 @@ public function configForRead(array $config): array ); } + /** + * Get the write configuration for a read / write connection. + * + * @internal + */ + public function configForWrite(array $config): array + { + return $this->getWriteConfig($this->parseConfig($config, $config['name'] ?? null)); + } + /** * Create a PDO-backed connection from its configuration. */ @@ -213,6 +242,9 @@ protected function createReadWriteConnection(array $config): PdoConnection { $connection = $this->createSingleConnection($this->getWriteConfig($config)); + // The PDO resolver and its metadata must describe the same selected endpoint. + $config['read'] = $this->getReadWriteConfig($config, 'read'); + return $connection ->setReadPdo($this->createReadPdo($config)) ->setReadPdoConfig($this->getReadConfig($config)); diff --git a/src/database/src/DatabaseManager.php b/src/database/src/DatabaseManager.php index 9f265e8cc5..f85a5635da 100755 --- a/src/database/src/DatabaseManager.php +++ b/src/database/src/DatabaseManager.php @@ -292,12 +292,21 @@ public function purge(UnitEnum|string|null $name = null): void } } + /** + * Return the current execution's idle database sessions to their pools. + */ + public function releaseIdleConnections(): void + { + ConnectionResolver::releaseIdleConnections(); + } + /** * Disconnect from the given database. * - * In pooled mode, this disconnects the current coroutine's driver resources - * (if one exists), forcing a reconnect on the next query. Does not clear - * context or affect the pool - the connection is still released at coroutine end. + * In pooled mode, close the current execution's driver resources and retain + * its logical connection, which reconnects on next use. Session leases + * invalidate their physical holder and settle the slot at the next release. + * Whole-connection pools retain their wrapper until execution ends. * * In non-pooled mode, disconnects the connection stored in the manager's cache * for SimpleConnectionResolver or in a CachedConnectionResolver's cache. diff --git a/src/database/src/DatabaseTransactionState.php b/src/database/src/DatabaseTransactionState.php new file mode 100644 index 0000000000..8b5f5e24ab --- /dev/null +++ b/src/database/src/DatabaseTransactionState.php @@ -0,0 +1,41 @@ + */ + public Collection $committed; + + /** @var Collection */ + public Collection $pending; + + /** @var array */ + public array $current = []; + + /** + * Create transaction state owned by one execution. + */ + public function __construct() + { + $this->committed = new Collection; + $this->pending = new Collection; + } + + /** + * Get the current execution's transaction state. + */ + public static function current(): self + { + return CoroutineContext::get(self::CONTEXT_KEY) + ?? CoroutineContext::set(self::CONTEXT_KEY, new self); + } +} diff --git a/src/database/src/DatabaseTransactionsManager.php b/src/database/src/DatabaseTransactionsManager.php index f496ed14b1..739fdb3511 100755 --- a/src/database/src/DatabaseTransactionsManager.php +++ b/src/database/src/DatabaseTransactionsManager.php @@ -17,12 +17,6 @@ */ class DatabaseTransactionsManager { - protected const string COMMITTED_CONTEXT_KEY = '__database.transactions.committed'; - - protected const string PENDING_CONTEXT_KEY = '__database.transactions.pending'; - - protected const string CURRENT_CONTEXT_KEY = '__database.transactions.current'; - /** * Get all committed transactions for the current coroutine. * @@ -30,17 +24,7 @@ class DatabaseTransactionsManager */ protected function getCommittedTransactionsInternal(): Collection { - return CoroutineContext::get(self::COMMITTED_CONTEXT_KEY, new Collection); - } - - /** - * Set committed transactions for the current coroutine. - * - * @param Collection $transactions - */ - protected function setCommittedTransactions(Collection $transactions): void - { - CoroutineContext::set(self::COMMITTED_CONTEXT_KEY, $transactions); + return DatabaseTransactionState::current()->committed; } /** @@ -50,37 +34,7 @@ protected function setCommittedTransactions(Collection $transactions): void */ protected function getPendingTransactionsInternal(): Collection { - return CoroutineContext::get(self::PENDING_CONTEXT_KEY, new Collection); - } - - /** - * Set pending transactions for the current coroutine. - * - * @param Collection $transactions - */ - protected function setPendingTransactions(Collection $transactions): void - { - CoroutineContext::set(self::PENDING_CONTEXT_KEY, $transactions); - } - - /** - * Get current transaction map for the current coroutine. - * - * @return array - */ - protected function getCurrentTransaction(): array - { - return CoroutineContext::get(self::CURRENT_CONTEXT_KEY, []); - } - - /** - * Set current transaction for a connection. - */ - protected function setCurrentTransactionForConnection(string $connection, ?DatabaseTransactionRecord $transaction): void - { - $current = $this->getCurrentTransaction(); - $current[$connection] = $transaction; - CoroutineContext::set(self::CURRENT_CONTEXT_KEY, $current); + return DatabaseTransactionState::current()->pending; } /** @@ -88,7 +42,7 @@ protected function setCurrentTransactionForConnection(string $connection, ?Datab */ protected function getCurrentTransactionForConnection(string $connection): ?DatabaseTransactionRecord { - return $this->getCurrentTransaction()[$connection] ?? null; + return DatabaseTransactionState::current()->current[$connection] ?? null; } /** @@ -96,17 +50,16 @@ protected function getCurrentTransactionForConnection(string $connection): ?Data */ public function begin(string $connection, int $level): void { - $pending = $this->getPendingTransactionsInternal(); + $state = DatabaseTransactionState::current(); $newTransaction = new DatabaseTransactionRecord( $connection, $level, - $this->getCurrentTransactionForConnection($connection) + $state->current[$connection] ?? null ); - $pending->push($newTransaction); - $this->setPendingTransactions($pending); - $this->setCurrentTransactionForConnection($connection, $newTransaction); + $state->pending->push($newTransaction); + $state->current[$connection] = $newTransaction; } /** @@ -118,9 +71,10 @@ public function commit(string $connection, int $levelBeingCommitted, int $newTra { $this->stageTransactions($connection, $levelBeingCommitted); - $currentForConnection = $this->getCurrentTransactionForConnection($connection); + $state = DatabaseTransactionState::current(); + $currentForConnection = $state->current[$connection] ?? null; if ($currentForConnection !== null) { - $this->setCurrentTransactionForConnection($connection, $currentForConnection->parent); + $state->current[$connection] = $currentForConnection->parent; } if (! $this->afterCommitCallbacksShouldBeExecuted($newTransactionLevel) @@ -129,18 +83,15 @@ public function commit(string $connection, int $levelBeingCommitted, int $newTra } // Clear pending transactions for this connection at or above the committed level - $pending = $this->getPendingTransactionsInternal()->reject( + $state->pending = $state->pending->reject( fn ($transaction) => $transaction->connection === $connection && $transaction->level >= $levelBeingCommitted )->values(); - $this->setPendingTransactions($pending); - - $committed = $this->getCommittedTransactionsInternal(); - [$forThisConnection, $forOtherConnections] = $committed->partition( + [$forThisConnection, $forOtherConnections] = $state->committed->partition( fn ($transaction) => $transaction->connection === $connection ); - $this->setCommittedTransactions($forOtherConnections->values()); + $state->committed = $forOtherConnections->values(); $this->executeCommitCallbacks($forThisConnection); @@ -152,21 +103,18 @@ public function commit(string $connection, int $levelBeingCommitted, int $newTra */ public function stageTransactions(string $connection, int $levelBeingCommitted): void { - $pending = $this->getPendingTransactionsInternal(); - $committed = $this->getCommittedTransactionsInternal(); + $state = DatabaseTransactionState::current(); - $toStage = $pending->filter( + $toStage = $state->pending->filter( fn ($transaction) => $transaction->connection === $connection && $transaction->level >= $levelBeingCommitted ); - $this->setCommittedTransactions($committed->merge($toStage)); + $state->committed = $state->committed->merge($toStage); - $this->setPendingTransactions( - $pending->reject( - fn ($transaction) => $transaction->connection === $connection - && $transaction->level >= $levelBeingCommitted - ) + $state->pending = $state->pending->reject( + fn ($transaction) => $transaction->connection === $connection + && $transaction->level >= $levelBeingCommitted ); } @@ -181,15 +129,14 @@ public function rollback(string $connection, int $newTransactionLevel): void return; } - $this->setPendingTransactions( - $this->getPendingTransactionsInternal()->reject( - fn ($transaction) => $transaction->connection === $connection - && $transaction->level > $newTransactionLevel - )->values() - ); + $state = DatabaseTransactionState::current(); + $state->pending = $state->pending->reject( + fn ($transaction) => $transaction->connection === $connection + && $transaction->level > $newTransactionLevel + )->values(); $transactions = new Collection; - $currentForConnection = $this->getCurrentTransactionForConnection($connection); + $currentForConnection = $state->current[$connection] ?? null; while ($currentForConnection !== null && $currentForConnection->level > $newTransactionLevel) { @@ -200,14 +147,14 @@ public function rollback(string $connection, int $newTransactionLevel): void $currentForConnection = $currentForConnection->parent; } - $this->setCurrentTransactionForConnection($connection, $currentForConnection); + $state->current[$connection] = $currentForConnection; - [$stagedTransactions, $remainingCommitted] = $this->getCommittedTransactionsInternal()->partition( + [$stagedTransactions, $remainingCommitted] = $state->committed->partition( fn (DatabaseTransactionRecord $committed): bool => $transactions->contains( fn (DatabaseTransactionRecord $transaction): bool => $transaction === $committed ) ); - $this->setCommittedTransactions($remainingCommitted->values()); + $state->committed = $remainingCommitted->values(); $this->executeRollbackCallbacks( $transactions @@ -223,14 +170,13 @@ public function rollback(string $connection, int $newTransactionLevel): void */ protected function removeAllTransactionsForConnection(string $connection): void { - [$committedForConnection, $committedForOtherConnections] = $this - ->getCommittedTransactionsInternal() - ->partition( - fn (DatabaseTransactionRecord $transaction): bool => $transaction->connection === $connection - ); + $state = DatabaseTransactionState::current(); + [$committedForConnection, $committedForOtherConnections] = $state->committed->partition( + fn (DatabaseTransactionRecord $transaction): bool => $transaction->connection === $connection + ); $currentTransactions = new Collection; - $currentForConnection = $this->getCurrentTransactionForConnection($connection); + $currentForConnection = $state->current[$connection] ?? null; for ($current = $currentForConnection; $current !== null; $current = $current->parent) { $currentTransactions->push($current); @@ -243,15 +189,13 @@ protected function removeAllTransactionsForConnection(string $connection): void ->values(); // User callbacks must observe the transaction records as already detached. - $this->setCurrentTransactionForConnection($connection, null); + $state->current[$connection] = null; - $this->setPendingTransactions( - $this->getPendingTransactionsInternal()->reject( - fn ($transaction) => $transaction->connection === $connection - )->values() - ); + $state->pending = $state->pending->reject( + fn ($transaction) => $transaction->connection === $connection + )->values(); - $this->setCommittedTransactions($committedForOtherConnections->values()); + $state->committed = $committedForOtherConnections->values(); $this->executeRollbackCallbacks($transactions); } @@ -264,14 +208,14 @@ protected function removeAllTransactionsForConnection(string $connection): void protected function removeCommittedTransactionsThatAreChildrenOf( DatabaseTransactionRecord $transaction ): Collection { - $committed = $this->getCommittedTransactionsInternal(); + $state = DatabaseTransactionState::current(); - [$removedTransactions, $remaining] = $committed->partition( + [$removedTransactions, $remaining] = $state->committed->partition( fn ($committed) => $committed->connection === $transaction->connection && $committed->parent === $transaction ); - $this->setCommittedTransactions($remaining); + $state->committed = $remaining; foreach ($removedTransactions as $removedTransaction) { $removedTransactions = $removedTransactions->concat( @@ -435,9 +379,10 @@ public function getCommittedTransactions(): Collection */ public static function hasNonCoroutinePendingTransactions(): bool { - $pending = CoroutineContext::getFromNonCoroutine(self::PENDING_CONTEXT_KEY); + /** @var null|DatabaseTransactionState $state */ + $state = CoroutineContext::getFromNonCoroutine(DatabaseTransactionState::CONTEXT_KEY); - return $pending instanceof Collection && $pending->isNotEmpty(); + return $state !== null && $state->pending->isNotEmpty(); } /** @@ -448,11 +393,25 @@ public static function hasNonCoroutinePendingTransactions(): bool */ public static function copyToNonCoroutineState(): void { - CoroutineContext::copyToNonCoroutine([ - self::COMMITTED_CONTEXT_KEY, - self::PENDING_CONTEXT_KEY, - self::CURRENT_CONTEXT_KEY, - ]); + CoroutineContext::setNonCoroutine( + DatabaseTransactionState::CONTEXT_KEY, + DatabaseTransactionState::current(), + ); + } + + /** + * Restore the test lifecycle's transaction state in the current coroutine. + * + * Tests only. This deliberately shares transaction ownership across setup, + * test and teardown; normal child coroutines must own separate transactions. + */ + public static function copyFromNonCoroutineState(): void + { + $state = CoroutineContext::getFromNonCoroutine(DatabaseTransactionState::CONTEXT_KEY); + + if ($state !== null) { + CoroutineContext::set(DatabaseTransactionState::CONTEXT_KEY, $state); + } } /** @@ -463,10 +422,6 @@ public static function copyToNonCoroutineState(): void */ public static function clearNonCoroutineState(): void { - CoroutineContext::clearFromNonCoroutine([ - self::COMMITTED_CONTEXT_KEY, - self::PENDING_CONTEXT_KEY, - self::CURRENT_CONTEXT_KEY, - ]); + CoroutineContext::clearFromNonCoroutine([DatabaseTransactionState::CONTEXT_KEY]); } } diff --git a/src/database/src/PdoConnection.php b/src/database/src/PdoConnection.php index d9f601495a..5e1b575cba 100755 --- a/src/database/src/PdoConnection.php +++ b/src/database/src/PdoConnection.php @@ -36,6 +36,11 @@ class PdoConnection extends Connection */ protected PDO|Closure|null $readPdo = null; + /** + * The owner notified when the current driver resources are forgotten. + */ + protected ?Closure $resourceReleaser = null; + /** * The default fetch mode of the connection. */ @@ -137,52 +142,63 @@ public function selectResultSets(string $query, array $bindings = [], bool $useR */ public function cursor(string $query, array $bindings = [], bool $useReadPdo = true, array $fetchUsing = []): Generator { - $statement = $this->run($query, $bindings, function ($query, $bindings) use ($useReadPdo) { - if ($this->pretending()) { - return null; - } + ++$this->sessionPinDepth; + $statement = null; - // First we will create a statement for the query. Then, we will set the fetch - // mode and prepare the bindings for the query. Once that's done we will be - // ready to execute the query against the database and return the cursor. - $statement = $this->prepared($this->getPdoForSelect($useReadPdo) - ->prepare($query)); + try { + $statement = $this->run($query, $bindings, function ($query, $bindings) use ($useReadPdo) { + if ($this->pretending()) { + return null; + } - $this->bindValues( - $statement, - $this->prepareBindings($bindings) - ); + // First we will create a statement for the query. Then, we will set the fetch + // mode and prepare the bindings for the query. Once that's done we will be + // ready to execute the query against the database and return the cursor. + $statement = $this->prepared($this->getPdoForSelect($useReadPdo) + ->prepare($query)); - // Next, we'll execute the query against the database and return the statement - // so we can return the cursor. The cursor will use a PHP generator to give - // back one row at a time without using a bunch of memory to render them. - $statement->execute(); + $this->bindValues( + $statement, + $this->prepareBindings($bindings) + ); - return $statement; - }); + // Next, we'll execute the query against the database and return the statement + // so we can return the cursor. The cursor will use a PHP generator to give + // back one row at a time without using a bunch of memory to render them. + $statement->execute(); - if ($statement === null) { - return; - } + return $statement; + }); - if ($fetchUsing !== []) { - // fetchAll() supplies default column and class arguments that setFetchMode() - // demands explicitly, so a mode-only call keeps the same meaning when streamed. - if (count($fetchUsing) === 1) { - $mode = $fetchUsing[0] & ~(PDO::FETCH_GROUP | PDO::FETCH_UNIQUE | PDO::FETCH_CLASSTYPE | PDO::FETCH_PROPS_LATE); + if ($statement === null) { + return; + } + + if ($fetchUsing !== []) { + // fetchAll() supplies default column and class arguments that setFetchMode() + // demands explicitly, so a mode-only call keeps the same meaning when streamed. + if (count($fetchUsing) === 1) { + $mode = $fetchUsing[0] & ~(PDO::FETCH_GROUP | PDO::FETCH_UNIQUE | PDO::FETCH_CLASSTYPE | PDO::FETCH_PROPS_LATE); - if ($mode === PDO::FETCH_COLUMN) { - $fetchUsing[] = 0; - } elseif ($mode === PDO::FETCH_CLASS && ($fetchUsing[0] & PDO::FETCH_CLASSTYPE) === 0) { - $fetchUsing[] = stdClass::class; + if ($mode === PDO::FETCH_COLUMN) { + $fetchUsing[] = 0; + } elseif ($mode === PDO::FETCH_CLASS && ($fetchUsing[0] & PDO::FETCH_CLASSTYPE) === 0) { + $fetchUsing[] = stdClass::class; + } } - } - $statement->setFetchMode(...$fetchUsing); - } + $statement->setFetchMode(...$fetchUsing); + } - foreach ($statement as $record) { - yield $record; + foreach ($statement as $record) { + yield $record; + } + } finally { + try { + $statement?->closeCursor(); + } finally { + --$this->sessionPinDepth; + } } } @@ -420,6 +436,69 @@ protected function resolveReadPdo(): PDO return $this->resolvePdo(); } + /** + * Resolve a physical handle without applying caller-specific session state. + * + * @internal + */ + public function resolveRawPdo(bool $read = false): PDO + { + return $read ? $this->resolveReadPdo() : $this->resolvePdo(); + } + + /** + * Adopt resolved handles and endpoint metadata without changing caller state. + * + * @internal + */ + public function attachPdoResources(self $source): void + { + if ($source->pdo instanceof PDO) { + $this->pdo = $source->pdo; + } + + if ($source->readPdo instanceof PDO) { + $this->readPdo = $source->readPdo; + } + + $this->config = $source->config; + $this->readConnectionConfig = $source->readConnectionConfig; + } + + /** + * Restore lazy resolvers after returning the physical resources to their owner. + * + * @param Closure(): PDO $pdo + * @param null|Closure(): PDO $readPdo + * + * @internal + */ + public function detachPdoResources(Closure $pdo, ?Closure $readPdo): void + { + if ($this->foreignKeyConstraintSuppressionDepth > 0) { + $this->markCurrentSessionStateUnknown(); + } + + $this->resourceReleaser = null; + $this->pdo = $pdo; + $this->readPdo = $readPdo; + $this->lockForPopping = null; + $this->maxBindings = null; + $this->errorCount = 0; + } + + /** + * Set the resource owner's callback for forgotten driver resources. + * + * @param Closure(): void $callback + * + * @internal + */ + public function setResourceReleaser(Closure $callback): void + { + $this->resourceReleaser = $callback; + } + /** * Synchronize the desired state for a physical database session. */ @@ -676,6 +755,10 @@ public function setReadPdoConfig(array $config): static protected function forgetDriverResources(): void { $this->setPdo(null)->setReadPdo(null); + + $releaser = $this->resourceReleaser; + $this->resourceReleaser = null; + $releaser?->__invoke(); } /** @@ -712,11 +795,17 @@ protected function disconnectDriverResources(): void protected function replaceDriverResources(Connection $fresh): void { /** @var self $fresh */ - $fresh->getPdo(); - $fresh->getReadPdo(); + try { + $fresh->getPdo(); + $fresh->getReadPdo(); + + $pdo = $fresh->getRawPdo(); + $readPdo = $fresh->getRawReadPdo(); + } finally { + // Its grammar retains the temporary connection until cyclic GC runs. + $fresh->setPdo(null)->setReadPdo(null); + } - $pdo = $fresh->getRawPdo(); - $readPdo = $fresh->getRawReadPdo(); $database = $fresh->database; $configuredDatabase = $fresh->configuredDatabase; $tablePrefix = $fresh->tablePrefix; diff --git a/src/database/src/Pool/ConnectionLease.php b/src/database/src/Pool/ConnectionLease.php new file mode 100644 index 0000000000..690b854369 --- /dev/null +++ b/src/database/src/Pool/ConnectionLease.php @@ -0,0 +1,160 @@ +pooledConnection = $pooledConnection; + $this->pdoResolver = fn (): PDO => $this->resolvePdo(); + $this->readPdoResolver = $physicalConnection->getRawReadPdo() !== null + ? fn (): PDO => $this->resolvePdo(read: true) + : null; + $this->connection = $factory->makeWithPdoResolvers( + $physicalConnection->getConfig(), + $this->pdoResolver, + $this->readPdoResolver, + ); + $this->connection->attachPdoResources($physicalConnection); + $this->connection->setReconnector($this->reconnect(...)); + $pooledConnection->attachLease($this); + } + + /** + * Resolve a physical handle and publish its generation to the logical owner. + */ + protected function resolvePdo(bool $read = false): PDO + { + if ($this->pooledConnection === null) { + /** @var PooledConnection $pooledConnection */ + $pooledConnection = $this->pool->borrow(); + $this->pooledConnection = $pooledConnection; + $pooledConnection->attachLease($this); + } + + $pooledConnection = $this->pooledConnection; + + try { + /** @var PdoConnection $physicalConnection */ + $physicalConnection = $pooledConnection->getDriverConnection(); + $pdo = $physicalConnection->resolveRawPdo($read); + + // A PDO-first extension may resolve its constructor argument eagerly. + if (isset($this->connection)) { // @phpstan-ignore isset.initializedProperty (PDO resolvers may run during construction.) + $this->connection->attachPdoResources($physicalConnection); + $pooledConnection->dispatchConnectionEstablishedEvent(); + + // A listener may have reconnected and replaced the published handle. + $pdo = $this->connection->resolveRawPdo($read); + } + + return $pdo; + } catch (Throwable $exception) { + if (isset($this->connection)) { // @phpstan-ignore isset.initializedProperty (PDO resolvers may run during construction.) + $this->discardAfterFailure($exception); + } + + throw $exception; + } + } + + /** + * Replace the held physical session and reconnect the logical owner. + */ + public function reconnect(): PdoConnection + { + $this->connection->disconnect(); + + if ($this->pooledConnection !== null) { + $this->pooledConnection->reconnect(); + $this->pooledConnection->attachLease($this); + } + + $this->connection->setPdo($this->pdoResolver)->setReadPdo($this->readPdoResolver); + $this->connection->getPdo(); + + return $this->connection; + } + + /** + * Release the held session when no operation requires it. + */ + public function releaseIfIdle(): void + { + if (! $this->connection->hasPinnedSession()) { + $this->release(); + } + } + + /** + * Settle the currently held physical session. + */ + public function release(): void + { + $this->pooledConnection?->release(); + } + + /** + * Discard the currently held physical session. + */ + public function discard(): void + { + $this->pooledConnection?->discard(); + } + + /** + * Remove references to a session before its slot becomes available again. + */ + public function detach(): void + { + $this->pooledConnection = null; + $this->connection->detachPdoResources($this->pdoResolver, $this->readPdoResolver); + } + + /** + * Discard a failed acquisition without replacing its exception. + */ + protected function discardAfterFailure(Throwable $exception): void + { + try { + $this->discard(); + } catch (CanceledException $cancellation) { + if (! $exception instanceof CanceledException) { + throw $cancellation; + } + } catch (Throwable) { + // Preserve the acquisition or connection-listener failure. + } + } +} diff --git a/src/database/src/Pool/DatabasePool.php b/src/database/src/Pool/DatabasePool.php index 2b57953dbf..86216b1254 100644 --- a/src/database/src/Pool/DatabasePool.php +++ b/src/database/src/Pool/DatabasePool.php @@ -6,6 +6,7 @@ use Hypervel\ConnectionPool\BorrowRateTracker; use Hypervel\ConnectionPool\ConnectionPool; +use Hypervel\ConnectionPool\PoolOptions; use Hypervel\Contracts\ConnectionPool\Connection as PoolConnection; use Hypervel\Contracts\ConnectionPool\UsageTracker; use Hypervel\Contracts\Container\Container; @@ -31,6 +32,8 @@ class DatabasePool extends ConnectionPool { protected array $config; + protected bool $usesSessionLeases; + protected ?Timer $heartbeatTimer = null; protected ?int $heartbeatTimerId = null; @@ -64,19 +67,13 @@ public function __construct(Container $container, string $name) $poolConfig = $config; if ($connectionName->isRead() && $factory->hasReadConfig($config)) { - $poolConfig = $factory->configForRead($config); - $this->ensureNotDerivedInMemorySqlitePool($connectionName, $poolConfig); - - if ($factory->getExtension($config, $connectionName->base) === null) { - $config = $poolConfig; - } else { - // Extensions own endpoint selection, but the pool still uses - // the selected read record's pool options and SQLite metadata. - $config[Connection::READ_WRITE_TYPE_CONFIG_KEY] = ConnectionName::READ; - } + $poolConfig = $this->readPoolConfig($factory, $connectionName, $config); + $config[Connection::READ_WRITE_TYPE_CONFIG_KEY] = ConnectionName::READ; } $this->config = $config; + $this->usesSessionLeases = $factory->getExtension($config, $connectionName->base) === null + && $this->hasConsistentLogicalIdentity($factory, $connectionName, $config); $poolOptions = Arr::except( Arr::get($poolConfig, 'pool', []), @@ -125,6 +122,44 @@ public function getSharedInMemorySqlitePdo(): ?PDO return $this->sharedInMemorySqlitePdo; } + /** + * Determine whether callers can return physical sessions before execution ends. + */ + public function usesSessionLeases(): bool + { + return $this->usesSessionLeases; + } + + /** + * Determine whether selectable endpoints share the same logical database identity. + */ + protected function hasConsistentLogicalIdentity(ConnectionFactory $factory, ConnectionName $name, array $config): bool + { + $role = $name->isRead() && $factory->hasReadConfig($config) ? 'read' : 'write'; + + if (! isset($config[$role][0])) { + return true; + } + + $identity = null; + + foreach ($config[$role] as $record) { + $candidate = array_replace($config, [$role => $record]); + $endpoint = $role === 'read' + ? $factory->configForRead($candidate) + : $factory->configForWrite($candidate); + $candidateIdentity = [$endpoint['driver'], $endpoint['database'], $endpoint['prefix']]; + + if ($identity !== null && $identity !== $candidateIdentity) { + return false; + } + + $identity = $candidateIdentity; + } + + return true; + } + /** * Create a new pooled connection. */ @@ -152,7 +187,12 @@ protected function createSharedInMemorySqlitePdo(): PDO $factory = $this->container->make('db.factory'); $connection = $factory->makeSharedInMemorySqliteConnection($this->config, $this->name); - return $connection->getPdo(); + try { + return $connection->resolveRawPdo(); + } finally { + // Only the pool should retain this PDO after its bootstrap holder is discarded. + $connection->setPdo(null)->setReadPdo(null); + } } /** @@ -177,6 +217,33 @@ protected function isInMemorySqlite(array $config): bool return SQLiteDatabase::isInMemory($database); } + /** + * Resolve consistent pool settings without choosing a physical read endpoint. + */ + protected function readPoolConfig(ConnectionFactory $factory, ConnectionName $name, array $config): array + { + $records = isset($config['read'][0]) ? $config['read'] : [$config['read']]; + $poolConfig = null; + $poolOptions = null; + + foreach ($records as $record) { + $readConfig = $factory->configForRead(array_replace($config, ['read' => $record])); + $this->ensureNotDerivedInMemorySqlitePool($name, $readConfig); + $options = PoolOptions::fromArray(Arr::except($readConfig['pool'] ?? [], ['testing_enabled'])); + + if ($poolOptions !== null && $poolOptions != $options) { + throw new InvalidArgumentException( + "Read records for database connection [{$name->requested}] must use the same effective pool options." + ); + } + + $poolConfig ??= $readConfig; + $poolOptions ??= $options; + } + + return $poolConfig; + } + /** * Ensure a derived read pool does not point at an in-memory SQLite database. */ diff --git a/src/database/src/Pool/PooledConnection.php b/src/database/src/Pool/PooledConnection.php index 52278153d9..8ac998e2d4 100644 --- a/src/database/src/Pool/PooledConnection.php +++ b/src/database/src/Pool/PooledConnection.php @@ -12,8 +12,10 @@ use Hypervel\Contracts\Log\StdoutLoggerInterface; use Hypervel\Coroutine\Coroutine as FrameworkCoroutine; use Hypervel\Database\Connection; +use Hypervel\Database\ConnectionName; use Hypervel\Database\Connectors\ConnectionFactory; use Hypervel\Database\Events\ConnectionEstablished; +use Hypervel\Database\PdoConnection; use Hypervel\Engine\Channel; use Hypervel\Engine\Coroutine; use Hypervel\Engine\Exceptions\CoroutineCreateException; @@ -37,6 +39,10 @@ class PooledConnection implements PoolConnection protected ?Connection $connection = null; + protected ?ConnectionLease $lease = null; + + protected bool $driverConnectionConfigured = false; + protected ConnectionFactory $factory; protected LoggerInterface $logger; @@ -81,6 +87,28 @@ public function getConnection(): Connection * Get the active connection, reconnecting if necessary. */ public function getActiveConnection(): Connection + { + if ($this->lease !== null) { + return $this->lease->connection; + } + + $connection = $this->getDriverConnection(); + + if (! $this->driverConnectionConfigured) { + $this->configureConnection($connection); + $connection->setReconnector($this->refresh(...)); + $this->driverConnectionConfigured = true; + } + + return $connection; + } + + /** + * Get the internal driver resource holder for the current generation. + * + * @internal + */ + public function getDriverConnection(): Connection { if ($this->check()) { $this->availableForReuse = false; @@ -95,26 +123,59 @@ public function getActiveConnection(): Connection return $this->connection; } + /** + * Create a caller-owned logical connection for this borrowed slot. + * + * @internal + */ + public function newLease(): ConnectionLease + { + /** @var PdoConnection $connection */ + $connection = $this->getDriverConnection(); + $lease = new ConnectionLease($this->pool, $this, $this->factory, $connection); + $this->configureConnection($lease->connection); + + return $lease; + } + + /** + * Attach a logical owner without retaining its state between borrowers. + * + * @internal + */ + public function attachLease(ConnectionLease $lease): void + { + $this->lease = $lease; + $this->connection?->unsetEventDispatcher(); + $this->connection?->unsetTransactionManager(); + $this->driverConnectionConfigured = false; + $lease->connection->setResourceReleaser($this->forgetDriverConnection(...)); + } + + /** + * Forget physical resources already disconnected by their logical owner. + */ + protected function forgetDriverConnection(): void + { + /** @var null|PdoConnection $connection */ + $connection = $this->connection; + + // The connection's grammar retains it until cyclic garbage collection. + $connection?->setPdo(null)->setReadPdo(null); + $this->connection = null; + $this->connectionEstablishedEventPending = false; + $this->markInvalid(); + } + /** * Reconnect to the database. */ public function reconnect(): bool { - $this->close(); + $this->closeDriverConnection(); $sharedPdo = $this->pool->getSharedInMemorySqlitePdo(); - - if ($sharedPdo !== null) { - // In-memory SQLite: use shared PDO so all pool slots see same data - $this->connection = $this->factory->makeSqliteFromSharedPdo( - $sharedPdo, - $this->config, - $this->config['name'] ?? null - ); - } else { - // Normal path: factory creates a fresh connection with new driver resources. - $this->connection = $this->factory->make($this->config, $this->config['name'] ?? null); - } + $this->connection = $this->makeConnection(); if (! $this->connection->isReusable()) { $this->markInvalid(); @@ -128,21 +189,12 @@ public function reconnect(): bool throw new RuntimeException('Database connection is not reusable after reconnecting.'); } - // Configure event dispatcher for query events - if ($this->container->bound('events')) { - $this->connection->setEventDispatcher($this->container->make('events')); - } - - // Configure transaction manager for after-commit callbacks - if ($this->container->has('db.transactions')) { - $this->connection->setTransactionManager($this->container->make('db.transactions')); + if (! $this->pool->usesSessionLeases()) { + $this->configureConnection($this->connection); + $this->connection->setReconnector($this->refresh(...)); + $this->driverConnectionConfigured = true; } - // Set up reconnector for the connection - $this->connection->setReconnector(function ($connection) { - $this->refresh($connection); - }); - $now = hrtime(true) / 1e9; $this->lastUseTime = $now; $this->stampGeneration($now); @@ -153,6 +205,20 @@ public function reconnect(): bool return true; } + /** + * Configure services used by the caller-visible connection. + */ + protected function configureConnection(Connection $connection): void + { + if ($this->container->bound('events')) { + $connection->setEventDispatcher($this->container->make('events')); + } + + if ($this->container->has('db.transactions')) { + $connection->setTransactionManager($this->container->make('db.transactions')); + } + } + /** * Notify listeners after the resolver has registered the connection. * @@ -173,7 +239,10 @@ public function dispatchConnectionEstablishedEvent(): void $events = $this->container->make('events'); if ($events->hasListeners(ConnectionEstablished::class)) { - $events->dispatch(new ConnectionEstablished($this->connection)); + $connection = $this->lease->connection ?? $this->connection; + $connection->withPinnedSession( + static fn () => $events->dispatch(new ConnectionEstablished($connection)) + ); } } } @@ -302,8 +371,30 @@ private function cancelHeartbeatCoroutine(?int $coroutineId): void * Close the database connection. */ public function close(): bool + { + try { + if ($this->lease !== null) { + $this->lease->connection->disconnect(); + } else { + $this->closeDriverConnection(); + } + } finally { + $this->lease?->detach(); + $this->lease = null; + $this->connection = null; + $this->connectionEstablishedEventPending = false; + } + + return true; + } + + /** + * Close the internal resource holder without settling its borrowed slot. + */ + protected function closeDriverConnection(): void { $this->connectionEstablishedEventPending = false; + $this->driverConnectionConfigured = false; if ($this->connection instanceof Connection) { try { @@ -314,8 +405,47 @@ public function close(): bool $this->connection = null; } } + } - return true; + /** + * Clean up the owning connection and notify listeners before detachment. + */ + protected function prepareForRelease(): void + { + $connection = $this->lease->connection ?? $this->connection; + + if ($connection !== null) { + $errorCount = $connection->getErrorCount(); + + if ($this->lease === null) { + $connection->resetForPool(); + } + + if ($errorCount > self::MAX_ERROR_COUNT) { + $this->logger->warning('Connection has too many errors, marking as stale.'); + $this->markInvalid(); + } + + if ($connection->transactionLevel() > 0) { + $connection->rollBack(0); + $this->logger->error('Database transaction was not committed or rolled back before release.'); + } + } + + $this->lastReleaseTime = hrtime(true) / 1e9; + $events = $this->pool->getOptions()->events; + + if (in_array(ConnectionReleasing::class, $events, true) + && $this->container->bound('events') + ) { + // Event::fake() can replace the dispatcher after this connection was created. + /** @var Dispatcher $dispatcher */ + $dispatcher = $this->container->make('events'); + + if ($dispatcher->hasListeners(ConnectionReleasing::class)) { + $dispatcher->dispatch(new ConnectionReleasing($this)); + } + } } /** @@ -327,39 +457,10 @@ public function release(): void $ordinaryFailure = null; try { - if ($this->connection instanceof Connection) { - $errorCount = $this->connection->getErrorCount(); - - // Reset wrapper state before another coroutine borrows it. - $this->connection->resetForPool(); - - // Check error count and mark as stale if too high - if ($errorCount > self::MAX_ERROR_COUNT) { - $this->logger->warning('Connection has too many errors, marking as stale.'); - $this->markInvalid(); - } - - // Roll back any uncommitted transactions (including nested savepoints) - if ($this->connection->transactionLevel() > 0) { - $this->connection->rollBack(0); - $this->logger->error('Database transaction was not committed or rolled back before release.'); - } - } - - $this->lastReleaseTime = hrtime(true) / 1e9; - - // Dispatch release event if configured - $events = $this->pool->getOptions()->events; - if (in_array(ConnectionReleasing::class, $events, true) - && $this->container->bound('events') - ) { - // Event::fake() can replace the dispatcher after this connection was created. - /** @var Dispatcher $dispatcher */ - $dispatcher = $this->container->make('events'); - - if ($dispatcher->hasListeners(ConnectionReleasing::class)) { - $dispatcher->dispatch(new ConnectionReleasing($this)); - } + if ($this->lease !== null) { + $this->lease->connection->withPinnedSession($this->prepareForRelease(...)); + } else { + $this->prepareForRelease(); } } catch (CanceledException $cancellation) { $cancellationFailure = $cancellation; @@ -374,6 +475,9 @@ public function release(): void } catch (Throwable $loggingException) { $ordinaryFailure = $loggingException; } + } finally { + $this->lease?->detach(); + $this->lease = null; } try { @@ -488,26 +592,41 @@ private function stampGeneration(float $now): void } /** - * Refresh the database connection resources. + * Create driver resources for a new physical connection generation. */ - protected function refresh(Connection $connection): void + protected function makeConnection(): Connection { $sharedPdo = $this->pool->getSharedInMemorySqlitePdo(); - try { - if ($sharedPdo !== null) { - // For shared in-memory SQLite, rebind to the same PDO. - // Creating a fresh PDO would give us a new empty database. - $fresh = $this->factory->makeSqliteFromSharedPdo( - $sharedPdo, - $this->config, - $this->config['name'] ?? null - ); - } else { - $fresh = $this->factory->make($this->config, $this->config['name'] ?? null); - } + if ($sharedPdo !== null) { + // Creating a fresh PDO would discard the shared in-memory database. + return $this->factory->makeSqliteFromSharedPdo( + $sharedPdo, + $this->config, + $this->config['name'] ?? null + ); + } - $connection->refreshFrom($fresh); + $config = $this->config; + $name = $config['name'] ?? null; + + if (($config[Connection::READ_WRITE_TYPE_CONFIG_KEY] ?? null) === ConnectionName::READ + && $this->factory->hasReadConfig($config) + && $this->factory->getExtension($config, $name) === null + ) { + $config = $this->factory->configForRead($config); + } + + return $this->factory->make($config, $name); + } + + /** + * Refresh the database connection resources. + */ + protected function refresh(Connection $connection): void + { + try { + $connection->refreshFrom($this->makeConnection()); } catch (Throwable $exception) { $this->markInvalid(); diff --git a/src/docs/database.md b/src/docs/database.md index 92fbd8dd62..91592afd7c 100644 --- a/src/docs/database.md +++ b/src/docs/database.md @@ -9,6 +9,7 @@ - [PostgreSQL Server Options](#postgresql-server-options) - [Read and Write Connections](#read-and-write-connections) - [Connection Pooling](#connection-pooling) + - [Releasing and Pinning Connections](#releasing-and-pinning-connections) - [Configuring Database Session State](#configuring-database-session-state) - [Extending Database Connections](#extending-database-connections) - [Static Analysis](#static-analysis) @@ -227,12 +228,14 @@ Eloquent models created or retrieved through `::write` retain that connection fo #### The `sticky` Option -The `sticky` option is an *optional* value that can be used to allow the immediate reading of records that have been written to the database during the current request cycle. If the `sticky` option is enabled and a "write" operation has been performed against the database during the current request cycle, any further "read" operations will use the "write" connection. This ensures that any data written during the request cycle can be immediately read back from the database during that same request. In Hypervel, sticky state is reset when the coroutine's connection is returned to the pool, so it will not leak into another request. It is up to you to decide if this is the desired behavior for your application. +The `sticky` option is an *optional* value that can be used to allow the immediate reading of records that have been written to the database during the current request cycle. If the `sticky` option is enabled and a "write" operation has been performed against the database during the current request cycle, any further "read" operations will use the "write" connection. This ensures that any data written during the request cycle can be immediately read back from the database during that same request. In Hypervel, sticky state belongs to the coroutine's connection and survives an early release of its physical session. Other coroutines have independent routing state. It is up to you to decide if this is the desired behavior for your application. ### Connection Pooling -Hypervel uses connection pools to keep database access efficient within long-lived Swoole workers. When a coroutine needs a database connection, Hypervel borrows a connection from the worker's pool, stores it for the current coroutine, and returns it to the pool when the coroutine ends. Before a connection is returned to the pool, Hypervel resets per-request state such as query logs, query duration tracking, transaction callbacks, read / write routing state, and uncommitted transactions. +Hypervel uses connection pools to keep database access efficient within long-lived Swoole workers. When a coroutine resolves a database connection, Hypervel borrows a physical session from the worker's pool. The coroutine keeps its own connection object, including query logs, callbacks and read / write routing state. Query and schema builders retain that object even if its idle session is returned to the pool and another session is borrowed later. When the coroutine ends, Hypervel rolls back unfinished transactions and returns its remaining borrowed sessions. + +A connection and the query builders created from it belong to the coroutine that resolved them. Build queries inside each child coroutine instead of passing builders or connections between coroutines, and do not keep them after their coroutine ends. Each connection may define its own `pool` configuration: @@ -270,6 +273,8 @@ For a connection with separate read and write hosts, each base pool slot may laz When a read side is configured, explicit `::read` connections use a separate read-side pool built from the merged read configuration, including the base `pool` settings unless the read configuration overrides them. Drivers registered through `DB::extend` still receive the complete connection configuration so they can select their own endpoints; the pool's options come from the merged read configuration. Without a read side, `::read` uses the base pool. Explicit `::write` connections do not create a separate pool, but a coroutine that uses both `mysql` and `mysql::write` at the same time may borrow two slots from the base pool. Most applications do not need these suffixes in normal query paths because Hypervel already routes reads, writes, transactions, and sticky reads automatically. +If your `read` configuration contains a list of connection records, Hypervel chooses a record for each new physical connection and chooses again when reconnecting. All records in an explicit `::read` pool must have the same effective `pool` options, including inherited defaults. Conflicting options throw an exception when the pool is created, since the records share one pool capacity and lifecycle policy. Derived read pools cannot use in-memory SQLite databases. + Heartbeat and max lifetime recycling apply to Hypervel's worker pool whether the connection points directly at the database or through a proxy / pooler. They help long-running workers avoid stale sockets and rotate old idle connection generations before those connections are used by a request. Hypervel's default database configuration also includes a `pgsql-pooled` connection. This connection is intended for PostgreSQL transaction poolers such as PgBouncer and uses separate `DB_POOLED_*` environment variables. It also sets `migrations_connection` to `pgsql`, allowing your application to use the pooled connection at runtime while migration commands use the direct PostgreSQL connection. @@ -278,6 +283,57 @@ You may use `migrations_connection` on any database connection to instruct migra When you need a direct connection for migrations or schema operations, configure it as a normal connection and reference it with `migrations_connection`. Hypervel does not use Laravel's `::direct` connection suffix. + +#### Releasing and Pinning Connections + +A request may spend much longer waiting for an external service than running database queries. Holding a database session throughout that wait prevents other requests from using it, even though the database has no work to do. + +Hypervel's [HTTP client](/docs/{{version}}/http-client) automatically returns the current execution's idle database sessions to their pools before sending an outgoing request, including retries and redirects. This happens after request callbacks, so connections used by those callbacks can also be released. The next database operation borrows a session automatically. Existing connection objects, builders, query logs and sticky reads remain valid. No database connection is opened merely to release it. + +Registered [session configurators](#configuring-database-session-state) apply the current execution's settings before a borrowed PDO is used, updating the physical session only when its desired state changes. Ordinary database queries need no changes to take advantage of early release. + +Before other external work, you may release idle sessions explicitly using `DB::releaseIdleConnections()`. This is also useful before starting concurrent work: each child coroutine owns its own connections and cannot release a session held by its parent. + +```php +use Hypervel\Support\Facades\DB; +use Hypervel\Support\Facades\Http; + +use function Hypervel\Coroutine\parallel; + +$order = DB::table('orders')->find($orderId); + +DB::releaseIdleConnections(); + +$responses = parallel([ + fn () => Http::get($inventoryUrl), + fn () => Http::get($shippingUrl), +]); +``` + +Automatic release occurs before sending a request, not on every read of a streamed response. If your stream consumer performs database work between chunks, you may call `releaseIdleConnections()` before waiting for more data. Faked HTTP requests do not release connections. If you provide your own Guzzle client using `setClient`, call `DB::releaseIdleConnections()` yourself before sending requests. + +Active queries, transactions, open cursors and `Schema::withoutForeignKeyConstraints` callbacks retain their sessions automatically. In particular, an HTTP call inside `DB::transaction()` keeps the transaction's connection. Completed `chunk` or `lazy` query batches may release between batches. + +If your code requires the same physical session across an external call, wrap the entire operation in `DB::withPinnedSession()`. For a named connection, call `withPinnedSession()` on that connection. For example, a PostgreSQL session-level advisory lock must be released on the session that acquired it: + +```php +$connection = DB::connection('pgsql'); + +$response = $connection->withPinnedSession(function () use ($connection, $accountId, $paymentUrl, $payload) { + $connection->selectFromWriteConnection('select pg_advisory_lock(?)', [$accountId]); + + try { + return Http::post($paymentUrl, $payload); + } finally { + $connection->selectFromWriteConnection('select pg_advisory_unlock(?)', [$accountId]); + } +}); +``` + +Pinning is also needed for temporary tables, retained raw PDOs or statements, and manual session changes spanning a release boundary. It applies to a manual `disableForeignKeyConstraints` / `enableForeignKeyConstraints` pair; prefer the scoped `withoutForeignKeyConstraints` method. Each pin protects its connection until the callback returns or throws, and pins may be nested. Complete any lazy work inside the callback rather than returning it for later execution. Pinning prevents early release but does not prevent replacing a broken connection during the normal lost-connection retry. + +Connections registered through `DB::extend` retain their complete driver object until execution ends; early release leaves them alone. The same applies when selectable write records (or read records in an explicit `::read` pool) specify different database names or table prefixes. PDO drivers registered through `Connection::resolverFor` participate in early release when those values agree. + ### Configuring Database Session State @@ -380,6 +436,8 @@ Connection::resolverFor('mysql', function ( }); ``` +For a custom PDO driver name, also bind `db.connector.{driver}` in your service provider. The connector must implement `Hypervel\Database\Connectors\ConnectorInterface`; its `connect` method receives the configuration and returns a PDO instance. Existing driver names use Hypervel's built-in connectors unless you replace their binding. + If your driver uses an HTTP client, native extension, or another non-PDO transport, extend the driver-neutral `Connection` class and register it during application boot using the `DB::extend` method: ```php diff --git a/src/docs/filesystem.md b/src/docs/filesystem.md index e3082f9303..26265654ce 100644 --- a/src/docs/filesystem.md +++ b/src/docs/filesystem.md @@ -453,7 +453,7 @@ If the file you are retrieving contains JSON, you may use the `json` method to r $orders = Storage::json('orders.json'); ``` -The `exists` method may be used to determine if a file exists on the disk: +The `exists` method may be used to determine if a file or directory exists on the disk: ```php if (Storage::disk('s3')->exists('file.jpg')) { @@ -461,7 +461,7 @@ if (Storage::disk('s3')->exists('file.jpg')) { } ``` -The `missing` method may be used to determine if a file is missing from the disk: +The `missing` method may be used to determine if a file or directory is missing from the disk: ```php if (Storage::disk('s3')->missing('file.jpg')) { @@ -481,6 +481,8 @@ if (Storage::disk('s3')->directoryMissing('photos')) { } ``` +The `Hypervel\Contracts\Filesystem\Filesystem` contract includes `exists`, `fileExists`, and `directoryExists`, so these checks are available when a disk is injected through the contract. Custom implementations must provide all three methods. + ### Images diff --git a/src/docs/http-client.md b/src/docs/http-client.md index fc5df54469..55c81d161c 100644 --- a/src/docs/http-client.md +++ b/src/docs/http-client.md @@ -41,6 +41,8 @@ Hypervel provides an expressive, minimal API around the [Guzzle HTTP client](htt The HTTP client supports Guzzle 7 and 8. +Before sending a real request, Hypervel returns the current execution's idle database sessions to their pools. Transactions and explicitly pinned sessions remain held. Code that needs the same database session across an HTTP call may use `DB::withPinnedSession()`. See [releasing and pinning database connections](/docs/{{version}}/database#releasing-and-pinning-connections). + ## Making Requests @@ -177,11 +179,17 @@ The `jsonLines` method skips blank lines and throws a `JsonException` if a recor For plain text or a custom JSON decoder, you may use the `lines` method instead. It removes LF and CRLF line endings, preserves empty lines, and includes the final line even when it has no newline. For binary data and other formats, you may read the underlying PSR-7 response body directly. -Both methods continue reading from the body's current position. They do not rewind it. If you call `body` or `json` first, you must rewind the stream before reading its lines. Memory usage grows with the longest line, not the total response size. +Both methods continue reading from the body's current position. They do not rewind it. Streaming bodies cannot be rewound, so choose between processing their lines and reading the entire body with `body` or `json`. Memory usage grows with the longest line, not the total response size. + +Inside a Swoole coroutine with native cURL hooks enabled, the HTTP client receives chunks as they arrive, buffers a limited amount, and pauses network reads when your application falls behind. [Named connections](#connections) reuse idle connections between requests while keeping request headers, credentials, and cookies separate. Each active stream owns its transport; concurrent streams do not multiplex onto the same connection. + +For streaming requests, `timeout` bounds the wait for response headers. After headers arrive, a long-running stream may continue beyond that timeout. The `read_timeout` option limits idle gaps during headers and body reads, defaults to 60 seconds, and accepts `0` to wait without an idle limit. `connect_timeout` continues to limit connection establishment. Closing a response or canceling its consuming coroutine releases its active transfer. -The default streaming handler requires PHP's `allow_url_fopen` setting. If this setting is disabled, real streaming requests throw a `RuntimeException`; faked requests are unaffected. Custom handlers and clients are responsible for providing their own streaming support. +If cURL negotiates authentication for a streamed request, such as digest authentication with Guzzle 7 or the `CURLOPT_HTTPAUTH` and `CURLOPT_PROXYAUTH` cURL options, the response is returned when its first body bytes arrive or the transfer completes. The `timeout` option also covers that wait. -Unlike buffered requests, the default streaming handler does not use shared cURL connections or multiplexing. Streaming requests fail if their connection options require either feature. +For streamed responses, `on_stats` runs when the response is returned. The `on_trailers` callback runs once the transfer has completed, after `on_headers`. Small responses may already be complete when returned; for longer responses, trailers arrive as the body is consumed. + +Outside a hooked coroutine, or when you provide `stream_context` or a custom `stream_factory`, the client uses Guzzle's PHP-stream handler. This fallback requires `allow_url_fopen`, cannot use cURL destination pins or required transport sharing, and retains Guzzle's timeout behavior. In Guzzle 7, an omitted `read_timeout` uses the stream context or PHP socket timeout. Custom handlers and clients remain responsible for their own streaming support; faked responses do not need a transport. ### Request Data @@ -781,6 +789,8 @@ return $responses[0]->ok() && $responses[2]->ok(); ``` +If the parent coroutine has already used the database, call `DB::releaseIdleConnections()` before `parallel()` to make its idle sessions available while it waits. The HTTP requests run in child coroutines and cannot release the parent's connections. Transactions and pinned sessions still remain held; see [database connection pooling](/docs/{{version}}/database#releasing-and-pinning-connections). + Each response can be accessed based on the order it was added to the array. If you wish, you can name the requests using array keys, which allows you to access the corresponding responses by name: ```php @@ -955,6 +965,8 @@ defer(function () { Hypervel's HTTP client supports named connection presets for services your application calls frequently. Synchronous requests on a registered connection share one low-level Guzzle transport handler, which retains reusable cURL handles and keep-alive connection state. Every pending request still receives a fresh Guzzle client and middleware stack, so request-specific middleware, callbacks, and options never become frozen onto the first request. Asynchronous requests use isolated handlers because a worker-lived cURL multi-handler cannot be driven safely by concurrent coroutines. +For streamed responses, each named connection keeps up to 32 idle connections per worker for reuse. Connections are opened as needed; this does not limit how many requests can run at once. + To register a connection, typically in the `boot` method of your application's `AppServiceProvider`, call the `registerConnection` method: ```php @@ -1063,7 +1075,7 @@ If the proxy cannot be resolved or refuses the connection, a `Hypervel\Http\Clie ### Limitations -Pinning relies on Guzzle's cURL handler and libcurl 7.75 or newer. For this reason, restricted requests may not use the `stream` option, set their own `proxy` or raw `curl` options, or use a custom handler from the `setHandler` method; such requests throw a `DisallowedDestinationException`. To write a large response to a file, use the `sink` method instead of streaming it. +Pinning relies on Guzzle's cURL transport and libcurl 7.75 or newer. Streaming is supported inside a Swoole coroutine with native cURL hooks enabled. Restricted requests may not use the PHP-stream fallback, set their own `proxy` or raw `curl` options, or use a custom handler from the `setHandler` method; such requests throw a `DisallowedDestinationException`. To write a large response directly to a file, use the `sink` method. If the installed cURL transport lacks a required capability, a `Hypervel\Http\Client\Destinations\DestinationPolicyException` is thrown. These failures are never retried, and your retry callback is not called, because another attempt cannot change the installed transport. diff --git a/src/docs/json-schema.md b/src/docs/json-schema.md index aee7d3b229..61e6e89543 100644 --- a/src/docs/json-schema.md +++ b/src/docs/json-schema.md @@ -79,6 +79,12 @@ $schema = JsonSchema::object([ ])->withoutAdditionalProperties(); ``` +To inspect or reuse an object's property schemas, call `getProperties`. This returns the named `Type` instances rather than their serialized JSON Schema arrays: + +```php +$properties = $schema->getProperties(); +``` + ### Array Schemas diff --git a/src/docs/porting-from-laravel.md b/src/docs/porting-from-laravel.md index a373c18a02..9c16f76b88 100644 --- a/src/docs/porting-from-laravel.md +++ b/src/docs/porting-from-laravel.md @@ -652,6 +652,8 @@ Hypervel's `Str::orderedUuid()` returns a UUIDv7, while Laravel returns a timest Hypervel's `Filesystem::hash()` method uses `xxh128` by default. Pass `md5` explicitly when a port requires Laravel-compatible digests. +Custom filesystem contract implementations must also provide `fileExists()` and `directoryExists()`. The existing `exists()` method continues to accept either a file or a directory. See [retrieving files](/docs/{{version}}/filesystem#retrieving-files). + Unlike Laravel, Hypervel honors `read-only` on scoped disk records. Remove that option from any scoped disk that must accept writes. Rename any configured disk called `ondemand`; Hypervel reserves that name for [on-demand disk fakes](/docs/{{version}}/filesystem#on-demand-disks). @@ -677,6 +679,8 @@ SQLite JSON-path updates replace assigned objects and retain JSON null. Review a Database connections are persistent, pooled worker resources. Define every connection in `config/database.php` before the application boots. Dynamic connection creation through `DB::build()` and `DB::connectUsing()` is not supported. Review pool sizing and any database session state against the [database documentation](/docs/{{version}}/database#connection-pooling). +Outgoing framework HTTP requests release idle database sessions automatically. Wrap code that depends on the same session across an HTTP call, such as temporary tables, session locks or retained raw PDOs, in `DB::withPinnedSession()`. Active transactions remain pinned automatically. See [releasing and pinning connections](/docs/{{version}}/database#releasing-and-pinning-connections). + When a package constructs `DatabaseStore`, `DatabaseSessionHandler`, `DatabaseQueue`, or `DatabaseBatchRepository` directly, pass the database connection resolver and configured connection name instead of retaining a resolved connection. Framework-configured drivers already use this form. Laravel's base `Connection` class exposes PDO methods. Hypervel's base `Connection` is driver-neutral, while its built-in SQL connections extend `PdoConnection`. Ported code that calls `getPdo`, `getReadPdo`, or another PDO-specific method should accept or narrow to `PdoConnection`. See [extending database connections](/docs/{{version}}/database#extending-database-connections) when porting a custom driver. diff --git a/src/docs/responses.md b/src/docs/responses.md index f526a346a4..15f3356b23 100644 --- a/src/docs/responses.md +++ b/src/docs/responses.md @@ -16,6 +16,7 @@ - [File Downloads](#file-downloads) - [File Responses](#file-responses) - [Streamed Responses](#streamed-responses) + - [Canceling Disconnected Streams](#canceling-disconnected-streams) - [Consuming Streamed Responses](#consuming-streamed-responses) - [Streamed JSON Responses](#streamed-json-responses) - [Event Streams (SSE)](#event-streams) @@ -463,6 +464,23 @@ Route::post('/chat', function () { }); ``` + +### Canceling Disconnected Streams + +An iterable response may stop producing data when its client connection closes. This is useful when producing the next chunk waits on an external service: + +```php +use Hypervel\Http\IterableStreamedResponse; + +return (new IterableStreamedResponse($chunks))->cancelOnDisconnect(); +``` + +The `stream` method returns an `IterableStreamedResponse` for generator callbacks, and `eventStream` uses this response type as well. Cancellation is opt-in and applies while the response is being produced. Other requests and work after response production are unaffected. Use `finally` to release resources owned by your generator, and allow `Swoole\Coroutine\CanceledException` to propagate. + +Connection-close cancellation works with coroutines enabled in Swoole base mode and in process mode with connection-stable dispatch modes (`2`, `4`, or `8`). The default process-mode settings support it. Your application's close callback continues to run. + +Current Swoole releases do not notify PHP when a client cancels one HTTP/2 stream while keeping its connection open. During a quiet upstream wait, that cancellation is detected only when a write fails or the operation finishes or times out. Closing the entire HTTP/2 connection cancels its opted-in producers. Configure appropriate upstream timeouts for quiet streams. + ### Consuming Streamed Responses diff --git a/src/docs/watcher.md b/src/docs/watcher.md index 5ae1539ca2..e8c83125f8 100644 --- a/src/docs/watcher.md +++ b/src/docs/watcher.md @@ -81,6 +81,7 @@ return [ 'watch' => [ 'app/**/*.php', 'config/**/*.php', + 'resources/skills/**', '.env', ], @@ -120,6 +121,8 @@ A directory entry watches every file within that directory recursively, while a Glob patterns use Symfony Finder's glob syntax. A single `*` matches within one directory, while `**` may match across directories. You may also use `?` to match one character, braces to match one of several values, and brackets to match a character range. +Agent skill files under `resources/skills` are watched by default. If your skills live elsewhere, add a glob such as `custom/skills/**` to the `watch` array or pass it through `--path`. Using a glob also identifies the watch root as a directory before it exists. + A directory given directly as a watch path may be a symbolic link. The watcher follows that root, but does not traverse symbolic links found inside a watched directory. @@ -175,6 +178,10 @@ If `find` cannot finish listing the watched files, such as when a watched direct The `FswatchDriver` uses operating system events instead of repeatedly scanning your files, giving it the lowest steady-state resource usage on local filesystems. This driver requires the `fswatch` executable and depends on your operating system delivering file events. On Linux, Hypervel registers only the directories required by your watch patterns, reducing inotify usage. On macOS, each watch root is observed recursively, so you should avoid unnecessarily broad roots. +With the `FswatchDriver`, creating a directory under a recursive watch root also triggers a restart, even if it contains no matching files. Prefer specific roots such as `app/**/*.php` over `**/*.php` to avoid restarts from unrelated directories. + +On Linux, fswatch can miss files written immediately after a missing watch root is created. Moving or renaming directories can also lose events, including later edits inside them, or report edits under an old path. Restart the watcher command to rebuild its watches; restarting only the managed server does not do this. Prefer `ScanFileDriver` or `FindDriver` when your workflow creates watch roots that did not exist when the watcher started, or moves or renames directories. + Polling is generally safer when files live in containers, virtual machines, or network mounts that do not reliably forward operating system events. diff --git a/src/filesystem/README.md b/src/filesystem/README.md index 41110d010b..7b32cd144d 100644 --- a/src/filesystem/README.md +++ b/src/filesystem/README.md @@ -11,6 +11,8 @@ Hypervel omits Laravel's legacy `Storage::cloud()` / `filesystem.cloud` default- Hypervel's `Filesystem::hash()` method uses `xxh128` by default instead of `md5`. Pass `md5` explicitly when Laravel-compatible digests are required. +Custom implementations of `Hypervel\Contracts\Filesystem\Filesystem` must provide `fileExists()` and `directoryExists()` as well as `exists()`, which accepts either kind of path. + The configured disk name `ondemand` is reserved. Disks explicitly set under that name, including fakes, intercept anonymous `build()` calls; configured disks with that name are rejected on resolution. Hypervel pools S3 and Google Cloud Storage SDK clients rather than complete disk adapters. Disks with equivalent client construction config share the expensive client pool while retaining their own bucket, root, visibility, and callback behavior. Pooled disks expose raw internals only through borrow-scoped `withClient()`, `withDriver()`, and `withAdapter()` callbacks. diff --git a/src/filesystem/src/ReadThroughFilesystem.php b/src/filesystem/src/ReadThroughFilesystem.php index f9d08ae515..1ccf6cc972 100644 --- a/src/filesystem/src/ReadThroughFilesystem.php +++ b/src/filesystem/src/ReadThroughFilesystem.php @@ -122,7 +122,7 @@ public function readStreamRange(string $path, ?int $start, ?int $end): mixed */ protected function readerFor(string $path): Cloud { - return $this->primary->fileExists($this->readThroughPrefixer->prefixPath($path)) // @phpstan-ignore method.notFound + return $this->primary->fileExists($this->readThroughPrefixer->prefixPath($path)) ? $this->primary : $this->fallback; } diff --git a/src/foundation/src/Testing/Concerns/InteractsWithTestCaseLifecycle.php b/src/foundation/src/Testing/Concerns/InteractsWithTestCaseLifecycle.php index de62085eb8..e5b5a6ef68 100644 --- a/src/foundation/src/Testing/Concerns/InteractsWithTestCaseLifecycle.php +++ b/src/foundation/src/Testing/Concerns/InteractsWithTestCaseLifecycle.php @@ -293,7 +293,8 @@ protected function callBeforeApplicationDestroyedCallbacks() * RefreshDatabase and DatabaseTransactions store transaction state in Context. * Since setUpTraits runs in a temporary coroutine (separate from the test method's * coroutine), we must copy this state to non-coroutine context. The test coroutine - * will then copy from non-coroutine context via copyFromNonCoroutine(). + * restores the same transaction state explicitly; normal context copying + * omits it so child coroutines cannot inherit their parent's transactions. */ protected function preserveTransactionContext(): void { diff --git a/src/foundation/src/Testing/Concerns/RunTestsInCoroutine.php b/src/foundation/src/Testing/Concerns/RunTestsInCoroutine.php index 1b882d2c4a..bb9ea30e4b 100644 --- a/src/foundation/src/Testing/Concerns/RunTestsInCoroutine.php +++ b/src/foundation/src/Testing/Concerns/RunTestsInCoroutine.php @@ -53,6 +53,7 @@ protected function invokeTestMethod(string $methodName, array $testArguments): m if ($this->copyNonCoroutineContext) { CoroutineContext::copyFromNonCoroutine(); + DatabaseTransactionsManager::copyFromNonCoroutineState(); } $shouldBootFramework = $this->shouldBootFrameworkForTest(); diff --git a/src/grpc/src/Server/Server.php b/src/grpc/src/Server/Server.php index 2eea497b27..1f23858679 100644 --- a/src/grpc/src/Server/Server.php +++ b/src/grpc/src/Server/Server.php @@ -135,7 +135,7 @@ public function onRequest(SwooleRequest $swooleRequest, SwooleResponse $swooleRe } $emissionStarted = true; - ResponseBridge::send($response, $swooleResponse, protocol: 'HTTP/2', request: $request ?? null); + ResponseBridge::send($response, $swooleResponse, protocol: 'HTTP/2', request: $request ?? null, streamId: $swooleRequest->streamId ?? 0); } catch (CanceledException $exception) { $cancelled = true; @@ -158,7 +158,7 @@ public function onRequest(SwooleRequest $swooleRequest, SwooleResponse $swooleRe try { $response = $this->responses->error($this->exceptions->map($throwable)); $emissionStarted = true; - ResponseBridge::send($response, $swooleResponse, protocol: 'HTTP/2', request: $request ?? null); + ResponseBridge::send($response, $swooleResponse, protocol: 'HTTP/2', request: $request ?? null, streamId: $swooleRequest->streamId ?? 0); } catch (CanceledException $exception) { $cancelled = true; diff --git a/src/http-server/composer.json b/src/http-server/composer.json index 085d40467b..3d1ed685dc 100644 --- a/src/http-server/composer.json +++ b/src/http-server/composer.json @@ -37,6 +37,7 @@ "hypervel/coordinator": "^0.4", "hypervel/coroutine": "^0.4", "hypervel/http": "^0.4", + "hypervel/server": "^0.4", "symfony/http-foundation": "^8.1" }, "config": { diff --git a/src/http-server/src/ResponseBridge.php b/src/http-server/src/ResponseBridge.php index f2a41a6d2a..df1109f610 100644 --- a/src/http-server/src/ResponseBridge.php +++ b/src/http-server/src/ResponseBridge.php @@ -6,9 +6,11 @@ use Hypervel\Contracts\Http\HasTrailers; use Hypervel\Http\IterableStreamedResponse; +use Hypervel\Server\ResponseCancellation; use ReflectionProperty; use RuntimeException; use SplTempFileObject; +use Swoole\Coroutine\CanceledException; use Swoole\Http\Response as SwooleResponse; use Symfony\Component\HttpFoundation\BinaryFileResponse; use Symfony\Component\HttpFoundation\Request; @@ -44,6 +46,7 @@ public static function send( bool $withBody = true, string $protocol = 'HTTP/1.1', ?Request $request = null, + int $streamId = 0, ): void { if ($response instanceof HasTrailers && $response instanceof BinaryFileResponse) { throw new RuntimeException('Binary file responses cannot emit trailers.'); @@ -73,7 +76,22 @@ public static function send( static::announceTrailers($response); static::sendStatusAndHeaders($response, $swooleResponse); - static::sendStreamedContent($response, $swooleResponse); + $registration = null; + + if ($response instanceof IterableStreamedResponse && $response->shouldCancelOnDisconnect()) { + // A disconnect during controller work may precede this registration. + if (! $swooleResponse->isWritable()) { + throw new CanceledException('The client disconnected before response production.'); + } + + $registration = ResponseCancellation::register($swooleResponse->fd, $streamId); + } + + try { + static::sendStreamedContent($response, $swooleResponse); + } finally { + $registration?->release(); + } return; } diff --git a/src/http-server/src/Server.php b/src/http-server/src/Server.php index 45869f25c9..5ccd408a7a 100644 --- a/src/http-server/src/Server.php +++ b/src/http-server/src/Server.php @@ -144,6 +144,7 @@ public function onRequest(SwooleRequest $swooleRequest, SwooleResponse $swooleRe withBody: ! isset($rawMethod) || $rawMethod !== 'HEAD', protocol: is_string($protocol) ? $protocol : 'HTTP/1.1', request: $request ?? null, + streamId: $swooleRequest->streamId ?? 0, ); } catch (CanceledException $throwable) { $cancellation = $throwable; diff --git a/src/http/composer.json b/src/http/composer.json index 55eec81310..e6852fd1cf 100644 --- a/src/http/composer.json +++ b/src/http/composer.json @@ -36,7 +36,9 @@ "hypervel/container": "^0.4", "hypervel/context": "^0.4", "hypervel/contracts": "^0.4", + "hypervel/coroutine": "^0.4", "hypervel/database": "^0.4", + "hypervel/engine": "^0.4", "hypervel/macroable": "^0.4", "hypervel/object-pool": "^0.4", "hypervel/pagination": "^0.4", diff --git a/src/http/src/Client/CurlStreamingBody.php b/src/http/src/Client/CurlStreamingBody.php new file mode 100644 index 0000000000..34b9f25e16 --- /dev/null +++ b/src/http/src/Client/CurlStreamingBody.php @@ -0,0 +1,565 @@ +>|RequestInterface|ResponseInterface> */ + protected ?array $pendingTrailerArguments = null; + + protected RequestInterface $request; + + /** + * Take ownership of a configured transfer and its bounded receive buffer. + */ + public function __construct( + protected CurlStreamingHandler $handler, + protected CurlFactoryInterface $factory, + protected ?CurlStreamingConnection $connection, + protected ?EasyHandle $easy, + protected BufferStream $buffer, + protected bool &$paused, + protected array $options, + protected float $startedAt, + protected ?float $headerDeadline, + protected float $readTimeout, + ) { + $this->request = $easy->request; + } + + /** + * Receive the final headers and expose a caller-owned streaming body. + */ + public function response(): ResponseInterface + { + $response = null; + + try { + if (curl_multi_add_handle($this->connection->handle, $this->easy->handle) !== CURLM_OK) { + throw new RuntimeException('Unable to start the streaming request.'); + } + + $this->attached = true; + $idleDeadline = $this->readTimeout > 0 ? $this->startedAt + $this->readTimeout : null; + $auth = $this->options['curl'] ?? []; + $negotiatesAuth = isset($auth[CURLOPT_HTTPAUTH]) || isset($auth[CURLOPT_PROXYAUTH]); + + while (! $this->completed) { + $this->advance(); + $response = $this->easy?->response; + + if ($response !== null + && ($response->getStatusCode() >= 200 || $response->getStatusCode() === 101) + && (! $negotiatesAuth || ! $this->buffer->eof()) + ) { + break; + } + + if (! $this->completed) { + $this->wait($idleDeadline, headers: true); + } + } + + $response = $this->easy->response ?? $this->completedResponse; + + // A pre-header retry creates and exposes its own streaming response. + if ($response->getBody() instanceof self) { + return $response; + } + + $response = $response->withBody($this); + $this->exposed = true; + + if ($this->headerDeadline !== null && hrtime(true) / 1e9 >= $this->headerDeadline) { + $this->timedOut = true; + + throw $this->responseException('Timed out while receiving the response headers', $response, timeout: true); + } + + $onHeaders = $this->options['on_headers'] ?? null; + unset($this->options['on_headers']); + + if ($onHeaders !== null) { + try { + $onHeaders($response, $this->request); + } catch (CanceledException $exception) { + throw $exception; + } catch (Throwable $exception) { + throw $this->responseException('An error was encountered during the on_headers event', $response, $exception); + } + } + + $trailerArguments = $this->pendingTrailerArguments; + $this->pendingTrailerArguments = null; + + if ($trailerArguments !== null) { + $onTrailers = $this->options['on_trailers']; + unset($this->options['on_trailers']); + + try { + $onTrailers(...$trailerArguments); + } catch (CanceledException $exception) { + throw $exception; + } catch (Throwable $exception) { + throw $this->responseException('An error was encountered during the on_trailers event', $response, $exception); + } + } + + $this->reportStats($response); + + return $response; + } catch (Throwable $exception) { + try { + $this->reportStats($this->exposed ? $response : null, $exception); + } finally { + $this->close(); + } + + throw $exception; + } + } + + /** + * Create a transport failure that retains the received response. + */ + protected function responseException(string $message, ResponseInterface $response, ?Throwable $previous = null, bool $timeout = false): TransferException + { + if (class_exists(ResponseException::class)) { + return $timeout + ? new ResponseTimeoutException($message, $this->request, $response, $previous) + : new ResponseException($message, $this->request, $response, $previous); + } + + return new GuzzleRequestException($message, $this->request, $response, $previous); // @phpstan-ignore argument.type (Guzzle 7 accepts the response before the cause.) + } + + /** + * Read available bytes without waiting to fill the requested length. + */ + public function read(int $length): string + { + if ($this->closed) { + throw new RuntimeException('Cannot read a closed streaming response.'); + } + + if ($length < 0) { + throw new RuntimeException('Length must be nonnegative.'); + } + + if ($length === 0) { + return ''; + } + + try { + if ($this->buffer->eof() && ! $this->completed) { + $deadline = $this->readTimeout > 0 ? hrtime(true) / 1e9 + $this->readTimeout : null; + + if ($this->paused) { + // Resuming may synchronously fill the buffer and pause the sink again. + $this->paused = false; + + if (curl_pause($this->easy->handle, CURLPAUSE_CONT) !== CURLE_OK) { + throw new RuntimeException('Unable to resume the streaming response.'); + } + } + + while ($this->buffer->eof() && ! $this->completed) { + $this->advance(); + + if ($this->buffer->eof() && ! $this->completed) { + $this->wait($deadline); + } + } + } + + $chunk = $this->buffer->read($length); + $this->position += strlen($chunk); + + return $chunk; + } catch (Throwable $exception) { + $this->close(); + + throw $exception; + } + } + + /** + * Drive native I/O and settle a completed transfer once. + */ + protected function advance(): void + { + $result = curl_multi_exec($this->connection->handle, $running); + + if (Coroutine::isCanceled()) { + throw new CanceledException('The streaming request was canceled.'); + } + + if ($result !== CURLM_OK) { + throw new RuntimeException('Unable to read the streaming response: ' . curl_multi_strerror($result)); + } + + $message = curl_multi_info_read($this->connection->handle); + + if ($message === false) { + return; + } + + $easy = $this->easy; + $easy->errno = $message['result']; + $this->detachTransfer(); + $this->completed = true; + $this->easy = null; + + $retry = function (RequestInterface $request, array $options): PromiseInterface { + if ($this->exposed) { + return Create::rejectionFor(new RuntimeException('Cannot retry a streaming request after exposing its response.')); + } + + $options = array_replace($this->options, $options); + + // Factory retry options contain temporary statistics and trailer collectors. + foreach (['on_headers', 'on_stats', 'on_trailers'] as $name) { + if (isset($this->options[$name])) { + $options[$name] = $this->options[$name]; + } else { + unset($options[$name]); + } + } + + if ($this->headerDeadline !== null) { + $remaining = $this->headerDeadline - hrtime(true) / 1e9; + + if ($remaining <= 0) { + return Create::rejectionFor(new ConnectException('The streaming request timed out before response headers.', $request)); + } + + $options['timeout'] = $remaining; + } + + // The replacement transfer owns callbacks and header-time statistics. + unset($this->options['on_headers'], $this->options['on_stats'], $this->options['on_trailers']); + + return ($this->handler)($request, $options); + }; + + $stats = null; + $trailerArguments = null; + + if (! $this->exposed && isset($easy->options['on_trailers'])) { + // Completion before exposure must not publish trailers before on_headers. + $easy->options['on_trailers'] = static function (array|ResponseInterface|RequestInterface ...$arguments) use (&$trailerArguments): void { + $trailerArguments = $arguments; + }; + } + + if (isset($this->options['on_stats'])) { + $easy->options['on_stats'] = static function (TransferStats $transfer) use (&$stats): void { + $stats = $transfer; + }; + } + + try { + $this->completedResponse = CurlFactory::finish($retry, $easy, $this->factory)->wait(); + } catch (TransferException $exception) { + // Completion callbacks run after the native cancellation check above. + if ($exception->getPrevious() instanceof CanceledException) { + throw $exception->getPrevious(); + } + + throw $exception; + } finally { + $this->completedStats = $stats; + $this->pendingTrailerArguments = $trailerArguments; + } + } + + /** + * Wait for network progress within the current phase's deadline. + */ + protected function wait(?float &$idleDeadline, bool $headers = false): void + { + $deadline = $idleDeadline; + + if ($headers && $this->headerDeadline !== null) { + $deadline = $deadline === null ? $this->headerDeadline : min($deadline, $this->headerDeadline); + } + + $remaining = $deadline === null ? 1.0 : $deadline - hrtime(true) / 1e9; + + if ($remaining <= 0) { + $this->timedOut = true; + + if ($headers) { + throw new ConnectException('The streaming request timed out before response headers.', $this->request); + } + + throw new RuntimeException('The streaming response read timed out.'); + } + + $selected = curl_multi_select($this->connection->handle, min(1.0, $remaining)); + + if (Coroutine::isCanceled()) { + throw new CanceledException('The streaming request was canceled.'); + } + + if ($selected === -1) { + // libcurl can briefly have no descriptors while scheduling its next action. + usleep(1000); + } elseif ($selected > 0 && $this->readTimeout > 0) { + $idleDeadline = hrtime(true) / 1e9 + $this->readTimeout; + } + } + + /** + * Report streaming statistics once, at the handler-return boundary. + */ + protected function reportStats(?ResponseInterface $response, ?Throwable $error = null): void + { + $callback = $this->options['on_stats'] ?? null; + unset($this->options['on_stats']); + $stats = $this->completedStats; + $this->completedStats = null; + + if ($callback !== null) { + $handlerStats = $stats?->getHandlerStats() ?? []; + + if ($this->easy !== null) { + $handlerStats = curl_getinfo($this->easy->handle); + $handlerStats['appconnect_time'] = curl_getinfo($this->easy->handle, CURLINFO_APPCONNECT_TIME); + } + + $handlerError = $stats?->getHandlerErrorData() ?? 0; + + $callback(new TransferStats( + $this->request, + $response ?? $stats?->getResponse(), + hrtime(true) / 1e9 - $this->startedAt, + $handlerError === 0 ? ($error ?? 0) : $handlerError, + $handlerStats, + )); + } + } + + /** + * Remove the active handle before returning its native connection cache. + */ + protected function detachTransfer(): void + { + $connection = $this->connection; + $this->connection = null; + + if ($connection === null) { + return; + } + + if ($this->attached) { + $this->attached = false; + + if (curl_multi_remove_handle($connection->handle, $this->easy->handle) !== CURLM_OK) { + throw new RuntimeException('Unable to detach the streaming request.'); + } + } + + $this->handler->release($connection); + } + + /** + * Close the stream and settle its transfer without waiting for more data. + */ + public function close(): void + { + if ($this->closed) { + return; + } + + $this->closed = true; + + try { + $this->detachTransfer(); + } finally { + if ($this->easy !== null) { + $easy = $this->easy; + $this->easy = null; + $this->factory->release($easy); + } + + $this->buffer->close(); + $this->completedResponse = null; + $this->completedStats = null; + $this->pendingTrailerArguments = null; + $this->options = []; + } + } + + /** + * Detach the body, which has no PHP stream resource to return. + */ + public function detach(): mixed + { + $this->close(); + + return null; + } + + /** + * Get the unknown size of the incremental response body. + */ + public function getSize(): ?int + { + return null; + } + + /** + * Get the number of bytes consumed by the caller. + */ + public function tell(): int + { + return $this->position; + } + + /** + * Determine whether the completed body has been consumed. + */ + public function eof(): bool + { + return $this->closed || ($this->completed && $this->buffer->eof()); + } + + /** + * Determine whether the stream can seek. + */ + public function isSeekable(): bool + { + return false; + } + + /** + * Reject seeking on an incremental response. + */ + public function seek(int $offset, int $whence = SEEK_SET): void + { + throw new RuntimeException('Cannot seek a streaming response.'); + } + + /** + * Reject rewinding an incremental response. + */ + public function rewind(): void + { + $this->seek(0); + } + + /** + * Determine whether the response accepts writes. + */ + public function isWritable(): bool + { + return false; + } + + /** + * Reject writes to the response body. + */ + public function write(string $string): int + { + throw new RuntimeException('Cannot write to a streaming response.'); + } + + /** + * Determine whether the stream is open for reading. + */ + public function isReadable(): bool + { + return ! $this->closed; + } + + /** + * Read the remaining response body. + */ + public function getContents(): string + { + $contents = ''; + + while (! $this->eof()) { + $contents .= $this->read(8192); + } + + return $contents; + } + + /** + * Get metadata for the forward-only response stream. + */ + public function getMetadata(?string $key = null): mixed + { + $metadata = ['timed_out' => $this->timedOut, 'seekable' => false, 'eof' => $this->eof()]; + + return $key === null ? $metadata : ($metadata[$key] ?? null); + } + + /** + * Read the remaining response body as a string. + */ + public function __toString(): string + { + return $this->getContents(); + } + + /** + * Release a response abandoned by its consumer. + */ + public function __destruct() + { + try { + $this->close(); + } catch (Throwable $exception) { + PoolErrorReporter::report($exception); + } + } +} diff --git a/src/http/src/Client/CurlStreamingConnection.php b/src/http/src/Client/CurlStreamingConnection.php new file mode 100644 index 0000000000..e4c616b3aa --- /dev/null +++ b/src/http/src/Client/CurlStreamingConnection.php @@ -0,0 +1,26 @@ +handle = curl_multi_init(); + curl_multi_setopt($this->handle, CURLMOPT_MAXCONNECTS, 1); + } +} diff --git a/src/http/src/Client/CurlStreamingHandler.php b/src/http/src/Client/CurlStreamingHandler.php new file mode 100644 index 0000000000..faa2c8c36e --- /dev/null +++ b/src/http/src/Client/CurlStreamingHandler.php @@ -0,0 +1,216 @@ + */ + protected array $idleConnections = []; + + /** + * Create a streaming handler with optional named-connection retention. + */ + public function __construct(array $options = [], protected int $maxIdleConnections = 0) + { + $sharing = CurlShareHandleState::fromOption($options['transport_sharing'] ?? null); + $this->factory = new CurlFactory( + $maxIdleConnections, + $sharing->mode ?? TransportSharing::NONE, + $sharing, + ); + } + + /** + * Add coroutine streaming while retaining the caller's fallback handler. + */ + public static function wrap(callable $fallback, array $options = [], int $maxIdleConnections = 0): Closure + { + $streaming = null; + + return static function (RequestInterface $request, array $requestOptions) use ($fallback, $options, $maxIdleConnections, &$streaming): PromiseInterface { + if (empty($requestOptions['stream']) || ! self::supports($requestOptions)) { + return $fallback($request, $requestOptions); + } + + $streaming ??= new self($options, $maxIdleConnections); + + return $streaming($request, $requestOptions); + }; + } + + /** + * Determine whether the current execution can use native streaming. + */ + public static function supports(array $options): bool + { + return Coroutine::inCoroutine() + && (Runtime::getHookFlags() & SWOOLE_HOOK_NATIVE_CURL) !== 0 + && function_exists('curl_multi_exec') + && ! array_key_exists('stream_context', $options) + && (! isset($options['stream_factory']) || $options['stream_factory'] instanceof HttpFactory); + } + + /** + * Start a transfer and return its response without draining its body. + * + * @return PromiseInterface + */ + public function __invoke(RequestInterface $request, array $options): PromiseInterface + { + // Guzzle has consumed the stack; retaining it would keep its response alive. + unset($options['handler']); + + try { + HostValidator::assertRequestHost($request); + + if (isset($options['delay'])) { + Sleep::usleep((int) ($options['delay'] * 1000)); + } + + $timeout = $this->timeout($options['timeout'] ?? 0, 'timeout'); + $readTimeout = $this->timeout($options['read_timeout'] ?? CurlStreamingBody::DEFAULT_READ_TIMEOUT, 'read_timeout'); + $startedAt = hrtime(true) / 1e9; + $deadline = $timeout > 0 ? $startedAt + $timeout : null; + + foreach (['on_headers', 'on_stats'] as $name) { + if (isset($options[$name]) && ! is_callable($options[$name])) { + throw new InvalidArgumentException("{$name} must be callable."); + } + } + + $buffer = new BufferStream; + $handle = null; + $paused = false; + $sink = FnStream::decorate($buffer, [ + 'write' => static function (string $chunk) use ($buffer, &$handle, &$paused): int { + if ($buffer->write($chunk) === 0) { + // The chunk is accepted; stop native reads at the buffer's high-water mark. + $paused = true; + + /** @var WeakReference $handle Assigned before the transfer starts. */ + if (curl_pause($handle->get(), CURLPAUSE_RECV) !== CURLE_OK) { + throw new RuntimeException('Unable to pause the streaming response.'); + } + } + + return strlen($chunk); + }, + ]); + $transferOptions = $options; + unset($transferOptions['on_headers'], $transferOptions['on_stats']); + $transferOptions['sink'] = $sink; + // Streaming timeout ends at headers; body waits use read_timeout. + $transferOptions['timeout'] = 0; + $easy = $this->factory->create($request, $transferOptions); + // Guzzle 7 may discard a handle without clearing its callbacks; avoid a cycle. + $handle = WeakReference::create($easy->handle); + + try { + $connection = $this->acquire($easy); + $body = new CurlStreamingBody( + $this, + $this->factory, + $connection, + $easy, + $buffer, + $paused, + $options, + $startedAt, + $deadline, + $readTimeout, + ); + } catch (Throwable $exception) { + $this->factory->release($easy); + + throw $exception; + } + + return Create::promiseFor($body->response()); + } catch (CanceledException $exception) { + throw $exception; + } catch (Throwable $exception) { + return Create::rejectionFor($exception); + } + } + + /** + * Obtain an exclusive transport without sharing another proxy's tunnel. + */ + protected function acquire(EasyHandle $easy): CurlStreamingConnection + { + for ($index = count($this->idleConnections) - 1; $index >= 0; --$index) { + $connection = $this->idleConnections[$index]; + + if ($connection->proxyTunnelSignature === $easy->proxyTunnelSignature) { + array_splice($this->idleConnections, $index, 1); + + return $connection; + } + } + + return new CurlStreamingConnection($easy->proxyTunnelSignature); + } + + /** + * Retain an idle transport after its request has been detached. + */ + public function release(CurlStreamingConnection $connection): void + { + if ($this->maxIdleConnections === 0) { + return; + } + + if (count($this->idleConnections) >= $this->maxIdleConnections) { + array_shift($this->idleConnections); + } + + $this->idleConnections[] = $connection; + } + + /** + * Normalize a nonnegative request timeout in seconds. + */ + protected function timeout(mixed $value, string $name): float + { + if (! is_numeric($value) || ! is_finite((float) $value) || $value < 0) { + throw new InvalidArgumentException("{$name} must be a finite nonnegative number."); + } + + return (float) $value; + } +} diff --git a/src/http/src/Client/Factory.php b/src/http/src/Client/Factory.php index 84162e9c2a..808fbc4c0c 100644 --- a/src/http/src/Client/Factory.php +++ b/src/http/src/Client/Factory.php @@ -686,7 +686,11 @@ public function getConnectionHandler(string $name): callable */ protected function createConnectionHandler(array $options): callable { - return Utils::chooseHandler($options); + return CurlStreamingHandler::wrap( + Utils::chooseHandler($options), + $options, + CurlStreamingHandler::MAX_IDLE_CONNECTIONS, + ); } /** diff --git a/src/http/src/Client/PendingRequest.php b/src/http/src/Client/PendingRequest.php index 0eb4120273..ca726f9a8f 100644 --- a/src/http/src/Client/PendingRequest.php +++ b/src/http/src/Client/PendingRequest.php @@ -20,8 +20,10 @@ use GuzzleHttp\Promise\PromiseInterface; use GuzzleHttp\TransferStats; use GuzzleHttp\UriTemplate\UriTemplate; +use GuzzleHttp\Utils; use Hypervel\Contracts\Container\Transient; use Hypervel\Contracts\Support\Arrayable; +use Hypervel\Database\ConnectionResolver; use Hypervel\Http\Client\Destinations\CurlCapabilities; use Hypervel\Http\Client\Destinations\DestinationPolicy; use Hypervel\Http\Client\Destinations\DestinationPolicyException; @@ -1051,6 +1053,11 @@ function (&$response) use ($attempt, &$shouldRetry) { } ); } catch (TransferException $e) { + // Guzzle wraps failures raised by transport callbacks. + if ($e->getPrevious() instanceof CanceledException) { + throw $e->getPrevious(); + } + if (($response = $this->responseFromException($e)) !== null) { $this->marshalTransportExceptionWithResponse($e, $response); } elseif (method_exists($e, 'getRequest')) { @@ -1170,6 +1177,10 @@ protected function makePromise(string $method, string $url, array $options = [], return $this->runAfterResponseCallbacks($response); }) ->otherwise(function (Throwable $e) { + if ($e instanceof TransferException && $e->getPrevious() instanceof CanceledException) { + throw $e->getPrevious(); + } + if ($e instanceof CanceledException) { throw $e; } @@ -1726,12 +1737,14 @@ public function buildHandlerStack(): HandlerStack $handler = $this->factory->getConnectionHandler($this->connection); } + $handler ??= CurlStreamingHandler::wrap(Utils::chooseHandler()); + $stack = $this->pushHandlers(HandlerStack::create($handler)); if ($this->handler === null && ! ini_get('allow_url_fopen')) { // Faked responses return before reaching this transport-only guard. $stack->push(static fn (callable $handler): Closure => static function (RequestInterface $request, array $options) use ($handler): PromiseInterface { - if ($options['stream'] ?? false) { + if (($options['stream'] ?? false) && ! CurlStreamingHandler::supports($options)) { throw new RuntimeException('Streaming responses require allow_url_fopen when using the default HTTP handler.'); } @@ -1763,6 +1776,7 @@ public function pushHandlers(HandlerStack $handlerStack): HandlerStack $stack->push(Middleware::prepareBody(), 'prepare_body'); $stack->push($this->buildRecorderHandler()); $stack->push($this->buildStubHandler()); + $stack->push($this->buildDatabaseReleaseHandler()); // Innermost, so it vets every physical request (each redirect included) and never sees faked ones. $stack->push($this->buildDestinationPolicyHandler()); }); @@ -1939,6 +1953,20 @@ protected function sinkStubHandler(mixed $sink): Closure }; } + /** + * Return idle database sessions before waiting for an external service. + */ + protected function buildDatabaseReleaseHandler(): Closure + { + return static function (callable $handler): Closure { + return static function (RequestInterface $request, array $options) use ($handler): PromiseInterface { + ConnectionResolver::releaseIdleConnections(); + + return $handler($request, $options); + }; + }; + } + /** * Build the destination policy handler. */ @@ -1975,7 +2003,7 @@ protected function sendToPolicyDestination( ): PromiseInterface { // The pins only bind the cURL handler, and raw cURL or proxy options could route around them. $unsupported = match (true) { - ! empty($options['stream']) => 'cannot stream responses; use [sink] instead', + ! empty($options['stream']) && ! CurlStreamingHandler::supports($options) => 'cannot stream responses with the fallback transport; use [sink] instead', $this->handler !== null => 'cannot use a custom handler', ($options['proxy'] ?? '') !== '' => 'cannot set the [proxy] option; select proxies in the destination policy', ! empty($options['curl']) => 'cannot set raw [curl] options', diff --git a/src/http/src/IterableStreamedResponse.php b/src/http/src/IterableStreamedResponse.php index 77500d79e7..437aaed960 100644 --- a/src/http/src/IterableStreamedResponse.php +++ b/src/http/src/IterableStreamedResponse.php @@ -19,6 +19,8 @@ class IterableStreamedResponse extends StreamedResponse /** @var null|iterable */ private array|Traversable|null $chunks = null; + protected bool $cancelOnDisconnect = false; + /** * Create a new iterable streamed response. * @@ -31,6 +33,24 @@ public function __construct(iterable $chunks, int $status = 200, array $headers $this->setChunks($chunks); } + /** + * Cancel response production when the client connection closes. + */ + public function cancelOnDisconnect(bool $cancel = true): static + { + $this->cancelOnDisconnect = $cancel; + + return $this; + } + + /** + * Determine whether disconnects should cancel response production. + */ + public function shouldCancelOnDisconnect(): bool + { + return $this->cancelOnDisconnect; + } + /** * Set the chunks associated with the response. * diff --git a/src/inertia/src/InertiaState.php b/src/inertia/src/InertiaState.php index c526c438e6..bdfbe7d801 100644 --- a/src/inertia/src/InertiaState.php +++ b/src/inertia/src/InertiaState.php @@ -107,10 +107,10 @@ class InertiaState implements ReplicableContext */ public static function current(): self { - if (CoroutineContext::has(self::CONTEXT_KEY)) { - /** @var self $state */ - $state = CoroutineContext::get(self::CONTEXT_KEY); + /** @var null|self $state */ + $state = CoroutineContext::get(self::CONTEXT_KEY); + if ($state !== null) { return $state; } diff --git a/src/json-schema/src/JsonSchema.php b/src/json-schema/src/JsonSchema.php index 30775c568a..dbc9f63293 100644 --- a/src/json-schema/src/JsonSchema.php +++ b/src/json-schema/src/JsonSchema.php @@ -17,7 +17,7 @@ use InvalidArgumentException; /** - * @method static ObjectType object(Closure|array $properties = []) + * @method static ObjectType object(Closure|array $properties = []) * @method static AnyOfType anyOf(Closure|array $schemas) * @method static IntegerType integer() * @method static NumberType number() diff --git a/src/json-schema/src/JsonSchemaTypeFactory.php b/src/json-schema/src/JsonSchemaTypeFactory.php index 1894b0397d..b076303cca 100644 --- a/src/json-schema/src/JsonSchemaTypeFactory.php +++ b/src/json-schema/src/JsonSchemaTypeFactory.php @@ -13,7 +13,7 @@ class JsonSchemaTypeFactory extends JsonSchema implements JsonSchemaContract /** * Create a new object schema instance. * - * @param array|(Closure(JsonSchemaTypeFactory): array) $properties + * @param array|(Closure(JsonSchemaTypeFactory): array) $properties */ public function object(Closure|array $properties = []): Types\ObjectType { diff --git a/src/json-schema/src/Types/ObjectType.php b/src/json-schema/src/Types/ObjectType.php index 1227c14898..3248a951ee 100644 --- a/src/json-schema/src/Types/ObjectType.php +++ b/src/json-schema/src/Types/ObjectType.php @@ -14,12 +14,22 @@ class ObjectType extends Type /** * Create a new object type instance. * - * @param array $properties + * @param array $properties */ public function __construct(protected array $properties = []) { } + /** + * Get the named property schemas. + * + * @return array + */ + public function getProperties(): array + { + return $this->properties; + } + /** * Disallow additional properties. */ @@ -33,7 +43,7 @@ public function withoutAdditionalProperties(): static /** * Set the type's default value. * - * @param null|array $value + * @param null|array $value */ public function default(?array $value): static { diff --git a/src/reflection/src/ClassMetadataCache.php b/src/reflection/src/ClassMetadataCache.php index 68d12f215d..0537d460e0 100644 --- a/src/reflection/src/ClassMetadataCache.php +++ b/src/reflection/src/ClassMetadataCache.php @@ -32,6 +32,11 @@ class ClassMetadataCache */ protected static array $properties = []; + /** + * @var array> + */ + protected static array $traits = []; + /** * @var array> */ @@ -109,6 +114,22 @@ public static function properties(object|string $target): array ??= static::reflectClass($class)->getProperties(); } + /** + * Determine if the class uses a trait directly, through another trait or through a parent. + * + * @param class-string|object $target + * @param class-string $trait + * + * @throws ReflectionException + */ + public static function usesTrait(object|string $target, string $trait): bool + { + $class = static::className($target); + static::$traits[$class] ??= static::resolveTraits(static::reflectClass($class)); + + return isset(static::$traits[$class][$trait]); + } + /** * Get the cached class attribute metadata for the given class. * @@ -187,6 +208,26 @@ public static function hasPropertyAttribute(ReflectionProperty $property, string = $property->getAttributes($attributeClass) !== []; } + /** + * Collect the traits used by a class and its parents. + * + * @param ReflectionClass $reflection + * @return array + */ + protected static function resolveTraits(ReflectionClass $reflection): array + { + $traits = []; + + do { + foreach ($reflection->getTraits() as $name => $trait) { + $traits[$name] = true; + $traits += static::resolveTraits($trait); + } + } while (($reflection = $reflection->getParentClass()) !== false); + + return $traits; + } + /** * Resolve the class attribute metadata for the given class. * @@ -244,6 +285,7 @@ public static function flushState(): void static::$methods = []; static::$defaultProperties = []; static::$properties = []; + static::$traits = []; static::$attributes = []; static::$classAttributePresence = []; static::$propertyAttributePresence = []; diff --git a/src/routing/src/ResponseFactory.php b/src/routing/src/ResponseFactory.php index bf8a3b3500..7fb861f518 100644 --- a/src/routing/src/ResponseFactory.php +++ b/src/routing/src/ResponseFactory.php @@ -19,6 +19,7 @@ use Hypervel\Support\Traits\Macroable; use ReflectionFunction; use SplFileInfo; +use Swoole\Coroutine\CanceledException; use Symfony\Component\HttpFoundation\BinaryFileResponse; use Symfony\Component\HttpFoundation\StreamedJsonResponse; use Symfony\Component\HttpFoundation\StreamedResponse; @@ -93,10 +94,13 @@ public function jsonp(string $callback, mixed $data = [], int $status = 200, arr /** * Create a new event stream response. + * + * @return IterableStreamedResponse */ public function eventStream(Closure $callback, array $headers = [], StreamedEvent|string|null $endStreamWith = ''): StreamedResponse { - return $this->stream(function () use ($callback, $endStreamWith) { + /** @var IterableStreamedResponse $response */ + $response = $this->stream(function () use ($callback, $endStreamWith) { try { foreach ($callback() as $message) { yield $this->formatStreamedEvent($message); @@ -105,6 +109,8 @@ public function eventStream(Closure $callback, array $headers = [], StreamedEven if (filled($endStreamWith)) { yield $this->formatStreamedEvent($endStreamWith); } + } catch (CanceledException $e) { + throw $e; } catch (Throwable $e) { report($e); } @@ -113,6 +119,8 @@ public function eventStream(Closure $callback, array $headers = [], StreamedEven 'Cache-Control' => 'no-cache', 'X-Accel-Buffering' => 'no', ])); + + return $response; } /** @@ -183,6 +191,8 @@ public function streamDownload(callable $callback, ?string $name = null, array $ $withWrappedException = function () use ($callback) { try { $callback(); + } catch (CanceledException $e) { + throw $e; } catch (Throwable $e) { throw new StreamedResponseException($e); } diff --git a/src/sentry/src/Features/Storage/FilesystemAdapterDecorator.php b/src/sentry/src/Features/Storage/FilesystemAdapterDecorator.php index 8984697e53..75359f7beb 100644 --- a/src/sentry/src/Features/Storage/FilesystemAdapterDecorator.php +++ b/src/sentry/src/Features/Storage/FilesystemAdapterDecorator.php @@ -48,22 +48,6 @@ public function assertDirectoryEmpty(string $path): static return $this; } - /** - * Determine if a file exists. - */ - public function fileExists(string $path): bool - { - return $this->withSentry(__FUNCTION__, func_get_args(), $path, compact('path')); - } - - /** - * Determine if a directory exists. - */ - public function directoryExists(string $path): bool - { - return $this->withSentry(__FUNCTION__, func_get_args(), $path, compact('path')); - } - /** * Get the checksum for a file. */ diff --git a/src/sentry/src/Features/Storage/FilesystemDecorator.php b/src/sentry/src/Features/Storage/FilesystemDecorator.php index 34ae436570..9c15d8a9f6 100644 --- a/src/sentry/src/Features/Storage/FilesystemDecorator.php +++ b/src/sentry/src/Features/Storage/FilesystemDecorator.php @@ -83,13 +83,29 @@ public function path(string $path): string } /** - * Determine if a file exists. + * Determine if a file or directory exists. */ public function exists(string $path): bool { return $this->withSentry(__FUNCTION__, func_get_args(), $path, compact('path')); } + /** + * Determine if a file exists. + */ + public function fileExists(string $path): bool + { + return $this->withSentry(__FUNCTION__, func_get_args(), $path, compact('path')); + } + + /** + * Determine if a directory exists. + */ + public function directoryExists(string $path): bool + { + return $this->withSentry(__FUNCTION__, func_get_args(), $path, compact('path')); + } + /** * Get the contents of a file. */ diff --git a/src/server/src/ResponseCancellation.php b/src/server/src/ResponseCancellation.php new file mode 100644 index 0000000000..8108a54bbc --- /dev/null +++ b/src/server/src/ResponseCancellation.php @@ -0,0 +1,79 @@ +> */ + protected static array $responses = []; + + /** + * Identify the execution producing one response on a connection. + */ + private function __construct( + protected int $connection, + protected int $stream, + protected int $coroutine, + ) { + } + + /** + * Register the current coroutine for the duration of response production. + */ + public static function register(int $connection, int $stream = 0): ?self + { + $coroutine = Coroutine::id(); + + if ($coroutine < 0) { + return null; + } + + return self::$responses[$connection][$coroutine] = new self($connection, $stream, $coroutine); + } + + /** + * Cancel every active producer on a closed connection. + */ + public static function cancel(int $connection): void + { + $responses = self::$responses[$connection] ?? []; + unset(self::$responses[$connection]); + + foreach ($responses as $response) { + Coroutine::cancelById($response->coroutine, true); + } + } + + /** + * Remove this registration without disturbing a newer response. + */ + public function release(): void + { + if ((self::$responses[$this->connection][$this->coroutine] ?? null) !== $this) { + return; + } + + unset(self::$responses[$this->connection][$this->coroutine]); + + if (self::$responses[$this->connection] === []) { + unset(self::$responses[$this->connection]); + } + } + + /** + * Flush all static state. + */ + public static function flushState(): void + { + self::$responses = []; + } +} diff --git a/src/server/src/Server.php b/src/server/src/Server.php index cfee0dece2..0798bf4130 100644 --- a/src/server/src/Server.php +++ b/src/server/src/Server.php @@ -90,10 +90,10 @@ protected function initServers(ServerConfig $config): void if (! $this->server instanceof SwooleServer) { $this->server = $this->makeServer($type, $host, $port, $config->getMode(), $sockType); $callbacks = array_replace($this->defaultCallbacks(), $config->getCallbacks(), $callbacks); - $this->registerSwooleEvents($this->server, $callbacks, $name); if ($this->server->set(array_replace($config->getSettings(), $server->getSettings())) === false) { throw new ServerException("Failed to configure server [{$name}]."); } + $this->registerSwooleEvents($this->server, $callbacks, $name); ServerManager::add($name, [$type, current($this->server->ports)]); // Trigger BeforeMainServerStart event, this event only triggers once before main server start. @@ -182,6 +182,18 @@ protected function makeServer(int $type, string $host, int $port, int $mode, int */ protected function registerSwooleEvents(SwoolePort|SwooleServer $server, array $events, string $serverName): void { + $mainServer = $server instanceof SwooleServer ? $server : $this->getServer(); + $settings = $mainServer->setting ?? []; + // These dispatch modes keep connection events in the response's worker. + $cancelResponses = (isset($events[Event::ON_REQUEST]) || isset($events[Event::ON_HANDSHAKE])) + && ($settings['enable_coroutine'] ?? true) + && ($mainServer->mode === SWOOLE_BASE + || ($mainServer->mode === SWOOLE_PROCESS && in_array((int) ($settings['dispatch_mode'] ?? 2), [2, 4, 8], true))); + + if ($cancelResponses) { + $events[Event::ON_CLOSE] ??= null; + } + foreach ($events as $event => $callback) { if (! Event::isSwooleEvent($event)) { continue; @@ -204,6 +216,18 @@ protected function registerSwooleEvents(SwoolePort|SwooleServer $server, array $ $callback = [$class, $method]; } + if ($cancelResponses && $event === Event::ON_CLOSE) { + $callback = static function (SwooleServer $server, int $connection, int $reactor) use ($callback): void { + try { + ResponseCancellation::cancel($connection); + } finally { + if ($callback !== null) { + $callback($server, $connection, $reactor); + } + } + }; + } + if ( ($event === Event::ON_REQUEST || $event === Event::ON_HANDSHAKE) && is_callable($callback) diff --git a/src/support/src/Facades/DB.php b/src/support/src/Facades/DB.php index 82a18d2084..c56a4b5ecf 100644 --- a/src/support/src/Facades/DB.php +++ b/src/support/src/Facades/DB.php @@ -30,6 +30,7 @@ * @method static void purge(\UnitEnum|string|null $name = null) * @method static void purgeConnections() * @method static \Hypervel\Database\Connection reconnect(\UnitEnum|string|null $name = null) + * @method static void releaseIdleConnections() * @method static \Hypervel\Database\DatabaseManager setApplication(\Hypervel\Contracts\Foundation\Application $app) * @method static void setDefaultConnection(string|null $name) * @method static void setReconnector(callable $reconnector) @@ -119,6 +120,7 @@ * @method static \Hypervel\Database\Connection useWriteConnectionWhenReading(bool $value = true) * @method static mixed withoutPretending(\Closure $callback) * @method static mixed withoutTablePrefix(\Closure $callback) + * @method static mixed withPinnedSession(\Closure $callback) * @method static void bindValues(\PDOStatement $statement, array $bindings) * @method static void configureSessionUsing(\Hypervel\Database\SessionConfigurator $configurator) * @method static \PDO getPdo() diff --git a/src/support/src/Facades/Response.php b/src/support/src/Facades/Response.php index 27c817000c..0d9659cdba 100644 --- a/src/support/src/Facades/Response.php +++ b/src/support/src/Facades/Response.php @@ -8,7 +8,7 @@ /** * @method static \Symfony\Component\HttpFoundation\BinaryFileResponse download(\SplFileInfo|string $file, string|null $name = null, array $headers = [], string $disposition = 'attachment') - * @method static \Symfony\Component\HttpFoundation\StreamedResponse eventStream(\Closure $callback, array $headers = [], \Hypervel\Http\StreamedEvent|string|null $endStreamWith = '') + * @method static \Hypervel\Http\IterableStreamedResponse eventStream(\Closure $callback, array $headers = [], \Hypervel\Http\StreamedEvent|string|null $endStreamWith = '') * @method static \Symfony\Component\HttpFoundation\BinaryFileResponse file(\SplFileInfo|string $file, array $headers = []) * @method static void flushMacros() * @method static void flushState() diff --git a/src/support/src/Facades/Storage.php b/src/support/src/Facades/Storage.php index 39f9371ca7..38d3476455 100644 --- a/src/support/src/Facades/Storage.php +++ b/src/support/src/Facades/Storage.php @@ -44,7 +44,9 @@ * @method static bool delete(array|string $paths) * @method static bool deleteDirectory(string $directory) * @method static array directories(string|null $directory = null, bool $recursive = false) + * @method static bool directoryExists(string $path) * @method static bool exists(string $path) + * @method static bool fileExists(string $path) * @method static array files(string|null $directory = null, bool $recursive = false) * @method static string|null get(string $path) * @method static string getVisibility(string $path) @@ -70,10 +72,8 @@ * @method static void buildTemporaryUrlsUsing(\Closure|null $callback) * @method static string|false checksum(string $path, array $options = []) * @method static bool copyToDisk(\Hypervel\Contracts\Filesystem\Filesystem|UnitEnum|string $disk, string $from, string|null $to = null) - * @method static bool directoryExists(string $path) * @method static bool directoryMissing(string $path) * @method static \Symfony\Component\HttpFoundation\StreamedResponse download(string $path, string|null $name = null, array $headers = []) - * @method static bool fileExists(string $path) * @method static bool fileMissing(string $path) * @method static array getConfig() * @method static \Hypervel\Image\Image image(string $path) diff --git a/src/support/src/Traits/StaticInstance.php b/src/support/src/Traits/StaticInstance.php index 7223dfbdd4..b4d954c623 100644 --- a/src/support/src/Traits/StaticInstance.php +++ b/src/support/src/Traits/StaticInstance.php @@ -14,11 +14,7 @@ trait StaticInstance public static function instance(array $params = [], bool $refresh = false, string $suffix = ''): static { $key = static::class . $suffix; - $instance = null; - - if (CoroutineContext::has($key)) { - $instance = CoroutineContext::get($key); - } + $instance = CoroutineContext::get($key); if ($refresh || ! $instance instanceof static) { $instance = new static(...$params); diff --git a/src/testbench/src/Attributes/WithEnv.php b/src/testbench/src/Attributes/WithEnv.php index 1d8cfa19ff..7de8276612 100644 --- a/src/testbench/src/Attributes/WithEnv.php +++ b/src/testbench/src/Attributes/WithEnv.php @@ -10,7 +10,6 @@ use Hypervel\Testbench\Contracts\Attributes\Invokable; use Hypervel\Testbench\Foundation\Bootstrap\LoadEnvironmentVariablesFromArray; use Hypervel\Testbench\Foundation\Env; -use Hypervel\Testbench\Foundation\UndefinedValue; use function Hypervel\Testbench\parse_environment_variables; @@ -37,15 +36,17 @@ public function __construct( public function __invoke(ApplicationContract $app): Closure { $key = $this->key; - $value = Env::get($key, new UndefinedValue); + $repository = Env::getRepository(); + $value = $repository->get($key); self::setEnvironmentVariable($app, $key, $this->value); - return static function () use ($app, $key, $value): void { - if ($value instanceof UndefinedValue) { - Env::forget($key); + // Keep the writer that owns this value even if a scoped environment helper replaces the repository. + return static function () use ($repository, $key, $value): void { + if ($value === null) { + $repository->clear($key); } else { - self::setEnvironmentVariable($app, $key, $value); + $repository->set($key, $value); } }; } diff --git a/src/testbench/src/Concerns/CreatesApplication.php b/src/testbench/src/Concerns/CreatesApplication.php index 3783637e13..e562963800 100644 --- a/src/testbench/src/Concerns/CreatesApplication.php +++ b/src/testbench/src/Concerns/CreatesApplication.php @@ -417,7 +417,7 @@ protected function resolveApplicationEnvironmentVariables(ApplicationContract $a if ($this instanceof PHPUnitTestCase && method_exists($this, 'beforeApplicationDestroyed')) { /* @phpstan-ignore function.alreadyNarrowedType */ $this->beforeApplicationDestroyed(static function () use ($attributeCallbacks) { - $attributeCallbacks->handle(); + $attributeCallbacks->reverse()->handle(); }); } } @@ -444,7 +444,6 @@ protected function maskInheritedApplicationEnvironment(): Closure unset($_SERVER['APP_ENV'], $_ENV['APP_ENV']); putenv('APP_ENV'); - Env::flushRepository(); return static function () use ($originalServerValue, $originalEnvironmentValue, $originalProcessValue): void { if ($originalServerValue instanceof UndefinedValue) { @@ -464,8 +463,6 @@ protected function maskInheritedApplicationEnvironment(): Closure } else { putenv("APP_ENV={$originalProcessValue}"); } - - Env::flushRepository(); }; } diff --git a/src/testbench/src/Foundation/Application.php b/src/testbench/src/Foundation/Application.php index e61376157d..d5943f8171 100644 --- a/src/testbench/src/Foundation/Application.php +++ b/src/testbench/src/Foundation/Application.php @@ -261,7 +261,6 @@ protected function maskInheritedStandaloneEnvironment(): Closure unset($_SERVER['APP_ENV'], $_ENV['APP_ENV']); putenv('APP_ENV'); - Env::flushRepository(); return static function () use ($originalServerValue, $originalEnvironmentValue, $originalProcessValue): void { if ($originalServerValue instanceof UndefinedValue) { @@ -281,8 +280,6 @@ protected function maskInheritedStandaloneEnvironment(): Closure } else { putenv("APP_ENV={$originalProcessValue}"); } - - Env::flushRepository(); }; } diff --git a/src/testing/src/PHPUnit/AfterEachTestSubscriber.php b/src/testing/src/PHPUnit/AfterEachTestSubscriber.php index 62a71b10bc..33b3a8a00e 100644 --- a/src/testing/src/PHPUnit/AfterEachTestSubscriber.php +++ b/src/testing/src/PHPUnit/AfterEachTestSubscriber.php @@ -261,6 +261,7 @@ protected function flushFrameworkState(): void \Hypervel\Routing\SortedMiddleware::flushCache(); \Hypervel\Routing\UrlGenerator::flushState(); \Hypervel\Server\ServerManager::flushState(); + \Hypervel\Server\ResponseCancellation::flushState(); \Hypervel\ServerProcess\ProcessCollector::flushState(); \Hypervel\ServerProcess\ProcessManager::flushState(); \Hypervel\Session\Middleware\AuthenticateSession::flushState(); diff --git a/src/view/src/Compilers/BladeCompiler.php b/src/view/src/Compilers/BladeCompiler.php index 93db3f772e..9f8282fc74 100644 --- a/src/view/src/Compilers/BladeCompiler.php +++ b/src/view/src/Compilers/BladeCompiler.php @@ -236,11 +236,7 @@ protected function getOpenAndClosingPhpTokens(string $contents): Collection */ public function getPath(): string { - if (CoroutineContext::has(static::PATH_CONTEXT_KEY)) { - return CoroutineContext::get(static::PATH_CONTEXT_KEY, ''); - } - - return $this->path; + return CoroutineContext::get(static::PATH_CONTEXT_KEY) ?? $this->path; } /** @@ -1053,7 +1049,6 @@ public function precompiler(callable $precompiler): void */ public function usingEchoFormat(string $format, callable $callback): string { - $hadOverride = CoroutineContext::has(static::ECHO_FORMAT_CONTEXT_KEY); $previous = CoroutineContext::get(static::ECHO_FORMAT_CONTEXT_KEY); CoroutineContext::set(static::ECHO_FORMAT_CONTEXT_KEY, $format); @@ -1061,7 +1056,7 @@ public function usingEchoFormat(string $format, callable $callback): string try { return call_user_func($callback); } finally { - if ($hadOverride) { + if ($previous !== null) { CoroutineContext::set(static::ECHO_FORMAT_CONTEXT_KEY, $previous); } else { CoroutineContext::forget(static::ECHO_FORMAT_CONTEXT_KEY); diff --git a/src/watcher/config/watcher.php b/src/watcher/config/watcher.php index a9a96d6f2b..2731ec7bef 100644 --- a/src/watcher/config/watcher.php +++ b/src/watcher/config/watcher.php @@ -48,6 +48,7 @@ 'watch' => [ 'app/**/*.php', 'config/**/*.php', + 'resources/skills/**', '.env', ], diff --git a/src/watcher/src/Driver/FswatchDriver.php b/src/watcher/src/Driver/FswatchDriver.php index 37666f5fcb..3b42f84524 100644 --- a/src/watcher/src/Driver/FswatchDriver.php +++ b/src/watcher/src/Driver/FswatchDriver.php @@ -128,18 +128,20 @@ protected function processOutput( $matched = false; foreach ($watchTargets as $watchTarget) { - if (! str_starts_with($file, $watchTarget['prefix'])) { + if ($file === rtrim($watchTarget['prefix'], '/')) { + $relativePath = $watchTarget['base']; + } elseif (str_starts_with($file, $watchTarget['prefix'])) { + $remainder = substr($file, strlen($watchTarget['prefix'])); + $relativePath = $watchTarget['base'] === '.' + ? $remainder + : $watchTarget['base'] . '/' . $remainder; + } else { continue; } - $remainder = substr($file, strlen($watchTarget['prefix'])); - $relativePath = $watchTarget['base'] === '.' - ? $remainder - : $watchTarget['base'] . '/' . $remainder; - // A recursive operand can observe a path whose configured base did not exist at startup. foreach ($watchPaths as $watchPath) { - if ($watchPath->matches($relativePath)) { + if ($watchPath->matches($relativePath) || $this->matchesDirectoryChange($watchPath, $relativePath, $file)) { $matched = true; break 2; } @@ -156,6 +158,28 @@ protected function processOutput( } } + /** + * Match directory arrivals that may contain files omitted from native events. + */ + protected function matchesDirectoryChange(WatchPath $watchPath, string $relativePath, string $file): bool + { + $root = $watchPath->type === WatchPathType::File + ? dirname($watchPath->path) + : (rtrim($watchPath->path, '/') ?: '.'); + + if ( + $relativePath !== $root + && ! str_starts_with($root, $relativePath . '/') + && ! ($watchPath->recursive && ($root === '.' || str_starts_with($relativePath, $root . '/'))) + ) { + return false; + } + + clearstatcache(true, $file); + + return is_dir($file); + } + /** * Stop the active fswatch processes. */ @@ -262,25 +286,42 @@ protected function resolveWatchTargets(array $watchPaths): array ? dirname($watchPath->path) : rtrim($watchPath->path, '/'); $base = $base === '' ? '.' : $base; - $literalPath = $base === '.' ? base_path() : base_path($base); - $canonicalPath = realpath($literalPath); - // A missing root keeps one literal operand/prefix because fswatch may activate it - // as a symlink that a recursive parent process does not follow. - $operand = $canonicalPath === false ? $literalPath : $canonicalPath; - - if (! isset($targets[$operand])) { - $targets[$operand] = [ - 'operand' => $operand, - 'recursive' => false, - 'canonical' => $canonicalPath === false ? null : $canonicalPath, - ]; + $bases = [$base]; + + if (! $this->isDarwin() && ($watchPath->type === WatchPathType::Directory || ! is_dir(base_path($base)))) { + // Shallow ancestor watches report root arrivals without recursively watching the project. + $ancestor = $base; + + while (($parent = dirname($ancestor)) !== $ancestor) { + $bases[] = $parent; + $ancestor = $parent; + + if (is_dir(base_path($ancestor))) { + break; + } + } } - $targets[$operand]['recursive'] = $targets[$operand]['recursive'] || $watchPath->recursive; + foreach ($bases as $index => $targetBase) { + $literalPath = $targetBase === '.' ? base_path() : base_path($targetBase); + $canonicalPath = realpath($literalPath); + // Keep missing roots as explicit operands for activation through symlinked ancestors. + $operand = $canonicalPath === false ? $literalPath : $canonicalPath; + + if (! isset($targets[$operand])) { + $targets[$operand] = [ + 'operand' => $operand, + 'recursive' => false, + 'canonical' => $canonicalPath === false ? null : $canonicalPath, + ]; + } + + $targets[$operand]['recursive'] = $targets[$operand]['recursive'] || ($index === 0 && $watchPath->recursive); - $prefix = rtrim($operand, '/') . '/'; - $entryKey = $prefix . "\0" . $base; - $entries[$entryKey] ??= ['prefix' => $prefix, 'base' => $base]; + $prefix = rtrim($operand, '/') . '/'; + $entryKey = $prefix . "\0" . $targetBase; + $entries[$entryKey] ??= ['prefix' => $prefix, 'base' => $targetBase]; + } } return [ diff --git a/src/websocket-server/src/Server.php b/src/websocket-server/src/Server.php index 92ac47ee53..fdd1f388d9 100644 --- a/src/websocket-server/src/Server.php +++ b/src/websocket-server/src/Server.php @@ -199,7 +199,7 @@ public function onHandshake(Request $request, SwooleResponse $response): void } try { - ResponseBridge::send($httpResponse, $response, request: $httpRequest); + ResponseBridge::send($httpResponse, $response, request: $httpRequest, streamId: $request->streamId ?? 0); } catch (CanceledException $exception) { throw $exception; } catch (Throwable $throwable) { diff --git a/tests/Benchmarks/Database/README.md b/tests/Benchmarks/Database/README.md new file mode 100644 index 0000000000..ffc0bb866b --- /dev/null +++ b/tests/Benchmarks/Database/README.md @@ -0,0 +1,39 @@ +# Database lifecycle benchmark + +This developer-only harness measures ordinary coroutine-owned database lifecycles: initial connection resolution, queries, transactions, and deferred pool cleanup. File-backed SQLite isolates framework CPU and allocation costs without network latency; it does not predict a database server's throughput. It uses the real pool and resolver with Testbench pooling enabled. + +Run only during an owner-confirmed idle window. Run the same harness in independently installed baseline and modified worktrees, using the same dependency versions, PHP configuration and machine. Record each revision and any uncommitted changes with the reports. Alternate the order of baseline and modified runs to reduce ordering bias. + +```shell +php tests/Benchmarks/Database/benchmark.php --concurrency=1 --queries=0 > /tmp/db-resolution.json +php tests/Benchmarks/Database/benchmark.php --concurrency=1 --queries=1 > /tmp/db-first-query.json +php tests/Benchmarks/Database/benchmark.php --concurrency=1 --queries=10 > /tmp/db-queries.json +php tests/Benchmarks/Database/benchmark.php --concurrency=32 --queries=10 --wait-us=1000 > /tmp/db-concurrent.json +php tests/Benchmarks/Database/benchmark.php --concurrency=32 --queries=10 --wait-us=1000 --transaction > /tmp/db-transactions.json +``` + +The short simulated external wait lets requests overlap and hold real pool slots. Without a wait, SQLite may complete each coroutine before the next starts; use the reported peak active requests and borrowed slots rather than treating configured concurrency as observed concurrency. Compare CPU per request as well as wall-clock throughput when waits are enabled. + +On a checkout supporting early release, add `--release` to the concurrent scenario. It returns idle sessions before the wait while retaining the logical connection for subsequent queries. Combining it with `--transaction` measures the pinned path. Baselines without the API reject `--release`; ordinary lifecycle scenarios remain directly comparable. + +For contention, set `--pool-size` below concurrency and enable `--pool-metrics`: + +```shell +php -d opcache.enable_cli=1 -d opcache.jit=disable tests/Benchmarks/Database/benchmark.php --requests=128 --samples=1 --concurrency=32 --pool-size=4 --queries=2 --wait-us=50000 --pool-metrics +``` + +Compare baseline, current, current with `--release`, and current with `--release --transaction`. The optional instrumented pool records checkout and physical hold p50/p95, including reacquisitions, and the harness records complete request p50/p95. Checkout includes creation and checkout bookkeeping as well as queue waits; after warmup, queueing dominates the contended cases. Hold measurements finish before the slot is made available to another coroutine. Keep this instrumentation disabled for allocation and CPU microbenchmarks because its sample arrays add overhead. + +To verify automatic release through the real HTTP client, start the separate verified-TLS origin described in `../HttpTransport/README.md`, then replace the simulated wait with: + +```shell +php -d opcache.enable_cli=1 -d opcache.jit=disable tests/Benchmarks/Database/benchmark.php --requests=128 --samples=1 --concurrency=32 --pool-size=4 --queries=2 --pool-metrics --http-url='https://127.0.0.1:PORT/?chunks=1&bytes=128&pace_us=0&delay_us=50000' --http-ca=/tmp/hypervel-http-benchmark/ca.pem +``` + +`--http-url` replaces `--wait-us`. Compare the unchanged baseline with the current default and with `--transaction`; do not use `--release` when measuring the automatic boundary. The origin delays its response by 50 ms while the caller either retains or releases its database session. TLS and HTTP processing add cost, so keep these results separate from simulated-wait measurements. + +Each scenario warms the pool, then reports repeated samples. Collection runs only between samples, never within the measured request loop. Reports include CPU time, resolution/query/deferred-cleanup timing, peak heap and ending heap growth, GC runs/collected objects/roots/collector time, and pool ownership after completion. GC roots are sampled after each bounded batch; peak heap captures growth between collections. Memory figures describe the worker, not independent per-request peaks. Harness scheduling and measurement overhead are present in both versions. + +For comparisons, invoke one measured sample per process and alternate baseline/current order within each scenario for at least ten pairs. Report median and spread. Enable CLI opcache for both, and use identical JIT and GC settings; the report includes those settings. The warmup precedes the measured sample in each process. + +Use `--requests`, `--samples`, `--concurrency`, `--queries`, `--wait-us` and `--pool-size` to vary workload. Keep raw reports outside the repository. Do not use throughput thresholds as CI assertions or add production GC calls to improve benchmark results. diff --git a/tests/Benchmarks/Database/benchmark.php b/tests/Benchmarks/Database/benchmark.php new file mode 100644 index 0000000000..9ab93cb34b --- /dev/null +++ b/tests/Benchmarks/Database/benchmark.php @@ -0,0 +1,305 @@ +#!/usr/bin/env php +acquisitions[] = ($acquired - $started) / 1e6; + $this->borrowedAt[spl_object_id($connection)] = $acquired; + + return $connection; + } + + /** + * Record the physical hold before returning the slot to waiting coroutines. + */ + public function release(PoolConnection $connection): void + { + $identity = spl_object_id($connection); + $this->holds[] = (hrtime(true) - $this->borrowedAt[$identity]) / 1e6; + unset($this->borrowedAt[$identity]); + parent::release($connection); + } +} + +/** + * Summarize measured milliseconds without retaining per-request reports. + */ +function percentiles(array $values): array +{ + sort($values, SORT_NUMERIC); + + return [ + 'p50' => $values[(int) ceil(count($values) * 0.5) - 1], + 'p95' => $values[(int) ceil(count($values) * 0.95) - 1], + ]; +} + +$options = getopt('', ['requests:', 'concurrency:', 'queries:', 'samples:', 'wait-us:', 'http-url:', 'http-ca:', 'pool-size:', 'pool-metrics', 'release', 'transaction', 'help']); + +if (isset($options['help'])) { + echo "Usage: php tests/Benchmarks/Database/benchmark.php [options]\n" + . " --requests=10000 --concurrency=32 --queries=10 --samples=7 --wait-us=0\n" + . " --pool-size=N Default: concurrency; use a smaller pool to measure contention\n" + . " --pool-metrics Record checkout, hold and request latency percentiles\n" + . " --http-url=URL Replace the simulated wait with a real HTTP request\n" + . " --http-ca=PATH Trusted local CA for the HTTP origin\n" + . " --release Release idle sessions before the simulated external wait\n" + . " --transaction Wrap each request's queries in a transaction\n" + . "Run measurements only in an owner-confirmed idle window. Output is JSON.\n"; + exit(0); +} + +$settings = ['requests' => 10000, 'concurrency' => 32, 'queries' => 10, 'samples' => 7, 'wait-us' => 0]; + +foreach ($settings as $name => $default) { + $value = filter_var($options[$name] ?? $default, FILTER_VALIDATE_INT); + $minimum = in_array($name, ['queries', 'wait-us'], true) ? 0 : 1; + + if ($value === false || $value < $minimum) { + throw new InvalidArgumentException("--{$name} must be an integer of at least {$minimum}."); + } + + $settings[$name] = $value; +} + +$release = isset($options['release']); +$transaction = isset($options['transaction']); +$poolMetrics = isset($options['pool-metrics']); +$httpUrl = $options['http-url'] ?? null; +$httpCertificate = $options['http-ca'] ?? true; +$settings['pool-size'] = filter_var($options['pool-size'] ?? $settings['concurrency'], FILTER_VALIDATE_INT); + +if ($settings['pool-size'] === false || $settings['pool-size'] < 1) { + throw new InvalidArgumentException('--pool-size must be an integer of at least 1.'); +} + +if ($release && ! method_exists(DatabaseManager::class, 'releaseIdleConnections')) { + throw new RuntimeException('This checkout does not support early release; compare ordinary lifecycle scenarios against it.'); +} + +Bootstrapper::bootstrap(); +$databasePath = tempnam(sys_get_temp_dir(), 'hypervel-database-benchmark-'); + +if ($databasePath === false) { + throw new RuntimeException('Cannot create the benchmark database.'); +} + +$application = null; + +try { + $application = Application::create(options: ['load_environment_variables' => false]); + $application->make('config')->set('database.connections.benchmark', [ + 'driver' => 'sqlite', + 'database' => $databasePath, + 'prefix' => '', + 'foreign_key_constraints' => true, + 'pool' => [ + 'testing_enabled' => true, + 'min_retained_connections' => 0, + 'max_connections' => $settings['pool-size'], + 'heartbeat_interval' => null, + 'idle_check_interval' => null, + ], + ]); + + if ($poolMetrics) { + $application->bind(DatabasePool::class, MeasuredDatabasePool::class); + } + + $database = $application->make(DatabaseManager::class); + $http = $httpUrl === null ? null : (new Factory)->registerConnection('benchmark'); + $pool = $application->make(PoolManager::class)->pool('benchmark'); + $reports = []; + $failure = null; + + run(function () use ($database, $http, $httpUrl, $httpCertificate, $pool, $settings, $release, $transaction, $poolMetrics, &$reports, &$failure): void { + try { + for ($sample = -1; $sample < $settings['samples']; ++$sample) { + if ($pool instanceof MeasuredDatabasePool) { + $pool->acquisitions = $pool->holds = []; + } + + $latencies = []; + // Collect between samples, never inside the measured request loop. + gc_collect_cycles(); + memory_reset_peak_usage(); + $gcBefore = gc_status(); + $heapBefore = memory_get_usage(); + $cpuBefore = getrusage(); + $started = hrtime(true); + $active = 0; + $peakActive = 0; + $peakBorrowed = 0; + $peakRoots = $gcBefore['roots']; + $resolutionNanoseconds = 0; + $queryNanoseconds = 0; + $cleanupNanoseconds = 0; + $requests = $sample < 0 ? max(100, $settings['concurrency']) : $settings['requests']; + + $request = function () use ( + $database, + $http, + $httpUrl, + $httpCertificate, + $pool, + $settings, + $release, + $transaction, + $poolMetrics, + &$latencies, + &$active, + &$peakActive, + &$peakBorrowed, + &$resolutionNanoseconds, + &$queryNanoseconds, + &$cleanupNanoseconds, + ): void { + ++$active; + $peakActive = max($peakActive, $active); + $requestStarted = hrtime(true); + $cleanupStarted = 0; + // Registered first, so database release runs before this measurement. + Coroutine::defer(static function () use (&$active, &$cleanupStarted, &$cleanupNanoseconds, $requestStarted, $poolMetrics, &$latencies): void { + $cleanupNanoseconds += hrtime(true) - $cleanupStarted; + + if ($poolMetrics) { + $latencies[] = (hrtime(true) - $requestStarted) / 1e6; + } + + --$active; + }); + + try { + $resolving = hrtime(true); + $connection = $database->connection('benchmark'); + $resolutionNanoseconds += hrtime(true) - $resolving; + $peakBorrowed = max($peakBorrowed, $pool->getBorrowedCount()); + + if ($transaction) { + $connection->beginTransaction(); + } + + for ($query = 0; $query < max(1, $settings['queries']); ++$query) { + if ($settings['queries'] > 0) { + $queryStarted = hrtime(true); + $connection->selectOne('select 1 as value'); + $queryNanoseconds += hrtime(true) - $queryStarted; + } + + if ($query === 0) { + if ($release) { + $database->releaseIdleConnections(); + } + + if ($http !== null) { + $http->connection('benchmark')->withOptions([ + 'verify' => $httpCertificate, + 'proxy' => '', + 'timeout' => 5, + ])->get($httpUrl)->throw()->body(); + } elseif ($settings['wait-us'] > 0) { + usleep($settings['wait-us']); + } + } + } + + if ($transaction) { + $connection->commit(); + } + } finally { + $cleanupStarted = hrtime(true); + } + }; + + for ($completed = 0; $completed < $requests; $completed += $settings['concurrency']) { + parallel(array_fill(0, min($settings['concurrency'], $requests - $completed), $request)); + $peakRoots = max($peakRoots, gc_status()['roots']); + } + + $elapsed = hrtime(true) - $started; + $cpuAfter = getrusage(); + $gcAfter = gc_status(); + + if ($sample >= 0) { + $reports[] = [ + 'requests_per_second' => $requests * 1e9 / $elapsed, + 'cpu_microseconds_per_request' => ( + ($cpuAfter['ru_utime.tv_sec'] + $cpuAfter['ru_stime.tv_sec'] - $cpuBefore['ru_utime.tv_sec'] - $cpuBefore['ru_stime.tv_sec']) * 1e6 + + $cpuAfter['ru_utime.tv_usec'] + $cpuAfter['ru_stime.tv_usec'] - $cpuBefore['ru_utime.tv_usec'] - $cpuBefore['ru_stime.tv_usec'] + ) / $requests, + 'resolution_ns_per_request' => $resolutionNanoseconds / $requests, + 'query_ns_per_request' => $queryNanoseconds / $requests, + 'deferred_cleanup_ns_per_request' => $cleanupNanoseconds / $requests, + 'peak_active_requests' => $peakActive, + 'peak_borrowed_slots' => $peakBorrowed, + 'borrowed_slots_after_sample' => $pool->getBorrowedCount(), + 'heap_growth_bytes' => memory_get_usage() - $heapBefore, + 'peak_heap_bytes' => memory_get_peak_usage(), + 'gc_runs' => $gcAfter['runs'] - $gcBefore['runs'], + 'gc_collected' => $gcAfter['collected'] - $gcBefore['collected'], + 'gc_roots' => $gcAfter['roots'], + 'peak_sampled_gc_roots' => $peakRoots, + 'gc_collector_seconds' => $gcAfter['collector_time'] - $gcBefore['collector_time'], + ] + ($pool instanceof MeasuredDatabasePool ? [ + 'checkout_ms' => percentiles($pool->acquisitions), + 'physical_hold_ms' => percentiles($pool->holds), + 'request_ms' => percentiles($latencies), + ] : []); + } + } + } catch (Throwable $exception) { + $failure = $exception; + } + }); + + if ($failure !== null) { + throw $failure; + } + + echo json_encode([ + 'php' => PHP_VERSION, + 'swoole' => swoole_version(), + 'opcache_cli' => ini_get('opcache.enable_cli'), + 'jit' => ini_get('opcache.jit'), + 'gc_enabled' => gc_enabled(), + 'settings' => $settings + ['release' => $release, 'transaction' => $transaction, 'pool-metrics' => $poolMetrics, 'http-url' => $httpUrl, 'http-ca' => $httpCertificate], + 'samples' => $reports, + ], JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR), PHP_EOL; +} finally { + try { + $application?->make(PoolManager::class)->purgeAll(); + $application?->terminate(); + } finally { + unlink($databasePath); + } +} diff --git a/tests/Benchmarks/HttpTransport/Fixtures/server.php b/tests/Benchmarks/HttpTransport/Fixtures/server.php new file mode 100644 index 0000000000..f674e1ddd2 --- /dev/null +++ b/tests/Benchmarks/HttpTransport/Fixtures/server.php @@ -0,0 +1,63 @@ +set([ + 'worker_num' => 1, + 'enable_coroutine' => true, + 'ssl_cert_file' => $argv[1], + 'ssl_key_file' => $argv[2], + 'log_level' => SWOOLE_LOG_WARNING, +]); +$connections = []; +$accepted = 0; +$active = 0; +$server->on('workerStart', static function (Server $server): void { + echo 'READY ' . $server->ports[0]->port . "\n"; +}); +$server->on('connect', static function (Server $server, int $connection) use (&$connections, &$accepted): void { + $connections[$connection] = ++$accepted; +}); +$server->on('close', static function (Server $server, int $connection) use (&$connections): void { + unset($connections[$connection]); +}); +$server->on('request', static function (Request $request, Response $response) use (&$connections, &$active): void { + ++$active; + + try { + $chunks = (int) $request->get['chunks']; + $bytes = (int) $request->get['bytes']; + $pace = (int) $request->get['pace_us']; + $delay = (int) ($request->get['delay_us'] ?? 0); + + if ($delay > 0) { + Coroutine::sleep($delay / 1e6); + } + + $response->header('Content-Type', 'text/event-stream'); + $response->header('X-Benchmark-Connection', (string) $connections[$request->fd]); + $response->header('X-Benchmark-Active', (string) $active); + $chunk = 'data: ' . str_repeat('x', $bytes - 8) . "\n\n"; + + for ($index = 0; $index < $chunks; ++$index) { + if (! $response->write($chunk)) { + return; + } + + if ($pace > 0 && $index + 1 < $chunks) { + Coroutine::sleep($pace / 1e6); + } + } + + $response->end(); + } finally { + --$active; + } +}); +$server->start(); diff --git a/tests/Benchmarks/HttpTransport/README.md b/tests/Benchmarks/HttpTransport/README.md new file mode 100644 index 0000000000..d5b77dfe62 --- /dev/null +++ b/tests/Benchmarks/HttpTransport/README.md @@ -0,0 +1,46 @@ +# HTTP transport benchmark + +This developer harness compares the public HTTP client against a separate local HTTPS origin. TLS verification stays enabled. It measures cold startup and repeated bursts, named connection reuse, unnamed streams, buffered requests, incremental event delivery and slow consumers. It is not a CI performance test or a prediction of remote provider throughput. + +Run only in an owner-confirmed idle window. Use independently installed baseline and modified worktrees with identical dependencies, PHP settings, opcache, JIT and GC. Copy this harness into an older checkout when necessary; do not change its production source. + +Create a temporary certificate trusted only by the benchmark client: + +```shell +mkdir -p /tmp/hypervel-http-benchmark +openssl req -x509 -newkey rsa:2048 -nodes \ + -keyout /tmp/hypervel-http-benchmark/key.pem \ + -out /tmp/hypervel-http-benchmark/ca.pem -days 2 \ + -subj '/CN=127.0.0.1' -addext 'subjectAltName=IP:127.0.0.1' +php tests/Benchmarks/HttpTransport/Fixtures/server.php \ + /tmp/hypervel-http-benchmark/ca.pem /tmp/hypervel-http-benchmark/key.pem +``` + +The origin listens on an automatically assigned loopback port and prints `READY `. Keep it running in a separate terminal, then use that port below. Stop it when measurements finish and remove the temporary certificate and key. + +```shell +php -d opcache.enable_cli=1 -d opcache.jit=disable \ + tests/Benchmarks/HttpTransport/benchmark.php \ + --endpoint=https://127.0.0.1:PORT --ca=/tmp/hypervel-http-benchmark/ca.pem \ + --requests=256 --samples=1 --concurrency=8 > /tmp/http-benchmark.json +``` + +Alternate baseline/current order within each scenario for at least ten pairs and report median and spread. Each process reports its initial cold burst separately; that burst includes lazy PHP initialization as well as new TLS connections. Measured samples follow that warmup. Throughput includes the configured gap between bursts. CPU measures the client process, excluding the separate origin. + +Useful scenarios: + +| Scenario | Options | +|---|---| +| Paced events | Defaults: 32 events of 128 bytes, 1 ms between events, 5 ms between bursts | +| Unpaced response | `--pace-us=0` | +| Serial warm reuse | `--concurrency=1 --pace-us=0` | +| Operation-owned transport | `--unnamed` | +| Ordinary buffered request | `--buffered` | +| Slow consumer, 1 MiB per response | `--chunks=256 --bytes=4096 --pace-us=0 --consumer-us=1000` | +| Sustained high concurrency | `--concurrency=128 --requests=512 --samples=10` | + +The default reader is `Response::lines()`. On older StreamHandler implementations it can withhold paced events until the response finishes. Compare that API behavior directly, but do not describe its lower CPU as equivalent incremental delivery. `--bytewise` reproduces the byte-at-a-time reader used by applications to avoid that batching; compare baseline bytewise delivery with current `lines()` separately. The origin emits complete ASCII SSE records, and the harness checks their count and length. + +Reports include start-to-first-event and complete-request p50/p95, client CPU, throughput, actual peak client and origin requests, and distinct/new connection IDs assigned by the origin at accept time. Named streaming transfers can retain 32 idle transports, so larger bursts still create new connections; this does not cap active requests. Buffered requests use Guzzle's separate idle easy-handle cache, whose synchronous default is three handles, not three physical connections. Per-request credentials change to exercise transport reuse without retaining a single credential identity. + +Linux `/proc` supplies current RSS and open descriptors before and after each sample and after explicit handler cleanup. On the measured Linux build, each idle streaming transport retains three descriptors: a connection socket and both ends of a wakeup pipe. Include this cost when comparing retention bounds and sizing worker descriptor limits. Heap peaks and GC counts cover the client. The latency arrays and cumulative set of observed connection IDs also consume memory; account for that bookkeeping when comparing samples rather than treating total process memory as transport-only allocation. No collection runs inside the measured request loop. Keep raw reports outside the repository, and do not turn timing values into test assertions. diff --git a/tests/Benchmarks/HttpTransport/benchmark.php b/tests/Benchmarks/HttpTransport/benchmark.php new file mode 100644 index 0000000000..66e8637741 --- /dev/null +++ b/tests/Benchmarks/HttpTransport/benchmark.php @@ -0,0 +1,239 @@ +#!/usr/bin/env php + 256, 'concurrency' => 8, 'samples' => 5, 'chunks' => 32, 'bytes' => 128, 'pace-us' => 1000, 'burst-gap-us' => 5000, 'consumer-us' => 0]; + +foreach ($settings as $name => $default) { + $value = filter_var($options[$name] ?? $default, FILTER_VALIDATE_INT); + $minimum = str_ends_with($name, '-us') ? 0 : ($name === 'bytes' ? 8 : 1); + + if ($value === false || $value < $minimum) { + throw new InvalidArgumentException("--{$name} must be an integer of at least {$minimum}."); + } + + $settings[$name] = $value; +} + +/** + * Return latency percentiles in milliseconds. + */ +function percentiles(array $values): array +{ + sort($values, SORT_NUMERIC); + + return [ + 'p50' => $values[(int) ceil(count($values) * 0.5) - 1], + 'p95' => $values[(int) ceil(count($values) * 0.95) - 1], + ]; +} + +/** + * Read current Linux process resources without including the origin process. + */ +function resources(): array +{ + $status = file_get_contents('/proc/self/status'); + preg_match('/^VmRSS:\s+(\d+) kB$/m', $status, $resident); + + return [ + 'heap_bytes' => memory_get_usage(), + 'rss_bytes' => (int) $resident[1] * 1024, + 'open_fds' => count(scandir('/proc/self/fd')) - 2, + ]; +} + +/** + * Reproduce the byte-at-a-time reader used to avoid StreamHandler batching. + */ +function bytewiseLines(Response $response): Generator +{ + $stream = $response->toPsrResponse()->getBody(); + $line = ''; + + while (! $stream->eof()) { + $byte = $stream->read(1); + + if ($byte === "\n") { + yield $line; + $line = ''; + } else { + $line .= $byte; + } + } + + if ($line !== '') { + yield $line; + } +} + +$factory = (new Factory)->registerConnection('benchmark'); +$named = ! isset($options['unnamed']); +$buffered = isset($options['buffered']); +$bytewise = isset($options['bytewise']); +$seenConnections = []; +$reports = []; +$cold = null; +$failure = null; +$before = resources(); + +run(function () use ($factory, $endpoint, $certificate, $settings, $named, $buffered, $bytewise, &$seenConnections, &$reports, &$cold, &$failure): void { + try { + for ($sample = -1; $sample < $settings['samples']; ++$sample) { + gc_collect_cycles(); + memory_reset_peak_usage(); + $before = resources(); + $cpuBefore = getrusage(); + $gcBefore = gc_status(); + $started = hrtime(true); + $active = 0; + $peakActive = 0; + $peakOriginActive = 0; + $connections = []; + $newConnections = 0; + $firstEvents = []; + $latencies = []; + $requests = $sample < 0 ? $settings['concurrency'] : $settings['requests']; + + $request = function () use ($factory, $endpoint, $certificate, $settings, $named, $buffered, $bytewise, &$active, &$peakActive, &$peakOriginActive, &$connections, &$newConnections, &$seenConnections, &$firstEvents, &$latencies): void { + $started = hrtime(true); + ++$active; + $peakActive = max($peakActive, $active); + $response = null; + + try { + $pending = $named ? $factory->connection('benchmark') : $factory->withOptions([]); + $response = $pending->withToken(bin2hex(random_bytes(16)))->withOptions([ + 'stream' => ! $buffered, + 'verify' => $certificate, + 'proxy' => '', + 'version' => '1.1', + 'timeout' => 10, + 'read_timeout' => 10, + ])->get($endpoint, [ + 'chunks' => $settings['chunks'], + 'bytes' => $settings['bytes'], + 'pace_us' => $settings['pace-us'], + ])->throw(); + $identity = $response->header('X-Benchmark-Connection'); + $connections[$identity] = true; + $newConnections += (int) ! isset($seenConnections[$identity]); + $seenConnections[$identity] = true; + $peakOriginActive = max($peakOriginActive, (int) $response->header('X-Benchmark-Active')); + $events = 0; + + foreach ($bytewise ? bytewiseLines($response) : $response->lines() as $line) { + if ($line === '') { + continue; + } + + if ($events++ === 0) { + $firstEvents[] = (hrtime(true) - $started) / 1e6; + } + + if (strlen($line) !== $settings['bytes'] - 2) { + throw new RuntimeException('The origin returned an incomplete event.'); + } + + if ($settings['consumer-us'] > 0) { + usleep($settings['consumer-us']); + } + } + + if ($events !== $settings['chunks']) { + throw new RuntimeException('The origin returned an unexpected event count.'); + } + } finally { + $response?->close(); + --$active; + $latencies[] = (hrtime(true) - $started) / 1e6; + } + }; + + for ($completed = 0; $completed < $requests; $completed += $settings['concurrency']) { + parallel(array_fill(0, min($settings['concurrency'], $requests - $completed), $request)); + + if ($settings['burst-gap-us'] > 0 && $completed + $settings['concurrency'] < $requests) { + usleep($settings['burst-gap-us']); + } + } + + $elapsed = hrtime(true) - $started; + $cpuAfter = getrusage(); + $gcAfter = gc_status(); + $report = [ + 'requests_per_second' => $requests * 1e9 / $elapsed, + 'cpu_microseconds_per_request' => ( + ($cpuAfter['ru_utime.tv_sec'] + $cpuAfter['ru_stime.tv_sec'] - $cpuBefore['ru_utime.tv_sec'] - $cpuBefore['ru_stime.tv_sec']) * 1e6 + + $cpuAfter['ru_utime.tv_usec'] + $cpuAfter['ru_stime.tv_usec'] - $cpuBefore['ru_utime.tv_usec'] - $cpuBefore['ru_stime.tv_usec'] + ) / $requests, + 'first_event_ms' => percentiles($firstEvents), + 'request_ms' => percentiles($latencies), + 'peak_active_requests' => $peakActive, + 'peak_origin_requests' => $peakOriginActive, + 'origin_connections' => count($connections), + 'new_origin_connections' => $newConnections, + 'resources_before' => $before, + 'resources_after' => resources(), + 'peak_heap_bytes' => memory_get_peak_usage(), + 'gc_runs' => $gcAfter['runs'] - $gcBefore['runs'], + 'gc_collected' => $gcAfter['collected'] - $gcBefore['collected'], + 'gc_roots' => $gcAfter['roots'], + ]; + + if ($sample < 0) { + $cold = $report; + } else { + $reports[] = $report; + } + } + } catch (Throwable $exception) { + $failure = $exception; + } +}, SWOOLE_HOOK_ALL); + +$factory->forgetConnectionHandlers(); +gc_collect_cycles(); + +if ($failure !== null) { + throw $failure; +} + +echo json_encode([ + 'php' => PHP_VERSION, + 'swoole' => swoole_version(), + 'opcache_cli' => ini_get('opcache.enable_cli'), + 'jit' => ini_get('opcache.jit'), + 'gc_enabled' => gc_enabled(), + 'settings' => $settings + ['named' => $named, 'buffered' => $buffered, 'bytewise' => $bytewise], + 'cold' => $cold, + 'samples' => $reports, + 'resources_before' => $before, + 'resources_after_cleanup' => resources(), +], JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR), PHP_EOL; diff --git a/tests/Cache/Fixtures/ArrayFilesystem.php b/tests/Cache/Fixtures/ArrayFilesystem.php index cf377b303e..744353003e 100644 --- a/tests/Cache/Fixtures/ArrayFilesystem.php +++ b/tests/Cache/Fixtures/ArrayFilesystem.php @@ -23,7 +23,23 @@ public function path(string $path): string public function exists(string $path): bool { - return array_key_exists($path, $this->files) || $this->files($path) !== []; + return $this->fileExists($path) || $this->directoryExists($path); + } + + /** + * Determine if a file exists. + */ + public function fileExists(string $path): bool + { + return array_key_exists($path, $this->files); + } + + /** + * Determine if a directory exists. + */ + public function directoryExists(string $path): bool + { + return $this->files($path) !== []; } public function get(string $path): ?string diff --git a/tests/Database/ConnectionResolverTest.php b/tests/Database/ConnectionResolverTest.php index d5b62009bd..9a58e178a7 100644 --- a/tests/Database/ConnectionResolverTest.php +++ b/tests/Database/ConnectionResolverTest.php @@ -129,7 +129,7 @@ public function testNestedOverrideRestoresExactPriorValue(): void public function testNonCoroutineConnectionIsRetainedUntilTerminalRelease(): void { $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $firstWrapper = m::mock(PooledConnection::class); $secondWrapper = m::mock(PooledConnection::class); $firstConnection = m::mock(Connection::class); @@ -163,7 +163,7 @@ public function testTerminalReleaseOwnsEveryRequestedConnectionRole(): void $resolver = $this->makeResolver('mysql', $poolManager); foreach (['mysql' => null, 'mysql::read' => 'read', 'mysql::write' => 'write'] as $name => $role) { - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $connection = m::mock(Connection::class); @@ -191,7 +191,7 @@ public function testTerminalReleaseOwnsEveryRequestedConnectionRole(): void public function testBorrowedConnectionRetainsItsRequestedRole(): void { $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $resolver = $this->makeResolver('sqlite', $poolManager); $pool->allows('getSharedInMemorySqlitePdo')->andReturnNull(); @@ -214,7 +214,7 @@ public function testBorrowedConnectionRetainsItsRequestedRole(): void public function testSharedInMemorySqliteAliasesReuseOneConnectionOwner(): void { $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $connection = m::mock(Connection::class); @@ -237,10 +237,10 @@ public function testSharedInMemorySqliteAliasesReuseOneConnectionOwner(): void $resolver->releaseConnections(); } - public function testTerminalStateIsDetachedBeforeReleaseCanReenterTheResolver(): void + public function testTerminalStateRemainsAvailableUntilItsOwnerIsReleased(): void { $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $firstWrapper = m::mock(PooledConnection::class); $secondWrapper = m::mock(PooledConnection::class); $firstConnection = m::mock(Connection::class); @@ -258,14 +258,16 @@ public function testTerminalStateIsDetachedBeforeReleaseCanReenterTheResolver(): $resolver = $this->makeResolver('mysql', $poolManager); $resolver->setDefaultConnection('reporting'); - $firstWrapper->expects('release')->andReturnUsing(function () use ($resolver, $secondConnection): void { - $this->assertSame('mysql', $resolver->getDefaultConnection()); - $this->assertSame($secondConnection, $resolver->connection('mysql')); + $firstWrapper->expects('release')->andReturnUsing(function () use ($resolver, $firstConnection): void { + $this->assertSame('reporting', $resolver->getDefaultConnection()); + $this->assertSame($firstConnection, $resolver->connection('mysql')); }); $this->assertSame($firstConnection, $resolver->connection('mysql')); $resolver->releaseConnections(); + $this->assertSame('mysql', $resolver->getDefaultConnection()); + $this->assertSame($secondConnection, $resolver->connection('mysql')); $resolver->releaseConnections(); } @@ -280,7 +282,7 @@ public function testTerminalReleaseExhaustsConnectionsAndPreservesTheFirstFailur ['first', $firstException], ['second', $secondException], ] as [$name, $exception]) { - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with($name)->andReturn($pool); @@ -316,7 +318,7 @@ public function testTerminalReleasePrioritizesTheFirstCancellationAndStillExhaus ['second', $firstCancellation], ['third', $secondCancellation], ] as [$name, $failure]) { - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with($name)->andReturn($pool); @@ -345,7 +347,7 @@ public function testTerminalDiscardExhaustsExactConnections(): void $resolver = $this->makeResolver('first', $poolManager); foreach (['first', 'second'] as $name) { - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with($name)->andReturn($pool); @@ -366,7 +368,7 @@ public function testConnectionRetrievalFailureDiscardsTheExactWrapper(): void { $exception = new RuntimeException('Connection retrieval failed.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with('mysql')->andReturn($pool); @@ -389,7 +391,7 @@ public function testListenerFailureDiscardsTheExactWrapperWithoutDeferringReleas { $exception = new RuntimeException('Connection listener failed.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->with('mysql')->andReturn($pool); @@ -418,7 +420,7 @@ public function testWriteRoleConfigurationFailureDiscardsTheExactWrapper(): void { $exception = new RuntimeException('Write role configuration failed.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $connection = m::mock(Connection::class); @@ -445,7 +447,7 @@ public function testDiscardFailureDoesNotReplaceTheSetupFailure(): void $setupException = new RuntimeException('Connection retrieval failed.'); $discardException = new RuntimeException('Discard failed.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with('mysql')->andReturn($pool); @@ -469,7 +471,7 @@ public function testDiscardCancellationReplacesAnOrdinarySetupFailure(): void $setupException = new RuntimeException('Connection retrieval failed.'); $discardCancellation = new CanceledException('Discard was canceled.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with('mysql')->andReturn($pool); @@ -493,7 +495,7 @@ public function testSetupCancellationRemainsPrimaryOverAnOrdinaryDiscardFailure( $setupCancellation = new CanceledException('Connection setup was canceled.'); $discardException = new RuntimeException('Discard failed.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with('mysql')->andReturn($pool); @@ -517,7 +519,7 @@ public function testSetupCancellationRemainsPrimaryOverDiscardCancellation(): vo $setupCancellation = new CanceledException('Connection setup was canceled.'); $discardCancellation = new CanceledException('Discard was canceled.'); $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $poolManager->expects('pool')->once()->with('mysql')->andReturn($pool); @@ -539,7 +541,7 @@ public function testSetupCancellationRemainsPrimaryOverDiscardCancellation(): vo public function testCoroutineConnectionRemainsDeferOwned(): void { $poolManager = m::mock(PoolManager::class); - $pool = m::mock(DatabasePool::class); + $pool = m::mock(DatabasePool::class, ['usesSessionLeases' => false]); $wrapper = m::mock(PooledConnection::class); $connection = m::mock(Connection::class); diff --git a/tests/Database/DatabaseConnectionFactoryTest.php b/tests/Database/DatabaseConnectionFactoryTest.php index 3b2be9740a..9b25cd2309 100755 --- a/tests/Database/DatabaseConnectionFactoryTest.php +++ b/tests/Database/DatabaseConnectionFactoryTest.php @@ -315,6 +315,31 @@ public function testReadWriteConnectionSetsReadConnectionConfig(): void $this->assertSame(':memory:', $config['database']); } + public function testReadEndpointMetadataMatchesTheSelectedPdoConfiguration(): void + { + $container = new Container; + $pdo = m::mock(PDO::class); + $connector = m::mock(ConnectorInterface::class); + $connector->expects('connect')->withArgs( + static fn (array $config): bool => $config['host'] === 'first-reader' + && $config['options'][PDO::ATTR_EMULATE_PREPARES] === true + )->andReturn($pdo); + $container->instance('db.connector.pgsql', $connector); + $factory = new AlternatingReadConnectionFactory($container); + $connection = $factory->make([ + 'driver' => 'pgsql', + 'database' => 'app', + 'write' => [], + 'read' => [ + ['host' => 'first-reader', 'options' => [PDO::ATTR_EMULATE_PREPARES => true]], + ['host' => 'second-reader', 'options' => [PDO::ATTR_EMULATE_PREPARES => false]], + ], + ], 'read-options'); + + $this->assertSame($pdo, $connection->getReadPdo()); + $this->assertSame(['true', 'false'], $connection->prepareBindings([true, false])); + } + // REMOVED: Laravel's external-pooler configuration tests. Hypervel configures // the direct endpoint as a normal named connection and uses migrations_connection // instead of the ::direct suffix. @@ -636,6 +661,21 @@ class FactorySqliteConnection extends SQLiteConnection { } +class AlternatingReadConnectionFactory extends ConnectionFactory +{ + protected int $readSelection = 0; + + /** + * Select consecutive read records to expose repeated endpoint selection. + */ + protected function getReadWriteConfig(array $config, string $type): array + { + return $type === 'read' && isset($config[$type][0]) + ? $config[$type][$this->readSelection++ % count($config[$type])] + : parent::getReadWriteConfig($config, $type); + } +} + class FactoryTestConnectionFactory extends ConnectionFactory { /** diff --git a/tests/Database/DatabaseConnectionLeaseLifecycleTest.php b/tests/Database/DatabaseConnectionLeaseLifecycleTest.php new file mode 100644 index 0000000000..0fbaeda602 --- /dev/null +++ b/tests/Database/DatabaseConnectionLeaseLifecycleTest.php @@ -0,0 +1,153 @@ +make('config')->set('app.stdout_log.level', []); + $app->make('config')->set('database.connections.leases', [ + 'driver' => 'sqlite', + 'database' => ':memory:', + 'pool' => [ + 'testing_enabled' => true, + 'max_connections' => 1, + 'wait_timeout' => 0.1, + 'events' => [ConnectionReleasing::class], + ], + ]); + } + + #[DataProvider('terminalReleases')] + public function testTerminalCallbacksResolveTheirOwningConnection(bool $coroutine, bool $listenerFails): void + { + $resolver = $this->app->make('db.resolver'); + $pool = $this->app->make(PoolManager::class)->pool('leases'); + $connection = null; + $callbacks = []; + $registeredAfterCleanup = null; + Event::listen(ConnectionReleasing::class, static function (ConnectionReleasing $event) use (&$callbacks, $listenerFails): void { + $resolved = DB::connection(); + $callbacks[] = ['release', $resolved, $resolved->selectOne('select 2 as value')->value]; + + if ($listenerFails) { + throw new RuntimeException('Release listener failed after querying its owner.'); + } + }); + $execute = static function () use (&$connection, &$callbacks): void { + DB::setDefaultConnection('leases'); + $connection = DB::connection(); + $connection->beginTransaction(); + $connection->afterRollBack(static function () use (&$callbacks): void { + $resolved = DB::connection(); + $callbacks[] = ['rollback', $resolved, $resolved->selectOne('select 1 as value')->value]; + }); + }; + + try { + if ($coroutine) { + $this->runInCoroutine(static function () use ($execute, &$registeredAfterCleanup): void { + Coroutine::defer(static function () use (&$registeredAfterCleanup): void { + $registeredAfterCleanup = array_key_exists('leases', DB::getConnections()); + }); + $execute(); + }); + } else { + $execute(); + $resolver->releaseConnections(); + $registeredAfterCleanup = array_key_exists('leases', DB::getConnections()); + $this->assertSame(config('database.default'), DB::getDefaultConnection()); + } + + $this->assertSame([['rollback', $connection, 1], ['release', $connection, 2]], $callbacks); + $this->assertFalse($registeredAfterCleanup); + $this->assertSame(0, $connection->transactionLevel()); + $this->assertSame(0, $pool->getBorrowedCount()); + $this->assertSame(1, $pool->getIdleCount()); + } finally { + $resolver->discardConnections(); + $pool->close(); + } + } + + /** + * Provide terminal cleanup triggers and a failed release callback. + */ + public static function terminalReleases(): array + { + return [ + 'coroutine end' => [true, false], + 'task end' => [false, false], + 'failed task release listener' => [false, true], + ]; + } + + public function testTaskCleanupSettlesTheCurrentLeaseAfterRepeatedEarlyReleases(): void + { + $resolver = $this->app->make('db.resolver'); + $connection = DB::connection('leases'); + $pool = $this->app->make(PoolManager::class)->pool('leases'); + + try { + foreach ([1, 2] as $value) { + $this->assertSame($value, $connection->selectOne('select ? as value', [$value])->value); + DB::releaseIdleConnections(); + $this->assertSame(1, $pool->getIdleCount()); + } + + $connection->selectOne('select 1'); + $resolver->releaseConnections(); + $resolver->releaseConnections(); + + $this->assertSame(1, $pool->getIdleCount()); + $this->assertNotSame($connection, DB::connection('leases')); + $resolver->discardConnections(); + $this->assertSame(0, $pool->getManagedCount()); + } finally { + $resolver->discardConnections(); + $pool->close(); + } + } + + public function testPurgingAndReplacingAConnectionDoesNotLoseItsPreviousOwner(): void + { + $resolver = $this->app->make('db.resolver'); + $pools = $this->app->make(PoolManager::class); + $connection = DB::connection('leases'); + $connection->selectOne('select 1'); + $previousPool = $pools->pool('leases'); + + try { + DB::purge('leases'); + $replacement = DB::connection('leases'); + $replacement->selectOne('select 2'); + $resolver->releaseConnections(); + + $this->assertNotSame($connection, $replacement); + $this->assertTrue($previousPool->isClosed()); + $this->assertSame(0, $previousPool->getManagedCount()); + $this->assertSame(1, $pools->pool('leases')->getIdleCount()); + } finally { + $resolver->discardConnections(); + $pools->purgeAll(); + } + } +} diff --git a/tests/Database/DatabaseConnectionLeaseTest.php b/tests/Database/DatabaseConnectionLeaseTest.php new file mode 100644 index 0000000000..e17f254d4a --- /dev/null +++ b/tests/Database/DatabaseConnectionLeaseTest.php @@ -0,0 +1,651 @@ +make('config')->set('database.connections.leases', [ + 'driver' => 'sqlite', + 'database' => ':memory:', + 'prefix' => '', + 'foreign_key_constraints' => true, + 'pool' => [ + 'testing_enabled' => true, + 'min_retained_connections' => 1, + 'max_connections' => 1, + 'wait_timeout' => 1.0, + 'events' => [ConnectionReleasing::class], + ], + ]); + $app->make('config')->set('database.connections.physical_leases', [ + 'driver' => 'pgsql', + 'database' => 'app', + 'read' => ['host' => 'replica'], + 'write' => ['host' => 'primary'], + 'sticky' => true, + 'pool' => ['testing_enabled' => true, 'max_connections' => 1], + ]); + $app->make('config')->set('database.connections.identity_leases', [ + 'driver' => 'pgsql', + 'database' => 'app', + 'sticky' => true, + 'pool' => ['testing_enabled' => true, 'max_connections' => 1], + ]); + $app->make('config')->set('database.connections.insert_id_leases', [ + 'driver' => 'mysql', + 'database' => 'app', + 'pool' => ['testing_enabled' => true, 'max_connections' => 1], + ]); + $app->instance('db.connector.pgsql', new LeasePdoConnector); + $app->instance('db.connector.mysql', new LeasePdoConnector); + $app->instance('db.connector.mariadb', new LeasePdoConnector); + } + + public function testRetainedBuildersAndCallerStateSurviveAnotherBorrower(): void + { + $connection = DB::connection('leases'); + $schema = $connection->getSchemaBuilder(); + $schema->create('records', static fn (Blueprint $table) => $table->integer('id')); + $builder = $connection->table('records'); + $builder->insert(['id' => 7]); + $connection->enableQueryLog(); + $queries = 0; + $connection->beforeExecuting(static function () use (&$queries): void { + ++$queries; + }); + + DB::releaseIdleConnections(); + $this->assertSame(1, $this->pool()->getIdleCount()); + + [$other] = parallel([static function (): Connection { + $connection = DB::connection('leases'); + $connection->selectOne('select 1'); + $connection->setTablePrefix('other_'); + $connection->setDatabaseName('other'); + + return $connection; + }]); + + $this->assertNotSame($connection, $other); + $this->assertSame($connection, DB::connection('leases')); + $this->assertSame('', $connection->getTablePrefix()); + $this->assertSame(':memory:', $connection->getDatabaseName()); + $this->assertSame(7, $builder->value('id')); + $this->assertTrue($schema->hasTable('records')); + $this->assertTrue($connection->hasModifiedRecords()); + $this->assertTrue($connection->logging()); + $this->assertGreaterThanOrEqual(2, $queries); + $this->assertCount($queries, $connection->getQueryLog()); + } + + public function testConcurrentExternalWaitsDoNotRetainTheOnlyPoolSlot(): void + { + $ready = new Channel(2); + $resume = new Channel(2); + $waitForProvider = static function () use ($ready, $resume): int { + $connection = DB::connection('leases'); + $value = $connection->selectOne('select 1 as value')->value; + DB::releaseIdleConnections(); + $ready->push(true); + + if ($resume->pop(3.0) !== true) { + throw new RuntimeException('The simulated provider was not resumed.'); + } + + return $value + $connection->selectOne('select 2 as value')->value; + }; + + $results = parallel([ + $waitForProvider, + $waitForProvider, + function () use ($ready, $resume): void { + try { + $this->assertTrue($ready->pop(3.0)); + $this->assertTrue($ready->pop(3.0)); + $this->assertSame(1, $this->pool()->getIdleCount()); + } finally { + $resume->push(true); + $resume->push(true); + } + }, + ]); + + $this->assertSame([3, 3, null], $results); + $this->assertSame(1, $this->pool()->getIdleCount()); + } + + public function testConcurrentHttpRequestsReleaseSessionsBeforeWaitingForHeaders(): void + { + $server = new Socket(AF_INET, SOCK_STREAM, 0); + $this->assertTrue($server->bind('127.0.0.1', 0)); + $this->assertTrue($server->listen()); + $url = 'http://127.0.0.1:' . $server->getsockname()['port']; + $request = function () use ($url): int { + $connection = DB::connection('leases'); + $value = $connection->selectOne('select 1 as value')->value; + $response = (new Factory)->timeout(3)->beforeSending(function () use ($connection): void { + $connection->selectOne('select 1'); + $this->assertSame(0, $this->pool()->getIdleCount()); + })->get($url); + $this->assertSame('OK', $response->body()); + $this->assertSame($connection, DB::connection('leases')); + + return $value + $connection->selectOne('select 2 as value')->value; + }; + + try { + $results = parallel([ + $request, + $request, + function () use ($server): void { + $clients = []; + + try { + for ($index = 0; $index < 2; ++$index) { + $client = $server->accept(3); + $this->assertInstanceOf(Socket::class, $client); + $clients[] = $client; + $headers = ''; + + while (! str_contains($headers, "\r\n\r\n")) { + $chunk = $client->recv(3); + $this->assertIsString($chunk); + $this->assertNotSame('', $chunk); + $headers .= $chunk; + } + } + + $this->assertSame(1, $this->pool()->getIdleCount()); + $this->assertSame(9, DB::connection('leases')->selectOne('select 9 as value')->value); + + foreach ($clients as $client) { + $client->sendAll("HTTP/1.1 200 OK\r\nContent-Length: 2\r\nConnection: close\r\n\r\nOK"); + } + } finally { + foreach ($clients as $client) { + $client->close(); + } + } + }, + ]); + } finally { + $server->close(); + } + + $this->assertSame([3, 3, null], $results); + $this->assertSame(1, $this->pool()->getIdleCount()); + } + + public function testHttpRetriesAndRedirectsReleaseSessionsAcquiredByRequestCallbacks(): void + { + $calls = 0; + $response = (new Factory)->retry(2, 0)->beforeSending(function (): void { + DB::connection('leases')->selectOne('select 1'); + $this->assertSame(0, $this->pool()->getIdleCount()); + })->setHandler(function () use (&$calls): PromiseInterface { + $this->assertSame(1, $this->pool()->getIdleCount()); + + return Create::promiseFor(match (++$calls) { + 1 => new Response(500), + 2 => new Response(302, ['Location' => 'http://example.test/redirected']), + default => new Response(200, body: 'OK'), + }); + })->get('http://example.test'); + + $this->assertSame(3, $calls); + $this->assertSame('OK', $response->body()); + } + + #[DataProvider('pinnedScopes')] + public function testSessionDependentScopesPreventEarlyRelease(string $scope): void + { + $connection = DB::connection('leases'); + $check = function () use ($connection): void { + $this->assertSame(1, $connection->selectOne('select 1 as value')->value); + DB::releaseIdleConnections(); + $this->assertSame(0, $this->pool()->getIdleCount()); + (new Factory)->setHandler(function (): PromiseInterface { + $this->assertSame(0, $this->pool()->getIdleCount()); + + return Create::promiseFor(new Response(200)); + })->get('http://example.test'); + }; + + match ($scope) { + 'explicit' => $connection->withPinnedSession( + fn () => $connection->withPinnedSession($check) + ), + 'transaction' => $connection->transaction($check), + 'foreign keys' => $connection->getSchemaBuilder()->withoutForeignKeyConstraints($check), + }; + + DB::releaseIdleConnections(); + $this->assertSame(1, $this->pool()->getIdleCount()); + } + + /** + * Provide scopes whose work requires the same physical session. + */ + public static function pinnedScopes(): array + { + return [['explicit'], ['transaction'], ['foreign keys']]; + } + + public function testAnAbandonedCursorReleasesItsPin(): void + { + $connection = DB::connection('leases'); + $cursor = $connection->cursor('select 1 as value union all select 2'); + $this->assertSame(1, $cursor->current()->value); + DB::releaseIdleConnections(); + $this->assertSame(0, $this->pool()->getIdleCount()); + + unset($cursor); + DB::releaseIdleConnections(); + + $this->assertSame(1, $this->pool()->getIdleCount()); + $this->assertSame(2, $connection->selectOne('select 2 as value')->value); + } + + public function testConnectionAndQueryListenersKeepTheLogicalOwnerAttached(): void + { + $established = null; + $released = null; + Event::listen(ConnectionEstablished::class, function (ConnectionEstablished $event) use (&$established): void { + if ($event->connection->getName() !== 'leases') { + return; + } + + $established = $event->connection; + DB::releaseIdleConnections(); + $this->assertSame(0, $this->pool()->getIdleCount()); + $this->assertSame(1, $event->connection->selectOne('select 1 as value')->value); + }); + Event::listen(QueryExecuted::class, function (QueryExecuted $event): void { + if ($event->connection->getName() === 'leases') { + DB::releaseIdleConnections(); + $this->assertSame(0, $this->pool()->getIdleCount()); + } + }); + Event::listen(ConnectionReleasing::class, function (ConnectionReleasing $event) use (&$released): void { + $released = $event->connection->getConnection(); + $this->assertSame(2, $released->selectOne('select 2 as value')->value); + }); + + $connection = DB::connection('leases'); + DB::releaseIdleConnections(); + + $this->assertSame($connection, $established); + $this->assertSame($connection, $released); + $this->assertSame(1, $this->pool()->getIdleCount()); + } + + public function testAnEagerPdoResolverCanConstructTheLogicalDriver(): void + { + Connection::resolverFor( + 'sqlite', + static fn (PDO|Closure $pdo, string $database, string $prefix, array $config): LeaseSqliteConnection => new LeaseSqliteConnection($pdo instanceof Closure ? $pdo() : $pdo, $database, $prefix, $config) + ); + + $connection = DB::connection('leases'); + $this->assertInstanceOf(LeaseSqliteConnection::class, $connection); + $this->assertSame(1, $connection->selectOne('select 1 as value')->value); + DB::releaseIdleConnections(); + $this->assertSame(1, $this->pool()->getIdleCount()); + $this->assertSame(2, $connection->selectOne('select 2 as value')->value); + } + + #[DataProvider('differentConfiguredIdentities')] + public function testDifferentConfiguredIdentitiesRetainWholeConnectionOwnership(string $name, string $role, array $records): void + { + config()->set('database.connections.identity_leases.read', ['host' => 'replica']); + config()->set('database.connections.identity_leases.write', ['host' => 'primary']); + config()->set("database.connections.identity_leases.{$role}", $records); + $connection = DB::connection($name); + $pool = $this->app->make(PoolManager::class)->pool($name); + $connection->statement('create table ' . $connection->getTablePrefix() . 'records (id integer)'); + $builder = $connection->table('records'); + $builder->insert(['id' => 7]); + + DB::releaseIdleConnections(); + + $this->assertSame(0, $pool->getIdleCount()); + $this->assertSame($connection, DB::connection($name)); + $this->assertSame(7, $builder->value('id')); + } + + /** + * Provide pools whose selectable endpoints need different logical connections. + */ + public static function differentConfiguredIdentities(): array + { + $prefixes = [['prefix' => 'first_'], ['prefix' => 'second_']]; + $drivers = [['driver' => 'mysql'], ['driver' => 'mariadb']]; + + return [ + 'read prefixes' => ['identity_leases::read', 'read', $prefixes], + 'write prefixes' => ['identity_leases', 'write', $prefixes], + 'read drivers' => ['identity_leases::read', 'read', $drivers], + 'write drivers' => ['identity_leases', 'write', $drivers], + ]; + } + + #[DataProvider('reconnectedPdoRoles')] + public function testConnectionListenerReconnectKeepsTheReplacementPdo(bool $read): void + { + $connection = DB::connection('physical_leases'); + $connection->disconnect(); + DB::releaseIdleConnections(); + $replacement = null; + $original = null; + $reconnected = false; + + Event::listen(ConnectionEstablished::class, static function (ConnectionEstablished $event) use ($read, &$original, &$replacement, &$reconnected): void { + if ($event->connection->getName() === 'physical_leases' && ! $reconnected) { + $reconnected = true; + $original = $read ? $event->connection->getRawReadPdo() : $event->connection->getRawPdo(); + $event->connection->reconnect(); + + if (! $read) { + $replacement = $event->connection->getPdo(); + } + } + }); + + $resolved = $read ? $connection->getReadPdo() : $connection->getPdo(); + + $this->assertInstanceOf(PDO::class, $original); + $this->assertNotSame($original, $resolved); + + if (! $read) { + $this->assertSame($replacement, $resolved); + } + + $this->assertSame(1, $this->app->make(PoolManager::class)->pool('physical_leases')->getBorrowedCount()); + DB::releaseIdleConnections(); + $this->assertSame(1, $this->app->make(PoolManager::class)->pool('physical_leases')->getIdleCount()); + } + + /** + * Provide the physical handle resolved when a connection listener reconnects. + */ + public static function reconnectedPdoRoles(): array + { + return ['write' => [false], 'read' => [true]]; + } + + public function testDisconnectDropsThePhysicalPdoAndReusesTheLogicalConnection(): void + { + $connection = DB::connection('physical_leases'); + $pdo = WeakReference::create($connection->getPdo()); + $readPdo = WeakReference::create($connection->getReadPdo()); + $this->assertNotSame($pdo->get(), $readPdo->get()); + + DB::disconnect('physical_leases'); + + $this->assertNull($pdo->get()); + $this->assertNull($readPdo->get()); + $this->assertNull($connection->getRawPdo()); + $this->assertNull($connection->getRawReadPdo()); + $this->assertSame($connection, DB::connection('physical_leases')); + $this->assertSame(2, $connection->selectOne('select 2 as value')->value); + } + + public function testCapturedMySqlInsertIdsSurviveEarlyRelease(): void + { + $connection = DB::connection('insert_id_leases'); + $connection->statement('create table records (id integer primary key)'); + $connection->insert('insert into records default values'); + + DB::releaseIdleConnections(); + + $this->assertSame('1', $connection->getLastInsertId()); + $this->assertSame(1, $this->app->make(PoolManager::class)->pool('insert_id_leases')->getIdleCount()); + $connection->insert('insert into records default values'); + $this->assertSame('2', $connection->getLastInsertId()); + } + + public function testReadWriteRolesAndStickyRoutingSurviveEarlyRelease(): void + { + $connection = DB::connection('physical_leases'); + $this->assertNotSame($connection->getPdo(), $connection->getReadPdo()); + $connection->statement('create table records (id integer)'); + + DB::releaseIdleConnections(); + + [$otherWasSticky] = parallel([static function (): bool { + $other = DB::connection('physical_leases'); + + return $other->getPdo() === $other->getReadPdo(); + }]); + + $this->assertFalse($otherWasSticky); + $this->assertSame($connection->getPdo(), $connection->getReadPdo()); + DB::releaseIdleConnections(); + + $read = DB::connection('physical_leases::read'); + $write = DB::connection('physical_leases::write'); + $this->assertSame('replica', $read->getConfig('host')); + $this->assertSame('primary', $write->getConfig('host')); + $this->assertSame($write->getPdo(), $write->getReadPdo()); + DB::releaseIdleConnections(); + + $this->assertSame(1, $read->selectOne('select 1 as value')->value); + $this->assertSame(2, $write->selectOne('select 2 as value')->value); + $this->assertSame('replica', $read->getConfig('host')); + $this->assertSame('primary', $write->getConfig('host')); + } + + public function testConfigFirstExtensionsRetainWholeConnectionOwnership(): void + { + $constructions = 0; + DB::extend('physical_leases', static function (array $config) use (&$constructions): PostgresConnection { + ++$constructions; + + return new PostgresConnection(new PDO('sqlite::memory:'), $config['database'], '', $config); + }); + $connection = DB::connection('physical_leases'); + $connection->selectOne('select 1'); + + DB::releaseIdleConnections(); + + $this->assertSame(0, $this->app->make(PoolManager::class)->pool('physical_leases')->getIdleCount()); + $this->assertSame($connection, DB::connection('physical_leases')); + $this->assertSame(2, $connection->selectOne('select 2 as value')->value); + $this->assertSame(1, $constructions); + } + + public function testSessionCapabilitiesAreResolvedAgainAfterEarlyRelease(): void + { + Connection::resolverFor( + 'pgsql', + static fn (PDO|Closure $pdo, string $database, string $prefix, array $config): LeaseCapabilityConnection => new LeaseCapabilityConnection($pdo, $database, $prefix, $config) + ); + $connection = DB::connection('physical_leases'); + $connection->getPdo()->exec('pragma user_version = 100'); + $this->assertSame(100, $connection->maxBindings()); + + DB::releaseIdleConnections(); + + parallel([static function (): void { + DB::connection('physical_leases')->getPdo()->exec('pragma user_version = 200'); + }]); + + $this->assertSame(200, $connection->maxBindings()); + } + + public function testSessionConfiguratorsRunOnTheLogicalOwnerAfterEachBorrowerChangesState(): void + { + $configurator = new LeaseSessionConfigurator; + PdoConnection::configureSessionUsing($configurator); + CoroutineContext::set('__lease_test.account', '1'); + $connection = DB::connection('leases'); + $this->assertSame(1, $connection->selectOne('pragma user_version')->user_version); + DB::releaseIdleConnections(); + + [$other] = parallel([function (): Connection { + CoroutineContext::set('__lease_test.account', '2'); + $connection = DB::connection('leases'); + $this->assertSame(2, $connection->selectOne('pragma user_version')->user_version); + + return $connection; + }]); + + $this->assertSame(1, $connection->selectOne('pragma user_version')->user_version); + $this->assertSame([ + [spl_object_id($connection), '1'], + [spl_object_id($other), '2'], + [spl_object_id($connection), '1'], + ], $configurator->applied); + } + + #[DataProvider('connectionListenerFailures')] + public function testConnectionListenerFailuresSettleTheLeaseOnce(bool $reacquire, bool $cancel): void + { + $connection = null; + + if ($reacquire) { + $connection = DB::connection('leases'); + $connection->disconnect(); + DB::releaseIdleConnections(); + } + + $failure = $cancel ? new CanceledException('listener canceled') : new RuntimeException('listener failed'); + $fail = true; + Event::listen(ConnectionEstablished::class, static function (ConnectionEstablished $event) use ($failure, &$fail): void { + if ($event->connection->getName() === 'leases' && $fail) { + $fail = false; + throw $failure; + } + }); + + try { + if ($reacquire) { + $connection->getPdo(); + } else { + DB::connection('leases'); + } + + $this->fail('Expected connection publication to fail.'); + } catch (Throwable $exception) { + $this->assertSame($failure, $exception); + } + + $this->assertSame(0, $this->pool()->getManagedCount()); + $connection ??= DB::connection('leases'); + $this->assertSame(1, $connection->selectOne('select 1 as value')->value); + DB::releaseIdleConnections(); + $this->assertSame(1, $this->pool()->getManagedCount()); + $this->assertSame(1, $this->pool()->getIdleCount()); + } + + /** + * Provide publication failures at initial acquisition and reacquisition. + */ + public static function connectionListenerFailures(): array + { + return [ + 'initial failure' => [false, false], + 'initial cancellation' => [false, true], + 'reacquisition failure' => [true, false], + 'reacquisition cancellation' => [true, true], + ]; + } + + /** + * Get the pool used by the test's connections. + */ + protected function pool(): DatabasePool + { + return $this->app->make(PoolManager::class)->pool('leases'); + } +} + +class LeaseSqliteConnection extends SQLiteConnection +{ +} + +class LeaseCapabilityConnection extends PostgresConnection +{ + /** + * Read a session-specific capability from the fixture's SQLite database. + */ + protected function resolveMaxBindings(): int + { + return (int) $this->getPdo()->query('pragma user_version')->fetchColumn(); + } +} + +class LeasePdoConnector implements ConnectorInterface +{ + /** + * Provide independent physical PDOs without an external database server. + */ + public function connect(array $config): PDO + { + return new PDO('sqlite::memory:'); + } +} + +class LeaseSessionConfigurator implements SessionConfigurator +{ + public array $applied = []; + + /** + * Resolve the current borrower's desired session state. + */ + public function state(PdoConnection $connection): ?string + { + return $connection->getName() === 'leases' + ? CoroutineContext::get('__lease_test.account') + : null; + } + + /** + * Apply a visible SQLite session value and record its logical owner. + */ + public function apply(PDO $pdo, string $state, PdoConnection $connection): void + { + $pdo->exec('pragma user_version = ' . $state); + $this->applied[] = [spl_object_id($connection), $state]; + } +} diff --git a/tests/Database/DatabasePdoConnectionTest.php b/tests/Database/DatabasePdoConnectionTest.php index e7869a54c3..4c1dda55af 100755 --- a/tests/Database/DatabasePdoConnectionTest.php +++ b/tests/Database/DatabasePdoConnectionTest.php @@ -26,6 +26,7 @@ use PHPUnit\Framework\Attributes\DataProvider; use RuntimeException; use stdClass; +use WeakReference; class DatabasePdoConnectionTest extends TestCase { @@ -875,6 +876,67 @@ public function testLostManagedCommitTerminallyDetachesTransactionState(): void $this->assertNull($connection->getRawReadPdo()); } + #[DataProvider('lostTransactionOperations')] + public function testLostTransactionCleanupPreservesAConnectionOpenedByARollbackCallback(string $operation): void + { + $failure = new PDOException('server has gone away'); + $pdo = m::mock(PDO::class); + $pdo->expects('beginTransaction')->andReturnTrue(); + $pdo->shouldNotReceive('prepare'); + + if ($operation === 'rollback') { + $pdo->expects('inTransaction')->andReturnTrue(); + $pdo->expects('rollBack')->andThrow($failure); + } else { + $pdo->expects('commit')->andThrow($failure); + } + + $connection = $this->getMockConnection([], $pdo); + $connection->setTransactionManager(new DatabaseTransactionsManager); + $replacement = new PDO('sqlite::memory:'); + $reconnections = 0; + $connection->setReconnector(static function (PdoConnection $connection) use ($replacement, &$reconnections): void { + ++$reconnections; + $connection->setPdo($replacement); + }); + $result = null; + $registerCallback = static function (Connection $connection) use (&$result): void { + $connection->afterRollBack(static function () use ($connection, &$result): void { + $result = $connection->selectOne('select 1 as value')->value; + }); + }; + + try { + if ($operation === 'rollback') { + $connection->beginTransaction(); + $registerCallback($connection); + $connection->rollBack(); + } else { + $connection->transaction($registerCallback); + } + + $this->fail('Expected the lost transaction operation to fail.'); + } catch (PDOException $exception) { + $this->assertSame($failure, $exception); + } + + $this->assertSame(1, $result); + $this->assertSame(1, $reconnections); + $this->assertSame($replacement, $connection->getRawPdo()); + $this->assertSame(0, $connection->transactionLevel()); + } + + /** + * Provide operations that terminally forget a lost transaction connection. + */ + public static function lostTransactionOperations(): array + { + return [ + 'explicit rollback' => ['rollback'], + 'managed commit' => ['commit'], + ]; + } + public function testNonLostPhysicalRollbackFailureKeepsActiveStateAndMarksTheSessionUnknown(): void { $failure = new RuntimeException('rollback failure'); @@ -1035,6 +1097,26 @@ public function testDisconnectPreservesManagerFailureAfterLostPhysicalRollbackFa $this->assertNull($connection->getRawReadPdo()); } + public function testRefreshTransfersOwnershipOfBothPhysicalHandles(): void + { + $connection = new PdoConnection(new PDO('sqlite::memory:')); + $fresh = new PdoConnection(new PDO('sqlite::memory:')); + $fresh->setReadPdo(new PDO('sqlite::memory:')); + $writePdo = WeakReference::create($fresh->getRawPdo()); + $readPdo = WeakReference::create($fresh->getRawReadPdo()); + + $connection->refreshFrom($fresh); + unset($fresh); + + $this->assertSame($writePdo->get(), $connection->getPdo()); + $this->assertSame($readPdo->get(), $connection->getReadPdo()); + + $connection->disconnect(); + + $this->assertNull($writePdo->get()); + $this->assertNull($readPdo->get()); + } + public function testRefreshRetainsConfiguredBaselinesWhenPreparationFails(): void { $oldPdo = new PDOStub; @@ -1048,11 +1130,12 @@ public function testRefreshRetainsConfiguredBaselinesWhenPreparationFails(): voi $connection->setTablePrefix('tenant_'); $failure = new RuntimeException('read connection failed'); $fresh = new PdoConnection( - new PDOStub, + new PDO('sqlite::memory:'), 'fresh_database', 'fresh_', ['name' => 'test', 'driver' => 'sqlite', 'endpoint' => 'fresh'] ); + $replacementPdo = WeakReference::create($fresh->getRawPdo()); $fresh->setReadPdo(static fn (): never => throw $failure); try { @@ -1063,6 +1146,7 @@ public function testRefreshRetainsConfiguredBaselinesWhenPreparationFails(): voi } $this->assertSame($oldPdo, $connection->getRawPdo()); + $this->assertNull($replacementPdo->get()); $this->assertSame('old', $connection->getConfig('endpoint')); $connection->resetForPool(); diff --git a/tests/Database/DatabaseReadPoolTest.php b/tests/Database/DatabaseReadPoolTest.php new file mode 100644 index 0000000000..5b9aafd3d3 --- /dev/null +++ b/tests/Database/DatabaseReadPoolTest.php @@ -0,0 +1,132 @@ + [ + 'driver' => 'pgsql', + 'database' => 'app', + 'read' => [['host' => 'first-reader'], ['host' => 'second-reader']], + 'pool' => ['min_retained_connections' => 0, 'max_connections' => 2], + ]]); + $factory = new ReadPoolConnectionFactory($this->app); + $this->app->instance('db.factory', $factory); + $hosts = []; + $connector = m::mock(ConnectorInterface::class); + $connector->expects('connect')->times(3)->andReturnUsing(static function (array $config) use (&$hosts): PDO { + $hosts[] = $config['host']; + + return new PDO('sqlite::memory:'); + }); + $this->app->instance('db.connector.pgsql', $connector); + $pool = new DatabasePool($this->app, 'read_pool::read'); + $first = $second = null; + + try { + $first = $pool->borrow(); + $second = $pool->borrow(); + $first->getConnection()->getPdo(); + $connection = $second->getConnection(); + $connection->getPdo(); + $connection->reconnect(); + + $this->assertSame(['first-reader', 'second-reader', 'first-reader'], $hosts); + $this->assertSame($connection, $second->getConnection()); + $this->assertSame('first-reader', $connection->getConfig('host')); + } finally { + $first?->release(); + $second?->release(); + $pool->close(); + } + } + + #[DataProvider('readPoolOptions')] + public function testReadRecordsRequireTheSameEffectivePoolOptions(array $secondOptions, bool $conflicting): void + { + config(['database.connections.read_pool_options' => [ + 'driver' => 'pgsql', + 'database' => 'app', + 'pool' => ['max_connections' => 5], + 'read' => [ + ['host' => 'first-reader', 'pool' => ['max_connections' => 2, 'connect_timeout' => 1]], + ['host' => 'second-reader', 'pool' => $secondOptions], + ], + ]]); + + if ($conflicting) { + $this->expectException(InvalidArgumentException::class); + $this->expectExceptionMessage('must use the same effective pool options'); + } + + $pool = new DatabasePool($this->app, 'read_pool_options::read'); + + try { + $this->assertSame(2, $pool->getOptions()->maxConnections); + $this->assertSame(1.0, $pool->getOptions()->connectTimeout); + } finally { + $pool->close(); + } + } + + /** + * Provide equivalent and conflicting read pool configurations. + */ + public static function readPoolOptions(): array + { + return [ + 'matching normalized values' => [[ + 'max_connections' => 2, + 'connect_timeout' => 1.0, + 'wait_timeout' => 3.0, + 'testing_enabled' => true, + ], false], + 'conflicting capacity' => [['max_connections' => 3, 'connect_timeout' => 1], true], + ]; + } + + public function testEveryReadRecordIsCheckedForAnInMemorySqliteDatabase(): void + { + config(['database.connections.read_pool_memory' => [ + 'driver' => 'sqlite', + 'database' => '/unused/write.sqlite', + 'read' => [ + ['database' => '/unused/read.sqlite'], + ['url' => 'sqlite:///:memory:'], + ], + ]]); + $this->app->instance('db.factory', new ReadPoolConnectionFactory($this->app)); + $this->expectException(InvalidArgumentException::class); + $this->expectExceptionMessage('cannot use a derived read pool for in-memory SQLite'); + + new DatabasePool($this->app, 'read_pool_memory::read'); + } +} + +class ReadPoolConnectionFactory extends ConnectionFactory +{ + protected int $readSelection = 0; + + /** + * Select consecutive replicas whenever a new read record is needed. + */ + protected function getReadWriteConfig(array $config, string $type): array + { + return $type === 'read' && isset($config[$type][0]) + ? $config[$type][$this->readSelection++ % count($config[$type])] + : parent::getReadWriteConfig($config, $type); + } +} diff --git a/tests/Database/DatabaseTransactionsManagerTest.php b/tests/Database/DatabaseTransactionsManagerTest.php index 517ee75ffc..bd64555322 100755 --- a/tests/Database/DatabaseTransactionsManagerTest.php +++ b/tests/Database/DatabaseTransactionsManagerTest.php @@ -4,14 +4,107 @@ namespace Hypervel\Tests\Database; +use Hypervel\Context\CoroutineContext; +use Hypervel\Coroutine\Coroutine; use Hypervel\Database\DatabaseTransactionRecord; use Hypervel\Database\DatabaseTransactionsManager; +use Hypervel\Engine\Channel; use Hypervel\Tests\TestCase; use RuntimeException; use Swoole\Coroutine\CanceledException; +use Throwable; + +use function Hypervel\Coroutine\parallel; class DatabaseTransactionsManagerTest extends TestCase { + public function testForkedTransactionsCannotRunOrCaptureParentCallbacks(): void + { + $manager = new DatabaseTransactionsManager; + $events = []; + $manager->begin('default', 1); + $manager->addCallback(static function () use (&$events): void { + $events[] = 'parent-before'; + }); + $ready = new Channel(1); + $resume = new Channel(1); + $finished = new Channel(1); + $failure = null; + + $child = Coroutine::fork(function () use ($manager, &$events, $ready, $resume, $finished, &$failure): void { + try { + $manager->begin('default', 1); + $manager->addCallback(static function () use (&$events): void { + $events[] = 'child'; + }); + $ready->push(true); + $this->assertTrue($resume->pop(1)); + $manager->commit('default', 1, 0); + } catch (Throwable $exception) { + $failure = $exception; + } finally { + $finished->push(true); + } + }); + + try { + $this->assertTrue($ready->pop(1)); + $manager->addCallback(static function () use (&$events): void { + $events[] = 'parent-after'; + }); + } finally { + $resume->push(true, 0); + $completed = $finished->pop(1); + + if ($completed !== true && Coroutine::exists($child)) { + Coroutine::cancelById($child, throwException: true); + $finished->pop(1); + } + } + + $this->assertTrue($completed); + + if ($failure !== null) { + throw $failure; + } + + $this->assertSame(['child'], $events); + $manager->commit('default', 1, 0); + $this->assertSame(['child', 'parent-before', 'parent-after'], $events); + } + + public function testExplicitTestLifecycleTransferPreservesCurrentTransactionOwnership(): void + { + $manager = new DatabaseTransactionsManager; + $events = []; + $manager->begin('default', 1); + DatabaseTransactionsManager::copyToNonCoroutineState(); + + try { + parallel([function () use ($manager, &$events): void { + CoroutineContext::copyFromNonCoroutine(); + $this->assertCount(0, $manager->getPendingTransactions()); + + DatabaseTransactionsManager::copyFromNonCoroutineState(); + $manager->begin('default', 2); + $manager->addCallback(static function () use (&$events): void { + $events[] = 'nested'; + }); + $manager->commit('default', 2, 1); + }]); + + $this->assertSame([], $events); + $manager->addCallback(static function () use (&$events): void { + $events[] = 'outer'; + }); + $manager->commit('default', 1, 0); + $this->assertSame(['nested', 'outer'], $events); + $this->assertFalse(DatabaseTransactionsManager::hasNonCoroutinePendingTransactions()); + } finally { + DatabaseTransactionsManager::clearNonCoroutineState(); + } + } + public function testBeginningTransactions(): void { $manager = new DatabaseTransactionsManager; diff --git a/tests/Database/PoolManagerTest.php b/tests/Database/PoolManagerTest.php index 22e59bc47d..94b26abe99 100644 --- a/tests/Database/PoolManagerTest.php +++ b/tests/Database/PoolManagerTest.php @@ -12,6 +12,7 @@ use Hypervel\Contracts\Container\Container as ContainerContract; use Hypervel\Contracts\Log\StdoutLoggerInterface; use Hypervel\Coordinator\Timer; +use Hypervel\Database\Connection as DatabaseConnection; use Hypervel\Database\Connectors\ConnectionFactory; use Hypervel\Database\Pool\DatabasePool; use Hypervel\Database\Pool\PoolManager; @@ -621,7 +622,8 @@ public function testReadConnectionUsesSeparatePoolWhenReadConfigComesFromUrl(): $this->assertSame('default::read', $pool->getName()); $this->assertNotSame($manager->pool('default'), $pool); $this->assertInstanceOf(PoolManagerTestPool::class, $pool); - $this->assertSame(['replica.test'], $pool->configForTest()['host']); + $this->assertSame(['replica.test'], $pool->configForTest()['read']['host']); + $this->assertSame('read', $pool->configForTest()[DatabaseConnection::READ_WRITE_TYPE_CONFIG_KEY]); } public function testReadConnectionUsesBasePoolWhenReadConfigIsMissingOrNull(): void diff --git a/tests/Http/Client/CurlStreamingHandlerTest.php b/tests/Http/Client/CurlStreamingHandlerTest.php new file mode 100644 index 0000000000..bf5068990e --- /dev/null +++ b/tests/Http/Client/CurlStreamingHandlerTest.php @@ -0,0 +1,572 @@ +expectException(ConnectionException::class); + $this->expectExceptionMessage($message); + + (new Factory)->withOptions(['stream' => true])->withHeaders($headers)->get($url); + } + + /** + * Provide URI and Host-header spellings that transports interpret differently. + */ + public static function ambiguousHosts(): array + { + return [ + ['http://127.0.0.1.:1/', [], 'must not be written as one to four'], + ['http://127.0.0.1:1/', ['Host' => "example.test\xc2\xa0"], 'must contain only printable ASCII'], + ]; + } + + public function testRedirectsCookiesAndDecompressionKeepThePublicClientBehavior(): void + { + $server = LoopbackHttpServer::start([ + ['status' => 302, 'headers' => ['Location' => '/final', 'Set-Cookie' => 'session=test; Path=/'], 'body' => 'redirect'], + ['headers' => ['Content-Encoding' => 'gzip'], 'body' => gzencode("{\"id\":1}\n{\"id\":2}\n")], + ]); + $response = (new Factory)->withOptions(['stream' => true])->get('http://127.0.0.1:' . $server->port); + + try { + $this->assertSame(200, $response->status()); + $this->assertSame([['id' => 1], ['id' => 2]], iterator_to_array($response->jsonLines())); + $this->assertSame('', $response->header('Content-Encoding')); + $server->request(); + $redirected = $server->request(); + $this->assertStringStartsWith('GET /final HTTP/1.1', $redirected); + $this->assertStringContainsString('Cookie: session=test', $redirected); + } finally { + $response->close(); + } + } + + #[DataProvider('authenticationResponses')] + public function testNativeAuthenticationExposesOnlyTheFinalPostResponse(int $initialStatus): void + { + $server = LoopbackHttpServer::start([ + ['status' => $initialStatus, 'headers' => ['WWW-Authenticate' => 'Digest realm="test", nonce="fixed", qop="auth", algorithm=MD5'], 'body' => 'intermediate'], + ['body' => 'final'], + ]); + $headers = []; + $stats = []; + $response = (new Factory)->withOptions([ + 'stream' => true, + 'curl' => [CURLOPT_HTTPAUTH => CURLAUTH_DIGEST, CURLOPT_USERPWD => 'account:secret'], + 'on_headers' => static function (ResponseInterface $response) use (&$headers): void { + $headers[] = $response->getStatusCode(); + }, + 'on_stats' => static function (TransferStats $transfer) use (&$stats): void { + $stats[] = $transfer->getResponse()->getStatusCode(); + }, + ])->withBody('prompt')->post('http://127.0.0.1:' . $server->port); + + try { + $this->assertSame([200], $headers); + $this->assertSame([200], $stats); + $this->assertSame('final', $response->body()); + $this->assertSame([200], $stats); + $this->assertStringEndsWith("\r\n\r\n", $server->request()); + $this->assertStringEndsWith("\r\n\r\nprompt", $server->request()); + } finally { + $response->close(); + } + } + + /** + * Provide challenge and successful negotiation responses before the real POST. + */ + public static function authenticationResponses(): array + { + return [[401], [200]]; + } + + public function testDigestAuthenticationWorksThroughThePublicApi(): void + { + $server = LoopbackHttpServer::start([ + ['status' => 401, 'headers' => ['WWW-Authenticate' => 'Digest realm="test", nonce="fixed", qop="auth", algorithm=MD5']], + ['body' => 'authenticated'], + ]); + $response = (new Factory)->withOptions(['stream' => true])->withDigestAuth('account', 'secret') + ->withBody('prompt')->post('http://127.0.0.1:' . $server->port); + + try { + $this->assertSame(200, $response->status()); + $this->assertSame('authenticated', $response->body()); + $server->request(); + $authenticated = $server->request(); + $this->assertStringContainsString('Authorization: Digest ', $authenticated); + $this->assertStringEndsWith("\r\n\r\nprompt", $authenticated); + } finally { + $response->close(); + } + } + + #[DataProvider('callbackFailures')] + public function testCallbackFailureKeepsItsResponseAndCauseAndClosesTheTransfer(int $status, string $callback): void + { + $server = LoopbackHttpServer::start([['status' => $status]]); + $failure = new RuntimeException('invalid provider response'); + $stats = []; + $trailers = 0; + + try { + (new Factory)->withOptions([ + 'stream' => true, + 'on_headers' => static function () use ($callback, $failure): void { + if ($callback === 'on_headers') { + throw $failure; + } + }, + 'on_trailers' => static function () use ($callback, $failure, &$trailers): void { + ++$trailers; + + if ($callback === 'on_trailers') { + throw $failure; + } + }, + 'on_stats' => static function (TransferStats $transfer) use (&$stats): void { + $stats[] = $transfer; + }, + ])->get('http://127.0.0.1:' . $server->port); + + $this->fail('Expected the response callback to fail.'); + } catch (ConnectionException|RequestException $exception) { + if ($status === 200) { + $this->assertInstanceOf(ConnectionException::class, $exception); + $this->assertSame("An error was encountered during the {$callback} event", $exception->getMessage()); + $this->assertSame($failure, $exception->getPrevious()->getPrevious()); + } else { + $this->assertInstanceOf(RequestException::class, $exception); + $this->assertSame($status, $exception->response->status()); + } + } + + $this->assertSame($callback === 'on_trailers' ? 1 : 0, $trailers); + $this->assertCount(1, $stats); + $this->assertSame($status, $stats[0]->getResponse()->getStatusCode()); + $this->assertSame($failure, $stats[0]->getHandlerErrorData()->getPrevious()); + $this->assertFalse($stats[0]->getResponse()->getBody()->isReadable()); + $this->assertNotNull($server->request()); + } + + /** + * Provide successful and failed responses rejected by response callbacks. + */ + public static function callbackFailures(): array + { + return [[200, 'on_headers'], [503, 'on_headers'], [200, 'on_trailers'], [503, 'on_trailers']]; + } + + public function testLateFinalHeadersRetainTheirResponseInTheExceptionAndStatistics(): void + { + $server = LoopbackHttpServer::start([['status' => 503, 'headers' => ['Retry-After' => '10']]]); + $delayed = false; + $stats = []; + + try { + (new Factory)->timeout(0.5)->withOptions([ + 'stream' => true, + 'progress' => static function (int $total, int $received) use (&$delayed): void { + if ($received > 0 && ! $delayed) { + $delayed = true; + // Headers exist, but native execution returns after their deadline. + usleep(600000); + } + }, + 'on_stats' => static function (TransferStats $transfer) use (&$stats): void { + $stats[] = $transfer; + }, + ])->get('http://127.0.0.1:' . $server->port); + + $this->fail('Expected the late response to time out.'); + } catch (RequestException $exception) { + $this->assertSame(503, $exception->response->status()); + $this->assertSame('10', $exception->response->header('Retry-After')); + } + + $this->assertCount(1, $stats); + $this->assertSame(503, $stats[0]->getResponse()->getStatusCode()); + $this->assertSame('Timed out while receiving the response headers', $stats[0]->getHandlerErrorData()->getMessage()); + $this->assertFalse($stats[0]->getResponse()->getBody()->isReadable()); + $this->assertTrue($stats[0]->getResponse()->getBody()->getMetadata('timed_out')); + } + + public function testPreHeaderRetryDoesNotRecheckTheDeadlineAfterTheReplacementCallback(): void + { + $server = LoopbackHttpServer::start([['body' => ''], ['body' => 'retried']]); + $nativeFactory = new CurlFactory(0); + $first = true; + $factory = m::mock(CurlFactoryInterface::class); + $factory->shouldReceive('create')->andReturnUsing(static function (RequestInterface $request, array $options) use ($nativeFactory, &$first): EasyHandle { + $easy = $nativeFactory->create($request, $options); + + if ($first) { + $first = false; + // Guzzle retries native success without a parsed response; reproduce that outcome. + curl_setopt($easy->handle, CURLOPT_HEADERFUNCTION, static fn (CurlHandle $handle, string $header): int => strlen($header)); + } + + return $easy; + }); + $factory->shouldReceive('release')->andReturnUsing($nativeFactory->release(...)); + $handler = new CurlStreamingHandler; + (new ReflectionProperty($handler, 'factory'))->setValue($handler, $factory); + $callbacks = 0; + $trailers = 0; + $response = (new Factory)->setHandler($handler)->timeout(0.5)->withOptions([ + 'stream' => true, + 'on_headers' => static function () use (&$callbacks): void { + ++$callbacks; + usleep(600000); + }, + 'on_trailers' => static function () use (&$trailers): void { + ++$trailers; + }, + ])->get('http://127.0.0.1:' . $server->port); + + try { + $this->assertSame('retried', $response->body()); + $this->assertSame(1, $callbacks); + $this->assertSame(1, $trailers); + $this->assertNotNull($server->request()); + $this->assertNotNull($server->request()); + } finally { + $response->close(); + } + } + + #[DataProvider('requestExecutionModes')] + public function testWrappedHeaderCancellationIsNotRecoveredOrRetried(bool $async): void + { + $server = LoopbackHttpServer::start(); + $failure = new CanceledException('header processing canceled'); + $events = new Dispatcher; + $events->listen(ConnectionFailed::class, fn () => $this->fail('Cancellation dispatched a connection failure.')); + $attempts = 0; + $request = (new Factory($events))->async($async) + ->beforeSending(static function () use (&$attempts): void { + ++$attempts; + }) + ->afterResponse(fn () => $this->fail('Cancellation recovered an HTTP response.')) + ->retry(3, when: fn () => $this->fail('Cancellation reached the retry policy.')) + ->withOptions(['on_headers' => static fn () => throw $failure]); + + try { + $result = $request->get('http://127.0.0.1:' . $server->port); + + if ($async) { + $result->wait(); + } + + $this->fail('Expected the callback cancellation.'); + } catch (CanceledException $exception) { + $this->assertSame($failure, $exception); + } + + $this->assertSame(1, $attempts); + $this->assertNotNull($server->request()); + } + + /** + * Provide synchronous and promise-based request execution. + */ + public static function requestExecutionModes(): array + { + return [[false], [true]]; + } + + #[DataProvider('trailerBodySizes')] + public function testCancellationDuringTrailerProcessingClosesTheBodyAndPreservesTheException(int $size): void + { + $server = LoopbackHttpServer::start([['body' => str_repeat('x', $size)]]); + $ready = new Channel(1); + $waiting = new Channel(1); + $cancellation = null; + $body = null; + $request = (new Factory)->withOptions([ + 'stream' => true, + 'on_headers' => static function (ResponseInterface $response) use (&$body): void { + $body = $response->getBody(); + }, + 'on_trailers' => static function () use ($ready, $waiting, &$cancellation): void { + $ready->push(Coroutine::id()); + + try { + $waiting->pop(2); + } catch (CanceledException $exception) { + $cancellation = $exception; + + throw $exception; + } + }, + ]); + + try { + $results = parallel([ + 'reader' => static function () use ($request, $server): ?CanceledException { + try { + $response = $request->get('http://127.0.0.1:' . $server->port); + $response->body(); + + return null; + } catch (CanceledException $exception) { + return $exception; + } + }, + 'cancel' => function () use ($ready): void { + $coroutine = $ready->pop(1); + $this->assertIsInt($coroutine); + usleep(1000); + $this->assertTrue(Coroutine::cancelById($coroutine, true)); + }, + ]); + + $this->assertInstanceOf(CanceledException::class, $results['reader']); + $this->assertSame($cancellation, $results['reader']); + $this->assertFalse($body->isReadable()); + } finally { + $body?->close(); + $ready->close(); + $waiting->close(); + } + } + + /** + * Provide transfers completing before exposure and during consumption. + */ + public static function trailerBodySizes(): array + { + return [[5], [65536]]; + } + + public function testAbandonedResponseBodyIsReleasedWithoutCyclicGarbageCollection(): void + { + $factory = new class(new CurlFactory(0)) implements CurlFactoryInterface { + public ?WeakReference $handle = null; + + /** + * Wrap the native handle factory. + */ + public function __construct(private CurlFactory $factory) + { + } + + /** + * Observe the handle without retaining request options or their callbacks. + */ + public function create(RequestInterface $request, array $options): EasyHandle + { + $easy = $this->factory->create($request, $options); + $this->handle = WeakReference::create($easy->handle); + + return $easy; + } + + /** + * Release the observed transfer through the real factory. + */ + public function release(EasyHandle $easy): void + { + $this->factory->release($easy); + } + }; + $handler = new CurlStreamingHandler; + (new ReflectionProperty($handler, 'factory'))->setValue($handler, $factory); + $garbageCollectionEnabled = gc_enabled(); + gc_disable(); + + try { + $server = LoopbackHttpServer::start(); + $response = (new Factory)->setHandler($handler)->withOptions(['stream' => true])->get('http://127.0.0.1:' . $server->port); + $body = WeakReference::create($response->toPsrResponse()->getBody()); + + unset($response); + + $this->assertNull($body->get()); + $this->assertNull($factory->handle->get()); + $this->assertNotNull($server->request()); + } finally { + if ($garbageCollectionEnabled) { + gc_enable(); + } + } + } + + public function testNativeWaitReportsDataArrivingBetweenPerformAndSelect(): void + { + // @TODO: Update the version gate when Swoole fixes curl_multi_select hiding ready data. + if (SWOOLE_VERSION_ID <= 60203) { + $this->markTestSkipped('Swoole 6.2.3 and earlier can consume ready cURL data and then wait for another event.'); + } + + $server = new Socket(AF_INET, SOCK_STREAM, 0); + $this->assertTrue($server->bind('127.0.0.1', 0)); + $this->assertTrue($server->listen()); + $url = 'http://127.0.0.1:' . $server->getsockname()['port']; + $sendFirst = new Channel(1); + $firstSent = new Channel(1); + $sendLast = new Channel(1); + + try { + [$selected] = parallel([ + function () use ($url, $sendFirst, $firstSent, $sendLast): int { + $headersReceived = false; + $body = ''; + $handle = curl_init($url); + $multi = curl_multi_init(); + curl_setopt_array($handle, [ + CURLOPT_PROXY => '', + CURLOPT_TIMEOUT => 3, + CURLOPT_HEADERFUNCTION => static function (CurlHandle $handle, string $header) use (&$headersReceived): int { + $headersReceived = $headersReceived || $header === "\r\n"; + + return strlen($header); + }, + CURLOPT_WRITEFUNCTION => static function (CurlHandle $handle, string $chunk) use (&$body): int { + $body .= $chunk; + + return strlen($chunk); + }, + ]); + curl_multi_add_handle($multi, $handle); + + try { + do { + curl_multi_exec($multi, $running); + + if ($running && ! $headersReceived) { + curl_multi_select($multi, 0.1); + } + } while ($running && ! $headersReceived); + + $this->assertTrue($headersReceived); + // Match CurlStreamingBody's perform-then-select window: the + // provider writes after perform, with its next event withheld. + $this->assertTrue($sendFirst->push(true)); + $this->assertTrue($firstSent->pop(2)); + + try { + $selected = curl_multi_select($multi, 0.1); + curl_multi_exec($multi, $running); + $this->assertSame('A', $body); + } finally { + $sendLast->push(true); + } + + do { + curl_multi_exec($multi, $running); + + if ($running) { + curl_multi_select($multi, 0.1); + } + } while ($running); + + $this->assertSame('AB', $body); + + return $selected; + } finally { + curl_multi_remove_handle($multi, $handle); + curl_multi_close($multi); + } + }, + function () use ($server, $sendFirst, $firstSent, $sendLast): void { + $client = $server->accept(2); + $this->assertInstanceOf(Socket::class, $client); + + try { + $request = ''; + + while (! str_contains($request, "\r\n\r\n")) { + $chunk = $client->recv(2); + $this->assertIsString($chunk); + $this->assertNotSame('', $chunk); + $request .= $chunk; + } + + $client->sendAll("HTTP/1.1 200 OK\r\nContent-Length: 2\r\nConnection: close\r\n\r\n"); + $this->assertTrue($sendFirst->pop(2)); + $this->assertSame(1, $client->sendAll('A')); + $firstSent->push(true); + $this->assertTrue($sendLast->pop(2)); + $this->assertSame(1, $client->sendAll('B')); + } finally { + $client->close(); + } + }, + ]); + + $this->assertGreaterThan(0, $selected, 'Already available provider data was consumed by the hook, then reported as a select timeout.'); + } finally { + $server->close(); + $sendFirst->close(); + $firstSent->close(); + $sendLast->close(); + } + } + + public function testSlowConsumersPauseDownloadsAndMayContinueBeyondTheHeaderTimeout(): void + { + $payload = str_repeat('streamed content ', 131072); + $server = LoopbackHttpServer::start([['body' => $payload]]); + $downloaded = 0; + $response = (new Factory)->timeout(0.5)->withOptions([ + 'stream' => true, + 'read_timeout' => 0, + 'progress' => static function (int $total, int $received) use (&$downloaded): void { + $downloaded = $received; + }, + ])->get('http://127.0.0.1:' . $server->port); + + try { + $body = $response->toPsrResponse()->getBody(); + $first = $body->read(1); + $pausedAt = $downloaded; + $this->assertGreaterThan(0, $pausedAt); + $this->assertLessThan(strlen($payload), $pausedAt); + $buffer = (new ReflectionProperty($body, 'buffer'))->getValue($body); + // BufferStream's 16 KiB high-water mark plus one CURL_MAX_WRITE_SIZE callback. + $this->assertLessThanOrEqual(32768, $buffer->getSize() + strlen($first)); + + // Allow the origin to progress while the caller holds its first byte. + usleep(600000); + $this->assertSame($pausedAt, $downloaded); + $this->assertSame($payload, $first . $body->getContents()); + } finally { + $response->close(); + } + } +} diff --git a/tests/Http/Client/Destinations/PublicDestinationPolicyTest.php b/tests/Http/Client/Destinations/PublicDestinationPolicyTest.php index c0c09f8e9f..83265d07db 100644 --- a/tests/Http/Client/Destinations/PublicDestinationPolicyTest.php +++ b/tests/Http/Client/Destinations/PublicDestinationPolicyTest.php @@ -208,13 +208,9 @@ public function testDirectPinningFallsBackAcrossAuthorizedAddresses(): void CURLOPT_PROXY => '', ] + $destination->curlOptions()); - try { - $this->assertSame('OK', curl_exec($curl)); - $this->assertSame(0, curl_errno($curl)); - $this->assertStringContainsString("Host: target.invalid:{$server->port}\r\n", (string) $server->request()); - } finally { - curl_close($curl); - } + $this->assertSame('OK', curl_exec($curl)); + $this->assertSame(0, curl_errno($curl)); + $this->assertStringContainsString("Host: target.invalid:{$server->port}\r\n", (string) $server->request()); } public function testDirectPinningPreservesTheHostWithoutSharedDnsResidue(): void @@ -239,29 +235,20 @@ public function testDirectPinningPreservesTheHostWithoutSharedDnsResidue(): void CURLOPT_SHARE => $share, ] + $destination->curlOptions()); - try { - $this->assertSame('OK', curl_exec($first)); - $this->assertSame(0, curl_errno($first)); - $this->assertStringContainsString("Host: target.invalid:{$port}\r\n", (string) $server->request()); - - $followUp = curl_init((string) $destination->uri); - curl_setopt_array($followUp, [ - CURLOPT_RETURNTRANSFER => true, - CURLOPT_TIMEOUT => 2, - CURLOPT_PROXY => '', - CURLOPT_SHARE => $share, - ]); - - try { - $this->assertFalse(curl_exec($followUp)); - $this->assertSame(CURLE_COULDNT_RESOLVE_HOST, curl_errno($followUp)); - } finally { - curl_close($followUp); - } - } finally { - curl_close($first); - curl_share_close($share); - } + $this->assertSame('OK', curl_exec($first)); + $this->assertSame(0, curl_errno($first)); + $this->assertStringContainsString("Host: target.invalid:{$port}\r\n", (string) $server->request()); + + $followUp = curl_init((string) $destination->uri); + curl_setopt_array($followUp, [ + CURLOPT_RETURNTRANSFER => true, + CURLOPT_TIMEOUT => 2, + CURLOPT_PROXY => '', + CURLOPT_SHARE => $share, + ]); + + $this->assertFalse(curl_exec($followUp)); + $this->assertSame(CURLE_COULDNT_RESOLVE_HOST, curl_errno($followUp)); } public function testProxyPinningPreservesTheProxyAndTargetHosts(): void @@ -283,18 +270,14 @@ public function testProxyPinningPreservesTheProxyAndTargetHosts(): void CURLOPT_PROXY => $destination->proxy, ] + $destination->curlOptions()); - try { - $this->assertSame('OK', curl_exec($curl)); - $this->assertSame(0, curl_errno($curl)); - $request = (string) $server->request(); - $this->assertStringStartsWith( - "GET http://target.invalid/probe HTTP/1.1\r\n", - $request, - ); - $this->assertStringContainsString("Host: target.invalid\r\n", $request); - } finally { - curl_close($curl); - } + $this->assertSame('OK', curl_exec($curl)); + $this->assertSame(0, curl_errno($curl)); + $request = (string) $server->request(); + $this->assertStringStartsWith( + "GET http://target.invalid/probe HTTP/1.1\r\n", + $request, + ); + $this->assertStringContainsString("Host: target.invalid\r\n", $request); } #[DataProvider('invalidUrls')] diff --git a/tests/Http/Fixtures/concurrent-stream-deadlines.php b/tests/Http/Fixtures/concurrent-stream-deadlines.php new file mode 100644 index 0000000000..b2868937b4 --- /dev/null +++ b/tests/Http/Fixtures/concurrent-stream-deadlines.php @@ -0,0 +1,93 @@ +bind('127.0.0.1', 0); + $listener->listen(); + $url = 'http://127.0.0.1:' . $listener->getsockname()['port']; + $arrived = new Channel(1); + $release = new Channel(1); + $finished = new Channel(2); + $result = new Channel(1); + $factory = (new Factory)->registerConnection('shared'); + + Coroutine::create(static function () use ($listener, $arrived, $release, $finished): void { + $client = $listener->accept(2); + try { + $request = ''; + while (! str_contains($request, "\r\n\r\n")) { + $chunk = $client->recv(2); + if ($chunk === false || $chunk === '') { + throw new RuntimeException('Request ended before its headers.'); + } + $request .= $chunk; + } + $arrived->push(true); + if ($release->pop(2) !== true) { + throw new RuntimeException('The first response was not released.'); + } + $client->sendAll("HTTP/1.1 200 OK\r\nContent-Length: 2\r\nConnection: close\r\n\r\nOK"); + } finally { + $client->close(); + $finished->push(true); + } + }); + + Coroutine::create(static function () use ($factory, $url, $result): void { + try { + $response = $factory->connection('shared')->timeout(0)->withOptions([ + 'stream' => true, 'stream_context' => [], 'read_timeout' => 2, 'proxy' => '', + ])->get($url); + try { + $result->push($response->body()); + } finally { + $response->close(); + } + } catch (Throwable $exception) { + $result->push($exception->getMessage()); + } + }); + + if ($arrived->pop(2) !== true) { + throw new RuntimeException('The first request did not reach the origin.'); + } + Coroutine::create(static function () use ($listener, $finished): void { + $client = $listener->accept(2); + try { + // Do not send headers. The second client's own timeout releases us. + while (($data = $client->recv(2)) !== '' && $data !== false); + } finally { + $client->close(); + $finished->push(true); + } + }); + + try { + $factory->connection('shared')->timeout(0.02)->withOptions([ + 'stream' => true, 'stream_context' => [], 'read_timeout' => 2, 'proxy' => '', + ])->get($url); + throw new RuntimeException('Expected request B to time out.'); + } catch (ConnectionException) { + // Native timeout rounding can return just before the monotonic deadline. + // Let a full B timeout interval pass before releasing A's headers. + Coroutine::sleep(0.03); + $release->push(true); + } + + echo json_encode($result->pop(2), JSON_THROW_ON_ERROR); + if ($finished->pop(2) !== true || $finished->pop(2) !== true) { + throw new RuntimeException('The origin requests did not finish.'); + } + $listener->close(); +}); diff --git a/tests/Http/Fixtures/concurrent-stream-headers.php b/tests/Http/Fixtures/concurrent-stream-headers.php new file mode 100644 index 0000000000..1babc5752c --- /dev/null +++ b/tests/Http/Fixtures/concurrent-stream-headers.php @@ -0,0 +1,82 @@ +bind('127.0.0.1', 0); + $listener->listen(); + $url = 'http://127.0.0.1:' . $listener->getsockname()['port']; + $payload = str_repeat('x', 16384); + $samples = []; + + try { + for ($sample = 0; $sample < 3; ++$sample) { + for ($batch = 0; $batch < 2; ++$batch) { + $arrived = new Channel($concurrency); + $done = new Channel($concurrency); + + Coroutine::create(static function () use ($listener, $arrived, $concurrency): void { + for ($index = 0; $index < $concurrency; ++$index) { + $client = $listener->accept(2); + $request = ''; + while (! str_contains($request, "\r\n\r\n")) { + $chunk = $client->recv(2); + if ($chunk === false || $chunk === '') { + throw new RuntimeException('Request ended before its headers.'); + } + $request .= $chunk; + } + $arrived->push($client); + } + }); + + for ($index = 0; $index < $concurrency; ++$index) { + Coroutine::create(static function () use ($url, $done): void { + $context = stream_context_create(['http' => ['timeout' => 2, 'header' => 'Connection: close']]); + $stream = fopen($url, 'r', false, $context); + if ($stream === false) { + throw new RuntimeException('HTTP open failed.'); + } + fclose($stream); + $done->push(true); + }); + } + + $clients = []; + for ($index = 0; $index < $concurrency; ++$index) { + $client = $arrived->pop(2); + if (! $client instanceof Socket) { + throw new RuntimeException('Not all requests reached the barrier.'); + } + $clients[] = $client; + } + // Every fopen has cleared the global and is waiting for headers. + foreach ($clients as $client) { + $client->sendAll("HTTP/1.1 200 OK\r\nX-Payload: {$payload}\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"); + $client->close(); + } + for ($index = 0; $index < $concurrency; ++$index) { + if ($done->pop(2) !== true) { + throw new RuntimeException('A client did not finish.'); + } + } + unset($clients, $client, $arrived, $done); + } + Coroutine::sleep(0.001); + http_clear_last_response_headers(); + gc_collect_cycles(); + $samples[] = memory_get_usage(); + } + } finally { + $listener->close(); + } + + echo json_encode($samples, JSON_THROW_ON_ERROR); +}); diff --git a/tests/Http/Fixtures/streaming-server.php b/tests/Http/Fixtures/streaming-server.php index 0a9b773334..347e6324f7 100644 --- a/tests/Http/Fixtures/streaming-server.php +++ b/tests/Http/Fixtures/streaming-server.php @@ -10,6 +10,7 @@ echo stream_socket_get_name($server, false), "\n"; fflush(STDOUT); $connection = null; +$start = null; $release = null; try { @@ -20,21 +21,96 @@ stream_set_timeout($connection, 5); while (($line = fgets($connection)) !== "\r\n" && $line !== false); + fwrite(STDOUT, "REQUEST\n"); - $first = $argv[1] === 'buffered' ? "{\"id\":1}\n" : ''; + $trailerBody = $argv[1] === 'streamed-trailers' ? str_repeat('hello', 8192) : 'hello'; + $scripted = match ($argv[1]) { + 'informational' => "HTTP/1.1 103 Early Hints\r\nLink: ; rel=preload\r\n\r\nHTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nfinal", + 'trailers', 'streamed-trailers' => "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\nTrailer: X-Checksum\r\n\r\n" . dechex(strlen($trailerBody)) . "\r\n{$trailerBody}\r\n0\r\nX-Checksum: abc\r\n\r\n", + default => null, + }; + + if ($scripted !== null) { + fwrite($connection, $scripted); + + return; + } + + if ($argv[1] === 'silent-headers') { + $start = stream_socket_accept($server, 5); + if ($start === false) { + throw new RuntimeException('The streaming test did not release the headers.'); + } + + stream_set_timeout($start, 5); + @fwrite($start, "ready\n"); + fgets($start); + + // Timeout and cancellation cases close the client before releasing us. + @fwrite($connection, "HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n"); + + return; + } + + if ($argv[1] === 'trickle-headers') { + fwrite($connection, "HTTP/1.1 200 OK\r\nX-Partial: "); + + for ($index = 0; $index < 8; ++$index) { + usleep(100000); + + // A header deadline may close the client before the line is complete. + if (@fwrite($connection, 'a') === false) { + return; + } + } + + @fwrite($connection, "\r\nContent-Length: 0\r\n\r\n"); + + return; + } + + $first = match ($argv[1]) { + 'buffered' => "{\"id\":1}\n", + 'truncated' => 'partial', + default => '', + }; $last = "{\"id\":2}\n"; - $length = strlen($first . $last); - fwrite($connection, "HTTP/1.1 200 OK\r\nContent-Type: application/x-ndjson\r\nContent-Length: {$length}\r\nConnection: close\r\n\r\n" . $first); + $length = $argv[1] === 'truncated' ? 100 : strlen($first . $last); + $chunked = $argv[1] === 'chunked'; + $framing = $chunked ? "Transfer-Encoding: chunked\r\n" : "Content-Length: {$length}\r\n"; + fwrite($connection, "HTTP/1.1 200 OK\r\nContent-Type: application/x-ndjson\r\n{$framing}Connection: close\r\n\r\n" . $first); fflush($connection); + if ($chunked) { + // Send body data only after the caller has received the response headers. + $start = stream_socket_accept($server, 5); + if ($start === false) { + throw new RuntimeException('The streaming test did not start the body.'); + } + + fwrite($connection, "9\r\n{\"id\":1}\n\r\n"); + fflush($connection); + } + // A second connection releases the final record independently of the reader. $release = stream_socket_accept($server, 5); if ($release === false) { throw new RuntimeException('The streaming test did not release the server.'); } - fwrite($connection, $last); + stream_set_timeout($release, 5); + @fwrite($release, "ready\n"); + fgets($release); + + if ($argv[1] === 'truncated') { + return; + } + + fwrite($connection, $chunked ? "9\r\n{$last}\r\n0\r\n\r\n" : $last); } finally { + if (is_resource($start)) { + fclose($start); + } if (is_resource($release)) { fclose($release); } diff --git a/tests/Http/HttpClientDestinationPolicyTest.php b/tests/Http/HttpClientDestinationPolicyTest.php index cd91d2d2dd..b7b685e18f 100644 --- a/tests/Http/HttpClientDestinationPolicyTest.php +++ b/tests/Http/HttpClientDestinationPolicyTest.php @@ -37,16 +37,19 @@ class HttpClientDestinationPolicyTest extends TestCase */ private array $connectionFailures = []; - public function testPinsTheRequestToTheVettedAddresses(): void + #[DataProvider('streamModes')] + public function testPinsTheRequestToTheVettedAddresses(bool $stream): void { $server = LoopbackHttpServer::start(); $policy = $this->loopbackPolicy('destination.invalid'); $response = $this->factory() ->withDestinationPolicy($policy) + ->withOptions(['stream' => $stream]) ->get("http://destination.invalid:{$server->port}/probe"); $this->assertSame('OK', $response->body()); + $this->assertSame('127.0.0.1', $response->handlerStats()['primary_ip']); $this->assertStringContainsString("Host: destination.invalid:{$server->port}\r\n", (string) $server->request()); $this->assertSame(["http://destination.invalid:{$server->port}/probe"], $policy->urls); } @@ -74,7 +77,8 @@ public function testPinsAHostWrittenWithATrailingDot(): void $this->assertStringContainsString("Host: destination.invalid.:{$server->port}\r\n", (string) $server->request()); } - public function testResolvesEveryRedirectAgain(): void + #[DataProvider('streamModes')] + public function testResolvesEveryRedirectAgain(bool $stream): void { $second = LoopbackHttpServer::start(); $first = LoopbackHttpServer::start([ @@ -84,6 +88,7 @@ public function testResolvesEveryRedirectAgain(): void $response = $this->factory() ->withDestinationPolicy($policy) + ->withOptions(['stream' => $stream]) ->get("http://first.invalid:{$first->port}/start"); $this->assertSame('OK', $response->body()); @@ -93,7 +98,8 @@ public function testResolvesEveryRedirectAgain(): void ], $policy->urls); } - public function testPinsAnApprovedProxyWithoutResolvingTheTarget(): void + #[DataProvider('streamModes')] + public function testPinsAnApprovedProxyWithoutResolvingTheTarget(bool $stream): void { $proxy = LoopbackHttpServer::start(); $policy = new FakeDestinationPolicy( @@ -104,6 +110,7 @@ public function testPinsAnApprovedProxyWithoutResolvingTheTarget(): void $response = $this->factory() ->withDestinationPolicy($policy) + ->withOptions(['stream' => $stream]) ->get('http://target.invalid/probe'); $this->assertSame('OK', $response->body()); @@ -111,6 +118,14 @@ public function testPinsAnApprovedProxyWithoutResolvingTheTarget(): void $this->assertSame(['proxy.invalid'], $policy->resolvedHosts); } + /** + * Provide buffered and incremental response transports. + */ + public static function streamModes(): array + { + return [[false], [true]]; + } + /** * @param Closure(PendingRequest): PendingRequest $configure */ @@ -136,9 +151,9 @@ public function testRejectsRequestsThePinsCannotCover(Closure $configure, string */ public static function unpinnableRequests(): iterable { - yield 'streamed response' => [ - static fn (PendingRequest $request): PendingRequest => $request->withOptions(['stream' => true]), - 'Destination-restricted requests cannot stream responses; use [sink] instead.', + yield 'fallback streamed response' => [ + static fn (PendingRequest $request): PendingRequest => $request->withOptions(['stream' => true, 'stream_context' => []]), + 'Destination-restricted requests cannot stream responses with the fallback transport; use [sink] instead.', ]; yield 'custom handler' => [ static fn (PendingRequest $request): PendingRequest => $request->setHandler( @@ -241,7 +256,7 @@ public function testAnUnresolvableProxyIsARetriedProxyConnectionFailure(bool $as } #[DataProvider('refusingProxies')] - public function testAProxyRefusingConnectionsIsARetriedProxyConnectionFailure(bool $async, bool $ipv6): void + public function testAProxyRefusingConnectionsIsARetriedProxyConnectionFailure(bool $async, bool $ipv6, bool $stream): void { [$refusing, $port] = $this->refusingPort($ipv6); $errors = []; @@ -252,6 +267,7 @@ public function testAProxyRefusingConnectionsIsARetriedProxyConnectionFailure(bo ); $request = $this->factory() ->withDestinationPolicy($policy) + ->withOptions(['stream' => $stream]) ->withOptions(['on_stats' => static function (TransferStats $stats) use (&$errors): void { $errors[] = $stats->getHandlerErrorData(); }]) @@ -276,10 +292,14 @@ public function testAProxyRefusingConnectionsIsARetriedProxyConnectionFailure(bo */ public static function refusingProxies(): iterable { - yield 'synchronous hostname' => [false, false]; - yield 'asynchronous hostname' => [true, false]; - yield 'synchronous IPv6 literal' => [false, true]; - yield 'asynchronous IPv6 literal' => [true, true]; + yield 'synchronous hostname' => [false, false, false]; + yield 'asynchronous hostname' => [true, false, false]; + yield 'synchronous IPv6 literal' => [false, true, false]; + yield 'asynchronous IPv6 literal' => [true, true, false]; + yield 'streamed synchronous hostname' => [false, false, true]; + yield 'streamed asynchronous hostname' => [true, false, true]; + yield 'streamed synchronous IPv6 literal' => [false, true, true]; + yield 'streamed asynchronous IPv6 literal' => [true, true, true]; } public function testARefusedDirectConnectionStaysAPlainConnectionException(): void diff --git a/tests/Http/HttpClientStreamingTest.php b/tests/Http/HttpClientStreamingTest.php index 848259a381..0ca1aad00b 100644 --- a/tests/Http/HttpClientStreamingTest.php +++ b/tests/Http/HttpClientStreamingTest.php @@ -5,10 +5,18 @@ namespace Hypervel\Tests\Http; use Closure; +use GuzzleHttp\ClientInterface; +use GuzzleHttp\Exception\TransferException; +use GuzzleHttp\Handler\StreamHandler; +use Hypervel\Engine\Coroutine as EngineCoroutine; +use Hypervel\Http\Client\ConnectionException; +use Hypervel\Http\Client\CurlStreamingHandler; use Hypervel\Http\Client\Factory; use Hypervel\Tests\TestCase; use PHPUnit\Framework\Attributes\DataProvider; +use Psr\Http\Message\ResponseInterface; use RuntimeException; +use Swoole\Coroutine\CanceledException; use Swoole\Coroutine\Channel; use Symfony\Component\Process\Process; use Throwable; @@ -78,19 +86,60 @@ public function testStreamingReadsAllowOtherCoroutinesToProgress(): void }); } - public function testBufferedFirstRecordArrivesBeforeTheNextServerWrite(): void + public function testConcurrentPhpStreamsReleaseTheirResponseHeaders(): void { + // @TODO: Update the version gate when Swoole releases the response-header ownership fix. if (SWOOLE_VERSION_ID <= 60203) { + $this->markTestSkipped('Swoole 6.2.3 and earlier do not isolate PHP response-header ownership between coroutines.'); + } + + $process = new Process([PHP_BINARY, __DIR__ . '/Fixtures/concurrent-stream-headers.php']); + $process->setTimeout(10); + $process->run(); + + $this->assertSame(0, $process->getExitCode(), $process->getErrorOutput()); + $samples = json_decode($process->getOutput(), true, flags: JSON_THROW_ON_ERROR); + $this->assertCount(3, $samples); + + // Allow allocator bookkeeping, but not a retained batch of 16 KiB headers. + $this->assertLessThan(131072, max($samples) - min($samples), $process->getOutput()); + } + + public function testConcurrentFallbackRequestsKeepIndependentDeadlines(): void + { + // @TODO: Remove this skip once Guzzle fixes shared StreamHandler request deadlines. + if (ClientInterface::MAJOR_VERSION >= 8) { + $this->markTestSkipped('Guzzle 8 stores concurrent StreamHandler request deadlines on the shared handler.'); + } + + $process = new Process([PHP_BINARY, __DIR__ . '/Fixtures/concurrent-stream-deadlines.php']); + $process->setTimeout(10); + $process->run(); + + $this->assertSame(0, $process->getExitCode(), $process->getErrorOutput()); + $this->assertSame('OK', json_decode($process->getOutput(), true, flags: JSON_THROW_ON_ERROR)); + } + + #[DataProvider('streamingTransports')] + public function testBufferedFirstRecordArrivesBeforeTheNextServerWrite(bool $native, string $mode): void + { + if (! $native && SWOOLE_VERSION_ID <= 60203) { $this->markTestSkipped('Swoole 6.2.3 and earlier lack the buffered-read fix: https://github.com/swoole/swoole-src/pull/6235.'); } - $this->withStreamingServer('buffered', function (string $address): void { + $this->withStreamingServer($mode, function (string $address) use ($native, $mode): void { $received = new Channel(1); try { $results = parallel([ - 'reader' => function () use ($address, $received): array { - $response = (new Factory)->withOptions(['stream' => true, 'read_timeout' => 3])->get('http://' . $address); + 'reader' => function () use ($address, $received, $native, $mode): array { + $response = (new Factory)->withOptions(['stream' => true, 'read_timeout' => 3]) + ->setHandler($native ? new CurlStreamingHandler : new StreamHandler) + ->get('http://' . $address); try { + if ($mode === 'chunked') { + $this->releaseServer($address); + } + $lines = $response->jsonLines(); $first = $lines->current(); $received->push($first); @@ -119,18 +168,361 @@ public function testBufferedFirstRecordArrivesBeforeTheNextServerWrite(): void }); } - public function testIdleStreamingReadTimeoutRaisesTheStreamReadError(): void + /** + * Provide the native streaming route and its PHP-stream fallback. + */ + public static function streamingTransports(): array { - if (SWOOLE_VERSION_ID <= 60203) { + return [ + 'native cURL with buffered data' => [true, 'buffered'], + 'native cURL with later chunked data' => [true, 'chunked'], + 'PHP stream fallback' => [false, 'buffered'], + ]; + } + + public function testInformationalHeadersAreNotExposedAsTheFinalResponse(): void + { + $this->withStreamingServer('informational', function (string $address): void { + $statuses = []; + $response = (new Factory)->withOptions([ + 'stream' => true, + 'on_headers' => static function (ResponseInterface $response) use (&$statuses): void { + $statuses[] = $response->getStatusCode(); + }, + ])->get('http://' . $address); + + try { + $this->assertSame([200], $statuses); + $this->assertSame('final', $response->body()); + } finally { + $response->close(); + } + }); + } + + public function testTruncatedBodiesFailInsteadOfReportingSuccessfulEndOfStream(): void + { + $this->withStreamingServer('truncated', function (string $address): void { + $response = (new Factory)->withOptions(['stream' => true])->get('http://' . $address); + + try { + $this->assertSame('partial', $response->toPsrResponse()->getBody()->read(8192)); + $this->releaseServer($address); + $response->body(); + + $this->fail('Expected a truncated-response error.'); + } catch (TransferException $exception) { + $this->assertStringContainsString('cURL error 18', $exception->getMessage()); + $this->assertFalse($response->toPsrResponse()->getBody()->isReadable()); + } finally { + $response->close(); + } + }); + } + + #[DataProvider('trailerResponses')] + public function testTrailersAreDeliveredOnceAfterHeadersAndTransferCompletion(string $mode, string $body): void + { + $this->withStreamingServer($mode, function (string $address) use ($mode, $body): void { + $trailers = []; + $events = []; + $response = (new Factory)->withOptions([ + 'stream' => true, + 'on_headers' => static function () use (&$events): void { + $events[] = 'headers'; + }, + 'on_trailers' => static function (array $headers) use (&$trailers, &$events): void { + $events[] = 'trailers'; + $trailers[] = $headers; + }, + ])->get('http://' . $address); + + try { + if ($mode === 'streamed-trailers') { + $this->assertSame([], $trailers); + } + + $this->assertSame($body, $response->body()); + $this->assertSame(['headers', 'trailers'], $events); + $this->assertSame([['x-checksum' => ['abc']]], $trailers); + $this->assertSame('', $response->body()); + $this->assertCount(1, $trailers); + } finally { + $response->close(); + } + }); + } + + /** + * Provide transfers that complete before exposure and during consumption. + */ + public static function trailerResponses(): array + { + return [ + 'small response' => ['trailers', 'hello'], + 'streamed response' => ['streamed-trailers', str_repeat('hello', 8192)], + ]; + } + + public function testCancelingASilentBodyReadFinishesBeforeTheProviderContinues(): void + { + $this->withStreamingServer('delayed', function (string $address): void { + $ready = new Channel(1); + $finished = new Channel(1); + + try { + $results = parallel([ + 'reader' => function () use ($address, $ready, $finished): ?CanceledException { + $response = (new Factory)->withOptions(['stream' => true])->get('http://' . $address); + + try { + $ready->push(EngineCoroutine::id()); + $response->lines()->current(); + + return null; + } catch (CanceledException $exception) { + return $exception; + } finally { + $response->close(); + $finished->push(true); + } + }, + 'cancel' => function () use ($address, $ready, $finished): bool { + try { + $coroutine = $ready->pop(1); + $this->assertIsInt($coroutine); + // Channel delivery resumes us before the reader enters its I/O wait. + usleep(1000); + $this->assertTrue(EngineCoroutine::cancelById($coroutine)); + + return $finished->pop(1) === true; + } finally { + $this->releaseServer($address); + } + }, + ]); + + $this->assertInstanceOf(CanceledException::class, $results['reader']); + $this->assertTrue($results['cancel'], 'The reader waited for provider data after cancellation.'); + } finally { + $ready->close(); + $finished->close(); + } + }); + } + + public function testAResponseCanBeConsumedAfterItsCreatingCoroutineEnds(): void + { + $this->withStreamingServer('buffered', function (string $address): void { + [$response] = parallel([ + static fn () => (new Factory)->withOptions(['stream' => true])->get('http://' . $address), + ]); + + try { + $lines = $response->jsonLines(); + $this->assertSame(['id' => 1], $lines->current()); + $this->releaseServer($address); + $lines->next(); + $this->assertSame(['id' => 2], $lines->current()); + $lines->next(); + $this->assertFalse($lines->valid()); + } finally { + $response->close(); + } + }); + } + + #[DataProvider('connectionRoutes')] + public function testNamedStreamsReuseConnectionsWithoutSharingCredentialsOrCookies(bool $proxy): void + { + $process = new Process([PHP_BINARY, dirname(__DIR__) . '/HttpServer/Fixtures/disconnect-server.php', '', 'process']); + $process->setTimeout(10); + $process->start(); + $processId = $process->getPid(); + + try { + $deadline = microtime(true) + 3; + + while (! str_contains($process->getOutput(), 'READY ') && $process->isRunning() && microtime(true) < $deadline) { + usleep(10000); + } + + $this->assertSame(1, preg_match('/READY (\d+)/', $process->getOutput(), $matches), $process->getErrorOutput()); + $endpoint = 'http://127.0.0.1:' . $matches[1]; + $url = ($proxy ? 'http://provider.invalid' : $endpoint) . '/identity'; + $failure = null; + + run(function () use ($url, $endpoint, $proxy, &$failure): void { + try { + $factory = (new Factory)->registerConnection('provider'); + $connections = []; + $tokens = ['first-account', 'second-account', 'first-account', null, 'second-account']; + + for ($index = 3; $index <= CurlStreamingHandler::MAX_IDLE_CONNECTIONS; ++$index) { + $tokens[] = 'account-' . $index; + } + + $tokens[] = 'account-' . CurlStreamingHandler::MAX_IDLE_CONNECTIONS; + $tokens[] = 'first-account'; + + foreach ($tokens as $token) { + $request = $factory->connection('provider')->withOptions(['stream' => true]); + + if ($proxy) { + $request->withOptions([ + 'proxy' => $endpoint, + 'curl' => [ + CURLOPT_HTTPPROXYTUNNEL => true, + CURLOPT_PROXYHEADER => $token === null ? [] : ['Proxy-Authorization: Bearer ' . $token], + ], + ]); + } + + if ($token !== null) { + $request->withToken($token); + } + + $response = $request->get($url); + + try { + $identity = $response->json(); + $connections[] = $identity['connection']; + $this->assertSame($token === null ? null : 'Bearer ' . $token, $identity['authorization']); + $this->assertSame($proxy && $token !== null ? 'Bearer ' . $token : null, $identity['proxy_authorization']); + $this->assertNull($identity['cookie']); + } finally { + $response->close(); + } + } + + $this->assertCount($proxy ? CurlStreamingHandler::MAX_IDLE_CONNECTIONS + 2 : 1, array_unique($connections)); + $this->assertSame($connections[0], $connections[2]); + $this->assertSame($connections[1], $connections[4]); + $this->assertSame($connections[count($connections) - 3], $connections[count($connections) - 2]); + + if ($proxy) { + $this->assertNotSame($connections[0], end($connections), 'The least recently used identity was not evicted.'); + } + } catch (Throwable $exception) { + $failure = $exception; + } + }, SWOOLE_HOOK_ALL); + + if ($failure !== null) { + throw $failure; + } + } finally { + posix_kill(-$processId, SIGKILL); + $process->stop(0); + } + } + + /** + * Provide direct requests and connection-authenticated proxy tunnels. + */ + public static function connectionRoutes(): array + { + return [[false], [true]]; + } + + public function testCancelingAHeaderWaitFinishesBeforeTheProviderResponds(): void + { + $this->withStreamingServer('silent-headers', function (string $address): void { + $started = new Channel(1); + $finished = new Channel(1); + + try { + $results = parallel([ + 'reader' => function () use ($address, $started, $finished): ?CanceledException { + $started->push(EngineCoroutine::id()); + + try { + (new Factory)->withOptions(['stream' => true])->get('http://' . $address); + + return null; + } catch (CanceledException $exception) { + return $exception; + } finally { + $finished->push(true); + } + }, + 'cancel' => function () use ($address, $started, $finished): void { + $coroutine = $started->pop(1); + $control = stream_socket_client('tcp://' . $address, $error, $message, 2); + $this->assertIsResource($control, $message); + + try { + stream_set_timeout($control, 2); + $this->assertSame("ready\n", fgets($control)); + $this->assertTrue(EngineCoroutine::cancelById($coroutine)); + $this->assertTrue($finished->pop(1), 'The header wait continued until the provider responded.'); + } finally { + fclose($control); + } + }, + ]); + + $this->assertInstanceOf(CanceledException::class, $results['reader']); + } finally { + $started->close(); + $finished->close(); + } + }); + } + + public function testSilentHeadersUseTheIdleTimeoutWhenTheTotalTimeoutIsDisabled(): void + { + $this->withStreamingServer('silent-headers', function (string $address): void { + try { + $this->expectException(ConnectionException::class); + $this->expectExceptionMessage('The streaming request timed out before response headers.'); + + (new Factory)->timeout(0)->withOptions(['stream' => true, 'read_timeout' => 0.2])->get('http://' . $address); + } finally { + $this->releaseServer($address); + } + }); + } + + public function testHeaderProgressResetsTheIdleTimeoutBeforeAHeaderLineIsComplete(): void + { + $this->withStreamingServer('trickle-headers', function (string $address): void { + $response = (new Factory)->timeout(0)->withOptions(['stream' => true, 'read_timeout' => 0.4])->get('http://' . $address); + + try { + $this->assertSame(200, $response->status()); + $this->assertSame('aaaaaaaa', $response->header('X-Partial')); + } finally { + $response->close(); + } + }); + } + + public function testHeaderProgressDoesNotResetTheTotalHeaderDeadline(): void + { + $this->withStreamingServer('trickle-headers', function (string $address): void { + $this->expectException(ConnectionException::class); + $this->expectExceptionMessage('The streaming request timed out before response headers.'); + + (new Factory)->timeout(0.4)->withOptions(['stream' => true, 'read_timeout' => 0.4])->get('http://' . $address); + }); + } + + #[DataProvider('idleTimeoutTransports')] + public function testIdleStreamingReadTimeoutRaisesTheStreamReadError(bool $native): void + { + if (! $native && SWOOLE_VERSION_ID <= 60203) { $this->markTestSkipped('Swoole 6.2.3 and earlier lack the read-timeout fix: https://github.com/swoole/swoole-src/pull/6236.'); } - $this->withStreamingServer('delayed', function (string $address): void { + $this->withStreamingServer('delayed', function (string $address) use ($native): void { $finished = new Channel(1); try { $results = parallel([ - 'reader' => function () use ($address, $finished): ?RuntimeException { - $response = (new Factory)->withOptions(['stream' => true, 'read_timeout' => 1])->get('http://' . $address); + 'reader' => function () use ($address, $finished, $native): ?RuntimeException { + $response = (new Factory)->withOptions(['stream' => true, 'read_timeout' => 1]) + ->setHandler($native ? new CurlStreamingHandler : new StreamHandler) + ->get('http://' . $address); try { $response->lines()->current(); @@ -149,13 +541,21 @@ public function testIdleStreamingReadTimeoutRaisesTheStreamReadError(): void ]); $this->assertInstanceOf(RuntimeException::class, $results['reader']); - $this->assertSame('Unable to read from stream', $results['reader']->getMessage()); + $this->assertSame($native ? 'The streaming response read timed out.' : 'Unable to read from stream', $results['reader']->getMessage()); } finally { $finished->close(); } }); } + /** + * Provide both transports for their respective read-timeout contracts. + */ + public static function idleTimeoutTransports(): array + { + return [[true], [false]]; + } + /** * Run a hooked client against an independently controlled loopback server. */ @@ -166,8 +566,13 @@ protected function withStreamingServer(string $mode, Closure $callback): void $process->start(); try { - $ready = $process->waitUntil(fn () => str_contains($process->getOutput(), "\n")); - $this->assertTrue($ready, $process->getErrorOutput()); + $deadline = microtime(true) + 5; + + while (! str_contains($process->getOutput(), "\n") && $process->isRunning() && microtime(true) < $deadline) { + usleep(10000); + } + + $this->assertStringContainsString("\n", $process->getOutput(), $process->getErrorOutput()); $address = trim($process->getOutput()); $failure = null; diff --git a/tests/Http/HttpConnectionTest.php b/tests/Http/HttpConnectionTest.php index 63959a64f8..f11c876a00 100644 --- a/tests/Http/HttpConnectionTest.php +++ b/tests/Http/HttpConnectionTest.php @@ -17,6 +17,12 @@ use PHPUnit\Framework\Attributes\DataProvider; use Psr\Http\Message\RequestInterface; use ReflectionMethod; +use Swoole\Coroutine; +use Swoole\Coroutine\Channel; +use Swoole\Coroutine\Http\Server; +use Swoole\Http\Request as ServerRequest; +use Swoole\Http\Response as ServerResponse; +use Throwable; use function Hypervel\Coroutine\parallel; use function Hypervel\Coroutine\run; @@ -54,6 +60,64 @@ public function testConnectionReusesOneLowLevelHandlerAcrossFreshClientsAndConcu $this->assertCount(10, $factory->invocations); } + public function testNamedBufferedRequestsReuseConnectionsAcrossConcurrentBursts(): void + { + // @TODO Unskip once Guzzle supports configurable idle-handle retention and Hypervel uses it for named buffered requests. + $this->markTestSkipped('Guzzle limits the synchronous handler to three idle handles.'); + + $bursts = []; + $failure = null; + + run(function () use (&$bursts, &$failure): void { + $server = new Server('127.0.0.1', 0, false, false); + $ready = new Channel(4); + $arrived = 0; + $factory = (new Factory)->registerConnection('api'); + + $server->handle('/', static function (ServerRequest $request, ServerResponse $response) use ($ready, &$arrived): void { + // Hold each burst until all four requests own an active connection. + if (++$arrived % 4 === 0) { + for ($index = 0; $index < 4; ++$index) { + $ready->push(true); + } + } + + if ($ready->pop(2) !== true) { + $response->status(503); + $response->end('Not all requests reached the barrier.'); + + return; + } + + $response->end((string) $request->server['remote_port']); + }); + Coroutine::create(fn (): bool => $server->start()); + $url = 'http://127.0.0.1:' . $server->port; + + try { + for ($burst = 0; $burst < 2; ++$burst) { + $bursts[] = parallel(array_fill(0, 4, fn (): string => $factory->connection('api') + ->withOptions(['proxy' => '', 'version' => '1.1']) + ->timeout(3)->get($url)->throw()->body())); + } + } catch (Throwable $exception) { + $failure = $exception; + } finally { + $factory->forgetConnectionHandlers(); + $server->shutdown(); + $ready->close(); + } + }); + + if ($failure !== null) { + throw $failure; + } + + $this->assertCount(4, array_unique($bursts[0])); + $this->assertCount(4, array_unique($bursts[1])); + $this->assertCount(4, array_unique(array_merge(...$bursts)), 'The second burst opened another connection instead of reusing all four warm connections.'); + } + public function testEachRequestKeepsItsOwnMiddlewareStack(): void { $factory = new RecordingHttpConnectionFactory; diff --git a/tests/Http/PackageMetadataTest.php b/tests/Http/PackageMetadataTest.php index 58ce9f7a20..2e9e73979a 100644 --- a/tests/Http/PackageMetadataTest.php +++ b/tests/Http/PackageMetadataTest.php @@ -35,6 +35,10 @@ public function testRuntimeAndOptionalDependenciesAreDeclared(): void $this->assertSame($rootComposer['require'][$dependency], $composer['require'][$dependency]); } + foreach (['hypervel/coroutine', 'hypervel/engine'] as $dependency) { + $this->assertArrayHasKey($dependency, $composer['require']); + } + $this->assertSame('*', $composer['require']['ext-filter']); $this->assertSame( 'Required to use Hypervel\Http\Testing\FileFactory::image().', diff --git a/tests/HttpServer/Fixtures/disconnect-server.php b/tests/HttpServer/Fixtures/disconnect-server.php new file mode 100644 index 0000000000..429fad04bf --- /dev/null +++ b/tests/HttpServer/Fixtures/disconnect-server.php @@ -0,0 +1,102 @@ +init(new ServerConfig([ + 'mode' => $argv[2] === 'base' ? SWOOLE_BASE : SWOOLE_PROCESS, + 'settings' => [ + 'worker_num' => 1, + 'reactor_num' => 1, + 'enable_coroutine' => true, + 'open_http2_protocol' => true, + 'hook_flags' => SWOOLE_HOOK_ALL, + 'log_level' => SWOOLE_LOG_WARNING, + ], + 'servers' => [ + [ + 'name' => 'http', + 'host' => '127.0.0.1', + 'port' => 0, + 'callbacks' => [ + Event::ON_WORKER_START => static function (NativeServer $server): void { + fwrite(STDOUT, 'READY ' . $server->ports[0]->port . "\n"); + }, + Event::ON_REQUEST => static function (Request $request, Response $native) use ($provider, &$proxyAuthorizations): void { + if ($request->server['request_method'] === 'CONNECT') { + $proxyAuthorizations[$request->fd] = $request->header['proxy-authorization'] ?? null; + $native->end(); + + return; + } + + if ($request->server['request_uri'] === '/health') { + $native->end('healthy'); + + return; + } + + if ($request->server['request_uri'] === '/identity') { + $native->cookie('session', 'previous-request'); + $native->end(json_encode([ + 'connection' => $request->fd, + 'authorization' => $request->header['authorization'] ?? null, + 'proxy_authorization' => $proxyAuthorizations[$request->fd] ?? null, + 'cookie' => $request->header['cookie'] ?? null, + ], JSON_THROW_ON_ERROR)); + + return; + } + + $response = (new IterableStreamedResponse((static function () use ($provider): iterable { + $upstream = null; + + try { + $upstream = (new Factory)->withOptions(['stream' => true])->get('http://' . $provider); + fwrite(STDOUT, "UPSTREAM_HEADERS\n"); + + yield from $upstream->lines(); + } finally { + $upstream?->close(); + fwrite(STDOUT, "STOPPED\n"); + } + })()))->cancelOnDisconnect(); + + ResponseBridge::send($response, $native, streamId: $request->streamId ?? 0); + }, + Event::ON_CLOSE => static function (NativeServer $server, int $connection) use (&$proxyAuthorizations): void { + unset($proxyAuthorizations[$connection]); + fwrite(STDOUT, "APPLICATION_CLOSE\n"); + }, + ], + ], + ], +])); +$server->start(); diff --git a/tests/HttpServer/PackageMetadataTest.php b/tests/HttpServer/PackageMetadataTest.php index 827dfde6b7..7fae22ce97 100644 --- a/tests/HttpServer/PackageMetadataTest.php +++ b/tests/HttpServer/PackageMetadataTest.php @@ -30,6 +30,7 @@ public function testDirectRuntimeDependenciesAreDeclared(): void 'hypervel/coordinator', 'hypervel/coroutine', 'hypervel/http', + 'hypervel/server', 'symfony/http-foundation', ] as $dependency) { $this->assertArrayHasKey($dependency, $composer['require']); @@ -38,6 +39,5 @@ public function testDirectRuntimeDependenciesAreDeclared(): void } $this->assertArrayNotHasKey('hypervel/engine', $composer['require']); - $this->assertArrayNotHasKey('hypervel/server', $composer['require']); } } diff --git a/tests/HttpServer/ResponseBridgeTest.php b/tests/HttpServer/ResponseBridgeTest.php index 06933ebda6..9a19135356 100644 --- a/tests/HttpServer/ResponseBridgeTest.php +++ b/tests/HttpServer/ResponseBridgeTest.php @@ -11,6 +11,7 @@ use Hypervel\Http\Request; use Hypervel\Http\Response as HypervelResponse; use Hypervel\HttpServer\ResponseBridge; +use Hypervel\Server\ResponseCancellation; use Hypervel\Testing\ParallelTesting; use Hypervel\Tests\TestCase; use Mockery as m; @@ -18,12 +19,16 @@ use ReflectionProperty; use RuntimeException; use SplTempFileObject; +use Swoole\Coroutine\CanceledException; +use Swoole\Coroutine\Channel; use Swoole\Http\Response as SwooleResponse; use Symfony\Component\HttpFoundation\BinaryFileResponse; use Symfony\Component\HttpFoundation\Cookie; use Symfony\Component\HttpFoundation\Response; use Symfony\Component\HttpFoundation\StreamedResponse; +use function Hypervel\Coroutine\parallel; + class ResponseBridgeTest extends TestCase { protected string $tempDirectory; @@ -1232,6 +1237,131 @@ public function testStatusFailureThrowsBeforeHeaders(): void ResponseBridge::send(new Response('body'), $swooleResponse); } + #[DataProvider('disconnectCancellationOptions')] + public function testDisconnectCancelsAllActiveStreamsOnOnlyThatConnection(bool $cancel, array $streams): void + { + $ready = new Channel(3); + $continue = new Channel(3); + $closed = []; + $produce = function (int $connection, int $stream) use ($ready, $continue, $cancel, &$closed): bool { + $response = new IterableStreamedResponse((function () use ($connection, $stream, $ready, $continue, &$closed): iterable { + try { + $ready->push(true); + $this->assertTrue($continue->pop(2)); + yield 'body'; + } finally { + $closed[] = [$connection, $stream]; + } + })()); + $native = $this->mockSwooleResponse(); + $native->fd = $connection; + + if ($cancel) { + $response->cancelOnDisconnect(); + $native->expects('isWritable')->andReturnTrue(); + } + + try { + ResponseBridge::send($response, $native, streamId: $stream); + + return false; + } catch (CanceledException) { + return true; + } + }; + + try { + $results = parallel([ + 'first' => fn (): bool => $produce(10, $streams[0]), + 'second' => fn (): bool => $produce(10, $streams[1]), + 'other' => fn (): bool => $produce(20, 1), + 'close' => function () use ($ready, $continue, $cancel): void { + for ($index = 0; $index < 3; ++$index) { + $this->assertTrue($ready->pop(1)); + } + + usleep(1000); + ResponseCancellation::cancel(10); + + for ($index = 0; $index < ($cancel ? 1 : 3); ++$index) { + $continue->push(true); + } + }, + ]); + + $this->assertSame($cancel, $results['first']); + $this->assertSame($cancel, $results['second']); + $this->assertFalse($results['other']); + $this->assertEqualsCanonicalizing([[10, $streams[0]], [10, $streams[1]], [20, 1]], $closed); + } finally { + $ready->close(); + $continue->close(); + } + } + + /** + * Provide default production and explicit disconnect cancellation. + */ + public static function disconnectCancellationOptions(): array + { + return [ + 'default opt-out' => [false, [1, 3]], + 'HTTP/2 streams' => [true, [1, 3]], + 'pipelined HTTP/1 responses' => [true, [0, 0]], + ]; + } + + public function testDisconnectAfterProductionDoesNotCancelLaterRequestWork(): void + { + $sent = new Channel(1); + $continue = new Channel(1); + + try { + $results = parallel([ + 'producer' => function () use ($sent, $continue): mixed { + $response = (new IterableStreamedResponse(['body']))->cancelOnDisconnect(); + $native = $this->mockSwooleResponse(); + $native->fd = 10; + $native->expects('isWritable')->andReturnTrue(); + ResponseBridge::send($response, $native); + $sent->push(true); + + return $continue->pop(2); + }, + 'close' => function () use ($sent, $continue): void { + $this->assertTrue($sent->pop(1)); + usleep(1000); + ResponseCancellation::cancel(10); + $continue->push(true); + }, + ]); + + $this->assertTrue($results['producer']); + } finally { + $sent->close(); + $continue->close(); + } + } + + public function testAlreadyDisconnectedClientDoesNotStartAnOptedInProducer(): void + { + $started = false; + $response = (new IterableStreamedResponse((static function () use (&$started): iterable { + $started = true; + yield 'body'; + })()))->cancelOnDisconnect(); + $native = $this->mockSwooleResponse(); + $native->expects('isWritable')->andReturnFalse(); + $native->shouldNotReceive('write'); + + try { + ResponseBridge::send($response, $native); + $this->fail('Expected cancellation before producing the response.'); + } catch (CanceledException) { + $this->assertFalse($started); + } + } + public function testHeaderFailureThrowsBeforeBody(): void { $swooleResponse = $this->mockSwooleResponse(); diff --git a/tests/HttpServer/ResponseCancellationTest.php b/tests/HttpServer/ResponseCancellationTest.php new file mode 100644 index 0000000000..db8f7ce236 --- /dev/null +++ b/tests/HttpServer/ResponseCancellationTest.php @@ -0,0 +1,225 @@ +setTimeout(10); + $server = null; + $serverProcessId = null; + $client = null; + $control = null; + + try { + $provider->start(); + $this->waitForOutput($provider, "\n"); + $providerAddress = trim($provider->getOutput()); + $server = new Process([PHP_BINARY, __DIR__ . '/Fixtures/disconnect-server.php', $providerAddress, $mode]); + $server->setTimeout(10); + $server->start(); + $serverProcessId = $server->getPid(); + $this->waitForOutput($server, 'READY '); + preg_match('/READY (\d+)/', $server->getOutput(), $matches); + $address = '127.0.0.1:' . $matches[1]; + + $client = stream_socket_client('tcp://' . $address, $error, $message, 2); + $this->assertIsResource($client, $message); + fwrite($client, "GET /stream HTTP/1.1\r\nHost: localhost\r\n\r\n"); + $this->waitForOutput($provider, 'REQUEST'); + + $control = stream_socket_client('tcp://' . $providerAddress, $error, $message, 2); + $this->assertIsResource($control, $message); + stream_set_timeout($control, 2); + $this->assertSame("ready\n", fgets($control)); + + if ($phase === 'delayed') { + $this->waitForOutput($server, 'UPSTREAM_HEADERS'); + } + + fclose($client); + $client = null; + $this->waitForOutput($server, 'STOPPED'); + $this->waitForOutput($server, 'APPLICATION_CLOSE'); + $this->assertTrue($provider->isRunning(), 'Provider completion, rather than the disconnect, unblocked the producer.'); + + $health = stream_socket_client('tcp://' . $address, $error, $message, 2); + $this->assertIsResource($health, $message); + + try { + stream_set_timeout($health, 2); + fwrite($health, "GET /health HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"); + $this->assertStringContainsString('healthy', stream_get_contents($health)); + } finally { + fclose($health); + } + + $this->assertSame('', $server->getErrorOutput()); + } finally { + if (is_resource($client)) { + fclose($client); + } + + if (is_resource($control)) { + fclose($control); + } + + if ($serverProcessId !== null) { + // The fixture owns its process group, including every Swoole child. + posix_kill(-$serverProcessId, SIGKILL); + } + + $server?->stop(0); + $provider->stop(0); + } + } + + /** + * Provide silent header and body waits under both supported server modes. + */ + public static function silentProviderPhases(): array + { + return [ + 'headers in process mode' => ['silent-headers', 'process'], + 'body in process mode' => ['delayed', 'process'], + 'headers in base mode' => ['silent-headers', 'base'], + 'body in base mode' => ['delayed', 'base'], + ]; + } + + public function testHttp2StreamResetLeavesSilentProductionUntilTheNextWriteAndPreservesSiblingStreams(): void + { + $provider = new Process([PHP_BINARY, dirname(__DIR__) . '/Http/Fixtures/streaming-server.php', 'delayed']); + $provider->setTimeout(10); + $server = null; + $serverProcessId = null; + $client = null; + $control = null; + + try { + $provider->start(); + $this->waitForOutput($provider, "\n"); + $providerAddress = trim($provider->getOutput()); + $server = new Process([PHP_BINARY, __DIR__ . '/Fixtures/disconnect-server.php', $providerAddress, 'process']); + $server->setTimeout(10); + $server->start(); + $serverProcessId = $server->getPid(); + $this->waitForOutput($server, 'READY '); + preg_match('/READY (\d+)/', $server->getOutput(), $matches); + $client = stream_socket_client('tcp://127.0.0.1:' . $matches[1], $error, $message, 2); + $this->assertIsResource($client, $message); + stream_set_timeout($client, 2); + + // Static HPACK indexes encode GET and http; the path and authority are literals. + fwrite($client, "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n" + . $this->http2Frame(4, 0, 0, '') + . $this->http2Frame(1, 5, 1, "\x82\x86\x04\x07/stream\x01\x09localhost")); + $this->waitForOutput($provider, 'REQUEST'); + $this->waitForOutput($server, 'UPSTREAM_HEADERS'); + $control = stream_socket_client('tcp://' . $providerAddress, $error, $message, 2); + $this->assertIsResource($control, $message); + stream_set_timeout($control, 2); + $this->assertSame("ready\n", fgets($control)); + + // RST_STREAM(CANCEL) affects stream 1; stream 3 proves the connection remains usable. + fwrite($client, $this->http2Frame(3, 0, 1, pack('N', 8)) + . $this->http2Frame(1, 5, 3, "\x82\x86\x04\x07/health\x01\x09localhost")); + $this->assertSame('healthy', $this->http2Body($client, 3)); + $this->assertStringNotContainsString('STOPPED', $server->getOutput()); + + fclose($control); + $control = null; + $this->waitForOutput($server, 'STOPPED'); + + fwrite($client, $this->http2Frame(1, 5, 5, "\x82\x86\x04\x07/health\x01\x09localhost")); + $this->assertSame('healthy', $this->http2Body($client, 5)); + } finally { + if (is_resource($client)) { + fclose($client); + } + + if (is_resource($control)) { + fclose($control); + } + + if ($serverProcessId !== null) { + posix_kill(-$serverProcessId, SIGKILL); + } + + $server?->stop(0); + $provider->stop(0); + } + } + + /** + * Encode one frame for the reset behavior unavailable through Swoole's HTTP/2 client. + */ + private function http2Frame(int $type, int $flags, int $stream, string $payload): string + { + return substr(pack('N', strlen($payload)), 1) . pack('CCN', $type, $flags, $stream) . $payload; + } + + /** + * Read one response body, acknowledging the server's initial settings. + * + * @param resource $client + */ + private function http2Body(mixed $client, int $stream): string + { + $body = ''; + + do { + $header = stream_get_contents($client, 9); + $this->assertSame(9, strlen($header), 'The HTTP/2 fixture stopped sending frames.'); + $length = unpack('N', "\0" . substr($header, 0, 3))[1]; + $type = ord($header[3]); + $flags = ord($header[4]); + $identity = unpack('N', substr($header, 5))[1]; + $payload = $length === 0 ? '' : stream_get_contents($client, $length); + $this->assertSame($length, strlen($payload)); + + if ($type === 4 && ($flags & 1) === 0) { + fwrite($client, $this->http2Frame(4, 1, 0, '')); + } + + if ($identity === $stream && $type === 0) { + $body .= $payload; + } + } while ($identity !== $stream || ($flags & 1) === 0 || ! in_array($type, [0, 1], true)); + + return $body; + } + + /** + * Wait for a fixture boundary while retaining its diagnostics on failure. + */ + private function waitForOutput(Process $process, string $line): void + { + $deadline = microtime(true) + 3; + + do { + if (str_contains($process->getOutput(), $line)) { + return; + } + + if (! $process->isRunning()) { + break; + } + + usleep(10000); + } while (microtime(true) < $deadline); + + $this->fail("The fixture did not reach [{$line}].\n" . $process->getOutput() . $process->getErrorOutput()); + } +} diff --git a/tests/Integration/Database/PooledConnectionTest.php b/tests/Integration/Database/PooledConnectionTest.php index 134ce913f2..176bad13fe 100644 --- a/tests/Integration/Database/PooledConnectionTest.php +++ b/tests/Integration/Database/PooledConnectionTest.php @@ -35,6 +35,7 @@ use ReflectionProperty; use RuntimeException; use Swoole\Coroutine\CanceledException; +use WeakReference; /** * Tests for PooledConnection — the adapter that wraps a database Connection @@ -884,8 +885,8 @@ public function testHeartbeatDoesNotComputeOrInvalidateSessionState(): void $configurator = new PoolSessionConfigurator; PdoConnection::configureSessionUsing($configurator); $pool = new DatabasePool($this->app, 'pool_test'); - $stateCallsAfterCreation = $configurator->stateCalls; - $applyCallsAfterCreation = $configurator->applyCalls; + $this->assertSame(0, $configurator->stateCalls); + $this->assertSame(0, $configurator->applyCalls); /** @var PooledConnection $pooledConnection */ $pooledConnection = $pool->borrow(); @@ -895,8 +896,8 @@ public function testHeartbeatDoesNotComputeOrInvalidateSessionState(): void $stateCallsBeforePing = $configurator->stateCalls; $applyCallsBeforePing = $configurator->applyCalls; - $this->assertGreaterThanOrEqual($stateCallsAfterCreation, $stateCallsBeforePing); - $this->assertSame($applyCallsAfterCreation, $applyCallsBeforePing); + $this->assertSame(1, $stateCallsBeforePing); + $this->assertSame(1, $applyCallsBeforePing); $this->assertTrue($pooledConnection->ping(1.0)); $this->assertSame($stateCallsBeforePing, $configurator->stateCalls); $this->assertSame($applyCallsBeforePing, $configurator->applyCalls); @@ -1445,6 +1446,16 @@ public function testReleaseResetsErrorCountForNextBorrowWindow(): void $nextPooledConnection->release(); } + public function testClosingAnUnusedSharedMemoryPoolClosesItsPhysicalSession(): void + { + $pool = new DatabasePool($this->app, 'pool_test'); + $pdo = WeakReference::create($pool->getSharedInMemorySqlitePdo()); + + $pool->close(); + + $this->assertNull($pdo->get()); + } + public function testSharedPdoPersistsAcrossInMemorySqliteBorrows(): void { $pool = new DatabasePool($this->app, 'pool_test'); diff --git a/tests/Integration/Database/Sqlite/ConnectionEstablishedTest.php b/tests/Integration/Database/Sqlite/ConnectionEstablishedTest.php index 9cd70020ad..83770ef830 100644 --- a/tests/Integration/Database/Sqlite/ConnectionEstablishedTest.php +++ b/tests/Integration/Database/Sqlite/ConnectionEstablishedTest.php @@ -128,16 +128,27 @@ public function testOnlyNewGenerationsNotifyAfterRegistration(): void }); $first = null; - $this->runInCoroutine(function () use (&$first): void { + $firstPdo = null; + $this->runInCoroutine(function () use (&$first, &$firstPdo): void { $first = DB::connection('established::write'); + $firstPdo = $first->getPdo(); }); - $this->runInCoroutine(function () use ($first): void { - $this->assertSame($first, DB::connection('established::write')); + $this->runInCoroutine(function () use ($first, $firstPdo): void { + $connection = DB::connection('established::write'); + $this->assertNotSame($first, $connection); + $this->assertSame($firstPdo, $connection->getPdo()); + $this->assertSame($firstPdo, $connection->getRawPdo()); }); $this->assertSame([$first], $connections); - $this->runInCoroutine(fn () => DB::connection('established::write')->reconnect()); - $this->assertSame([$first, $first], $connections); + $reconnected = null; + $this->runInCoroutine(function () use (&$reconnected, $firstPdo): void { + $reconnected = DB::connection('established::write'); + $reconnected->reconnect(); + $this->assertSame($reconnected, DB::connection('established::write')); + $this->assertNotSame($firstPdo, $reconnected->getRawPdo()); + }); + $this->assertSame([$first, $reconnected], $connections); $pool = $this->app->make(PoolManager::class)->pool('established'); $pooled = $pool->borrow(); @@ -152,7 +163,7 @@ public function testOnlyNewGenerationsNotifyAfterRegistration(): void $replacement = DB::connection('established::write'); }); $this->assertNotSame($first, $replacement); - $this->assertSame([$first, $first, $replacement], $connections); + $this->assertSame([$first, $reconnected, $replacement], $connections); } #[TestWith([true, false])] diff --git a/tests/JsonSchema/ObjectTypeTest.php b/tests/JsonSchema/ObjectTypeTest.php index e8407b1e79..007b763133 100644 --- a/tests/JsonSchema/ObjectTypeTest.php +++ b/tests/JsonSchema/ObjectTypeTest.php @@ -81,6 +81,17 @@ public function testItMayBeInitializedWithAClosureButMayHaveProperties(): void ], $type->toArray()); } + public function testItExposesNamedPropertySchemasWithoutSerializingThem(): void + { + $properties = [ + 'name' => JsonSchema::string()->required(), + '0' => JsonSchema::object(['active' => JsonSchema::boolean()]), + ]; + + $this->assertSame($properties, JsonSchema::object($properties)->getProperties()); + $this->assertSame([], JsonSchema::object()->getProperties()); + } + public function testNumericStringPropertyNamesRemainStringsInRequiredArray(): void { $type = JsonSchema::object([ diff --git a/tests/Routing/ResponseFactoryTest.php b/tests/Routing/ResponseFactoryTest.php index d33c6703f6..a47e95efe4 100644 --- a/tests/Routing/ResponseFactoryTest.php +++ b/tests/Routing/ResponseFactoryTest.php @@ -5,6 +5,8 @@ namespace Hypervel\Tests\Routing; use Generator; +use Hypervel\Container\Container; +use Hypervel\Contracts\Debug\ExceptionHandler; use Hypervel\Contracts\View\Factory as ViewFactory; use Hypervel\Http\IterableStreamedResponse; use Hypervel\Http\StreamedEvent; @@ -12,6 +14,8 @@ use Hypervel\Routing\ResponseFactory; use Hypervel\Tests\TestCase; use Mockery as m; +use PHPUnit\Framework\Attributes\DataProvider; +use Swoole\Coroutine\CanceledException; use Symfony\Component\HttpFoundation\StreamedResponse; class ResponseFactoryTest extends TestCase @@ -143,6 +147,40 @@ static function () { ], $chunks); } + #[DataProvider('streamingFactoryMethods')] + public function testStreamingFactoriesPassCancellationThroughWithoutReportingOrWrapping(string $method): void + { + $failure = new CanceledException('response canceled'); + $handler = m::mock(ExceptionHandler::class); + $handler->shouldNotReceive('report'); + Container::getInstance()->instance(ExceptionHandler::class, $handler); + $callback = $method === 'eventStream' + ? static function () use ($failure): iterable { + yield 'first'; + throw $failure; + } + : static fn () => throw $failure; + $response = $this->factory()->{$method}($callback); + ob_start(); + + try { + $response->sendContent(); + $this->fail('Expected response production to propagate cancellation.'); + } catch (CanceledException $exception) { + $this->assertSame($failure, $exception); + } finally { + ob_end_clean(); + } + } + + /** + * Provide response factories that handle producer exceptions. + */ + public static function streamingFactoryMethods(): array + { + return [['eventStream'], ['streamDownload']]; + } + private function factory(): ResponseFactory { return new ResponseFactory( diff --git a/tests/Sentry/Features/StorageIntegrationTest.php b/tests/Sentry/Features/StorageIntegrationTest.php index d6c2da6ca7..c0618daefe 100644 --- a/tests/Sentry/Features/StorageIntegrationTest.php +++ b/tests/Sentry/Features/StorageIntegrationTest.php @@ -218,6 +218,41 @@ public function testAdapterSpecificOperationsAndFluentAssertionsRemainInstrument $disk->deleteDirectory('empty'); } + public function testPlainFilesystemExistenceChecksRemainInstrumented(): void + { + $root = ParallelTesting::tempDir('SentryFilesystemExistence'); + $files = new Filesystem; + $files->deleteDirectory($root); + $files->ensureDirectoryExists($root); + + try { + $adapter = new LocalFilesystemAdapter($root); + $filesystem = new FilesystemAdapter(new Flysystem($adapter), $adapter); + $filesystem->put('file.txt', 'contents'); + $filesystem->makeDirectory('empty'); + $disk = new SentryFilesystem($filesystem, ['disk' => 'plain'], true, false); + $transaction = $this->startTransaction(); + + $this->assertTrue($disk->fileExists('file.txt')); + $this->assertFalse($disk->fileExists('empty')); + $this->assertTrue($disk->directoryExists('empty')); + $this->assertFalse($disk->directoryExists('file.txt')); + $this->assertTrue($disk->exists('empty')); + + $spans = array_slice($transaction->getSpanRecorder()->getSpans(), 1); + $this->assertSame([ + 'file.fileExists', + 'file.fileExists', + 'file.directoryExists', + 'file.directoryExists', + 'file.exists', + ], array_map(static fn ($span): ?string => $span->getOp(), $spans)); + $this->assertSame(['path' => 'file.txt', 'disk' => 'plain'], $spans[0]->getData()); + } finally { + $files->deleteDirectory($root); + } + } + public function testTemporaryUrlCapabilitiesAndCallbacksDelegateToWrappedAdapter(): void { $disks = config('filesystems.disks'); diff --git a/tests/Server/ServerTest.php b/tests/Server/ServerTest.php index db2b4390e6..bea4768b64 100644 --- a/tests/Server/ServerTest.php +++ b/tests/Server/ServerTest.php @@ -22,6 +22,7 @@ use Hypervel\Testing\ParallelTesting; use Hypervel\Tests\TestCase; use Mockery as m; +use PHPUnit\Framework\Attributes\DataProvider; use Psr\Log\LoggerInterface; use RuntimeException; use Swoole\Coroutine\CanceledException; @@ -249,16 +250,86 @@ public function testPortEventRegistrationFailureStopsConfiguration(): void { $nativePort = m::mock(SwoolePort::class); $nativePort->expects('on')->with(Event::ON_REQUEST, m::type('callable'))->andReturnFalse(); + $server = $this->server(m::mock(Container::class)); + $server->useNativeServer(m::mock(SwooleServer::class)); $this->expectException(ServerException::class); $this->expectExceptionMessageIs('Failed to register event [request] on server [test].'); - $this->server(m::mock(Container::class))->registerEvents($nativePort, [ + $server->registerEvents($nativePort, [ Event::ON_REQUEST => static function (SwooleRequest $request, SwooleResponse $response): void { }, ]); } + #[DataProvider('responseCancellationModes')] + public function testDisconnectHandlingPreservesApplicationCallbacksAndRespectsServerModes(int $mode, array $settings, bool $supported): void + { + $registered = []; + $received = []; + $applicationCallback = static function (SwooleServer $server, int $connection, int $reactor) use (&$received): void { + $received = [$server, $connection, $reactor]; + }; + $nativeServer = m::mock(SwooleServer::class); + $nativeServer->mode = $mode; + $nativeServer->setting = $settings; + $nativeServer->expects('on')->twice()->andReturnUsing( + static function (string $event, callable $callback) use (&$registered): bool { + $registered[$event] = $callback; + + return true; + }, + ); + + $this->server(m::mock(Container::class))->registerEvents($nativeServer, [ + Event::ON_REQUEST => static function (): void { + }, + Event::ON_CLOSE => $applicationCallback, + ]); + + $this->assertSame($supported, $registered[Event::ON_CLOSE] !== $applicationCallback); + $registered[Event::ON_CLOSE]($nativeServer, 123, 2); + $this->assertSame([$nativeServer, 123, 2], $received); + } + + /** + * Provide modes with and without connection ownership in the serving worker. + */ + public static function responseCancellationModes(): array + { + return [ + 'base mode' => [SWOOLE_BASE, [], true], + 'default process mode' => [SWOOLE_PROCESS, [], true], + 'IP dispatch' => [SWOOLE_PROCESS, ['dispatch_mode' => 4], true], + 'connection balancing' => [SWOOLE_PROCESS, ['dispatch_mode' => 8], true], + 'round robin' => [SWOOLE_PROCESS, ['dispatch_mode' => 1], false], + 'queue dispatch' => [SWOOLE_PROCESS, ['dispatch_mode' => 3], false], + 'coroutines disabled' => [SWOOLE_PROCESS, ['enable_coroutine' => false], false], + ]; + } + + public function testResponseDisconnectCallbackIsInstalledWithoutAnApplicationCallback(): void + { + $registered = []; + $nativeServer = m::mock(SwooleServer::class); + $nativeServer->mode = SWOOLE_PROCESS; + $nativeServer->expects('on')->twice()->andReturnUsing( + static function (string $event, callable $callback) use (&$registered): bool { + $registered[$event] = $callback; + + return true; + }, + ); + + $this->server(m::mock(Container::class))->registerEvents($nativeServer, [ + Event::ON_REQUEST => static function (): void { + }, + ]); + + $this->assertArrayHasKey(Event::ON_CLOSE, $registered); + $registered[Event::ON_CLOSE]($nativeServer, 123, 2); + } + public function testMainServerSettingsFailureStopsConfiguration(): void { $nativeServer = m::mock(SwooleServer::class); diff --git a/tests/Support/ClassMetadataCacheTest.php b/tests/Support/ClassMetadataCacheTest.php index af1766f34f..344446f9e5 100644 --- a/tests/Support/ClassMetadataCacheTest.php +++ b/tests/Support/ClassMetadataCacheTest.php @@ -74,6 +74,19 @@ public function testClassAttributeMetadataIsCached(): void $this->assertSame(ClassMetadataCacheAttributedFixture::class, $first?->declaringClass->getName()); } + public function testTraitMembershipIncludesNestedAndInheritedTraitsWithoutCachingMisses(): void + { + $fixture = new ClassMetadataCacheInheritedTraitFixture; + + $this->assertTrue(ClassMetadataCache::usesTrait($fixture, ClassMetadataCacheTrait::class)); + $this->assertTrue(ClassMetadataCache::usesTrait($fixture::class, ClassMetadataCacheNestedTrait::class)); + $this->assertTrue(ClassMetadataCache::usesTrait($fixture, ClassMetadataCacheLeafTrait::class)); + $traits = $this->staticProperty('traits'); + $this->assertFalse(ClassMetadataCache::usesTrait($fixture, RuntimeException::class)); + $this->assertSame($traits, $this->staticProperty('traits')); + $this->assertFalse(ClassMetadataCache::usesTrait(ClassMetadataCacheFixture::class, ClassMetadataCacheTrait::class)); + } + public function testMissingClassAttributeIsCachedAsNull(): void { $this->assertNull(ClassMetadataCache::getAttribute(ClassMetadataCacheFixture::class, ClassMetadataCacheAttribute::class)); @@ -183,11 +196,13 @@ public function testFlushStateClearsCachedMetadata(): void ClassMetadataCache::getAttribute(ClassMetadataCacheAttributedFixture::class, ClassMetadataCacheAttribute::class); ClassMetadataCache::hasClassAttribute(ClassMetadataCacheParentFixture::class, ClassMetadataCacheAttribute::class); ClassMetadataCache::hasClassAttribute(ClassMetadataCacheChildFixture::class, ClassMetadataCacheAttribute::class, ascend: true); + ClassMetadataCache::usesTrait(ClassMetadataCacheTraitFixture::class, ClassMetadataCacheTrait::class); ClassMetadataCache::flushState(); $this->assertSame([], $this->staticProperty('methods')); $this->assertSame([], $this->staticProperty('attributes')); $this->assertSame([], $this->staticProperty('classAttributePresence')); + $this->assertSame([], $this->staticProperty('traits')); $classAfter = ClassMetadataCache::reflectClass(ClassMetadataCacheFixture::class); $methodAfter = ClassMetadataCache::reflectMethod(ClassMetadataCacheFixture::class, 'greet'); @@ -260,6 +275,20 @@ class ClassMetadataCacheTraitFixture use ClassMetadataCacheTrait; } +trait ClassMetadataCacheLeafTrait +{ +} + +trait ClassMetadataCacheNestedTrait +{ + use ClassMetadataCacheLeafTrait; +} + +class ClassMetadataCacheInheritedTraitFixture extends ClassMetadataCacheTraitFixture +{ + use ClassMetadataCacheNestedTrait; +} + class ClassMetadataCachePropertyFixture { #[ClassMetadataCachePropertyAttribute] diff --git a/tests/Testbench/Attributes/WithEnvTest.php b/tests/Testbench/Attributes/WithEnvTest.php index 3433c8f1c8..42c9f5e3bf 100644 --- a/tests/Testbench/Attributes/WithEnvTest.php +++ b/tests/Testbench/Attributes/WithEnvTest.php @@ -10,6 +10,7 @@ use Hypervel\Testbench\TestCase; use Mockery as m; use PHPUnit\Framework\Attributes\Test; +use PHPUnit\Framework\Attributes\TestWith; class WithEnvTest extends TestCase { @@ -68,6 +69,36 @@ public function itDoesNotPersistDefinedEnvVariablesBetweenTests(): void $this->assertNull(Env::get('TESTING_USING_ATTRIBUTE')); } + #[Test] + #[TestWith([false])] + #[TestWith([true])] + public function itRestoresValuesAfterScopedEnvironmentChanges(bool $hasOriginalValue): void + { + $key = 'TESTBENCH_ATTRIBUTE_OWNERSHIP'; + + $this->withEnvironmentValue($key, null, function () use ($key, $hasOriginalValue): void { + $originalValue = $hasOriginalValue ? '"original"' : null; + + if ($originalValue !== null) { + Env::getRepository()->set($key, $originalValue); + } + + $restore = (new WithEnv($key, 'temporary'))(m::mock(ApplicationContract::class)); + + try { + $this->assertSame('temporary', Env::get($key)); + + $this->withEnvironmentValue('TESTBENCH_OTHER_ATTRIBUTE_SETTING', 'other', static fn () => null); + + $this->assertSame('temporary', Env::get($key)); + } finally { + $restore(); + } + + $this->assertSame($originalValue, Env::getRepository()->get($key)); + }); + } + #[Test] public function itCannotChangeDefinedEnvVariables(): void { diff --git a/tests/Testbench/Foundation/ApplicationTest.php b/tests/Testbench/Foundation/ApplicationTest.php index 5eab833b73..38d2772daa 100644 --- a/tests/Testbench/Foundation/ApplicationTest.php +++ b/tests/Testbench/Foundation/ApplicationTest.php @@ -164,7 +164,7 @@ static function (ApplicationContract $app): void { in_array(default_migration_path(), $app->make('migrator')->paths(), true), ); } finally { - Env::forget('TESTBENCH_WITHOUT_DEFAULT_MIGRATIONS'); + $forgotten = Env::forget('TESTBENCH_WITHOUT_DEFAULT_MIGRATIONS'); try { $app->terminate(); @@ -172,6 +172,9 @@ static function (ApplicationContract $app): void { $app->flush(); } } + + $this->assertTrue($forgotten); + $this->assertNull(Env::get('TESTBENCH_WITHOUT_DEFAULT_MIGRATIONS')); } /** diff --git a/tests/Testbench/TestCaseTest.php b/tests/Testbench/TestCaseTest.php index e11bfb1924..45bcb80d7d 100644 --- a/tests/Testbench/TestCaseTest.php +++ b/tests/Testbench/TestCaseTest.php @@ -13,6 +13,7 @@ use Hypervel\Testbench\Foundation\Env; use Hypervel\Testbench\PHPUnit\TestCase; use PHPUnit\Framework\Attributes\Test; +use PHPUnit\Framework\Attributes\TestWith; use RuntimeException; use Throwable; @@ -141,7 +142,9 @@ public function itRestoresExactApplicationEnvironmentStateAfterBootstrapFails(): } #[Test] - public function itKeepsWithEnvActiveAcrossRequestsUntilTestTeardown(): void + #[TestWith([true])] + #[TestWith([false])] + public function itKeepsWithEnvActiveAcrossRequestsUntilTestTeardown(bool $loadEnvironmentVariables): void { $key = 'TESTBENCH_REQUEST_ENV'; $serverValue = $this->snapshotEnvironmentValue($_SERVER, $key); @@ -154,7 +157,7 @@ public function itKeepsWithEnvActiveAcrossRequestsUntilTestTeardown(): void putenv($key); Env::flushRepository(); - $testCase->createTestApplication(); + $testCase->createTestApplication($loadEnvironmentVariables); $this->assertSame('active', $testCase->requestEnvironment()); $this->assertSame('active', $testCase->requestEnvironment()); @@ -232,6 +235,7 @@ protected function defineEnvironment(ApplicationContract $app): void } } +#[WithEnv('TESTBENCH_REQUEST_ENV', 'class-default')] class WithEnvRequestLifecycleTestCaseFixture extends \Hypervel\Testbench\TestCase { #[WithEnv('TESTBENCH_REQUEST_ENV', 'active')] @@ -239,8 +243,9 @@ public function testPlaceholder(): void { } - public function createTestApplication(): void + public function createTestApplication(bool $loadEnvironmentVariables): void { + $this->loadEnvironmentVariables = $loadEnvironmentVariables; $this->app = $this->createApplication(); $this->app->make(Router::class)->get( diff --git a/tests/Watcher/Driver/FswatchDriverTest.php b/tests/Watcher/Driver/FswatchDriverTest.php index dde9eb8dea..be05e5eabd 100644 --- a/tests/Watcher/Driver/FswatchDriverTest.php +++ b/tests/Watcher/Driver/FswatchDriverTest.php @@ -15,6 +15,7 @@ use Hypervel\Watcher\WatchPathType; use InvalidArgumentException; use PHPUnit\Framework\Attributes\DataProvider; +use PHPUnit\Framework\Attributes\TestWith; use RuntimeException; class FswatchDriverTest extends TestCase @@ -297,7 +298,7 @@ public function testCommandPreservesWatchPathsAsLiteralArguments(): void ...$resolvedPaths, ]; - $this->assertSame($expected, $driver->commandForTest()); + $this->assertSame($expected, $driver->commandsForTest()[$driver->isDarwin() ? 'all' : 'recursive']); } public function testTargetsNormalizeRootAndTrailingSlashes(): void @@ -348,9 +349,11 @@ public function testCommandUsesTheSameNulProtocolAndEventFiltersOnEveryPlatform( '--event', 'Renamed', $this->fixturePath, + ...(! $darwin && ! $recursive ? [base_path()] : []), ]; - $this->assertSame($expected, $driver->commandForTest()); + $group = $darwin ? 'all' : ($recursive ? 'recursive' : 'shallow'); + $this->assertSame($expected, $driver->commandsForTest()[$group]); } public static function commandProvider(): array @@ -404,6 +407,10 @@ public function testSharedOperandPromotesToRecursive(): void ); $this->assertSame([ + 'shallow' => [ + 'recursive' => false, + 'operands' => [$this->fixturePath], + ], 'recursive' => [ 'recursive' => true, 'operands' => [$this->fixturePath . '/app'], @@ -432,12 +439,16 @@ public function testContainmentOnlyRemovesShallowOperandsFromTheRecursiveGroup() $driver->processChunksWithTargets($channel, [$file . "\0"], $targets); $this->assertSame([ + 'shallow' => [ + 'recursive' => false, + 'operands' => [$this->fixturePath], + ], 'recursive' => [ 'recursive' => true, 'operands' => [$this->fixturePath . '/app'], ], ], $targets['groups']); - $this->assertSame(2, count($targets['entries'])); + $this->assertSame(3, count($targets['entries'])); $this->assertSame($file, $channel->pop()); $this->assertSame(0, $channel->getLength()); } finally { @@ -471,7 +482,7 @@ public function testShallowSymlinkOutsideRecursiveTreeRetainsItsOperandAndPublis $this->assertSame([ 'shallow' => [ 'recursive' => false, - 'operands' => [$this->fixturePath . '/outside'], + 'operands' => [$this->fixturePath, $this->fixturePath . '/outside'], ], 'recursive' => [ 'recursive' => true, @@ -507,12 +518,16 @@ public function testShallowSymlinkInsideRecursiveTreeDropsItsOperandButKeepsItsM $driver->processChunksWithTargets($channel, [$file . "\0"], $targets); $this->assertSame([ + 'shallow' => [ + 'recursive' => false, + 'operands' => [$this->fixturePath], + ], 'recursive' => [ 'recursive' => true, 'operands' => [$this->fixturePath . '/app'], ], ], $targets['groups']); - $this->assertSame(2, count($targets['entries'])); + $this->assertSame(3, count($targets['entries'])); $this->assertSame($file, $channel->pop()); $this->assertSame(0, $channel->getLength()); } finally { @@ -547,7 +562,7 @@ public function testNestedOperandsWithinOneGroupAreRetained(): void $recursiveDriver->targetsForTest()['groups']['recursive']['operands'], ); $this->assertSame( - [$this->fixturePath . '/app', $this->fixturePath . '/app/Http'], + [$this->fixturePath . '/app', $this->fixturePath, $this->fixturePath . '/app/Http'], $shallowDriver->targetsForTest()['groups']['shallow']['operands'], ); } @@ -571,7 +586,7 @@ public function testSiblingPathWithParentSegmentsRemainsASeparateOperand(): void $driver->processChunksWithTargets($channel, [$file . "\0"], $targets); $this->assertSame( - [base_path('../packages/foo')], + [base_path('../packages/foo'), base_path('../packages'), realpath(base_path('..'))], $targets['groups']['shallow']['operands'], ); $this->assertSame( @@ -618,7 +633,7 @@ public function testExactFilesShareTheirParentMappingAndMissingOperandsAreRetain $driver = new InspectableFswatchDriver(new Option(driver: FswatchDriver::class, watchPaths: $watchPaths)); $this->assertSame( - [$configPath, $this->fixturePath . '/missing'], + [$configPath, $this->fixturePath . '/missing', $this->fixturePath], $driver->operandsForTest(), ); $this->assertSame( @@ -631,6 +646,10 @@ public function testExactFilesShareTheirParentMappingAndMissingOperandsAreRetain 'prefix' => $this->fixturePath . '/missing/', 'base' => $missingRelativePath, ], + [ + 'prefix' => $this->fixturePath . '/', + 'base' => $this->fixtureRelativePath, + ], ], $driver->targetsForTest()['entries'], ); @@ -653,11 +672,11 @@ public function testMissingOperandKeepsItsLiteralPrefixThroughASymlinkedAncestor try { $driver->processChunksWithTargets($channel, [$file . "\0"], $targets); - $this->assertSame([$this->fixturePath . '/link/later'], $driver->operandsForTest()); - $this->assertSame([[ - 'prefix' => $this->fixturePath . '/link/later/', - 'base' => $missingBase, - ]], $targets['entries']); + $this->assertSame([$this->fixturePath . '/link/later', $this->fixturePath . '/real'], $driver->operandsForTest()); + $this->assertSame([ + ['prefix' => $this->fixturePath . '/link/later/', 'base' => $missingBase], + ['prefix' => $this->fixturePath . '/real/', 'base' => $this->fixtureRelativePath . '/link'], + ], $targets['entries']); $this->assertSame($file, $channel->pop()); } finally { $driver->stop(); @@ -729,14 +748,14 @@ public function testMissingShallowOperandThatBecomesOutsideSymlinkKeepsLiteralCo $this->assertSame([ 'shallow' => [ 'recursive' => false, - 'operands' => [$this->fixturePath . '/app/Generated'], + 'operands' => [$this->fixturePath, $this->fixturePath . '/app/Generated'], ], 'recursive' => [ 'recursive' => true, 'operands' => [$this->fixturePath . '/app'], ], ], $targets['groups']); - $this->assertSame(2, count($targets['entries'])); + $this->assertSame(3, count($targets['entries'])); } finally { $driver->stop(); $channel->close(); @@ -761,8 +780,8 @@ public function testAliasedMappingsPublishARecordOnce(): void try { $driver->processChunksWithTargets($channel, [$file . "\0"], $targets); - $this->assertSame(1, count($driver->operandsForTest())); - $this->assertSame(2, count($targets['entries'])); + $this->assertSame(2, count($driver->operandsForTest())); + $this->assertSame(3, count($targets['entries'])); $this->assertSame($file, $channel->pop()); $this->assertSame(0, $channel->getLength()); } finally { @@ -845,6 +864,134 @@ public function testAtomicFileReplacementIsObservedThroughTheWatchedParent(): vo $this->assertNull($failure); } + public function testDirectoryEventsRespectRecursiveRootsAndExcludeUnrelatedPaths(): void + { + $recursiveBase = $this->fixtureRelativePath . '/app'; + $shallowBase = $this->fixtureRelativePath . '/config'; + $fileBase = $this->fixtureRelativePath . '/settings/local'; + $this->files->makeDirectory(base_path($recursiveBase . '/Nested'), 0755, true); + $this->files->makeDirectory(base_path($shallowBase . '/Nested'), 0755, true); + $this->files->makeDirectory($this->fixturePath . '/storage'); + $this->files->put(base_path($recursiveBase . '/notes.txt'), 'ignored'); + $driver = new InspectableFswatchDriver(new Option(driver: FswatchDriver::class, watchPaths: [ + new WatchPath($recursiveBase, WatchPathType::Directory, $recursiveBase . '/**/*.php'), + new WatchPath($shallowBase, WatchPathType::Directory, $shallowBase . '/*.php'), + new WatchPath($fileBase . '/app.php', WatchPathType::File), + ])); + $targets = $driver->targetsForTest(); + $this->files->makeDirectory(base_path($fileBase . '/Nested'), 0755, true); + $channel = new Channel(10); + + try { + $driver->processChunksWithTargets($channel, [implode("\0", [ + base_path($recursiveBase), + base_path($recursiveBase . '/Nested'), + base_path($shallowBase), + base_path($shallowBase . '/Nested'), + base_path($recursiveBase . '/notes.txt'), + $this->fixturePath . '/storage', + base_path(dirname($fileBase)), + base_path($fileBase), + base_path($fileBase . '/Nested'), + ]) . "\0"], $targets); + + $this->assertSame(5, $channel->getLength()); + $this->assertSame(base_path($recursiveBase), $channel->pop()); + $this->assertSame(base_path($recursiveBase . '/Nested'), $channel->pop()); + $this->assertSame(base_path($shallowBase), $channel->pop()); + $this->assertSame(base_path(dirname($fileBase)), $channel->pop()); + $this->assertSame(base_path($fileBase), $channel->pop()); + } finally { + $driver->stop(); + $channel->close(); + } + } + + #[TestWith([false])] + #[TestWith([true])] + public function testCreatedDirectoriesAndSubsequentFilesAreObserved(bool $missingRoot): void + { + $root = $this->fixtureRelativePath . ($missingRoot ? '/resources/skills' : '/app'); + + if (! $missingRoot) { + $this->files->makeDirectory(base_path($root)); + } + + $ready = ($missingRoot ? $this->fixtureRelativePath : $root) . '/ready.php'; + $option = new Option(driver: FswatchDriver::class, watchPaths: [ + new WatchPath($ready, WatchPathType::File), + new WatchPath($root, WatchPathType::Directory, $root . '/**/*.php'), + ]); + + try { + $driver = new FswatchDriver($option); + } catch (InvalidArgumentException $exception) { + if ($exception->getMessage() === 'The FswatchDriver requires the `fswatch` executable.') { + $this->markTestSkipped('The fswatch executable is not available.'); + } + + throw $exception; + } + + $channel = new Channel(100); + $finished = new WaitGroup(1); + $failure = null; + + Coroutine::create(function () use ($channel, $driver, $finished, &$failure): void { + try { + $driver->watch($channel); + } catch (RuntimeException $exception) { + $failure = $exception; + } finally { + $finished->done(); + } + }); + + try { + $deadline = hrtime(true) + 5_000_000_000; + $received = false; + + while (! $received && hrtime(true) < $deadline) { + $this->files->put(base_path($ready), 'ready'); + $received = $channel->pop(0.25) === base_path($ready); + } + + $this->assertTrue($received, 'fswatch did not register the existing watch root.'); + + $directory = base_path($root . '/New'); + $this->files->makeDirectory($directory, 0755, true); + $this->files->put($directory . '/Initial.php', 'initial'); + $arrival = $missingRoot ? $this->fixturePath . '/resources' : $directory; + $deadline = hrtime(true) + 5_000_000_000; + $received = false; + + while (! $received && hrtime(true) < $deadline) { + $event = $channel->pop(0.25); + $received = is_string($event) && ($event === $arrival || str_starts_with($event, $arrival . '/')); + } + + $this->assertTrue($received, 'fswatch did not report the directory arrival.'); + + // A different filename cannot be satisfied by queued events from the initial creation. + $laterFile = $directory . '/Later.php'; + $deadline = hrtime(true) + 5_000_000_000; + $received = false; + + while (! $received && hrtime(true) < $deadline) { + $this->files->put($laterFile, 'later'); + $received = $channel->pop(0.25) === $laterFile; + } + + $this->assertTrue($received, 'fswatch did not observe subsequent changes inside the directory.'); + } finally { + $driver->stop(); + $this->assertTrue($finished->wait(1)); + $channel->close(); + } + + $this->assertNull($failure); + } + public function testWatchPassesTheCommandAsAnArgumentListAndDeliversEachBatchInOrder(): void { $literalPath = 'shell $(literal).php'; @@ -994,7 +1141,7 @@ private function option(): Option return new Option( driver: FswatchDriver::class, watchPaths: [ - new WatchPath($this->fixtureRelativePath, WatchPathType::Directory), + new WatchPath($this->fixtureRelativePath . '/watched.php', WatchPathType::File), ], ); } @@ -1030,19 +1177,6 @@ public function isDarwin(): bool return $this->darwin; } - /** - * Return the command for the configured paths. - * - * @return list - */ - public function commandForTest(): array - { - $targets = $this->targetsForTest(); - $group = array_values($targets['groups'])[0]; - - return $this->getCommand($group['operands'], $group['recursive']); - } - /** * Return commands for every configured process group. * diff --git a/types/Database/Connection.php b/types/Database/Connection.php index c80690bba2..073f5e2400 100644 --- a/types/Database/Connection.php +++ b/types/Database/Connection.php @@ -30,6 +30,7 @@ function testCallbackWrappersPreserveCallbackReturns(Connection $connection): vo { assertType("'foo'", $connection->withoutPretending(fn () => 'foo')); assertType("'foo'", $connection->withoutTablePrefix(fn () => 'foo')); + assertType("'foo'", $connection->withPinnedSession(fn () => 'foo')); } function testTransactionPreservesCallbackReturn(ConnectionInterface $connection): void diff --git a/types/Routing/ResponseFactory.php b/types/Routing/ResponseFactory.php new file mode 100644 index 0000000000..9ff5c38a71 --- /dev/null +++ b/types/Routing/ResponseFactory.php @@ -0,0 +1,22 @@ +eventStream($events)->cancelOnDisconnect()); + assertType(IterableStreamedResponse::class, $contract->eventStream($events)->cancelOnDisconnect()); + assertType(IterableStreamedResponse::class, response()->eventStream($events)->cancelOnDisconnect()); + assertType(IterableStreamedResponse::class, Response::eventStream($events)->cancelOnDisconnect()); +}