BE-816: atlas: pull fit generations from S3 by indietyp · Pull Request #9682 · hashintel/hash

indietyp · 2026-09-11T15:05:37Z

🌟 What is the purpose of this PR?

Pull a published generation onto a serving host and switch to it. To serve millions of points, we rely heavily on memory-mapped files. The issue is that you can't mmap object storage, and solutions that allow it, like NFS, have the downside that each page miss is a network fetch and thus has immense latency.

This PR introduces a way to poll a storage backend by specifying --download, e.g. hash-graph atlas serve --download s3://<bucket>/<prefix>, which polls the remote storage, watches for changes to the current pointer and then proceeds to download the file into local storage.

The organising idea is that the downloader and the generation manager share exactly one contract: the local current pointer. This decouples the processes and minimises the invasive procedure. Once we're finished, we simply change the local current pointer, which the generation manager then picks up.

For review, the properties we rely on are the following:

Activation waits until every artefact's digest matches the received metadata and the metadata's digest matches the generation ID (Download::acquire, then GenerationRoot::activate_verified). A partial or corrupt download never becomes the served generation. The host keeps serving what it had and tries again on the next poll.
Each poll samples the remote pointer once and uses that ID throughout (Download::synchronize). A promotion landing mid-poll cannot mix two generations' files, and the next poll picks it up whole. An unchanged id costs a local pointer read and a directory stat, so the one-second default cadence is cheap.
The downloader reuses a directory already on disk under the id only after it re-hashes under the root lock. A stale or hand-placed copy under the right name is never served on trust.
Shutdown finishes the in-flight synchronisation, including the blocking import and the pointer swap. Stopping the host mid-download leaves the root with the old generation or the new one, never half of either, so a restart needs no cleanup.
--download s3://… without --s3 fails at startup, before any task starts and without opening the source, so a misconfigured host does not come up serving 503 and logging poll failures.

🔍 What does this change?

flowchart LR
    S["S3 current"] -->|poll| D[Download]
    D -->|"active/&lt;id&gt;/ into staging, verify"| R[root]
    R -->|local current| M[GenerationManager]
    M -->|open · promote · retire| Q[requests]

file/generation/download/: Download::synchronize is one poll (read remote current, reuse or acquire, activate), Download::run is the loop with the shutdown future, and DownloadTask is what the host retains. GenerationDownloadBackend is the read side of the storage trait, so the tests run against a directory and a fault-injecting fake.
file/generation/remote.rs: RemoteRoot and RemoteGeneration name the generations/{current,previous,repository,active} layout once, and Upload builds its paths from the same type.
GenerationRoot::activate_verified: verification under the root lock, with ActivateError gaining Open and Integrity for what it can now find.
CLI and host: --download and --download-poll-interval (HASH_GRAPH_ATLAS_DOWNLOAD*) on serve, S3Args flattened into the graph binary's serve arguments, and ServeOptions.storage. Serving becomes Serve, whose into_parts takes a shutdown factory and returns the router, the maintenance future and an optional download future, and apps/hash-graph registers both futures on its lifecycle.
tests/generation_transfer.rs behind the test-utils feature: a publisher-to-downloader round trip and a corrupt-artifact case, plus the cleanup that keeps a failed assertion from leaving objects or multipart uploads behind. Each case runs in a fresh bucket on a local MinIO, #[ignore] by default.
The rust-tracing-practices skill records the rule the download and serve code follows: log an ordinary error with Display and an error_stack::Report with Debug.

⚠️ Breaking Changes & Known Issues

With a source configured, local current follows the source. The next poll re-points a hand-edited local pointer, so a rollback moves the remote current.
A local copy that fails its re-hash stays in place, and every poll fails on it until an operator removes that directory. The downloader overwrites nothing.
A failed poll logs at warn and retries at the poll cadence, with no backoff.
S3Args sets no request timeouts, so the SDK defaults are what bound a poll against an unreachable endpoint, and shutdown waits on it.
The downloader adds directories and removes none. Only BE-815's --unlink-expired-generations removes expired local generations, so a --download host without that flag keeps one directory per generation it has served.

❓ How to test this?

yarn compose up -d minio, and publish a generation into a bucket with BE-817's fit --upload s3://<bucket>/atlas.
On an empty root, hash-graph atlas serve --download s3://<bucket>/atlas --s3 --s3-endpoint http://localhost:9000 --s3-force-path-style --s3-region us-east-1 --s3-access-key-id dev-s3-access-key-id --s3-secret-access-key dev-s3-secret-access-key. The read API answers 503 until the first download completes and the manager opens it, then serves.
Publish a second generation the same way and wait one poll: the host switches, and in-flight sessions on the old generation keep answering until retention expires.
For the MinIO suite: yarn workspace @rust/hash-graph-atlas test:integration --test generation_transfer --run-ignored only.

vercel · 2026-09-11T15:05:50Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hash	Ready	Preview	Sep 11, 2026 7:14pm UTC
petrinaut	Ready	Preview	Sep 11, 2026 7:14pm UTC
petrinaut-docs	Ready	Preview	Sep 11, 2026 7:14pm UTC

1 Skipped Deployment

Project	Deployment	Actions	Updated
hashdotdesign-tokens	Ignored	Preview	Sep 11, 2026 7:14pm UTC

codecov · 2026-09-11T15:17:42Z

Codecov Report

❌ Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (bm/be-817-atlas-push-fit-generations-to-s3@c8e1087). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
apps/hash-graph/src/subcommand/atlas.rs	0.00%	15 Missing ⚠️

Additional details and impacted files

@@                              Coverage Diff                              @@
##             bm/be-817-atlas-push-fit-generations-to-s3    #9682   +/-   ##
=============================================================================
  Coverage                                              ?   12.27%           
=============================================================================
  Files                                                 ?       12           
  Lines                                                 ?     1198           
  Branches                                              ?       40           
=============================================================================
  Hits                                                  ?      147           
  Misses                                                ?     1048           
  Partials                                              ?        3

Flag	Coverage Δ
apps.hash-graph	`12.27% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

semgrep-code-hashintel · 2026-09-11T19:07:57Z

+        let (admin, connection) = Config::new()
+            .host("localhost")
+            .port(5432)
+            .user("postgres")
+            .password("postgres")


Semgrep identified a blocking 🔴 issue in your code:
A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. It is recommended to rotate the secret and retrieve them from a secure secret vault or Hardware Security Module (HSM), alternatively environment variables can be used if allowed by your company policy.

To resolve this comment:

🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.

💬 Ignore this finding

Reply with Semgrep commands to ignore this finding.

/fp <comment> for false positive

/ar <comment> for acceptable risk

/other <comment> for all other reasons

Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by hardcoded-password.

_{You can view more details about this finding in the Semgrep AppSec Platform.}

+        let (admin, connection) = Config::new()
+            .host("localhost")
+            .port(5432)
+            .user("postgres")
+            .password("postgres")


+            &DatabaseConnectionInfo::new(
+                DatabaseType::Postgres,
+                "postgres".to_owned(),
+                "postgres".to_owned(),


github-actions · 2026-09-11T20:26:19Z

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2002	$$28.7 \mathrm{ms} \pm 236 \mathrm{μs}\left({\color{gray}4.20 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$3.51 \mathrm{ms} \pm 30.4 \mathrm{μs}\left({\color{gray}0.880 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 1002	$$12.8 \mathrm{ms} \pm 151 \mathrm{μs}\left({\color{gray}1.49 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 3314	$$43.8 \mathrm{ms} \pm 557 \mathrm{μs}\left({\color{gray}0.246 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$14.2 \mathrm{ms} \pm 126 \mathrm{μs}\left({\color{gray}0.651 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 1527	$$24.9 \mathrm{ms} \pm 202 \mathrm{μs}\left({\color{gray}1.61 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 2078	$$30.0 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{gray}4.45 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.81 \mathrm{ms} \pm 29.9 \mathrm{μs}\left({\color{gray}1.15 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 1033	$$14.1 \mathrm{ms} \pm 165 \mathrm{μs}\left({\color{gray}3.62 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_medium

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 102	$$3.87 \mathrm{ms} \pm 27.6 \mathrm{μs}\left({\color{gray}0.525 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$3.01 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}-0.040 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 52	$$3.37 \mathrm{ms} \pm 16.4 \mathrm{μs}\left({\color{gray}-0.405 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 269	$$5.30 \mathrm{ms} \pm 41.4 \mathrm{μs}\left({\color{gray}0.678 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$3.68 \mathrm{ms} \pm 32.9 \mathrm{μs}\left({\color{gray}2.93 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 108	$$4.23 \mathrm{ms} \pm 32.2 \mathrm{μs}\left({\color{gray}0.523 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 133	$$4.57 \mathrm{ms} \pm 37.2 \mathrm{μs}\left({\color{gray}0.860 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.48 \mathrm{ms} \pm 22.4 \mathrm{μs}\left({\color{gray}1.02 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 63	$$4.16 \mathrm{ms} \pm 29.8 \mathrm{μs}\left({\color{gray}1.33 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_none

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 2	$$2.77 \mathrm{ms} \pm 23.0 \mathrm{μs}\left({\color{gray}0.656 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.71 \mathrm{ms} \pm 17.5 \mathrm{μs}\left({\color{gray}0.905 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 2	$$2.81 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}-0.743 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 8	$$3.09 \mathrm{ms} \pm 22.8 \mathrm{μs}\left({\color{gray}1.45 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$2.87 \mathrm{ms} \pm 16.8 \mathrm{μs}\left({\color{gray}0.343 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 3	$$3.21 \mathrm{ms} \pm 23.2 \mathrm{μs}\left({\color{gray}1.90 \mathrm{\%}}\right) $$	Flame Graph

policy_resolution_small

Function	Value	Mean	Flame graphs
resolve_policies_for_actor	user: empty, selectivity: high, policies: 52	$$3.13 \mathrm{ms} \pm 18.9 \mathrm{μs}\left({\color{gray}-0.063 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: low, policies: 1	$$2.81 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}0.125 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: empty, selectivity: medium, policies: 26	$$3.00 \mathrm{ms} \pm 22.0 \mathrm{μs}\left({\color{gray}-0.237 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: high, policies: 94	$$3.58 \mathrm{ms} \pm 30.1 \mathrm{μs}\left({\color{gray}0.133 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: low, policies: 1	$$3.08 \mathrm{ms} \pm 17.8 \mathrm{μs}\left({\color{gray}-0.095 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: seeded, selectivity: medium, policies: 27	$$3.36 \mathrm{ms} \pm 22.9 \mathrm{μs}\left({\color{gray}0.367 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: high, policies: 66	$$3.48 \mathrm{ms} \pm 17.8 \mathrm{μs}\left({\color{gray}-1.285 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: low, policies: 1	$$3.07 \mathrm{ms} \pm 21.0 \mathrm{μs}\left({\color{gray}-1.039 \mathrm{\%}}\right) $$	Flame Graph
resolve_policies_for_actor	user: system, selectivity: medium, policies: 29	$$3.42 \mathrm{ms} \pm 35.5 \mathrm{μs}\left({\color{gray}0.610 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_complete

Function	Value	Mean	Flame graphs
entity_by_id;one_depth	1 entities	$$31.9 \mathrm{ms} \pm 231 \mathrm{μs}\left({\color{gray}0.567 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	10 entities	$$71.7 \mathrm{ms} \pm 494 \mathrm{μs}\left({\color{gray}1.95 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	25 entities	$$35.4 \mathrm{ms} \pm 262 \mathrm{μs}\left({\color{gray}-0.870 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	5 entities	$$39.6 \mathrm{ms} \pm 247 \mathrm{μs}\left({\color{gray}0.770 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;one_depth	50 entities	$$41.6 \mathrm{ms} \pm 313 \mathrm{μs}\left({\color{gray}1.86 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	1 entities	$$33.5 \mathrm{ms} \pm 171 \mathrm{μs}\left({\color{gray}-0.086 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	10 entities	$$424 \mathrm{ms} \pm 1.19 \mathrm{ms}\left({\color{gray}0.621 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	25 entities	$$91.3 \mathrm{ms} \pm 784 \mathrm{μs}\left({\color{gray}1.31 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	5 entities	$$78.3 \mathrm{ms} \pm 378 \mathrm{μs}\left({\color{gray}-0.127 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;two_depth	50 entities	$$273 \mathrm{ms} \pm 766 \mathrm{μs}\left({\color{gray}0.012 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	1 entities	$$10.3 \mathrm{ms} \pm 65.1 \mathrm{μs}\left({\color{gray}-0.223 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	10 entities	$$10.5 \mathrm{ms} \pm 66.8 \mathrm{μs}\left({\color{gray}0.666 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	25 entities	$$10.6 \mathrm{ms} \pm 87.8 \mathrm{μs}\left({\color{gray}0.113 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	5 entities	$$10.6 \mathrm{ms} \pm 68.7 \mathrm{μs}\left({\color{gray}0.741 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id;zero_depth	50 entities	$$10.6 \mathrm{ms} \pm 79.7 \mathrm{μs}\left({\color{gray}0.952 \mathrm{\%}}\right) $$	Flame Graph

read_scaling_linkless

Function	Value	Mean	Flame graphs
entity_by_id	1 entities	$$10.4 \mathrm{ms} \pm 60.3 \mathrm{μs}\left({\color{gray}0.162 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10 entities	$$10.3 \mathrm{ms} \pm 61.3 \mathrm{μs}\left({\color{gray}-0.191 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	100 entities	$$10.5 \mathrm{ms} \pm 80.4 \mathrm{μs}\left({\color{gray}0.817 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	1000 entities	$$10.5 \mathrm{ms} \pm 83.3 \mathrm{μs}\left({\color{gray}0.369 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	10000 entities	$$10.8 \mathrm{ms} \pm 71.4 \mathrm{μs}\left({\color{gray}0.401 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity

Function	Value	Mean	Flame graphs
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/block/v/1`	$$10.8 \mathrm{ms} \pm 75.0 \mathrm{μs}\left({\color{gray}-0.311 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/book/v/1`	$$10.9 \mathrm{ms} \pm 67.1 \mathrm{μs}\left({\color{gray}1.12 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/building/v/1`	$$10.8 \mathrm{ms} \pm 85.8 \mathrm{μs}\left({\color{gray}-0.547 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/organization/v/1`	$$10.8 \mathrm{ms} \pm 61.4 \mathrm{μs}\left({\color{gray}-1.500 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/page/v/2`	$$10.9 \mathrm{ms} \pm 69.0 \mathrm{μs}\left({\color{gray}0.562 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/person/v/1`	$$10.9 \mathrm{ms} \pm 76.5 \mathrm{μs}\left({\color{gray}0.367 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/playlist/v/1`	$$10.9 \mathrm{ms} \pm 76.1 \mathrm{μs}\left({\color{gray}1.12 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/song/v/1`	$$11.0 \mathrm{ms} \pm 88.6 \mathrm{μs}\left({\color{gray}-0.347 \mathrm{\%}}\right) $$	Flame Graph
entity_by_id	entity type ID: `https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1`	$$10.9 \mathrm{ms} \pm 65.1 \mathrm{μs}\left({\color{gray}1.49 \mathrm{\%}}\right) $$	Flame Graph

representative_read_entity_type

Function	Value	Mean	Flame graphs
get_entity_type_by_id	Account ID: `bf5a9ef5-dc3b-43cf-a291-6210c0321eba`	$$8.02 \mathrm{ms} \pm 41.4 \mathrm{μs}\left({\color{gray}0.257 \mathrm{\%}}\right) $$	Flame Graph

representative_read_multiple_entities

Function	Value	Mean	Flame graphs
entity_by_property	traversal_paths=0	0	$$56.8 \mathrm{ms} \pm 403 \mathrm{μs}\left({\color{gray}2.29 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$109 \mathrm{ms} \pm 637 \mathrm{μs}\left({\color{gray}2.09 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$62.8 \mathrm{ms} \pm 494 \mathrm{μs}\left({\color{gray}2.48 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$71.4 \mathrm{ms} \pm 482 \mathrm{μs}\left({\color{gray}1.71 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$80.2 \mathrm{ms} \pm 545 \mathrm{μs}\left({\color{gray}2.40 \mathrm{\%}}\right) $$
entity_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$87.6 \mathrm{ms} \pm 573 \mathrm{μs}\left({\color{gray}2.56 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=0	0	$$46.4 \mathrm{ms} \pm 294 \mathrm{μs}\left({\color{gray}2.76 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=255	1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true	$$74.2 \mathrm{ms} \pm 506 \mathrm{μs}\left({\color{gray}1.83 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false	$$52.6 \mathrm{ms} \pm 348 \mathrm{μs}\left({\color{gray}2.91 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true	$$61.9 \mathrm{ms} \pm 519 \mathrm{μs}\left({\color{gray}3.36 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true	$$63.4 \mathrm{ms} \pm 392 \mathrm{μs}\left({\color{gray}1.07 \mathrm{\%}}\right) $$
link_by_source_by_property	traversal_paths=2	1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true	$$63.3 \mathrm{ms} \pm 392 \mathrm{μs}\left({\color{gray}1.50 \mathrm{\%}}\right) $$

scenarios

Function	Value	Mean	Flame graphs
full_test	query-limited	$$112 \mathrm{ms} \pm 649 \mathrm{μs}\left({\color{gray}-0.430 \mathrm{\%}}\right) $$	Flame Graph
full_test	query-unlimited	$$124 \mathrm{ms} \pm 545 \mathrm{μs}\left({\color{gray}-0.606 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-limited	$$22.9 \mathrm{ms} \pm 158 \mathrm{μs}\left({\color{gray}0.022 \mathrm{\%}}\right) $$	Flame Graph
linked_queries	query-unlimited	$$516 \mathrm{ms} \pm 1.12 \mathrm{ms}\left({\color{gray}1.36 \mathrm{\%}}\right) $$	Flame Graph

indietyp added this pull request to stack #9506 September 11, 2026 15:05

indietyp deployed to pull-request September 11, 2026 15:06 — with GitHub Actions Active

vercel Bot deployed to Preview – petrinaut September 11, 2026 15:08 View deployment

vercel Bot deployed to Preview – petrinaut-docs September 11, 2026 15:09 View deployment

vercel Bot deployed to Preview – hash September 11, 2026 15:14 View deployment

indietyp changed the title ~~bm/be 816 atlas pull fit generations from s3~~ BE-816: atlas: pull fit generations from S3 Sep 11, 2026

indietyp force-pushed the bm/be-816-atlas-pull-fit-generations-from-s3 branch from 384d83d to 25240e1 Compare September 11, 2026 16:48

indietyp deployed to pull-request September 11, 2026 16:49 — with GitHub Actions Active

vercel Bot deployed to Preview – petrinaut-docs September 11, 2026 16:52 View deployment

vercel Bot deployed to Preview – petrinaut September 11, 2026 16:52 View deployment

vercel Bot deployed to Preview – hash September 11, 2026 17:02 View deployment

indietyp added 5 commits September 11, 2026 21:05

feat: add generation download and s3 round-trip transfer

78c59bf

feat: support remote storage and downloads in atlas serve

9a781a9

chore: add mut qualifier to shutdown closure

68978f3

refactor: clarify atlas generation lifecycle

62571f6

test: add atlas service integration fixtures

ffad6be

indietyp force-pushed the bm/be-816-atlas-pull-fit-generations-from-s3 branch from 25240e1 to ffad6be Compare September 11, 2026 19:05

indietyp deployed to pull-request September 11, 2026 19:06 — with GitHub Actions Active

semgrep-code-hashintel Bot reviewed Sep 11, 2026

View reviewed changes

github-advanced-security Bot found potential problems Sep 11, 2026

View reviewed changes

Comment thread libs/@local/graph/atlas/src/test_utils/postgres.rs

Comment on lines +25 to +29

let (admin, connection) = Config::new()

.host("localhost")

.port(5432)

.user("postgres")

.password("postgres")

vercel Bot deployed to Preview – petrinaut September 11, 2026 19:08 View deployment

vercel Bot deployed to Preview – petrinaut-docs September 11, 2026 19:09 View deployment

vercel Bot deployed to Preview – hash September 11, 2026 19:14 View deployment

github-advanced-security Bot found potential problems Sep 11, 2026

View reviewed changes

Comment thread libs/@local/graph/atlas/src/serve/tests/integration.rs

&DatabaseConnectionInfo::new(

DatabaseType::Postgres,

"postgres".to_owned(),

"postgres".to_owned(),

Navigation Menu

Uh oh!

BE-816: atlas: pull fit generations from S3 - #9682

BE-816: atlas: pull fit generations from S3#9682
indietyp wants to merge 5 commits into
bm/be-817-atlas-push-fit-generations-to-s3from
bm/be-816-atlas-pull-fit-generations-from-s3

indietyp commented Sep 11, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Sep 11, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 11, 2026 •

edited

Loading

Uh oh!

semgrep-code-hashintel Bot Sep 11, 2026

Uh oh!

github-actions Bot commented Sep 11, 2026

@rust/hash-graph-benches – Integrations

policy_resolution_large

policy_resolution_medium

policy_resolution_none

policy_resolution_small

read_scaling_complete

read_scaling_linkless

representative_read_entity

representative_read_entity_type

representative_read_multiple_entities

scenarios

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

indietyp commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌟 What is the purpose of this PR?

🔍 What does this change?

⚠️ Breaking Changes & Known Issues

❓ How to test this?

Uh oh!

vercel Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

semgrep-code-hashintel Bot Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 11, 2026

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

policy_resolution_medium

policy_resolution_none

policy_resolution_small

read_scaling_complete

read_scaling_linkless

representative_read_entity

representative_read_entity_type

representative_read_multiple_entities

scenarios

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

indietyp commented Sep 11, 2026 •

edited

Loading

vercel Bot commented Sep 11, 2026 •

edited

Loading

codecov Bot commented Sep 11, 2026 •

edited

Loading