feat(insight): add FileExporter - #700
Draft
wangyb-A wants to merge 2 commits into
Draft
Conversation
added 2 commits
September 9, 2026 23:39
Port the JS FileExporter to the Java insight-plugin. Writes each
Workflow Insight record to a writable directory (EFS mount, S3 File
Gateway, or /tmp) with no extra dependency, using only java.nio.file.
Two modes mirror JS: NDJSON (default) appends one compact JSON line
per emission to a date-partitioned {YYYY-MM-DD}.ndjson keyed off the
record's emittedAt day; JSON writes one pretty-printed (2-space) file
per execution, overwriting on update. operationsFormat selects ARRAY
(default), BY_NAME, or BOTH renderings. Builder validates a required
directory and a positive maxRecordSizeBytes (no default cap, matching
JS). File names are sanitized to [a-zA-Z0-9._-] and the resolved path
is verified to stay within the configured directory.
Adds a public emittedAt() accessor and a toBothWireMap() rendering to
WorkflowInsightRecord, and a stringifyPretty() helper to Json. No
change to SDK core or WorkflowInsight. Tests use real temporary
directories, not mocked IO. README documents both modes.
Document non-atomic concurrent NDJSON append on shared EFS/NFS and recommend JSON mode or a single writer; append operationsByName last in toBothWireMap() to match the JS spread order and correct its comment; add a traversal-shaped emittedAt guard test and a deterministic concurrent NDJSON append test (real @tempdir IO).
Contributor
Author
|
Intent: Add a filesystem |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Issue Link, if available
Related: #679 (does not close it)
Description
Adds a
FileExporterto the Workflow Insight plugin so insight records can be written to the local filesystem, aligned with the JavaScript SDK'sfile-exportercontract where practical.The exporter has two write modes:
{directory}/{YYYY-MM-DD}.ndjson. WhenemittedAtis missing it falls back to today's UTC date.{directory}/{sanitize(executionName ?? executionArn)}.json, overwriting on each write.sanitizemaps[^a-zA-Z0-9._-]to_.Operation formats mirror the JS
operationsFormatoption:ARRAY(default): operations as an array.BY_NAME: operations grouped underoperationsByName.BOTH: the operations array plus an appendedoperationsByNamekey. The key is appended last (after all record fields and any truncation markers) to match JS{...record, operationsByName}field order.A builder validates a required non-blank
directoryand a positivemaxRecordSizeBytes(which has no default, matching JS). The exporter uses onlyjava.nio.file(no new dependency); truncation and per-exporterThrowableisolation are handled centrally inWorkflowInsight.emit, so no plugin/core change was needed. Path names are normalized and asserted to stay inside the configureddirectory(traversal defense in depth); IO failures surface asUncheckedIOException.EFS / shared-mount caveat (documented in the README): NDJSON
Files.write(..., APPEND)is atomic for a single writer on local disk but is NOT guaranteed atomic on a shared NFS/EFS mount written by more than one Lambda environment concurrently — concurrent writers can interleave partial lines or overwrite each other. The README recommends JSON mode (one file per execution, no shared append) or a single writer per NDJSON file (e.g. a per-environment subdirectory).Additive shared changes (public,
@Experimental, no signatures changed — same pattern as the S3/CloudWatch exporters consuming the record):WorkflowInsightRecord: added a publicemittedAt()accessor andtoBothWireMap()(builds ontoWireMap()and appendsoperationsByNamelast).Json: addedstringifyPretty()(2-spaceDefaultPrettyPrinterwith\nbreaks, matchingJSON.stringify(x, null, 2)).Demo/Screenshots
N/A — backend library change with no user-visible UI.
Checklist
Testing
Unit Tests
Yes.
FileExporterTest(15 tests, real@TempDirIO, no mocked filesystem):BY_NAMEandBOTHoperation formatsemittedAtfallback to todaymaxRecordSizeBytesnull and setflushno-optraversalShapedEmittedAtIsRejectedByResolveChild:emittedAt = "../../etc/passwd"drives a traversal-shaped NDJSON name; assertsIllegalStateException("escapes the configured directory") and no file written outsidedirectoryconcurrentNdjsonAppendsWriteCompleteParseableLines: 8 threads × 25 exports (200 total) against one shared NDJSON file; asserts exactly 200 non-blank lines, each parseable as a completeWorkflowInsightJSON object (no interleaving/truncation on local disk)Full reactor
mvn clean install -DskipITs: BUILD SUCCESS, 1956 tests, 0 failures, 0 errors;spotless:checkclean.Integration Tests
No. The plugin
emitpath (truncation + per-exporter isolation) is already covered by existingWorkflowInsighttests; the new exporter is covered end-to-end against real temp directories via the unit suite above.Examples
No new example added — the README FileExporter usage subsection documents both modes, safe naming, the no-default size cap, and the EFS-vs-local persistence guidance.