Preserve recursive FileTransfer paths

[cooktheryan]

Summary

• Preserve directory structure below targetPath when FileTransfer deploys files
• Keep single-file glob selections copying directly into destinationDirectory
• Support nested paths for renames, deletions, and opt-in tracked cleanup
• Extend documentation, release notes, and real host-artifact lifecycle coverage

Closes #236.

Validation

• go test -mod=mod ./... -count=1
• go test -mod=mod -tags host_artifacts_integration ./pkg/engine -run '^$' -count=1
• python -m sphinx -W -E -b html docs _output/docs
• End-to-end validation on a disposable AWS Fedora 43 instance with Podman 5.8.8:
  • recursive deployment preserved nested directories
  • targetPath was not duplicated in the destination
  • a single-file glob copied only test1.txt
• The AWS instance, temporary key, and security group were removed after validation

Summary by Sourcery

Preserve FileTransfer directory structure below target paths across container and host artifact deployments.

New Features:

• Preserve nested repository paths when deploying FileTransfer files while keeping single-file selections targeted directly at the destination directory.

Bug Fixes:

• Correct nested-path handling for FileTransfer renames, deletions, retries, and tracked cleanup without duplicating target paths.

Enhancements:

• Add path validation and safe creation of nested destination directories for host-managed artifacts.

Documentation:

• Update method documentation and release notes for recursive FileTransfer path preservation.

Tests:

• Expand lifecycle and integration coverage for nested FileTransfer deployment, renames, deletions, and cleanup.

[sourcery-ai]

Sorry @cooktheryan, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 2 days and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.

[sourcery-ai]

Reviewer's Guide

FileTransfer now carries validated repository-relative paths through Podman and host-artifact deployment, preserving nested directory structure for recursive selections and lifecycle operations while maintaining direct destination behavior for single-file and Systemd flows; tests, CI validation, examples, and documentation are updated accordingly.

Sequence diagram for recursive FileTransfer deployment

sequenceDiagram
    participant Git as Git changes
    participant FT as FileTransfer
    participant Podman as Podman
    participant Dest as Destination directory
    Git->>FT: MethodEngine
    FT->>FT: fileTransferRelativePath
    FT->>Podman: fileTransferPodman
    Podman->>Podman: generateSpec
    Podman->>Dest: rsync --mkpath to destination/relativePath
    Dest-->>FT: Nested file deployed

Loading

Sequence diagram for nested FileTransfer lifecycle operations

sequenceDiagram
    participant Git as Git changes
    participant FT as FileTransfer
    participant Host as Host artifact deployment
    participant Dest as Artifact destination
    Git->>FT: MethodEngine
    FT->>FT: fileTransferRelativePath
    FT->>Host: deployHostArtifact
    Host->>Host: executeHostArtifactPlan
    Host->>Host: artifactTargetPath
    alt apply
        Host->>Dest: Create parent directories and rename staged file
    else delete or rename
        Host->>Dest: Remove previous nested path
    end

Loading

File-Level Changes

Change	Details	Files
Preserve repository-relative paths during FileTransfer deployment, renames, and deletions while retaining direct placement for single-file selections.	Compute and validate paths relative to the configured target root. Pass nested destination paths through Podman rsync commands and cleanup operations. Use path-aware container names and create missing parent directories with rsync. Keep Systemd-backed transfers using basename destinations.	pkg/engine/filetransfer.go pkg/engine/container.go pkg/engine/systemd.go
Extend host artifact management to safely support nested FileTransfer destinations.	Add an explicit destination path to artifact plans and validate every path component. Create and verify real parent directories before reading, removing, or installing artifacts. Apply nested paths consistently to staging, fingerprinting, rename, deletion, and cleanup flows.	pkg/engine/host_artifacts.go pkg/engine/host_artifact_helper.go
Add coverage for nested lifecycle behavior and recursive deployment validation.	Test nested create, rename, delete, and failed-removal command generation. Test host-side nested installation, rename, and deletion. Wait for nested deployment in the container integration workflow.	pkg/engine/filetransfer_lifecycle_test.go pkg/engine/host_artifacts_test.go .github/workflows/docker-image.yml
Document and demonstrate recursive FileTransfer behavior.	Add a nested sample file. Update method documentation and release notes with the path-preservation behavior.	examples/filetransfer/nested/hello.txt docs/methods.rst docs/release_notes.rst

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#236	Allow files selected from a Twitch stream directory to be deployed recursively, preserving their paths beneath the configured target path.	✅
#236	Create required nested destination directories and preserve nested paths for file transfers, including renames and deletions.	✅

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.