7.0.1.1 Flows#94

Draft

davramov wants to merge 24 commits intoals-computing:mainfrom

davramov:flow-7011

Commits on Dec 17, 2025

initial commit for bl7.0.1.1
davramov
committed
Adding globus collection uuid for ALS Computing 7.0.1.1 to the config
davramov
committed
Updating transfer/prune flow to use the compute-dtn collection for 7.0.1.1
davramov
committed
fixing pytests for prefect 3
davramov
committed
Making config optional in new_7011_file_flow and dispatcher, since they are initialized in the functions if set to None
davramov
committed
Adding pytests for move and dispatcher
davramov
committed
Adding mocker patch to dispatcher pytest
davramov
committed
Adding documentation for 7.0.1.1 flows
davramov
committed
Making the dispatcher call process_new_7011_file_task, rather than the flow. Making the process_new_7011_file_flow call the task as well. This is for cleaner observation into the logs.
davramov
committed
removing main methods from move.py and dispatcher.py
davramov
committed
Updating pytest
davramov
committed
Updating bl7011/prefect.yaml to reflect the new file flow -> task change
davramov
committed
Adding 7.0.1.1 flow diagram to mkdocs
davramov
committed
Adding logger = get_run_logger() for better prefect logging
davramov
committed
Ensuring the test flow completely fails if the transfer failed by throwing a RuntimeError
davramov
committed
Adjusting default flight_check file path to point at the existing test/test_065.h5 fuke, since I am hitting a 550 read-only filesystem error when I try to create a new folder in the collection.
davramov
committed
Adding try block around the test transfer, with an except block to get more details about why (potentially) the transfer failed
davramov
committed
Updating 7.0.1.1 endpoint details in config.yml
davramov
committed
Fixing the 7011 nersc alsdev globus endpoint (was pointing at the wrong nersc endpoint previously)
davramov
committed
Adding flow_run_name to schedule_prefect_flow call in the prune method
davramov
committed
commit before rebasing
davramov
committed
Using Variable blocks instead of JSON blocks
davramov
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

7.0.1.1 Flows#94

7.0.1.1 Flows#94
davramov wants to merge 24 commits intoals-computing:mainfrom
davramov:flow-7011

Commits on Dec 17, 2025

initial commit for bl7.0.1.1

Adding globus collection uuid for ALS Computing 7.0.1.1 to the config

Updating transfer/prune flow to use the compute-dtn collection for 7.0.1.1

fixing pytests for prefect 3

Making config optional in new_7011_file_flow and dispatcher, since they are initialized in the functions if set to None

Adding pytests for move and dispatcher

Adding mocker patch to dispatcher pytest

Adding documentation for 7.0.1.1 flows

Making the dispatcher call process_new_7011_file_task, rather than the flow. Making the process_new_7011_file_flow call the task as well. This is for cleaner observation into the logs.

removing main methods from move.py and dispatcher.py

Updating pytest

Updating bl7011/prefect.yaml to reflect the new file flow -> task change

Adding 7.0.1.1 flow diagram to mkdocs

Adding logger = get_run_logger() for better prefect logging

Ensuring the test flow completely fails if the transfer failed by throwing a RuntimeError

Adjusting default flight_check file path to point at the existing test/test_065.h5 fuke, since I am hitting a 550 read-only filesystem error when I try to create a new folder in the collection.

Adding try block around the test transfer, with an except block to get more details about why (potentially) the transfer failed

Updating 7.0.1.1 endpoint details in config.yml

Fixing the 7011 nersc alsdev globus endpoint (was pointing at the wrong nersc endpoint previously)

Adding flow_run_name to schedule_prefect_flow call in the prune method

commit before rebasing

Using Variable blocks instead of JSON blocks

Commits on Jan 20, 2026

Making the dispatcher/move flows accepts a metadata field for the UID from Databroker

updating pytest to handle the new metadata field