Skip to content

Releases: Framework-R-D/phlex

Phlex v0.2.0

31 Mar 19:38
@knoepfel knoepfel
v0.2.0
c2a09bd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Phlex v0.2.0 Latest
Latest

Important

This is a prototype release. It is known to have incomplete functionality with respect to the Phlex conceptual design; backwards compatibility of API is not guaranteed. The following API is intended for use:

  • PHLEX_REGISTER_ALGORITHMS(...) opener for registering physics algorithms
  • PHLEX_REGISTER_PROVIDERS(...) opener for registering data-product providers
  • PHLEX_REGISTER_DRIVER(...) opener for registering a user-defined framework-driver function
  • Anything in the top-level phlex namespace

Any interface (namespaces, functions, classes, etc.) that contains the strings experimental, internal, or detail should not be directly used.

Phlex v0.2.0 is the second prototype release of the Phlex framework. The purpose of the v0.2.0 release is to allow early adopters of Phlex to continue exercising the basic functionalities provided by Phlex v0.1.0, while testing a few new features, such as:

  • A user-defined API for registering C++ framework drivers according to fixed hierarchies
  • A richer grammar for specifying data products that serve as input to algorithms
  • Python providers that can emit Python data products
  • Writing user-defined C++ data products to framework files

Instructions for building code that uses Phlex and running simple Phlex jobs can be found in the phlex-examples repository.

Note

Phlex v0.2.0 uses C++23. You will need a compiler with sufficient C++23 support to build and install this release (GCC 15, LLVM 21, etc.)

Specific Features

  1. API to register C++ fixed-hierarchy drivers

    • Users will be able to specify a fixed hierarchy by which data cells can be processed by the framework (see the interface described at the top of #443 for an example)
    • Hierarchies as read from input files are not yet supported
    • For most users, it will be sufficient to continue using the generate_layers driver plugin as provided in Phlex v0.1.0 and described at #145.

  2. Product specification with the creator name

    • Phlex v0.2.0 replaces the "<product_name>"_in("<data layer>") syntax with a more expressive syntax that specifies product constraints: 
      - "<product_name>"_in("<data layer>")
+ { .creator = "<product_creator>", .layer = "<data layer>", .suffix = "<product suffix>" }
      where the suffix field is only necessary when disambiguating between products of the same type created by an algorithm.

  3. Provide access to product creator name, suffix, and phase name through handles

    • The phlex::handle<T> interface now provides access to the product creator and product suffix (not provided in Phlex v0.1.0) in addition to the product itself and its data-cell index.

  4. Algorithms that can access multiple data products from different data layers of the same hierarchy path

    • A hierarchy path is a path defined as a chain of data-cell indices generated by data_cell_cursor::yield_child(...) invocations
    • In #443, allowed paths include job -> run -> subrun -> spill, job -> run, job -> run -> calibration_period, etc.
    • With Phlex v0.2.0, multiple data products that belong to the same hierarchy path may appear as arguments to a user's algorithm.

  5. Providers that can emit Python data products

  6. Writing user-defined C++ data products to framework files

    • Users interested in doing this should contact the FORM developers for explicit instructions.

  7. Switch from C++20 to C++23

    • Upgrading to C++23 brings the implementation into conformance with the Phlex CDR

  8. Enable writing of data products from providers to the output

    • This was not supported with Phlex v0.1.0. With Phlex v0.2.0, and data products provided by providers are suitable for writing to a FORM output file.

Included Pull Requests

  • Support the installation and use of act to test and run GitHub workflows by @greenc-FNAL in #222
  • Add Project Context & Workflow section to copilot-instructions.md by @Copilot in #236
  • Replace moribund cmake-format with gersemi by @greenc-FNAL in #169
  • Add minor clarifications and cautions to INSTALLATION.md by @marcpaterno in #237
  • Apply gersemi formatting by @greenc-FNAL in #239
  • Convert Boost JSON string to std::string_view by @knoepfel in #225
  • Fix #220 by @beojan in #241
  • run_phlex needs to depend publicly on the library its public headers use by @pcanal in #242
  • VSCode dev container setup and workspace config by @greenc-FNAL in #87
  • Switch to dynamic_cast instead of string comparison for product retrieval by @knoepfel in #203
  • Fix AI hallucination (no such mount option required=false) by @greenc-FNAL in #246
  • Simplify implementation for forming input arguments by @knoepfel in #244
  • Safer product_query construction by @knoepfel in #205
  • Increase ruff's target python version to 3.12 by @wlav in #247
  • New copilot instructions re the purpose of comments by @greenc-FNAL in #251
  • Use find_package arguments to mandate NumPy by @knoepfel in #219
  • Fix CodeQL workflow: correct alert collection bug and improve fork PR handling by @greenc-FNAL in #249
  • Fix workflow naming per @knoepfel report by @greenc-FNAL in #254
  • Add Python test coverage by @greenc-FNAL in #250
  • Export Python symbols globally for pymodule by @knoepfel in #253
  • Include python and py-numpy as roots of CI environment by @knoepfel in #255
  • fix(ci): Ensure CodeQL comment workflow only runs on success by @google-labs-jules[bot] in #258
  • Improve Workflow Robustness and Feedback by @google-labs-jules[bot] in #256
  • Update how type information is passed from Phlex to FORM by @pcanal in #252
  • Shift async_driver gear into park if exception from workflow graph by @knoepfel in #269
  • Refactor workflows for external repositories by @greenc-FNAL in #262
  • Remove unwanted ":" from headings in REUSABLE_WORKFLOWS.md by @greenc-FNAL in #271
  • Quote $GITHUB_OUTPUT where used by @greenc-FNAL in #277
  • Add workflow_dispatch to all relevant workflows by @greenc-FNAL in #272
  • Add markdownlint check and fix workflows by @greenc-FNAL in #274
  • Add jsonnet-format check and fix workflows by @greenc-FNAL in #273
  • Restore integer representation of nproc value by @knoepfel in #283
  • JSON parsing changes from new product query by @beojan in #282
  • Use native CMake functionality for stripping trailing whitespace by @knoepfel in #285
  • Upgrade Phlex to build with C++23 by @knoepfel in #284
  • Revert to cmake-build workflow from the main branch by @knoepfel in #289
  • Homogenize "count" interface by @knoepfel in #287
  • Update CI container by @knoepfel in #294
  • Restore configure-cmake action from main branch by @knoepfel in #295
  • Workflow standardization and improvements by @greenc-FNAL in #290
  • Fix workflow execution: coverage/build on workflow_dispatch, build on push to main by @Copilot in #296
  • Set auto-merge (subject to branch protections) on dependabot PRs by @greenc-FNAL in #297
  • Safety and correctness improvements by @greenc-FNAL in #300
  • Remove vestigial inheritance from std::enable_shared_from_this by @knoepfel in #293
  • Add phlex::identifier by @beojan in #288
  • Enable coverage monitoring for non-test Python modules by @Copilot in #304
  • Rename product_store::id() to product_store::index() by @kno...
Read more

Contributors

marcpaterno, greenc-FNAL, and 11 other contributors
Assets 2
Loading

Phlex v0.1.0

22 Dec 20:09
@knoepfel knoepfel
v0.1.0
14c7635

Choose a tag to compare

View all tags
Phlex v0.1.0

Important

This is a prototype release. It is known to have incomplete functionality with respect to the Phlex conceptual design; backwards compatibility of API is not guaranteed. The following API is intended for use:

  • PHLEX_REGISTER_ALGORITHMS(...) opener for registering physics algorithms
  • PHLEX_REGISTER_PROVIDERS(...) opener for registering data-product providers
  • Anything in the top-level phlex namespace

Any interface (namespaces, functions, classes, etc.) that contains the strings experimental, internal, or detail should not be directly used.

Phlex v0.1.0 is the first prototype release of the Phlex framework. The purpose of the v0.1.0 release is to allow early adopters of Phlex to exercise its basic functionalities, which include:

  • Specifying C++ algorithms as operators to the transform, observe, fold, and unfold higher-order functions
  • Specifying Python algorithms as operators to the transform and observe higher-order functions
  • Executing phlex programs using Jsonnet/JSON runtime configurations that load compiled plugins
  • Configuring a Phlex-provided driver to produce a user-defined hierarchy (a tree) of data layers
  • Writing data products with basic types to Phlex output files through FORM
  • Accessing data-cell information through phlex::handle<T> objects for data products being used by algorithms

Instructions for building code that uses Phlex and running simple Phlex jobs can be found in the phlex-examples repository.

Specific Features

Phlex v0.1.0 provides the following capabilities:

  1. API to register algorithms for C++ and Python
  2. Phlex executable
  3. Configuration 
    1. Supported language is Jsonnet
    2. Ability to configure which modules are loaded (includes ability to configure which algorithms specified in the module are registered)
    3. Ability to configure a Phlex-provided driver regarding which layers should be emitted
  4. Product specification
    1. C++ language type
    2. Global product "name"
    3. Data layer
  5. Higher-order functions
    1. Transform (C++, Python)
    2. Observer (C++, Python)
    3. Fold (C++)
    4. Unfold (C++)
  6. Access to product information (including data-cell ID) through handles
  7. Algorithms that can access multiple data products from the same data layer
  8. Python algorithm that can consume a data product produced by a C++ algorithm
  9. C++ algorithm that can consume a data product produced by a Python algorithm
  10. C++ providers that can emit C++ data products

Included Pull Requests

Read more

Contributors

marcpaterno, greenc-FNAL, and 11 other contributors
Loading