Feature templates

[beykyle]

Pending PR tasks

• Consistent use of type hints based on clear developer guidelines
• Get bfrescox tests passing. Please document and justify all test design decisions in TestData/README (e.g., adjusting threshold test value)
• Update bfrescoxpro testing to run the same tests as bfrescox
• Get all code passing standard adherence check (flake8 / check tox task)
• Since this is a large PR, some clean-up efforts could be skipped in this PR to keep manageable. Determine what will be skipped and create an Issue for each distinct task and add these as tasks in alpha release Issue

PR Self-review

• Review all changes
• Confirm all tox tasks running locally
• Build source distribution for both and universal wheel for bfrescox and confirm that contents are correct and minimal.
• Assess if code coverage levels are acceptable and assess test update needs to address insufficiencies
  • Is it acceptable that a test file not have 100% coverage?
• Confirm all actions passing

@jared321 PR Review

• Review all changes
• Confirm all tox tasks running locally and, for bfrescoxpro, on a variety of different machines with modules and different MPI implementations
• Confirm correct synchronization with main
• Confirm all actions passing

[codecov-commenter]

Codecov Report

❌ Patch coverage is 79.45205% with 210 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.81%. Comparing base (4fa9e27) to head (93bfc55).

Files with missing lines	Patch %	Lines
common/parse_parallelization_setup.py	6.25%	60 Missing ⚠️
common/_run_frescox_simulation.py	58.88%	37 Missing ⚠️
common/parse_performance_results.py	12.82%	34 Missing ⚠️
common/Configuration.py	70.27%	11 Missing ⚠️
common/generate_inelastic_template.py	83.92%	9 Missing ⚠️
...src/bfrescox/tests/TestRunUserProvidedTemplates.py	91.22%	5 Missing ⚠️
.../bfrescoxpro/tests/TestRunUserProvidedTemplates.py	92.42%	5 Missing ⚠️
common/generate_elastic_template.py	82.75%	5 Missing ⚠️
...escox_pypkg/src/bfrescox/tests/TestElasticSuite.py	93.75%	4 Missing ⚠️
...cox_pypkg/src/bfrescox/tests/TestInelasticSuite.py	93.75%	4 Missing ⚠️
... and 13 more

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #19       +/-   ##
===========================================
+ Coverage   58.75%   78.81%   +20.06%     
===========================================
  Files          17       35       +18     
  Lines         320     1213      +893     
===========================================
+ Hits          188      956      +768     
- Misses        132      257      +125     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[beykyle]

At this point, tests are passing for user defined templates, including tests which run a simulation and compare cross sections parsed from the output to expected values. See bfrescox_pypkg/src/bfrescox/tests/TestRunUserProvidedTemplates.py.

It should be fairly simple to copy and modify this test file to apply to a new test suite. We need test suites for problems with the template types that can be generated in bfrescox:

• elastic
• inelastic

There is an additional remaining task:

• get GH actions to pass

[beykyle]

Also, we could probably simplify testing with https://docs.python.org/3/library/tempfile.html

[beykyle]

I can smell a merge coming

[jared321]

I have reviewed the test suite setup and have the following comments/requests

• The bfrescoxpro MPI/OpenMP setups all specify just a single MPI process, which means that we aren't adequately testing MPI functionality. Running these with 2 MPI processes should be sufficient without running afoul of the limited number of processors in GH action runners.
• When I run the 48Ca user-provided with bfrescoxpro and 2 MPI processes, the test runs without stopping. It appears that Frescox gets stuck writing data endlessly to fort.38. Please determine if this is due to how we are running/configuring the test setup or if it is a bug in Frescox. If it cannot be fixed and we have to run this setup with just one MPI process, please note this in the test suite README notes and add details.
• We presently only test data results loaded from fort.16, which is the official source of Frescox results. However, the data in those files suffers from serious round-off error. This error is so large that we should be able to set our error checking tolerances to zero absolute difference. Please lower these tolerances as much as possible and add details to the test suite README notes about how threshold values were selected. Also include details regarding this testing limitation of fort.16.
• Tests should also load data from stdout and check it as well since the same data is written to stdout with several orders of magnitude more precision.

[beykyle]

* [ ]  When I run the 48Ca user-provided with `bfrescoxpro` and 2 MPI processes, the test runs without stopping.  It appears that Frescox gets stuck writing data endlessly to `fort.38`.  Please determine if this is due to how we are running/configuring the test setup or if it is a bug in Frescox.  If it cannot be fixed and we have to run this setup with just one MPI process, please note this in the test suite README notes and add details.

I can reproduce this, investigating

[beykyle]

• at least one test case per suite now uses "nMpiProcs" : 2
• the deadlock/race condition with multiple MPI ranks is was due to invalid settings in that test file, which have been fixed
• rtol is no longer specified, and atol is now 0 for the fort.16 tests
• Stdout tests #47 adds testing comparing stdout