Feat/integrate reporting endpoints & use in HEMS script

.gitignore

@@ -56,6 +56,3 @@ venv/*
 log
 HEMS data full
 *.pkl
-
-# Reporter parameters json files
-*reporter_param.json

README.rst

@@ -110,7 +110,7 @@ Post a measurement from a sensor:
 .. code-block:: python
 
     await client.post_sensor_data(
-        sensor_id=<sensor_id>,  # integer
+        sensor_id=1,
         start="2023-03-26T10:00+02:00",  # ISO datetime
         duration="PT6H",  # ISO duration
         values=[1, 2, 3, 4],  # list
@@ -162,38 +162,41 @@ Scheduling
 
 With FlexMeasures a schedule can be requested to optimize at what time the flexible assets can be activated to optimize for price of energy or emissions.
 
-The calculation of the schedule can take some time depending on the complexity of the calculations. A polling function is used to check if a schedule is available after triggering the schedule.
+The calculation of a schedule can take some time. On FlexMeasures v0.33.0 and
+newer, the convenience method waits on the generic job-status endpoint before
+retrieving the schedule values. It falls back to result-endpoint polling on
+older servers.
 
 Trigger and retrieve a schedule for multiple devices:
 
 .. code-block:: python
 
-    schedule = await flexmeasures_client.trigger_and_get_schedule(
-        asset_id=<asset_id>,  # the asset ID (int) of the asset that all relevant power sensors belong to (or live under, in case of a tree-like asset structure)
-        start="2023-03-26T10:00+02:00",  # ISO datetime
+    schedules = await client.trigger_and_get_schedule(
+        asset_id=3,
+        start="2026-09-05T08:00+02:00",
         duration="PT12H",  # ISO duration
         flex_context={
-            "consumption-price": {"sensor": <consumption_price_sensor_id>},  # int
+            "consumption-price": {"sensor": 7},
         },
-        flex-model=[
+        flex_model=[
             # Example flex-model for an electric truck at a regular Charge Point
             {
-                "sensor": <power_sensor_id>,  # int
+                "sensor": 8,
                 "power-capacity": "22 kVA",
                 "production-capacity": "0 kW",
                 "soc-at-start": "50 kWh",
                 "soc-max": "400 kWh",
                 "soc-min": "20 kWh",
                 "soc-targets": [
-                    {"value": "100 kWh", "datetime": "2023-03-03T11:00+02:00"},
+                    {"value": "100 kWh", "datetime": "2026-09-05T18:00+02:00"},
                 ],
             },
             # Example flex-model for curtailable solar panels
             {
-                "sensor": <another_power_sensor_id>,  # int
+                "sensor": 9,
                 "power-capacity": "20 kVA",
                 "consumption-capacity": "0 kW",
-                "production-capacity": {"sensor": <another_power_sensor_id>},  # int
+                "production-capacity": {"sensor": 9},
             },
         ],
     )
@@ -203,19 +206,19 @@ Alternatively, use a single-device flex-model (no list) and move the device's po
 
 .. code-block:: python
 
-    schedule = await flexmeasures_client.trigger_and_get_schedule(
-        sensor_id=<sensor_id>,  # int
-        start="2023-03-26T10:00+02:00",  # ISO datetime
+    schedule = await client.trigger_and_get_schedule(
+        sensor_id=8,
+        start="2026-09-05T08:00+02:00",
         duration="PT12H",  # ISO duration
         flex_context={
-            "consumption-price": {"sensor": <consumption_price_sensor_id>},  # int
+            "consumption-price": {"sensor": 7},
         },
-        flex-model={
+        flex_model={
             "soc-at-start": "50 kWh",
             "soc-max": "400 kWh",
             "soc-min": "20 kWh",
             "soc-targets": [
-                {"value": "100 kWh", "datetime": "2023-03-03T11:00+02:00"},
+                {"value": "100 kWh", "datetime": "2026-09-05T18:00+02:00"},
             ],
         },
     )
@@ -226,22 +229,26 @@ Trigger a schedule:
 
 .. code-block:: python
 
-    schedule_uuid = await flexmeasures_client.trigger_schedule(
+    schedule_uuid = await client.trigger_schedule(
         **kwargs,  # same kwargs as previous example
     )
 
 The ``trigger_schedule`` method returns a ``schedule_uuid``.
-This can be used to retrieve the schedule, using:
+On FlexMeasures v0.33.0 and newer, wait for the job once before retrieving one
+or more sensor results:
 
 .. code-block:: python
 
-    schedule = await flexmeasures_client.get_schedule(
-        sensor_id=<sensor_id>,  # int
-        schedule_id="<schedule_uuid>",  # uuid
+    await client.wait_for_job(schedule_uuid)
+
+    schedule = await client.get_schedule(
+        sensor_id=8,
+        schedule_id=schedule_uuid,
         duration="PT45M",  # ISO duration
     )
 
-The client will re-try until the schedule is available or the ``MAX_POLLING_STEPS`` of ``10`` is reached.
+For the complete scheduling API, including multi-device results, job timeouts,
+and compatibility with older servers, see :doc:`scheduling`.
 
 
 Forecasting
@@ -252,13 +259,15 @@ Trigger a forecast for a sensor and wait for the result:
 .. code-block:: python
 
     forecast = await client.trigger_and_get_forecast(
-        sensor_id=<sensor_id>,  # int
+        sensor_id=1,
         duration="PT24H",  # ISO duration – how far ahead to forecast
     )
     # Returns e.g. {"values": [1.2, 1.5, ...], "start": "...", "duration": "PT24H", "unit": "kW"}
 
-The client polls until the forecasting job is complete.  For more advanced options
-(training window, regressors, forecast frequency, etc.) see :doc:`forecasting`.
+On FlexMeasures v0.33.0 and newer, the client polls the generic job endpoint
+until the forecasting job is complete, then retrieves its values. For more
+advanced options (training window, regressors, forecast frequency, etc.) see
+:doc:`forecasting`.
 
 
 Development

docs/HEMS.rst

@@ -15,14 +15,14 @@ This is the resulting dashboard:
     :align: center
 |
 
-.. note:: The tutorial still uses the CLI for reporting. In future versions, we might make reporting available via the API, as well.
+.. note:: The tutorial talks to FlexMeasures over the API only, including for reporting. That requires a FlexMeasures server of version 1.1.0 or above, and a worker listening on the ``reporting`` queue.
 
 
 Set up your environment
 ========================
 
 To run the HEMS example (``HEMS_setup.py``), you'll need an environment in which both ``flexmeasures`` (the server) and ``flexmeasures-client`` is installed.
-The example requires FlexMeasures 1.0 or newer.
+The example requires FlexMeasures 1.1.0 or newer, since it triggers reports over the API.
 
 We use `uv <https://docs.astral.sh/uv/>`_ to manage dependencies. First, `install uv <https://docs.astral.sh/uv/getting-started/installation/>`_.
 
@@ -115,11 +115,11 @@ Open three terminals. In the first terminal, run the server:
     flexmeasures run
 
 In the second terminal, run a flexmeasures worker that listens to the
-forecasting, scheduling, and ingestion queues:
+forecasting, scheduling, ingestion, and reporting queues:
 
 .. code-block:: bash
 
-    flexmeasures jobs run-worker --queue "forecasting|scheduling|ingestion"
+    flexmeasures jobs run-worker --queue "forecasting|scheduling|ingestion|reporting"
 
 Note: you can run the same command in two terminals (2 workers), to speed up the computation!
 
@@ -130,17 +130,9 @@ In the third terminal, go to the HEMS directory:
     cd examples/HEMS
 
 .. note::
-   For the time being, report generation (see :ref:`hems-tutorial` note above) shells out to a ``flexmeasures`` CLI process, which by default is expected on ``PATH`` and configured against the same database as the server. If your FlexMeasures server runs elsewhere (e.g. inside a Docker Compose service), point report generation at it instead via two environment variables:
+   Reports are triggered over the API, so the client script needs nothing beyond its API credentials: no local CLI, no database access, and no bind-mount of ``examples/HEMS/configs/`` into the server. Those configuration files are read by the client and posted along with each report request. The server does need a worker on the ``reporting`` queue, as above, or reports will stay queued until the client gives up on them.
 
-   - ``FLEXMEASURES_CLI_CMD``: the command used to invoke the CLI
-   - ``FLEXMEASURES_CLI_CONFIG_DIR``: the directory the CLI process sees the ``examples/HEMS/configs/`` files at, if different from their local path
-
-   Here are steps if you use FlexMeasures' docker-compose:
-   - ``export FLEXMEASURES_CLI_CMD="docker compose -f full/path/to/docker-compose.yml exec -T server flexmeasures"``.
-   - Add this mount in docker-compose.yml under server.volumes, and restart it: ``- /full/path/to/flexmeasures-client/examples/HEMS/configs:/app/hems-configs:ro``
-   - ``export FLEXMEASURES_CLI_CONFIG_DIR="/app/hems-configs"``
-
-Another caveat is rate-limiting. Since v1.0, FlexMeasures only allows a limited number of schedule and forecasts per 5 minute interval.
+Another caveat is rate-limiting. Since v1.0, FlexMeasures only allows a limited number of schedule, forecast and report triggers per 5 minute interval.
 Either give your account a generous plan (see the docs), or simply set ``FLEXMEASURES_MODE="play"`` and restart the server.
 If you use docker-compose, you could do that like this:
 

docs/forecasting.rst

@@ -7,17 +7,19 @@ The FlexMeasures Client supports the forecasting API endpoints introduced in
 FlexMeasures v0.31.0:
 
 - ``POST /sensors/<id>/forecasts/trigger`` — queue a forecasting job
-- ``GET  /sensors/<id>/forecasts/<uuid>``  — poll for results
+- ``GET  /jobs/<uuid>``                    — inspect the job (v0.33.0+)
+- ``GET  /sensors/<id>/forecasts/<uuid>``  — retrieve the result
 
 These are exposed through three client methods:
 
 - :meth:`trigger_forecast` — trigger and return the job UUID
-- :meth:`get_forecast`     — poll until results are ready
-- :meth:`trigger_and_get_forecast` — convenience wrapper for both
+- :meth:`get_forecast` — retrieve results, with legacy result polling when needed
+- :meth:`trigger_and_get_forecast` — trigger, wait, and retrieve
 
 .. note::
 
-    These endpoints require a FlexMeasures server of version **0.31.0** or above.
+    Forecasting requires a FlexMeasures server of version **0.31.0** or above.
+    The generic job status endpoint is available from **v0.33.0**.
 
 
 Basic example
@@ -117,31 +119,70 @@ Trigger and retrieve separately to handle the job UUID yourself:
     )
     print(f"Job queued: {forecast_id}")
 
-    # Step 2 – poll until the job finishes
+    # Step 2 – wait for the job itself to finish (FlexMeasures v0.33.0+)
+    await client.wait_for_job(forecast_id)
+
+    # Step 3 – retrieve the forecast values
     forecast = await client.get_forecast(
         sensor_id=1,
         forecast_id=forecast_id,
     )
     print(forecast)
 
 
-Polling behaviour
------------------
+Waiting and legacy polling
+--------------------------
+
+On FlexMeasures v0.33.0 and newer, ``trigger_and_get_forecast`` waits through
+``GET /jobs/<uuid>`` and fetches the forecast values only after the job has
+finished. Its job wait uses exponential backoff and the following client
+defaults:
+
+- ``job_polling_interval`` (default 2 s) — delay before a repeated status check
+- ``job_polling_max_interval`` (default 30 s) — maximum delay between checks
+- ``job_polling_timeout`` (default 600 s) — total job wait budget
+
+Set these defaults when constructing the client. The convenience method's
+``polling_interval``, ``max_polling_interval``, and ``timeout`` arguments can
+override them for one forecast:
+
+For example:
+
+.. code-block:: python
+
+    client = FlexMeasuresClient(
+        ...,
+        job_polling_interval=5.0,
+        job_polling_timeout=1800.0,
+    )
+
+    forecast = await client.trigger_and_get_forecast(
+        sensor_id=1,
+        duration="PT24H",
+        max_polling_interval=60.0,
+    )
+
+``get_forecast`` still polls the result endpoint when it is called directly,
+and ``trigger_and_get_forecast`` retains that behaviour for servers older than
+v0.33.0. This legacy polling uses the client's general HTTP request settings.
+Those settings also apply to authentication, API discovery, asset and sensor
+operations, data transfer, trigger calls, result retrieval, and each individual
+job-status lookup:
 
-``get_forecast`` polls the server with a ``GET`` request and returns when the
-server responds with HTTP 200.  The polling respects the same client-level
-settings as scheduling:
+- ``request_timeout`` (default 40 s) — timeout for one HTTP attempt
+- ``request_retry_interval`` (default 10 s) — initial wait between retries
+- ``request_retry_timeout`` (default 200 s) — total request/retry budget
+- ``max_request_attempts`` (default 10) — maximum attempts in that loop
 
-- ``polling_interval`` (default 10 s) — time between retries
-- ``polling_timeout`` (default 200 s) — maximum total wait time
-- ``max_polling_steps`` (default 10)  — maximum number of poll attempts
+They do not control the cadence or total lifetime of a background-job wait;
+the ``job_polling_*`` settings above do that.
 
-Override them at client construction time:
+Configure those settings at client construction time:
 
 .. code-block:: python
 
     client = FlexMeasuresClient(
         ...,
-        polling_interval=5.0,   # check every 5 seconds
-        polling_timeout=300.0,  # wait up to 5 minutes
+        request_retry_interval=5.0,  # retry after 5 seconds
+        request_retry_timeout=300.0,  # allow retries for up to 5 minutes
     )

docs/index.rst

@@ -31,6 +31,8 @@ Contents
 
    Overview <readme>
    Forecasting <forecasting>
+   Scheduling <scheduling>
+   Reporting <reporting>
    Contributions & Help <contributing>
    License <license>
    Authors <authors>