Skip to content

Telemetry follow-up: tier 2, endpoint hardening, retention #1429

Description

@davidberenstein1957

Is your feature request related to a problem? Please describe.

#1200 ships a reduced version of the telemetry proposed in #1106: only the minimal tier, sent once per process to an open /telemetry endpoint, with no API token and no coordinates. This issue tracks what was left out so #1200 could stay small.

Describe the solution you'd like

  1. Tier 2 (extensive). Reintroduce the usage fields from Add internal and public telemetry #1106 (integration, output methods, notebook/IDE/CI detection, energy and emissions summary). Open questions: are the fields sent to /telemetry only, or also published as runs and emissions in a public CodeCarbon experiment, as Add internal and public telemetry #1106 proposed? The second needs a write token, and a token shipped in an open-source package is public, so it must only be able to write to that one experiment.
  2. Endpoint hardening. The in-process per-IP limit on /telemetry (60 requests per minute) can be bypassed: uvicorn runs with --proxy-headers --forwarded-allow-ips=* (carbonserver/docker/entrypoint.sh), so a client can set its own X-Forwarded-For. It is also per process. Options: trust only the load balancer's addresses, or rate-limit at the proxy.
  3. Retention. Decide how long telemetry rows are kept, document it, and add the purge job.
  4. Schema generation. Generate the server schema from the client one (or the reverse) instead of relying on the drift test.

Describe alternatives you've considered

Keeping tier 2 in #1200. Dropped to get the minimal tier reviewed and deployed first.

Additional context

Related: #1106 (original proposal), #1200 (minimal tier).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions