Skip to content

core: start-blueos-core: Add runtime tuning for improved stream performance#3816

Merged
patrickelectric merged 1 commit intobluerobotics:1.4from
joaoantoniocardoso:1.4-runtime-performance-tuning
Mar 4, 2026
Merged

core: start-blueos-core: Add runtime tuning for improved stream performance#3816
patrickelectric merged 1 commit intobluerobotics:1.4from
joaoantoniocardoso:1.4-runtime-performance-tuning

Conversation

@joaoantoniocardoso
Copy link
Member

@joaoantoniocardoso joaoantoniocardoso commented Mar 4, 2026

Summary

Adds a tune_performance() function to core/start-blueos-core that applies runtime sysctl tuning on every container start. These changes were essential for improved WebRTC performance and stream stability.

Changes

UDP buffer sizes (wmem_max, rmem_max, wmem_default, rmem_default, netdev_max_backlog)

From testing, the minimum UDP buffer size for stable streaming was actually 2.5 MB, but we set it to 16 MB instead to be future-proof (4K 60fps, 100 Mbps, etc). We can lower this if reviewers think it is too much.

Note that this potentially increases the RAM used by all services that use UDP, not just the streaming pipeline, since these are system-wide sysctl settings.

Swappiness (vm.swappiness = 10)

This did not come from test data. It was chosen as a better configuration than the current default (60), since the team had already mentioned during a meeting that it would be good to reduce swap use. On SD cards, swap I/O is extremely slow and creates latency spikes, so preferring file cache reclamation over swapping should improve overall responsiveness.

Real-time scheduler limits (sched_rt_runtime_us = 950000)

This is a safe rationale, not derived from test data. MCM uses many real-time (SCHED_RR/SCHED_FIFO) threads for streaming, and we do not want to let them starve the rest of the system. The setting reserves 5% of each CPU core's time (50 ms per second) for non-RT threads, preventing a scenario where aggressive RT scheduling locks out essential system processes.

Summary by Sourcery

Introduce runtime system tuning in start-blueos-core to improve streaming performance and system responsiveness by adjusting kernel networking and scheduling parameters on container startup.

Enhancements:

  • Apply UDP-related sysctl tuning at container start to increase buffer sizes and backlog for more stable high-bitrate streaming.
  • Lower system swappiness to reduce reliance on slow swap storage and improve interactivity on constrained media.
  • Configure real-time scheduler runtime limits to prevent RT threads from starving other system processes during intensive streaming workloads.

@sourcery-ai
Copy link

sourcery-ai bot commented Mar 4, 2026

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adds a tune_performance() routine to the blueos-core startup script that applies sysctl-based runtime tuning on each container start to improve UDP streaming performance, constrain real-time scheduling, and reduce swap usage.

File-Level Changes

Change Details Files
Add tune_performance() helper invoked during blueos-core startup to apply sysctl runtime tuning.
  • Introduce a new tune_performance() function in the start-blueos-core script.
  • Within tune_performance(), set UDP-related kernel networking sysctls (wmem_max, rmem_max, wmem_default, rmem_default, netdev_max_backlog) to higher values (around 16 MB buffers) to support high-bitrate, low-latency streaming.
  • Adjust vm.swappiness to a lower value (10) to reduce swap usage and favor page cache reclamation.
  • Configure sched_rt_runtime_us to reserve approximately 5% CPU time for non-real-time tasks, preventing RT-thread starvation.
  • Wire tune_performance() into the container startup flow so it runs on every container start.
core/start-blueos-core

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Consider making the sysctl tuning values configurable (e.g., via env vars or a small config file) so they can be adjusted per deployment without modifying the script.
  • Ensure tune_performance() handles failures gracefully when a sysctl key is unavailable or when the container lacks permission to set it, and that such failures don't prevent the rest of the startup sequence.
  • It may be helpful to group the tuning parameters as named constants or a data structure inside the script so that future adjustments to these values have a single, obvious place to change.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider making the sysctl tuning values configurable (e.g., via env vars or a small config file) so they can be adjusted per deployment without modifying the script.
- Ensure tune_performance() handles failures gracefully when a sysctl key is unavailable or when the container lacks permission to set it, and that such failures don't prevent the rest of the startup sequence.
- It may be helpful to group the tuning parameters as named constants or a data structure inside the script so that future adjustments to these values have a single, obvious place to change.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Free Tier Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Bugbot Autofix is ON, but it could not run because the branch was deleted or merged before autofix could start.

@joaoantoniocardoso joaoantoniocardoso force-pushed the 1.4-runtime-performance-tuning branch from 970524c to 80d48fc Compare March 4, 2026 18:05
@patrickelectric patrickelectric merged commit 428bb99 into bluerobotics:1.4 Mar 4, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants