feat(benchmark): add benchmark package

[ktro2828]

PR Type

• New Feature

Related Links

Description

This pull request introduces a new benchmarking library for accelerated image processing, including comprehensive benchmarking functionality, command-line benchmarking tools, and support for both synthetic and ROSBag image data. It establishes the project structure, adds core benchmarking logic, and provides utilities for configuration and processor management.

Core benchmarking functionality:

• Added the Benchmarker class for measuring and reporting image processing performance, including methods for timing, statistics, and storage ratio calculations (benchmarker.hpp, benchmarker.cpp). [1] [2]
• Implemented synthetic image generation via the make_synthetic_image function, enabling benchmarking with generated data (image.hpp, image.cpp). [1] [2]

Command-line benchmarking tool:

• Introduced the compression CLI tool for running compression benchmarks, supporting both synthetic images and images from ROSBag, with flexible configuration and reporting (compression.cpp).
• Added a user guide and instructions for installation and usage in the README.md.

ROSBag and configuration utilities:

• Provided the RosBagReader class for reading image messages from ROSBag files, supporting multiple storage backends (rosbag.hpp).
• Added utility functions for loading YAML configuration, applying parameters to processors, and printing processor information (utility.hpp).

Project setup:

• Created the CMakeLists.txt and package.xml files to define dependencies, build configuration, and installation for the new benchmarking package. [1] [2]

How to Use

• Build package

colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release --packages-up-to accelerated_image_processor_benchmark
source install/setup.bash

• Run compression benchmark: accbench compression <CONFIG_PATH> [...]

Compression benchmark with synthetic images

$ accbench compression ./src/accelerated_image_processor_ros/config/imgproc.param.yaml
------------------ Processor Information ------------------
ClassName: accelerated_image_processor::compression::NvJPEGCompressor
Parameters:
  quality: 80
-----------------------------------------------------------
>>> Preparing input images
Loading synthetic images:
  (Height, Width): (1080, 1920)
>>> Starting warmup [n = 10]
<<< ✨Finished warmup
>>> Starting iterations [n = 100]
<<< ✨Finished iterations
------------------ Benchmark Summary ------------------
Storage Reduction: 96.0227% (593MB -> 23MB)
Iteration ms: [Average]=0.860179, [50%tile]=0.834774, [90%tile]=0.871758, [FPS]=1162.55
-------------------------------------------------------

Compression benchmark with ROSBag

$ accbench compression ./src/accelerated_image_processor_ros/config/imgproc.param.yaml --bag /media/kotarouetake/tier4/rosbag_20230329/RinkaiFukutoshin-to-Sachiura-1/camera0/rosbag2_2023_03_29-01_17_43/rosbag2_2023_03_29-01_17_43_0.db3 --topic /sensing/camera/camera0/image_rect_color
------------------ Processor Information ------------------
ClassName: accelerated_image_processor::compression::NvJPEGCompressor
Parameters:
  quality: 80
-----------------------------------------------------------
>>> Preparing input images
Loading images from bag:
  Bag: /media/kotarouetake/tier4/rosbag_20230329/RinkaiFukutoshin-to-Sachiura-1/camera0/rosbag2_2023_03_29-01_17_43/rosbag2_2023_03_29-01_17_43_0.db3
  Storage ID: sqlite3
  Topic: /sensing/camera/camera0/image_rect_color
[INFO 1770334430.293311784] [rosbag2_storage]: Opened database '/media/kotarouetake/tier4/rosbag_20230329/RinkaiFukutoshin-to-Sachiura-1/camera0/rosbag2_2023_03_29-01_17_43/rosbag2_2023_03_29-01_17_43_0.db3' for READ_ONLY.
>>> Starting warmup [n = 10]
<<< ✨Finished warmup
>>> Starting iterations [n = 100]
<<< ✨Finished iterations
------------------ Benchmark Summary ------------------
Storage Reduction: 95.7606% (703MB -> 29MB)
Iteration ms: [Average]=0.671652, [50%tile]=0.670101, [90%tile]=0.679704, [FPS]=1488.87
-------------------------------------------------------

Review Procedure

Remarks

Pre-Review Checklist for the PR Author

PR Author should check the checkboxes below when creating the PR.

• Assign PR to reviewer

Checklist for the PR Reviewer

Reviewers should check the checkboxes below before approval.

• Commits are properly organized and messages are according to the guideline
• (Optional) Unit tests have been written for new behavior
• PR title describes the changes

Post-Review Checklist for the PR Author

PR Author should check the checkboxes below before merging.

• All open points are addressed and tracked via issues or tickets

CI Checks

• Build and test for PR: Required to pass before the merge.

[Copilot]

Pull request overview

This PR introduces a comprehensive benchmarking package for accelerated image processing, enabling performance measurement with both synthetic images and ROSBag data. The package provides a command-line tool for running compression benchmarks with flexible configuration and detailed performance reporting.

Changes:

• Added Benchmarker class for timing image processing operations and calculating performance metrics (average, percentile, FPS) and storage ratios
• Implemented synthetic image generation for testing without external data dependencies
• Created compression CLI tool with support for both synthetic images and ROSBag input sources

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/accelerated_image_processor_benchmark/CMakeLists.txt	Build configuration for the benchmark package, including dependencies and executable targets
src/accelerated_image_processor_benchmark/package.xml	ROS package metadata and dependencies
src/accelerated_image_processor_benchmark/README.md	User documentation for installation and usage
src/accelerated_image_processor_benchmark/include/accelerated_image_processor_benchmark/benchmarker.hpp	Header defining the Benchmarker class for performance measurement
src/accelerated_image_processor_benchmark/include/accelerated_image_processor_benchmark/image.hpp	Header declaring synthetic image generation function
src/accelerated_image_processor_benchmark/include/accelerated_image_processor_benchmark/rosbag.hpp	Header defining RosBagReader class for reading image messages
src/accelerated_image_processor_benchmark/include/accelerated_image_processor_benchmark/utility.hpp	Header declaring utility functions for configuration and processor management
src/accelerated_image_processor_benchmark/src/benchmarker.cpp	Implementation of benchmarking logic including timing, statistics, and reporting
src/accelerated_image_processor_benchmark/src/image.cpp	Implementation of synthetic image generation with random noise
src/accelerated_image_processor_benchmark/src/compression.cpp	CLI tool for running compression benchmarks
src/accelerated_image_processor_benchmark/src/utility.cpp	Implementation of configuration loading, image loading, and processor utilities

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.