SuperPoint LightGlue TensorRT

SuperPoint and LightGlue with TensorRT.

Demo

• This demo was tested on the Quadro P620 GPU.

Baseline

Image Size: 320 x 240	RTX3080	Quadro P620
SuperPoint (250 points)
SuperPoint (257 points)
LightGlue (256 dims)

Docker(Recommand)

docker pull yuefan2022/tensorrt-ubuntu22.04-cuda12.1:latest
docker run -it --env DISPLAY=$DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix --privileged --runtime nvidia --gpus all --volume ${PWD}:/workspace --workdir /workspace --name tensorrt yuefan2022/tensorrt-ubuntu20.04-cuda11.6:latest /bin/bash

Environment Required

• CUDA==12.1
• TensorRT==8.6.1.6
• OpenCV>=4.0
• EIGEN
• yaml-cpp

Convert Model(Optional)

The converted models (superpoint_v1_fixed.onnx, superpoint_lightglue_fixed.onnx) and the config.yaml naming them are no longer tracked in this repository; AllFeature-VSLAM downloads them from the HF repo vslamlab/allfeature-vslamlab (folder superpoint_models/). If you are using the pretrained model officially provided by SuperPoint and SuperGlue, you do not need to go through this step.

python convert2onnx/convert_superpoint_to_onnx.py --weight_file superpoint_pth_file_path --output_dir superpoint_onnx_file_dir
python convert2onnx/convert_superglue_to_onnx.py --weight_file superglue_pth_file_path --output_dir superglue_onnx_file_dir

Build and Run

git clone https://github.com/yuefanhao/SuperPoint-LightGlue-TensorRT.git
cd SuperPoint-LightGlue-TensorRT
mkdir build
cd build
cmake ..
make
# test on image pairs 100 times, the output image will be saved in the build dir
./superpoint_lightglue_image  <superpoint_models>/config.yaml <superpoint_models>/ ${PWD}/../image/image0.png ${PWD}/../image/image1.png
# test on the folder with image sequence, output images will be saved in the param assigned dir
./superpoint_lightglue_sequence  <superpoint_models>/config.yaml <superpoint_models>/ ${PWD}/../image/freiburg_sequence/ ${PWD}/../image/freiburg_sequence/match_images/

The default image size param is 320x240, if you need to modify the image size in the config file, you should delete the old .engine file in the models dir (engines are GPU/TensorRT-version specific and are rebuilt from the ONNX on first run).