Qualcomm AI Engine Direct - CDSP Direct Mode

[winskuo-quic]

Summary

• Support CDSP direct mode by defining ExecuTorch's customized rpc protocol.

• We have validated this PR with the following spec:

  • QNN 2.42.0
  • Hexagon SDK 6.4.02
  • Hexagon Tools Root 19.0.04 (This is tool chain under Hexagon SDK 6.4.02)
  • V79 & v81 device

• Please refer to README file under backends/qualcomm/runtime/backends/direct_mode/README.md for setup. Please be aware of the Note section if you observe the total execution time is slower the traditional mode.

Example Script

To build:
backends/qualcomm/scripts/build.sh --enable_hexagon

To run traditional mode (Same as usual):
python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedOperator.test_qnn_backend_adaptive_avg_pool2d --model SM8750 --device $DEVICE --build_folder build-android

To run direct mode (add --direct_build_folder build-hexagon):
python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedOperator.test_qnn_backend_adaptive_avg_pool2d --model SM8750 --device $DEVICE --build_folder build-android --direct_build_folder build-hexagon/

Test plan

add --direct_build_folder build-hexagon/ at end of any TestQNNQuantizedUtils, TestQNNQuantizedModel, TestQNNFloatingPointModel, TestQNNFloatingPointOperator

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17326

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Lint/Mac jobs intermittently fail

❌ 1 Awaiting Approval, 17 New Failures, 1 Unrelated Failure

As of commit 9acdb38 with merge base 47b8d1d ():

AWAITING APPROVAL - The following workflow needs approval before CI can run:

• periodic (gh)

NEW FAILURES - The following jobs have failed:

• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t c4b347138dc914ec3239676e724c727247104d927cdae6c17a21bfc9b5216bf3 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 7f1891d2885fd88efc01c82ade26ffcd05a2c1efab7e0c780256a3dd11937ef8 /exec failed with exit code 1
• pull / test-moshi-linux / linux-job (gh)
  RuntimeError: Command docker exec -t e0e8c92db90b4d0c002ec1b8a89c9d22e3e1ec860ed3f331bfc0bcfd8c2754f8 /exec failed with exit code 1
• pull / test-parakeet-xnnpack-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 7999061859afc5bfea590fd7614351e7795ec5f8ededfabab8cf2a0dc4cd8ece /exec failed with exit code 1
• pull / test-samsung-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5f5c19bc78aeb42f19aef49d581ad97ec9cb6f24cd706a32fe078839c3770289 /exec failed with exit code 1
• pull / test-samsung-quantmodels-linux / linux-job (gh)
  RuntimeError: Command docker exec -t dc44e130fc24d9e4311e90aed4608fb95608c72d39227cf214f68a75ba1ded60 /exec failed with exit code 1
• pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (gh)
  RuntimeError: Command docker exec -t ffc89e2e2a581a72d7c8e9caf73b7008ca36b9ecea02d2a0e32c860215429576 /exec failed with exit code 1
• pull / test-static-llama-qnn-linux (stories_110m) / linux-job (gh)
  RuntimeError: Command docker exec -t fe971b110ef2ec6e39d4b0c376291bb7b1dcd518db35ddd42121bad144540d4e /exec failed with exit code 1
• pull / test-static-llama-qnn-linux (stories_260k_bc) / linux-job (gh)
  RuntimeError: Command docker exec -t 1c911268d10f7d2d24ee52964d0e15af0376447cb38fc66e954e68309853c7ce /exec failed with exit code 1
• pull / test-vulkan-operators-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 429f8d73ae344fce8ee860d96d39bbf5384cabc8ae52db6f9253177e035fda1e /exec failed with exit code 127
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1db62b3b958a1976855094368fcf7bcf525b9140f3494b4dc08504c179d17b8 /exec failed with exit code 8
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8
• pull / unittest-arm-backend-with-no-deps (test_pytest_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 8ebd6151a4dc97ce5380234b4e1471eea4844f0bea35ca060414e03b00b84ccb /exec failed with exit code 1
• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t f1eedf9c1db684e75a00e72e6438d5a3c09a06d48d9027471ede7da89dc96fa0 /exec failed with exit code 3
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t b5fe3c8212f254f863aedfd7c8c6b49462b73cb1c7856345f26ff514008317c2 /exec failed with exit code 8
• pull / unittest-editable / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 8
• Test Metal Backend / test-metal-backend-modules / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 127

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-buck / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.