-
Notifications
You must be signed in to change notification settings - Fork 598
onnx-coreml backend #2319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
onnx-coreml backend #2319
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for the CoreML execution provider to the ONNX backend, enabling hardware acceleration on Apple Silicon devices. The changes register a new "onnx-coreml" backend option and configure it to use the MLProgram model format with compute plan profiling.
- Adds COREML as a new OnnxProvider enum value
- Implements CoreML provider configuration with MLProgram format and profiling enabled
- Registers the "onnx-coreml" backend with priority 59
- Updates CI pipeline to build with ONNX runtime and test the CoreML backend on macOS ARM64
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/neural/backends/network_onnx.cc | Adds CoreML provider enum, configuration logic, and backend registration |
| .circleci/config.yml | Adds ONNX runtime installation, build configuration, and CoreML backend testing on macOS ARM64 |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
091e474 to
0f71b12
Compare
|
Some preliminary tests using fp32: fp16 fp16 with PR26442 applied: |
Currently the onnxruntime coreml provider doesn't support everything required, the following three patches are needed for both fp32 and fp16 with fixed batch size (default for now).
microsoft/onnxruntime#26443(merged)microsoft/onnxruntime#26442
microsoft/onnxruntime#26462(merged)For variable batch size, hopefully the fix for issue microsoft/onnxruntime#26328 is simple.
If someone wants to try it out, the default onnxruntime branch should work. The last outstanding patch is for
Gatherfp16 support, which is the last kernel before the policy output, so doing it on the cpu shouldn't cause a huge performance drop.