ONNX

import sol
import numpy as np

model  = sol.optimize("myModel.onnx") # no input description needed, as provided by model itself!
# or if you want to override the shape of the model
model  = sol.optimize("myModel.onnx", [np.rand(1, 3, 224, 224), ...], {'named_tensor': np.rand(3, 2, 1)})

input  = np.random.rand(1, 3, 224, 224)
output = model(input)

F.A.Q.

How can I execute an ONNX model on an accelerator device?

By default the ONNX frontend returns a Numpy executable model. You can either set sol.optimize("model.onnx", framework='pytorch') to a framework that supports accelerator devices or use the sol.device.set(‘device_type’, device_idx) API for transparent offloading.

How can I train an ONNX model?

The ONNX format does not store information about trainable parameters. However, you can set sol.optimize(“model.onnx”, framework=“pytorch”) to load the ONNX model into PyTorch. Then use iterate over model.parameters() and set the param.requires_grad = True for all parameters that shall be trained.

Tested Models

ONNX Hub v1.10.0

Format: ModelName (OpSet(s))

  • AlexNet (7, 8, 9, 12)
  • CaffeNet (7, 8, 9, 12)
  • DenseNet-121 (6, 7, 8, 9)
  • DenseNet-121-12 (12)
  • EfficientNet-Lite4 (11)
  • Emotion FERPlus (7, 8)
  • GoogleNet (12)
  • GoogleNet (3, 6, 7, 8, 9)
  • Inception-2 (7, 8, 9)
  • LResNet100E-IR (8)
  • MNIST (7,8)
  • MNIST-12 (12)
  • MobileNet v2-1.0 (10)
  • MobileNet v2-1.0-fp32 (12)
  • MobileNet v2-7 (7)
  • R-CNN ILSVRC13 (7, 8, 9)
  • ResNet101 (7)
  • ResNet101-v2 (7)
  • ResNet152 (7)
  • ResNet152-v2 (7)
  • ResNet18 (7)
  • ResNet18-v2 (7)
  • ResNet34 (7)
  • ResNet34-v2 (7)
  • ResNet50 (7)
  • ResNet50-caffe2 (7, 8, 9)
  • ResNet50-fp32 (12)
  • ResNet50-v2 (7)
  • ShuffleNet-v1 (7, 8, 9)
  • ShuffleNet-v2 (10)
  • ShuffleNet-v2-fp32 (12)
  • SqueezeNet 1.0 (6, 7, 8, 9, 12)
  • SqueezeNet 1.1 (7)
  • Super_Resolution (10)
  • Tiny YOLOv2 (7,8)
  • VGG 16 (7)
  • VGG 16-bn (7)
  • VGG 16-fp32 (12)
  • VGG 19 (7)
  • VGG 19-bn (7)
  • VGG 19-caffe2 (7, 8, 9)
  • YOLOv2 (9)
  • YOLOv4 (11)
  • ZFNet-512 (7, 8, 9, 12)
  • version-RFB-320 (9)
  • version-RFB-640 (9)

Supported Layers

Please refer to https://github.com/onnx/onnx/blob/master/docs/Operators.md for how these functions are used. This documentation only contains which layers, functions and tensor functionality are currently implemented within SOL.