HLIR

SOL’s High Level Intermediate Representation (HLIR) is used to represent the computation graph. HLIR mainly consists of the classes sol::compiler::Layer, sol::compiler::LayerInput, sol::compiler::LayerOutput and sol::compiler::Operation. Each Layer requires at least one LayerOutput, no inputs and can have one Operation assigned.

Adding new Operations

To add new Operations to SOL, you need to inherit the class sol::compiler::Operation and implement the necessary function calls. You can’t use class variables but instead need to use editJson("key") = value to guarantee that the options get correctly stored within the SOL cache. Then this new Operation needs to be registered to SOL using:

class MyFancyOp final : sol::compiler::Operation {
	...
};

SOL_REGISTER_LAYERS(
	Operation::registerOperation<MyFancyOp>();
)

Further, you need a function that can be called from the parser to add your new Operations. The following shows a minimalistic example with one input and one output.

extern "C" {
	sol::compiler::LayerOutput* myNewCoolOp(LayerOutput* in, const int option) {
		auto op = new(_fl) MyFancyOp(in->editNetwork(), option);
		auto l  = new(_fl) Layer(op);
		auto li = l->addInput(IOType::Input, in);
		auto lo = l->addOutput(IOType::Output, li);
		return lo;
	}
}

In your Python code this then needs to be connected like the following example. Please look at the framework’s SDK description how the exact syntax for the framework parser hooks look like, as this differs between these:

lib = ctypes.cdll.load('.../libsol-backend-...so')

def parser_hook(in, option):
	return sol.hlir.Tensor(lib.myNewCoolLayer(sol.hlir.ptr(in), option))

HLIR Transformations

SOL runs different processing stages and applies different optimizations. Please ensure that your processing steps can be run multiple times.

sol::compiler::Optimizer Transformations

The optimizer transformations are general transformations to simplify and optimize the computation graph independent of the underlying device, i.e., to simplify computations like: A * 1 = A. These can be added using: sol::compiler::Optimizer::addTransformation(int, std::function<void(sol::compiler::Network*)>). The int is a priority, the lower the earlier it gets executed. Be advised you can’t use the same priority twice!

sol::compiler::Device Transformations

The device transformations are more specific. With sol::compiler::Device::addTransformation(int, std::function<void(sol::compiler::Device*)>) transformations can be added to ALL device types. By overriding sol::compiler::Device::transformPostAutotune() or sol::compiler::Device::transformPostOptimize() device type specific optimizations can be added. Of course you can check the sol::compiler::Device::deviceType() to apply transformations only to specific devices, but it’s better to implement these within the backends, see below.

sol::compiler::AutoTuner Transformations

When looking for optimal Backends for each layer, SOL executed the transformations that can be added with sol::compiler::AutoTuner::addTransformation(int, std::function<void(sol::compiler::Layer*)>). Further, the AutoTuner calls sol::compiler::Operation::autotune() once the Algo gets assigned to the Layer. This can be used to update/modify the hyper parameters of the Operation, i.e., if the data layouts have changed.

sol::compiler::Backend Transformations

To determine which Backend is optimal to use, SOL probes each Backend to get an sol::compiler::Algo object and an estimated execution time (either Heuristic, or measured). The best Algo will be used. The Algo object supports the editDims(IOType) API, which allows to modify the Dims of the LayerInputs and LayerOutputs. If modified, SOL will automatically add Reorder layers if necessary to automatically ensure correct data formats for the layer. This is, i.e., useful if the layer only supports NCHW data layouts, but the network provides NHWC as layer input.

When SOL has determined a Backend to be the optimal choice, it runs the sol::compiler::Backend::transformAlgo() function, allowing to transform the layer to the user’s need. I.e., to modify the hyperparameters, split the layers, or replace them with an equivalent but more optimal optimization. BE AWARE that if you create new Layers in this step you need add an Algo object to EACH new layer using sol::compiler::Layer::setAlgo().

Execution Order

The following is the execution order of all SOL transformations. Be aware that the Optimizer and Device transformations get executed multiple times, to ensure that whatever transformations Backends might apply, the generic optimizations get applied onto these.

  1. Network Parsing
    1. sol::compiler::Optimizer::transformations()
  2. Network Compilation
    1. AutoTuning Phase
      1. sol::compiler::AutoTuner::transformations()
      2. sol::compiler::Operation::autotune()
      3. sol::compiler::Optimizer::transformations()
      4. sol::compiler::Device::transformations()
      5. sol::compiler::Backend::transformAlgo()
      6. sol::compiler::Optimizer::transformations()
      7. sol::compiler::Device::transformations()
      8. sol::compiler::Device::transformPostAutotune()
    2. General Optimization Phase
      1. sol::compiler::Optimizer::transformations()
      2. sol::compiler::Device::transformations()
      3. sol::compiler::Device::transformPostOptimize()