Very interesting. You can achieve the same speed up improvement on cpus using the onnxruntime library. It also allows to use multiple threads and it also optimizes the graph for the given hardware which is a nice to have.