It is possible to make a fast model with keras with tensor flow as backend and use it to predict in C or C++?
I need to do the prediction inside of a C++ program, but I feel much more comfortable doing the model and the training in keras.
2
votes
3 Answers
3
votes
In case you don't need to utilize a GPU in the environment you are deploying to, you could also use my library, called frugally-deep. It is available on GitHub and published under the MIT License: https://github.com/Dobiasd/frugally-deep
frugally-deep allows running forward passes on already-trained Keras models directly in C++ without the need to link against TensorFlow or any other backend.
It not only supports prediction with sequential models but also with more complex models build with the functional API.
Additionally to supporting many common layer types it can keep up with (and sometimes even beat) the performance of TensorFlow on a single CPU. You can find up-to-date benchmark results for some common model in the repo.
By automatic testing frugally-deep guarantees that the output of a model used with it in C++ is exactly the same as if run with Keras in Python.
0
votes
Yes, it is possible. TensorFlow provides a stable C API as well as a C++ one.
For more details, you probably want to ask a more specific question.
