I have developed a Machine learning model (Random Forest classification model) in Azure Machine Learning Studio & deployed the same to Azure Container Instance (ACI) as a web service.
I then test the performance of the Webservice from within a notebook running in Azure ML studio with a 4 core, 8 GB ram back-end Azure compute. Using the %%timeit magic command in Jupyter notebook, I get an average speed of around 1.2 seconds/loop (for running inference on 1000 data points).
However, when I test the same webservice outside of Azure, in my local machine, the performance drops to ~5 seconds/loop (more than 4 times slower)
I am fairly new to web service deployment, so I am really not sure how to go about troubleshooting this (and could not find any helpful info on googling either). Do let me know if there are any specific configuration or environment details required to answer this (I have mostly just followed the azure documentation for deployment,e.g. the tutorial here)
