Experiment design for machine learning algorithm in production

Question

I have a machine learning algorithm ready. I would like to put it into production in a country of 70 cities. But before rolling it out to 70 cities, I would like to do experimentation in 1 city to evaluate it's performance in production. However, I'm now facing a question that what criteria should I set in terms if: 1. Time ( how many months I can keep it in production ) 2. Data ( how much data I would need in live environment order to evaluate the algorithms performance)

Can anyone guide with this machine learning experimentation in production environment ?

Edit: I'm applying machine learning for price optimization in US.

Answer 1

It depends on what is your output and how is affected by the time. In which area are you using ML? It is hard to directly say something.