Bypass Google Geocoding API IP Rate Limits

6
votes

We are a division of a large corporation and are running into a MAJOR problem with the Google maps and geocoding API. The problem is that all 70 divisions of our corporation are behind the same IP address. So a customer service agent in India would appear to be coming from the same IP address as a developer in Cleveland, OH who would appear to be coming from the same IP address as a division president in western Europe. With over 130,000 employees, we routinely get blocked for exceeding the IP rate limit.

Aside from individuals just browsing to websites that happen to be using Google maps with client-side geocoding requests, any division that attempts to do batch geocoding or provides their own application for showing maps will all contribute to the same limit! We actually use our own Where To Buy service internally, which is publicly available, (http://www.ridgid.com/Tools/Where-To-Buy/), to give customers information about local distributors.

While we are not running into an API limit (at least not yet, and when we do we can always buy the enterprise license), we are running into the IP rate limit and currently have no workaround. We are already following best practices in terms of caching up geocoding results and reducing wasteful calls. Thankfully this isn't a problem for our customers as they are not on the same IP as users within our corporation.

The question is whether there is ANY way we can get an exception from Google to our specific IP address to improve the IP-based limitations given our setup? This is really a question for Google. Thanks for your help!

6
google-mapsgoogle-maps-api-3google-geocoder
I would suggest contacting Google's Map for Business team directly: support.google.com/enterprisehelp/bin/… - andresf
I'll give it a shot. Although the google forums say to post questions here now. - jakejgordon
Yes, but in terms of licensing, you should contact the Google Maps for Business team. Your question is not really technical in nature--you're asking a business question. - andresf
I just found something interesting in the Google API Console. You can set a parameter for maximum requests per user per second to your API. The default is 1 request per second per user (aka IP address). While I don't think this is the issue we were running into, I think that updating this to be a higher number (e.g. 100 instead of 1) might help reduce the likelihood of running into problems from internal usage of our public site. - jakejgordon
Hypothetically we could get blocked just from individuals browsing to sites that are using google maps--regardless of whether we are using our own service or writing batch geocoding processes. I'm happy to retire this question from stackoverflow if we believe other corporations wouldn't benefit from hearing the answer to this problem. If the answer is "buy an enterprise license and Google will manually remove the rate limit for our IP" I still think this has benefit for others. Anyone else have thoughts? - jakejgordon

6 Answers

10
votes

A division of a large corporation with tens of thousands of employees should have an enterprise licence.

2
votes

I don't think there's a limit associated with IP if you are using the client-side geocoding as described here:

https://developers.google.com/maps/articles/geocodestrat

Server-side geocoding through the Geocoding Web Service has a quota of 2,500 requests per IP per day, so all requests in one day count against the quota. In addition, the Web Service is rate-limited, so that requests that come in too quickly result in blocking.

Client-side geocoding through the browser is rate limited per map session, so the geocoding > is distributed across all your users and scales with your userbase.

A "map session" means every time the Javascript Map API is loaded. However there might be a limit on that as well, whether or not it's associated with IP / domain / sub-domain / URL still needs further clarification.

And the "rate limited" on client side doesn't mean there is a daily quota or something, I had tested this method, on a single day, it can geocode more than 2500 addresses. You will have to think and test how the "rate limit" works, my observation is it let you burst to around 10 addresses in no time, then limit you to a rate of around 1 geocode per second for the next 150 request, further request from the same "session" would take exponentially longer time.

0
votes

One possiblity is to use a batch of reverse geocode instead of a single request. Google allows up to 24 waypoints in a single direction request. Instead of sending a single reverse geocode you can wait until you have 24 reverse geocode accumulated in a batch. Or you can use Yahoo map like in this answer: Issue in displaying static Google maps.

0
votes

I wondering if there is a lot of overlap in the requests, like everyone keeps looking up that same 50 locations? You write a caching proxy that makes requests to a internal database first then requests from Google when the results arn't found. This could be done at the software level changing the GeoCode call to a InternalGeoCode or maybe at the network level and catch all the requests to the Google Geocode servers and redirect them? I'm not 100% how hard this would be to implement or if it would help maybe the requests are all unique.

0
votes

If you are doing forward geocoding only, I suggest trying out GeocodeFarm. I just got an email from them the other day saying they would start releasing "reverse" geocoding soon too and that they are still working out the kinks. Maybe it would help you since they have no request rate limits in place and the only limit is your daily 2,500 limit, same as google... IDK. Just a suggestion..

http://www.geocodefarm.com == the link

0
votes

The limit in standard license is in 50 to 100 QPS (Queries per second) limit depending upon the service (i.e. Elevation service has a 50 QPS limit, Geocoding has a 100 QPS limit). Despite you may have a higher quota you cannot perform more than 50 requests per second otherwise the server will block your IP for about 2 hours (based on my experience). The solution is to cache the requests and perform them on a scheduled basis and not upon occurrence.