Import large CSV file into TileMill for use in MapBox

Question

I am beginning to experiment with using MapBox and TileMill, and what I would like to do is map 400,000 addresses in a CSV file which have been pre-geocoded. When I try to add this 100mb CSV file as a layer into MapBox, I receive an error telling me that the CSV file is greater than 20mb and apparently this is a problem.

Can someone point me in the right direction in terms of what is the best way to get these 400k records into TileMill? Eventually, I want to publish the map to the web, and I was planning to do that using MapBox. I saw a program for converting CSV to a shapefile, and wondering whether this is the best approach.

Answer 1

Hundreds of thousands of markers is a lot. In the free tier of Mapbox, there is a limit of two thousand features. Such a limit would not stop you from displaying those in Tilemill, but it would stop you from uploading them to mapbox.com.

For discussion of that limit, see here.

A simple strategy for reducing the markers is to restrict to the subset of features that lies within a smaller bounding box.

I don't think it will matter whether your features are expressed in geojson, shapefiles, csv, or other formats. The number of features is what's stopping you.

Answer 2

I have the same problem. I had to import a 22MB csv file into tilemill and got the same error.

Although I don't have a working answer for you, but I would think either:

1. Convert csv to SQLite export files http://www.mapbox.com/tilemill/docs/tutorials/sqlite-work/
2. Configure the buffer for tilemill (however I doubt this would be the best because my tilemill cant take 5 GB of memory when doing points/markers rendering, increasing the buffer would make things worse)

I will keep experimenting with the ideas, and update this thread as soon as I found something. Also, I am looking forward to the tillmill pros out here for the best working answer~!

Best