How to concat/merge multiple zipped shapefiles using geopandas and python?

Question

I have a ~60 zipped shapefiles from the US census for different states. I want to combine them all into one nationwide shapefile. I've tried so many different approaches from trying to download the file with read_file and a variety of other Python/pandas/geopandas examples that use csv files or .shp files themselves. I'd like to avoid unzipping the shapefile zips if possible. My understanding is that geopandas.read_file can work with zipped shapefiles just fine (e.g. https://www2.census.gov/geo/tiger/TIGER2010/TABBLOCK/2010/tl_2010_01_tabblock10.zip)

But I now have these files locally.

Here's the code I'm trying in my notebook:

from pathlib import Path
import pandas
import geopandas

folder = Path("/Users/kyle/Desktop/jupyter-env/blocks")
shapefiles = folder.glob("*.zip")
gdf = pandas.concat([
   geopandas.read_file(shp)
   for shp in shapefiles
]).pipe(geopandas.GeoDataFrame)
gdf.to_file(folder / 'compiled.shp')

I get the message ValueError: No objects to concatenate.

I must be missing something here. Do zip files work differently than csv files or something like that? Is this sort of thing possible: looping over a list of local or remote files and merging the zipped shapefiles into one big one?

jakub jakub · Accepted Answer · 2020-08-02T23:45:57

The GeoPandas documentation prefixes all of the zipfile paths with zip://.

from pathlib import Path
import pandas
import geopandas

folder = Path("/Users/kyle/Desktop/jupyter-env/blocks")
shapefiles = folder.glob("*.zip")
gdf = pandas.concat([
   geopandas.read_file("zip://" + str(shp))
   for shp in shapefiles
]).pipe(geopandas.GeoDataFrame)
gdf.to_file(folder / 'compiled.shp')

How to concat/merge multiple zipped shapefiles using geopandas and python?

1 Answers