datashader xarray.Image to holoviews Points

Question

This is the code:

import datashader as ds
import pandas as pd
from colorcet import fire
from datashader import transfer_functions as tf
from datashader.utils import lnglat_to_meters
import holoviews as hv
import geoviews as gv
from holoviews.operation.datashader import datashade, spread, aggregate
hv.extension('bokeh')  

df = pd.read_csv('...')

agg = ds.Canvas().points(df, 'x', 'y', agg=ds.count())
img = tf.shade(agg.where(agg['x']>0), cmap=fire)

url = 'https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg'
tile_opts = dict(width=1000,height=600,xaxis=None,yaxis=None,show_grid=False,bgcolor='black')
map_tiles = gv.WMTS(url).opts(style=dict(alpha=1.0), plot=tile_opts)
points = hv.Points(df, ['x', 'y'])
#points = img    # <-- Using this does not work
ds_points = spread(datashade(points, width=1000, height=600, cmap=fire), px=2)

map_tiles * ds_points

The above code creates a holoviews Points object based on data from a pandas dataframe and uses spread() and datashade() functions in holoviews to plot the points on a map. However, I want to do some transformations on the data before I plot it on the map. I tried to use the functionality already available in datashader, but I'm unable to figure out how I can convert the xarray.Image object created by datashader into a holoviews Point object which can be plotted on top of the map tiles.

EDIT

I'm not able to format code properly in comments, so I'll just put it here.

I tried doing the following as a degenerate case:

from custom_operation import CustomOperation
points = hv.Points(df, ['x', 'y'])
CustomOperation(rasterize(points))

where CustomOperation is defined as:

from holoviews.operation import Operation

class CustomOperation(Operation):
    def _process(self, element, key=None):
        return element

This produces the following error:

AttributeError: 'Image' object has no attribute 'get'

James A. Bednar James A. Bednar · Accepted Answer · 2019-10-04T19:17:14

The Image object created by Datashader is a regular grid/array of values that aggregates the original points by bin, so it is no longer possible to recover the original points. It would not be meaningful to use a HoloViews Points object on this already 2D-histogrammed data; a Points object expects a set of individual points, not a 2D array. Instead, you can use a HoloViews Image object, which accepts a 2D array like that generated by Datashader. The syntax would be something like hv.Image(img), though I can't test it with the above code because it's not runnable without the CSV file.

Note that if you take this approach, what will happen is that Datashader will render the points into a fixed-size grid, and HoloViews will then overlay that specific grid of values onto the map. Even if you zoom in or pan, you'll still see that same grid; it will never update to show a subset of the data at a higher resolution as your current code will, because the Datashader computations will have all completed and given you a fixed array before you start plotting anything with HoloViews or Bokeh. If you want dynamic zoom and updating, don't use the Datashader API (Canvas, .points, tf.shade, etc.) separately; you'll need to use either the HoloViews operations you are already using (datashade,spread, rasterize, etc.) or define a custom HoloViews operation to encapsulate the processing you want to do (which can include manually calling the Datashader API if you need to) and allow the processing to be dynamically applied each time the user pans or zooms.

datashader xarray.Image to holoviews Points

1 Answers