Patterns for Large Scale Temporal Processing of Geospatial Data Using Dask | Kirill Kouzoubov

Speaker: Kirill Kouzoubov

We describe an approach for efficiently embedding non-Dask algorithms into Dask processing pipeline. By constructing large contiguous memory array incrementally from a Dask graph we were able to achieve significant peak memory reductions. We have used this approach to generate cloud-free Sentinel-2 Geometric Median and Median Absolute Deviations mosaics over Africa (10m res).