»Geodata Processing and Webservices with Python and Azure«
2019-06-25, 11:45–12:00, Room 2
We demonstrate how to leverage Python’s two biggest strengths - data analysis and web development - together with a versatile cloud platform to deliver GIS functionalities like raster-based isochrones.
According to a recent survey among developers, Python is nowadays popular both for data analysis and web developers. As part of a geo community, data analysis heavily includes spatial data. At the same time, with cloud platforms like AWS, Azure or smaller ones like Digital Ocean, it nowadays is easier than ever to get the result of your data analysis coding to your customers and users.
In this talk, we want to demonstrate how we used the Python eco-system and the cloud platform Azure to bring raster-based isochrones to your favorite GIS tool.
Background: what are raster-based isochrones?
Isochrones are a set of isolines on a map showing points with the same travelling time with respect to a starting point. Isochrones are usually calculated with graph-based algorithms and software like pgRouting. Our approach takes a different path (no pun intended): Our pedestrian isochrones show the precise walking time of a neighborhood for any starting point. Rather than following a network of streets and paths, they are an aggregate of thousands of individual paths, bundled into one result. As opposed to other isochrone analyses, our approach takes into account desire paths and potential shortcuts across open spaces such as large squares.
<img alt="animated isochrone" src="https://i2.wp.com/geo.ebp.ch/wp-content/uploads/2016/10/Bern-Beispiel-1-4.gif?zoom=4.5&resize=375%2C332">
This is done by modeling the pedestrian movements in the same way as the flow of water in hydrology when calculating catchment areas. In order to calculate the pedestrian movements via isochrones, we need to model the earth’s surface according to the movement of a pedestrian, where each raster cell represents the “walkalbility” of its area.
Calculation in Python
In order to create the isochrones, we use several Python components: For the walkability or cost raster, we process the vector data from sources like OSM or any other topographic data source, if they are available. We mainly use FME, which has a nice integration of Python. For example, the rules on how to decide the walkability based on the topographic data is done in a so-called PythonCaller transformer. The result is a cost raster, which we cut into tiles of smaller rasters, similar to map tiles for web maps.
We use modules such as numpy and scipy to calculate the isochrones. Since most of the heavy computation is within numpy arrays, the implementation is rather fast. For example, it takes less than a second on an average laptop to compute an isochrone of size 1000x1000 pixel.
Last, but not least, we couldn’t live without indispensable spatial Python modules such as pyproj and gdal. For creating animated isochrones, animation, we use moviepy, which has a nice integration with numpy.
Web-Development in the Cloud
So, now that we have covered the data processing part, we’d like to expose our fancy calculation to the world. First, we wrap our code for isochrone within a [framework that run on WSGI, such as flask, bottle or falcon. The result is a simple REST API call, which can be consumed by any language and platform.
Both for backend processing as well as the web API, we actually don’t need to buy our own hardware and infrastructure, we can leverage the advantages of cloud platforms such as AWS or Azure to do the heavy lifting. For our purpose we used the following offers from Microsoft Azure:
- Azure storage for storing cost raster tiles.
- Azure VM for transforming and processing the geo data to cost raster
- Azure Web-App, or even Azure functions for running the API code.
- Azure API Management for an API gateway.
The final result is a developer platform, for wich integration exists on Python, R and ArcGIS.