Welcome.

I'm David Lacroix,
Geo data engineer @MeilleursAgents

the constraints

volume
coverage
flexibility
simplicity

How is it being done ?

WMTS + IGN js api @Geoportail

Geojson + Mapboxgl.js @Etalab

Vector tile + Mapboxgl.js @Koumoul

the realm of vector tiles

Source Gaffuri (2012)

light, fast and adaptable

Source Mapzen

a standard architecture

a bright future ahead

OGC - Vector Tiles Pilot

our stack

Postgres

↦

Flask

↦

MapboxGL

your only friend

find more information in that great tutorial

a calf on steroid

2.4.0: introduction of ST_AsMVT()
2.5.0: parallel implementation
3.0.0: performance boost + ST_TileEnvelope()

one less middleware

your web application is your tile server

#madewithmapbox

mapbox vector tile specification

try it out

$ sudo docker-compose up -d

https://github.com/DavidLacroix/postgis-mvt

a map speaks louder than words

http://localhost:8001/map_empty

geojson is trivial


                            map.on('load', function () {
                              map.addLayer({
                                "id": "haussmann",
                                "type": "fill",
                                "source": {
                                  "type": "geojson",
                                  // path to geojson (text, file, url...)
                                  "data": "static/resource/haussmann.geojson"
                                },
                                "paint": {
                                  "fill-color": "#088",
                                  "fill-opacity": 0.8
                                }
                              });
                            });

tiles are trickier


                                http://localhost:8001/my-layer/{z}/{x}/{y}
                                # z: zoom level (0-22)
                                # x: tilegrid X coordinate
                                # y: tilegrid Y coordinate

flask ↦ serving tiles


                        @app.route("/map_geojson")
                        def map_geojson():
                            return render_template('map_geojson.html', token=MAPBOX_TOKEN)

                        @app.route('/<string:layer>/<int:z>/<int:x>/<int:y>', methods=['GET'])
                        def generic_mvt(layer, z, x, y):
                            srid = int(request.args.get('srid', 4326))
                            tile = _load_tile(layer, x, y, z, srid=srid)
                            response = make_response(tile)
                            response.headers.add('Content-Type', 'application/octet-stream')
                            response.headers.add('Access-Control-Allow-Origin', '*')
                            return response

                        def _load_tile(layer_name, x, y, z, srid=4326):
                            tile = None

flask ↦ loading tile


                        def _load_tile(layer_name, x, y, z, srid=4326):
                            tile = None

                            xmin, ymin, xmax, ymax = _tms2bbox(x, y, z)

                            query = '''
                                ...
                            '''.format(
                                schema=TILE_SCHEMA,
                                table_name=layer_name,
                            )
                            query_parameters = {
                                'layer_name': layer_name,
                                'xmin': xmin, 'ymin': ymin, 'xmax': xmax, 'ymax': ymax,
                                'srid_bbox': srid
                            }

                            with psycopg2.connect(**DB_PARAMETERS) as connection:
                                with connection.cursor() as cursor:
                                    cursor.execute(query, query_parameters)
                                    res = cursor.fetchone()
                                    if 'mvt' in res and res['mvt'] is not None:
                                        tile = bytes(res['mvt'])

                            return tile

from tile to ground coordinates

postgis ↦ creating tile


                      SELECT ST_AsMVT(tile, %(layer_name)s, 4096, 'mvt_geom') AS mvt
                      FROM (
                        SELECT id,
                          value,
                          extrude,
                          ST_AsMVTGeom(
                            -- Geometry field
                            ST_Transform(t.geom, 3857),
                            -- MVT tile boundary
                            ST_Makebox2d(
                              -- Lower left coordinate
                              ST_Transform(ST_SetSrid(ST_MakePoint(%(xmin)s, %(ymin)s), 4326), 3857),
                              -- Upper right coordinate
                              ST_Transform(ST_SetSrid(ST_MakePoint(%(xmax)s, %(ymax)s), 4326), 3857)
                            ),
                            4096 /*Extent*/, 256 /*Buffer*/, TRUE /*Clip geom*/) AS mvt_geom
                        FROM {schema}.{table_name} t
                        WHERE
                          ST_Makebox2d(
                            ST_Transform(ST_SetSrid(ST_MakePoint(%(xmin)s, %(ymin)s), 4326), %(srid_bbox)s),
                            ST_Transform(ST_SetSrid(ST_MakePoint(%(xmax)s, %(ymax)s), 4326), %(srid_bbox)s)
                          ) && t.geom
                      ) AS tile

pick your layer

$ psql -h localhost -p 5555 -d postgres -U postgres -c "\d+ mvt.*"


                               View "mvt.apur_building"
 Column  |            Type             | Collation | Nullable | Default | Storage | Description 
---------+-----------------------------+-----------+----------+---------+---------+-------------
 id      | integer                     |           |          |         | plain   | 
 value   | integer                     |           |          |         | plain   | 
 extrude | integer                     |           |          |         | plain   | 
 geom    | geometry(MultiPolygon,4326) |           |          |         | main    | 

View definition:
 SELECT apur_building_raw.ogc_fid AS id,
     (regexp_match(apur_building_raw.c_perconst, '[0-9]{4}'))[1]::integer AS value,
     apur_building_raw.h_med::integer AS extrude,
     apur_building_raw.wkb_geometry AS geom
 FROM apur_building_raw;

best served hot

http://localhost:8001/map_apur


                          map.addLayer({
                            "id": "apur_building",
                            "type": "fill",
                            "source": {
                              "type": "vector",
                              "tiles": ["http://localhost:8001/apur_building/{z}/{x}/{y}"],
                              "minzoom": 11,
                              "maxzoom": 21
                            },
                            "source-layer": "apur_building",
                            "paint": {
                              "fill-color": "#ffab40",
                              "fill-opacity": 0.7
                            },
                          });

data driven styles

http://localhost:8001/map_apur?show_color=true


                          {"paint": {
                            "fill-color": [
                              "interpolate",
                              ["linear"],
                              ["number", ["get", "value"], 0],
                              0,
                              "#aaa",
                              1800,
                              "#dd2c00",
                              2010,
                              "#ffd600",
                            ],
                            "fill-opacity": .7
                          }}

reusing the data-source


                          // Separating data source to render the same data with different style 
                          data_source = {
                            "type": "vector",
                            "tiles": ["http://localhost:8001/apur_building/{z}/{x}/{y}?srid=4326"],
                            "minzoom": 12,
                            "maxzoom": 21
                          }
                          map.addSource('apur_building', data_source);

                          apur_building_line = {
                            "id": "apur_building_line",
                            "type": "line",
                            "source": 'apur_building',
                            "paint": {

labelling entities


                          map.addLayer({
                            "id": "apur_building_label",
                            "type": "symbol",
                            "source": "apur_building",
                            "source-layer": "apur_building",
                            "minzoom": 18,
                            "layout": {
                              "text-field": ["get", "value"],
                              "text-size": 14,
                              "text-font": ["Open Sans Semibold", "Arial Unicode MS Bold"],
                            },
                          });

handling interactions


                          map.on("mousemove", 'apur_building', function(e){
                            map.getCanvas().style.cursor = 'pointer';
                          
                            if (e.features.length > 0) {
                              idDisplay.textContent = e.features[0].properties.id;
                              valueDisplay.textContent = e.features[0].properties.value;
                              extrusionDisplay.textContent = e.features[0].properties.extrude;
                          
                              hoveredStateId = e.features[0].id;
                              // set the hover attribute to true
                              map.setFeatureState(
                                {source: 'apur_building', id: hoveredStateId, 'sourceLayer': 'apur_building'},
                                {hover: true}
                              );
                            }
                          });

displaying interactions


                          {"paint": {
                            "fill-outline-color": "#000099",
                            "fill-color": "#000099",
                            "fill-opacity": [
                            "case", ["boolean", ["feature-state", "hover"], false],
                            1,
                            0
                            ]
                          }}

Error: The feature id parameter must be provided and non-negative.

the FeatureID issue

refer to mapbox hover tutorial

fixed in postgis 3

https://trac.osgeo.org/postgis/ticket/4128

rebuild mapbox-gl-js

https://github.com/mapbox/vector-tile-js/blob/master/lib/vectortilefeature.js#L19


                      if ('id' in this.properties) {
                        this.id = this.properties.id;
                      }

Debugging tile

$ npm install -g @mapbox/vt2geojson

$ vt2geojson http://localhost:8001/apur_building/{z}/{x}/{y}

https://github.com/mapbox/vt2geojson

Scaling 🚀

caching tiles: NGINX, CDN
postgres master-slave replication
asynchrone tileserver: postile, martin

leaving the demo

$ sudo docker build -t data-master data-processing/

$ sudo docker run \
    --link=postgres-master \
    --network=global-network \
    --env-file .env \
    -it data-master