Map Match#
Processor MapMatch requires an additional, seperate license for street map premium data. Please contact the BDT team at bdt_support@esri.com if you are interested.
Table of Contents#
[ ]:
import bdt
bdt.auth("abfss://integration@bdtcicd.dfs.core.windows.net/.bdt/bdt.lic")
from bdt import processors as P
BDT has been successfully authorized!
Welcome to
___ _ ___ __ ______ __ __ _ __
/ _ ) (_) ___ _ / _ \ ___ _ / /_ ___ _ /_ __/ ___ ___ / / / /__ (_) / /_
/ _ | / / / _ `/ / // // _ `// __// _ `/ / / / _ \/ _ \ / / / '_/ / / / __/
/____/ /_/ \_, / /____/ \_,_/ \__/ \_,_/ /_/ \___/\___//_/ /_/\_\ /_/ \__/
/___/
BDT python version: v3.3.0-v3.3.0
BDT jar version: v3.3.0-v3.3.0
Part 1: What is Map Matching#
Map Matching is the process by which noisy gps points are snapped to their most likely road vector.
Map Matching intelligently chooses snap locations, factoring in much more than the distance to a road. Some additional factors are: 1. Context of the GPS Point (It’s position in the path) 2. Heading of the GPS Point
Part 2: Map Match Input Data#
The input data for Map Matching is noisy GPS points. The image of noisy points below is from the example dataset used for this notebook, which is called MapMatchExample. The data is in parquet format.
Map Match input data must have these five columns: 1. Unique Identifier - ID for each GPS Point 2. Ordinal Identifier - An ordinal value that indicates the ordering of the GPS points. For example, a timestamp 3. Path Identifier - Path ID for each GPS point 4. X - X value of each point. IMPORTANT: Must be in SRID 3857 Web Mercator 5. Y - Y value of each point. IMPORTANT: Must be in SRID 3857 Web Mercator
In the below cells, the parquet file with Lon & Lat pairs is loaded and preprocessed for Map Matching. Lon and Lat need to be converted to X and Y 3857 Web Mercator coordinates. The required columns are:
Unique Identifier - oid
Ordinal Identifier - dt
Path Identifier - vid
X - X
Y - Y
[ ]:
noisyPoints = spark \
.read \
.parquet("abfss://integration@bdtcicd.dfs.core.windows.net/sampleNotebooks/data/MapMatchExample/MapMatchExample/MapMatchExample.parquet")
noisyPoints.printSchema()
root
|-- OBJECTID: integer (nullable = true)
|-- EPOCH: integer (nullable = true)
|-- VID: string (nullable = true)
|-- lon: double (nullable = true)
|-- lat: double (nullable = true)
[ ]:
npProcessed = noisyPoints \
.selectExpr("OBJECTID as oid", "VID as vid", "EPOCH as dt", "LonToX(lon) as X", "LatToY(lat) as Y") \
.cache()
npProcessed.show()
+---+-----+----------+------------------+------------------+
|oid| vid| dt| X| Y|
+---+-----+----------+------------------+------------------+
| 1|V0001|1612921948|-9243240.292453075| 4867153.238361778|
| 2|V0001|1612921949|-9243267.110208966| 4867194.103599254|
| 3|V0001|1612921950|-9243304.144199805| 4867196.657582264|
| 4|V0001|1612921951| -9243348.84042251| 4867164.731751849|
| 5|V0001|1612921952|-9243382.043353071| 4867176.225009891|
| 6|V0001|1612921953|-9243392.259699337| 4867097.048807928|
| 7|V0001|1612921954|-9243347.563476631| 4867028.088847031|
| 8|V0001|1612921955|-9243347.563476631| 4866950.189697147|
| 9|V0001|1612921956|-9243310.529374475| 4866897.831207512|
| 10|V0001|1612921957|-9243231.353164012|4866882.5067635365|
| 11|V0001|1612921958|-9243154.731067942| 4866868.459365237|
| 12|V0001|1612921959|-9243098.541441768| 4866863.351278181|
| 13|V0001|1612921960|-9243033.412637843| 4866862.074256826|
| 14|V0001|1612921961|-9242960.621602051| 4866853.13496662|
| 15|V0001|1612921962|-9242929.972785888|4866865.9053213755|
| 16|V0001|1612921963|-9242857.181750096| 4866846.749868081|
| 17|V0001|1612921964| -9242862.28986757|4866899.1082334705|
| 18|V0001|1612921965|-9242878.891388506| 4866948.912664615|
| 19|V0001|1612921966|-9242890.384680657|4867006.3792912355|
| 20|V0001|1612921967|-9242910.817261871| 4867066.399909539|
+---+-----+----------+------------------+------------------+
only showing top 20 rows
Part 3: Map Match Processor#
To run the Map Match processor on the data, specify the required column names and a few additional parameters. These parameters are:
snapDist: The maximum snap distance from a noisy point to a road vector.
microPathSize: The size of the micropaths. Points are divided into ‘micropaths’ of this size.
pathDist: The maximum distance on the road network between two snap points.
alpha: Penalty weight for the match of the heading of the gps point and the heading of the snapped road.
beta: Penalty weight for the perpendicularity of the vector from the gps point to its snapped location on the road and the snapped path vector.
IMPORTANT: Map Matching requires the LMDB database to get the street network data. Reference the LMDB section of the docs for more information on running BDT3 with LMDB.
In the cell below, Map Matching is run on the noisyPoints data with the parameters specified as:
snapDist: 50 meters
microPathSize: 7 points
pathDist: 1000 meters
alpha: 10 (The default value. This is recommended as a starting point.)
beta: 1 (The default value. This is recommended as a starting point.)
[ ]:
outDF = P.mapMatch(npProcessed,
idField="oid",
timeField="dt",
pidField="vid",
xField="X",
yField="Y",
snapDist=50,
pathDist=1000,
microPathSize=7,
alpha=10,
beta=1,
cache=False)
outDF.show()
+----------+------------------+------------------+---+
| linkID| linkX| linkY|oid|
+----------+------------------+------------------+---+
| 762013802|-9243248.401894853| 4867150.273809605| 1|
| 762013802|-9243264.741249682| 4867194.969614891| 2|
| 28522038| -9243303.71955067| 4867195.604766801| 3|
| 28522038|-9243353.237028405| 4867175.632079383| 4|
| 28522038|-9243377.806959594| 4867165.721890539| 5|
| 28522157|-9243386.409073453| 4867099.140091309| 6|
| 28522157|-9243359.488456225| 4867023.826309632| 7|
| 948819477|-9243339.045947738| 4866952.799356237| 8|
| 948819489|-9243310.654122414| 4866896.748270372| 9|
| 948819488| -9243230.82541711| 4866886.41440023| 10|
|1151085819|-9243154.092614582|4866877.5065606795| 11|
|1151085819|-9243097.758933812| 4866873.340938837| 12|
| 28558143|-9243032.956288306| 4866867.841855132| 13|
|1190127800|-9242960.621602051| 4866863.633752934| 14|
|1190127800|-9242930.098716237| 4866862.866853412| 15|
|1190127800|-9242878.471287979| 4866860.727136911| 16|
| 125259681|-9242883.246483844| 4866896.135612449| 17|
|1193317833|-9242891.587308537| 4866946.597267412| 18|
|1193317834|-9242904.707156738| 4867002.722469071| 19|
|1193317834|-9242921.647450076|4867063.0368814645| 20|
+----------+------------------+------------------+---+
only showing top 20 rows
