Introduction

The OpenStreetMap project consists of raw map data, collected and aggregated by thousands of users. However, its open access policy sparked a number of collateral projects, which collectively cover many of the features typically offered by commercial mapping services.

The most obvious advantage in using OpenStreetMap-based software over a commercial solution is economical convenience, because OpenStreetMap comes as free (both as in beer and as in speech) software. The downside is that it takes a little configuration in order to setup a working web service.

This tutorial covers the configuration and maintenance of a web service which can answer questions such as:

• What is the closest street to a given pair of coordinates?
• What's the best way to get from point A to point B?
• How long does it take to get from point A to point B with a car, or by foot?

The software that makes this possible is an open-source project called Open Source Routing Machine (OSRM), which is based on the OpenStreetMap data. Functionalities to embed OpenStreetMaps in Web pages are already provided out-of-the-box by APIs such as OpenLayers.

Prerequisites

To follow this tutorial, you will need:

• A Ubuntu 14.04 Droplet.
• A sudo non-root user, which you can create by following this tutorial. This tutorial assumes your sudo non-root username is osrm.
• Recommended 4 Gigabytes of swap, as covered in How to add swap on Ubuntu 14.04.

Some of the operations in this tutorial, such as building OSRM from source and the pre-processing phases, are memory intensive. On low-memory Droplets, these operations may fail, which is why it is necessary to allocate a swap file.

The web service does not normally require additional swap while running, so this swapfile can be removed after the setup is complete. However, when serving large maps, a small swap file might actually be necessary, especially in low-memory Droplets. If so, follow the instructions in the above tutorial to make the swap file persistent.

Step 1 — Updates and Security

In this step, we will configure updates for the server.

It is important to keep our server up to date and safe over time. Therefore, we first upgrade all the packages in the system by running the following commands.

sudo apt-get update
sudo apt-get upgrade

Then, we turn on Ubuntu's unattended security upgrades. To do this, we first need to install the necessary packages.

sudo apt-get install unattended-upgrades

Next, turn on the automatic upgrades by editing the file /etc/apt/apt.conf.d/10periodic. We need to use sudo because this is a system-wide configuration file.

sudo nano /etc/apt/apt.conf.d/10periodic

Add the following line at the end of the file, then save and close it.

APT::Periodic::Unattended-Upgrade "1";

Step 2 — Download a Map Export

In this step, we will choose and download a map export.

To keep everything tidy and clean, it is a good idea to create a dedicated directory in which to build the code, process the map, etc.

mkdir osrm

Move into the new directory.

cd osrm

Our web service will be based on a map export from OpenStreetMap. There are several possible ways to obtain a map export.

The first option is to point your browser to the OpenStreetMap's export page, zoom on the interested area, and click on Export. (You may need to use one of the other sources if the native export does not work; Overpass API is a good choice.) This will allow you to finely choose which areas to include in your map and, in general, to cut down on the pre-processing times.

However, there are limits on the size of the export you can get from the OSRM website, so you might want to download a ready-made export. A number of services, such as Geofabrik and Mapzen offer updated and ready-made map exports of countries and selected metropolitan areas, which are good for most use cases.

Whichever method you use, copy the URL of the export and download it onto your Droplet.

wget -O map.osm url_of_the_export

You should now have a file named map.osm in your working directory.

Step 3 — Install Dependencies

In this step, we will install OSRM's dependencies.

OSRM must be compiled from source, so we first need to install the necessary build machinery. Fortunately, Ubuntu comes with a convenient meta-package called build-essential which contains the needed compiler toolchain. In addition, we will need git to grab the OSRM source code, and CMake to generate the build system. The following command will install all 3 of those.

sudo apt-get install build-essential git cmake

OSRM is a fairly complex piece of software and also relies on a number of libraries. We can install the required dependencies with the following command.

sudo apt-get install libboost-all-dev libtbb-dev liblua5.2-dev libluabind-dev libstxxl-dev libxml2 libxml2-dev libosmpbf-dev libbz2-dev libprotobuf-dev

These dependencies are used for various things. Lua is used to define custom speed profile scripts, e.g. defining that on a secondary road, in absence of limits, a car goes on average at 80 km/h, and that on a gravel road the average speed is 50 km/h. STXXL is a version of C++'s standard library which uses disk space as memory, used to manipulate large files. LibXML and Protocol buffers are used to load, write, and manipulate OSM files, and Boost and TBB are used for parallelization and to represent data structures.

Step 4 — Compile OSRM

In this step, we will get OSRM's source code and compile it.

Thanks to the fact that OSRM's build system of choice is CMake, once the dependencies are installed, it is fairly easy to generate build files and compile OSRM.

First, we clone the source code from the project's page. The project is composed of several repositories, handling different functionalities. We are interested in the back-end (the server-side part).

git clone https://github.com/Project-OSRM/osrm-backend.git

Next, move into the code directory.

cd osrm-backend

The next step is to use CMake to generate the build files. It is recommended to build OSRM in a dedicated build directory in the source code root, to avoid polluting the source directories with temporary build files.

Create a build directory.

mkdir build

Move into the build directory.

cd build

Finally, we will generate the build files with cmake. This command will generate a number of directories and Makefiles in the build directory tree.

cmake ..

If you get an error here, make sure you have enabled swap via the instructions in the prerequisites.

Next, compile and install OSRM.

sudo make install

Note: This may take 5 to 10 minutes.

Building with sudo is necessary here because the install target will copy some executables in the system's binary path, including some which we will need. Namely:

• osrm-extract which opens the map file and runs a first pre-processing step on the data.
• osrm-prepare which processes the output of osrm-extract and computes the traveling times for all the map edges according to a given Lua speed profile.
• osrm-routed the actual web service daemon, which allows us to query for distances and locations.

Step 5 — Configure STXXL

In this step, we will create a configuration file for STXXL.

Before running the web service, we need to pre-process our map export. Because we have installed the needed binaries in the system path, we can do this from anywhere. For the purpose of this tutorial, we will run the pre-processing in the root of the osrm directory we have created.

First, move to the osrm directory.

cd ~/osrm

The map pre-processing is quite memory intensive. For this reason, OSRM uses a library called STXXL to map its internal operations on the hard disk. STXXL relies on a configuration file called .stxxl, which lives in the same directory where you are running your software, to determine how much space is dedicated to the STXXL data structures. Depending on our Droplet's capacity and on the size of the map we wish to process, we need to write a suitable .stxxl configuration file, allocating enough memory for the operations.

Create and open .stxxl for editing.

nano .stxxl

The file must contain a single line with format disk=path,capacity,access, where path is the path where the allocation file will be placed, capacity is the capacity of the file, and access is a file access implementation.

Here is an example of a .stxxl file. You can paste this into .stxxl, but may want to change the size of the file based on the map you're using and the size of your Droplet. See the documentation for advanced options.

disk=/tmp/stxxl,10G,syscall

Save and close .stxxl.

Step 6 — Extract the Map

In this step, we will extract the map.

The first step of the pre-processing is the extraction of the map. The osrm-extract command expects the path of a map export as an argument, and assumes the presence of a suitable speed profile script under the name of profile.lua in the working directory. The speed profile is used to figure out which of the available routes can be used (for instance, a speed profile for a truck might forbid some streets).

The OSRM backend distribution includes a number of default speed profile scripts under the profiles directory of the repository. In this tutorial, we will use the car.lua profile, which is good for most use cases

Because the speed profile script might depend on some Lua functions defined in the profiles library, we also create a symbolic link to it in the same directory by running the following two commands.

ln -s osrm-backend/profiles/car.lua profile.lua
ln -s osrm-backend/profiles/lib

Our map export is called map.osm, so next, run:

osrm-extract map.osm

This step generates a bunch of files in the pre-processing directory, including map.osrm, which is the input to the next step.

Step 7 — Compute Travel Times

In this step, we will compute the travel times for the map.

The step is carried out by the osrm-prepare command, which again uses the speed profile script to compute the travel times for each edge of the map graph. To do this, run the following command.

osrm-prepare map.osrm

This step also produces some additional files needed by the web service, which we will set up in the next section.

Step 8 — Run and Test the Web Service

In this step, we will run OSRM and test that it works via a browser.

The OSRM backend comes with a final command, osrm-routed, which can read the processed map and allows querying it through a web service API. In order to test it, run:

osrm-routed map.osrm

Now, by pointing your browser at http://your_server_ip:5000, you should now be able to see the web service in action. It will show an error message like {"status_message":"Query string malformed close to position 0","status":400} because you are not using the correct query format.

As a test, choose a set of latitude and longitude coordinates within your map boundaries, and go to the following URL, replacing latitude and longitude with the coordinates you chose.

http://your_server_ip:5000/nearest?loc=latitude,longitude

You should see a JSON output similar to this:

{
    "name": "street_name",
    "mapped_coordinate": [
        latitude,
        longitude
    ],
    "status":0
}

If you get an error message instead of this, you may have chosen a set of coordinates outside of the map boundaries or your query syntax might be wrong. For more available queries, check out the server API.

You can now stop osrm-routed using CTRL+C.

Step 9 — Set up Nginx

In this step, we will set up Nginx to work with osrm-routed.

We now have a working web service, but querying it by specifying the port is awkward. Moreover, if we decide to serve different maps or maps processed with different speed profiles, we want to avoid having to remember a collection of ports.

Nginx is a high-performance web server which can also work as a proxy and act as a gateway for our web services. Setting it up to work with osrm-routed is fairly easy.

First, we install Nginx.

sudo apt-get install nginx

Next, we add a configuration file for our Web services. Nginx uses two directories for its site-specific configuration files: /etc/nginx/sites-available (all sites that can be served) and /etc/nginx/sites-enabled (all sites that are being served). The standard way of adding a site is to add its configuration file to sites-available, and then link it symbolically in sites-enabled.

So first, we will add a configuration fime for OSRM to sites-available.

sudo nano /etc/nginx/sites-available/osrm.conf

Our configuration file will define an upstream that points at our web service and a server which listens on port 80 and redirects a subset of the queries to our upstream.

Paste the follow configuration file into osrm.conf. You will need to specify two variables, which are highlighted below: your server IP and a path (which will be used to access the web service, as in http://your_server_ip/example_path).

upstream osrm {
    server 0.0.0.0:5000;
}

server {
    listen 80;
    server_name your_server_ip;

    location /example_path {
        proxy_pass http://osrm/;
        proxy_set_header Host $http_host;
    }
}

Once you have saved the file, move to the sites-enabled directory.

cd /etc/nginx/sites-enabled

Then we can link the osrm.conf file.

sudo ln -s /etc/nginx/sites-available/osrm.conf

Next, reload the configuration.

sudo service nginx reload

Finally, restart Nginx.

sudo service nginx restart

Now re-run osrm-routed.

osrm-routed ~/osrm/map.osrm

You should be able to access the Web service by pointing our browser to http://your_server_ip/example_path. Note that you no longer need to specify the port. You can now stop osrm-routed using CTRL+C.

By adding more upstreams and locations, and by running osrm-routed by specifying the port with -p or --port we can run more instances of the Web service, and bind them to different paths. This tutorial won't go into details about this, but you can check out the OSRM backend documentation for more information.

Step 10 — Install and Configure Supervisor

In this step, we will install and configure Supervisor to keep Nginx running, allowing our web service to be available through reboots.

Nginx now acts as a gateway for our web service. However, we have started it manually, so if we log out from the system, it would stop running. In order to make our web services survive through reboots and, in general, to make them recover from possible failures, we can use a tool called Supervisor.

Supervisor is a process control system which mostly takes care of keeping services running properly. Setting it up is quite easy. First, we install Supervisor itself.

sudo apt-get install supervisor

Then, we add our web service to the pool of services controlled by Supervisor by adding a new configuration file.

sudo nano /etc/supervisor/conf.d/osrm.conf

The configuration file must contain a definition of this form for every web service we wish to supervise, and the program name must be different for every web service.

Paste the below configuration into the osrm.conf file, then save and close it.

[program:osrm]
directory=/home/osrm/osrm
command=/usr/local/bin/osrm-routed -p 5000 map.osrm
user=osrm

What this configuration says is that we want the user osrm to keep the command /usr/local/bin/osrm-routed -p 5000 map.osrm running, and that it must be run from the specified directory, /home/osrm/osrm (which is how we can specify map.osrm as a relative path in the command). In this example we specified the port for osrm-routed using -p so that multiple programs can be added by increasing the port.

Once you save and close the file, restart Supervisor.

sudo service supervisor restart

We can then check the status of the web service by running:

sudo supervisorctl status

If everything is fine, we should see something similar to this

osrm          RUNNING    pid 12698, uptime 0:00:40

This means that our web service is running. Because the upstream is pointing to port 5000, Nginx will be able to serve it on the specified path.

Conclusion

This tutorial convers the installation of the OSRM backend, but further documentation is available at this page.

At the time of writing, the OSRM back-end is still in beta, and has some hard-coded limitations on the number of locations that can be included in a single query (currently 100, affecting the time matrix API). Such limitations can be disabled, but it is necessary to modify the source code of osrm-routed in order to do that.

Depending on the size of your map and on the capacity of your Droplet, you might need to allocate a larger swap file or increase the capacity in the .stxxl config file in order for the web service to operate properly. You may need to test a few configuration to see what works best for your setup.