Skip to content

Scaling ReadyMap

ReadyMap Architecture ReadyMap

ReadyMap is a highly scalable application that consists of many different components. This document aims to help you understand how ReadyMap works and how to scale it out to take advantage of your hardware.

ReadyMap Web Application

Client Applications interact with the ReadyMap Web Application using HTTP endpoint which provide access to the user interface, REST API and tile serving endpoints. The Web Server that powers ReadyMap consists of two classes of processes: Web Processes and Generate On Demand Processes.

Web Processes are lightweight workers that respond to requests like serving the user interface, REST API and map tiles that have already been cached. The number of web processes that are launched is controlled by the NUM_WEB_PROCESSES environment variable and the default is 6. Increasing this value greater than the default is not generally useful for most deployments.

Generate on Demand Processes are responsible for generating any map tiles that don't already exist in the Tile Storage directly from source data. The number of generate on demand processes that are launched is controlled by the NUM_GENERATE_ON_DEMAND_PROCESSES environment variable and the default is 1. Generating map tiles from a large amount of geospatial data can be a memory and CPU intensive process, so it is critical to separate the processes that generate the map tiles from those that serve tiles that are already cached, otherwise the web server could get completely overwhelmed generating map tiles and become unresponsive. Increasing the number of generate on demand processes will make client applications that are requesting tiles on demand more responsive.

ReadyMap Workers

ReadyMap has a collection of worker processes that process jobs that are placed on the work queue stored in Redis. There are different worker processes for different types of jobs in ReadyMap.

Default Workers: These workers are responsible for importing feature data into the database, tiling feature data and doing any file processing tasks. The number of workers is controlled by the NUM_WORKERS environment variable, defaulting to 2..

Tiling Workers: These workers orchestrate tile generation, delete tiles, and export tilesets. The NUM_TILERS environment variable controls their number, defaulting to 2. Tiling workers distribute tasks to Tile Generator Workers but do not speed up tiling jobs themselves. Think of them as managers assigning tasks, while the actual tile generation is done by the Tile Generator Workers.

Tile Generator Workers: These workers generate map tiles requested by the Tiling Workers. The NUM_TILE_GENERATORS environment variable controls their number, defaulting to 2. Increasing this number speeds up tiling jobs by distributing the work among more workers.

Allocating Resources

In general, the two settings that you want to tweak to horizontally scale ReadyMap are NUM_GENERATE_ON_DEMAND_PROCESSES and NUM_TILE_GENERATORS. You need to think about how your ReadyMap system is going to be used and what resources you have available to you to determine what values to choose. For example, assume you have a 16 core system. You might want to reserve 4 cores for the main web workers and general OS work, leaving you 12 cores to allocate to for Tile Generator Workers and Generate on Demand Processes. You could split them evenly and allocate 6 generate on demand workers and 6 tile generators. You also could decide that you are rarely going to be performing manual tile generation jobs and want generate on demand tiles to be as fast as possible so you allocate 11 generate on demand processes and 1 tile generator.

In addition to thinking about the CPU core allocation, you need to think about the memory available to you and what kind of layers you will be working with. ReadyMap uses the GDAL to read geospatial data. GDAL uses a block cache controlled by the GDAL_CACHEMAX environment variable (default is 2000MB in ReadyMap) and each generate on demand and tile generator process will have it's own block cache. Keep this in mind when scaling ReadyMap. For example, setting NUM_GENERATE_ON_DEMAND_PROCESSES to 50 could use up to 100GB of memory just in GDAL block cache. You can lower the GDAL_CACHEMAX to combat this, but you need to make sure that it's sufficiently high if you are dealing with large datasets.