Coordinate Response to COVID-19 with Interactive Maps

Managing a community response during a pandemic requires tools that inform and educate. Maps have done this across centuries, and we now have amazing digital tools that can do this with the click of a few buttons. Here's how our community is responding to the crisis, and hope others can leverage these tools to build their own applications to help inform and mobilize their communities.

Phase 1 - Get the tools up and running. FAST!

It was March 2020, and each day the news seemed to get worse. COVID-19 was out in our community, and people were looking for action. A project was started that allowed volunteers from across the community to "adopt a block" that they live in, and look out to help their neighbors during this time of need. A non-profit called ForRichmond started the effort, and it was to cut across geographical boundaries in central Virginia. Since we were also locked down under quarantine, the project was going to need to rely on digital tools. That's where I volunteered to help.

Given that massive impact, I recommended to get the application up and running as fast as possible. While we found some great applications to model on, we new that time was precious to get this started. So we began the effort with an MVP model, starting with a basic application to get running.

Step 1 - Build a Volunteer Form

The non-profit that we were working with already had a basic static website hosted by Squarespace. Were able to get a form page setup that could serve to sign-up volunteers and directly load into a g-sheet. This page was then linked to their main website, and shared on multiple social media channels.

Step 2 - Build a Map

The founder for the non-profit had already heard of a similar project in Philadelphia, and was able to connect me with the lead for it. This was based on Open Street Map. Based on research, I found there is also a commercial product called Mapbox that builds tools and services that add onto it. I signed up for the service, and was able to quickly build a map using the free tier. Whenever someone wanted to "adopt a block" by filling out the form on the website, we could then draw out the block on a map for reference. This map was then integrated into a webpage hosted on the Squarespace site for others to see the progress of how many blocks were adopted.

Step 3 (Ongoing) - Manually Maintain the Map

As volunteers sign-up in the form, someone needs to take the data out of the G-sheet, and then go into the Studio interface for Mapbox and map the street that was volunteered for. That takes some coordination through texts and e-mails. Once the changes are made, a new version of the map is then published that is then reflected in the site. It was a bit manual, but was easy to get up and running so hit the mark for the MVP. Here's a process flow for reference.

To see this in action, please check out this link.

https://www.forrichmond.org/loveyourneighbormap

Phase 2 - Digital Mapping

After getting the MVP up and running, I started to work on what the next version. Two key limitations were (1) lag between when someone would sign-up for a block and when it would show up due to the manual entry and (2) trying to understand intent from the volunteer based on a text entry field. The second point is tied to geography. In a classic downtown street grid, it's easy to describe what a block is. For example, the 3300 block of Main Street, or Center Ave between K & L Street. In suburban areas, blocks are far more vague as they don't follow a grid, rather they are made up of curvy roads and cul-de-sacs. To fix this, the interface needs to change, and have the blocks pre-defined that a user can explicitly select. Here's a view of the new version that is also built with Mapbox, along with how it is built. The basic concept is that all of the streets start off as grey, but as they are adopted, they change to blue.

Step 1 - Source the Data

The greater Richmond metro area has more than one million people living in it, and is spread across several hundred square miles of land mass. Mapping all of this data by hand would take years, but fortunately this information already existed - just was a matter of finding out where. Our research showed that the local municipalities have GIS departments that have already built digital maps for other purposes, and publish it as public data. Here's an example of one of the sites for the City of Richmond.

https://richmond-geo-hub-cor.hub.arcgis.com/maps/edit?content=cor%3A%3Acenterlines

This data can be downloaded, and then used in other projects. I downloaded the data for all four municipalities in our region that we wanted to get block level data for, so this enabled moving forward with the new approach of building a regional map of all of the residential blocks that could be adopted.

Step 2 - Convert Street Data to GeoJSON

While I was able to get versions of data, a consistent theme was that the datasets were in a different format than what was needed in Mapbox. KML data is the common format that was found in the GIS departments, however what's needed is geojson. That requires conversion. Also these downloaded files are quite large - in the 10-100MB range, and given that we're eventually going to need to use API's to load this data, I needed to keep the size manageable. I used an online utility to convert the KML structures to a more common csv format (see link below), then created multiple shards that broke the files into something easier to manage (approximately 5-10MB each shard).

https://www.convertcsv.com/xml-to-csv.htm

Step 3 - Filter and Enrich the Data

Next was a step to reduce the dataset for just the information we needed. This would help in the usability as well as the performance of the application. This application just needed residential streets. The maps that we received from the municipalities is used for street maintenance, so includes things like highways and overpasses were removed. This was done through a data filtering process written as a lambda function in AWS that could process data staged in S3 buckets.

The interactive map also needed to be seeded with attributes for each block. That makes rendering the maps easier within Mapbox. To accomplish this, there's a boolean that indicates if the block has been adopted, as well as who has adopted it. These attributes were added to the filtered data in the previous step, continuing to use a json format.

Step 4 - Load Data into Mapbox

To create the map that would be rendered by a browser, I needed to import the data into Mapbox into unique datasets. This can be done either by API or by the Studio interface, and I chose the later. There are limits on how large of an upload file can be processed at once, so the file sizes needed to be practical. This was done for all four data sets processed in the prior step.

Step 5 - Build a View of the Map Data

Once the data is loaded into Mapbox, tilesets need to be created that reflect each of the different groups of street data. These tilesets are what the browser renders within the map as a layer that the user can interact with, and are also referenced in the SDK within the browser code. Each layer has its own attributes - color, font size, line width, etc. This is all done within Mapbox studio and is called a style.

Step 6 - Add Region Layer to make Map Usable

An unexpected scope that I needed to add was a summary layer to the maps that we refer to as the regional layer. The underlying datasets in the prior steps totaled 100k city blocks (features), so when attempting to display them all at once, the user experience is terrible as it just turns into a big blob of a single color as all of the streets blend together.

To address this, I created a summary view of the region that was usable when trying to take the whole thing in. This was done by creating a dataset using Mapbox studio that outlined the regions. The starting point for the map is then zoomed out at this level which shows the broader area.

There is javascript embedded within the webpage that allows the user to click on one of the regions, then it zooms into that region. From there a more user friendly version of the map can be clicked on.

Step 7 - Build an API that can update the Map

Mapbox already has API's that can manage updates. To call the API, will need the unique identifier of the feature that is to be updated. Given that Mapbox sets that unique identifier, will need to extract the maps from Mapbox after they are loaded, and build a lookup utility that serves as a wrapper before the API is called that can apply the unique identifier. This was written in NodeJS, and hosted in AWS.

Step 8 - Integrate API into Digital Map

Once we have the API working, then integrate it into our. I built a simple AngularJS web app that then calls the API to assign a block to an individual volunteer. This enables multiple individuals to directly update the map, enabling the project to scale up.