COVID-19 Response dashboard documentation

Overview

This document outlines the metrics found in the Massachusetts COVID-19 Response dashboard, the context for each metric, and why these metrics matter to the dashboard’s audience. This document serves as a history of the changes made, and describes the "trail map" of future enhancements and fixes.

Dashboard audience

This dashboard aggregates a number of COVID-19 datastreams that the EOTSS Command Center has requested and makes it possible for Command Center personnel to monitor metrics about these datastreams.

Metrics and context

Below the metrics of each page of the dashboard are outlined along with context for why it matters to the audience.

Page 1 - Text alerts

Total sign-ups scorecard

The cumulative sum of all English and Spanish language text message sign-ups to date.

Sign-ups for texts in Spanish scorecard

The cumulative sum of only Spanish language text message sign-ups since April 10th.

Sign-ups in last 30 days scorecard

The sum of all English and Spanish language text message sign-ups over the last 30 days (excluding the current day) compared to the previous 30 days.

Weekly English sign-ups line graph

The number of English language text message signups. Shows the overall trend of English language text message sign-ups over time.

Weekly Spanish sign-ups line graph

The number of Spanish language text message signups. Shows the overall trend of Spanish language text message sign-ups over time.

COVID-19 text alerts page context

These graphs aim to show how many people are signing up for text message alerts.. It also shows internal Mass.gov teams the overall trend of those signing up for the alerts. This may be an indication of how successful the campaigns to let the public know of the text alert option are.

Page 2 - Text alert campaign engagement rate

Daily average engagement rate (AlertsMA campaigns only)

Shows engagement rate by day for texts associated with the #AlertsMA campaign. The date picker above the table defaults to the last 14 days (excluding the current day).

Clicks for non AlertsMA campaigns

Shows the number of clicks for texts not associated with #AlertsMA campaigns. The campaign drop down above the table lets you filter by specific campaigns.

Campaign engagement rates context

This page shows how often subscribers to daily COVID-19 update texts follow links that go out with the texts. It helps the Command Center understand how interested their audience is in specific topics, and it shows the extent to which subscribers engage with the texts they receive.

The data for this page comes from this spreadsheet, which requires some maintenance. We take in data as follows:

• The text team adds the planned and sent text messages including date sent, long and short URL, messaging, and more to Sheet 1 in the workbook

• We import that data to a new tab to perform manipulations like checking the long URL for mass.gov as the hostname

• We also import data from the Command Center dashboard to get the number of text subscribers per day and add that to our data manipulation tab

• Engagement rate calculation:

  • Sessions (from Google Analytics) for a particular day / Number of texts sent on that same day * Total Text Signups for that same day

  • The "Text sent today" column (from the above formula) filters out the duplicate texts (i.e. the ones sent in Spanish). The "total subscribers" number represents English and Spanish language subscribers.

This workbook uses a series of formulas that are maintained by an autuomated script, which listens for new data to be added, and makes sure that formulas are applied to them, too.

Page 3 - Web analytics

Year-over-year traffic comparison to all Mass.gov by month line chart

This chart shows the total number of pageviews (y-axis) to Mass.gov by month (x-axis). Each colored line represents traffic for each year (2020 to date, 2019, and 2018).

Traffic by page table

This table shows the total number of pageviews for pages on Mass.gov sorted from most pageviews to least pageviews this year to date, as well as the percentage change in pageviews compared to the previous year.

Year-over-year traffic comparison to public health pages line chart

The line chart shows the total number of pageviews for Department of Public Health pages. We filter for DPH using the “organization” custom dimension. Similar to the line chart above it, this chart shows the total number of pageviews (y-axis) to Mass.gov DPH pages by month (x-axis). The colored lines represent traffic for each year (2020 to date and 2019).

Year-over-year traffic comparison to unemployment pages line chart

The line chart shows the total number of pageviews for the Department of Unemployment Assistance pages. Similar to the line chart to the left of it, this chart shows the total number of pageviews (y-axis) to Mass.gov DUA pages by month (x-axis). The colored lines represent traffic for each year (2020 to date and 2019).

Web analytics page context

By surfacing general data about Mass.gov traffic, and the increases in traffic to specific areas of the site, Mass.gov internal teams hope to keep a pulse on what content is affected by COVID-19 traffic and to determine what content needs to be reviewed/further supported.

Page 4 - Audience

Traffic by device pie charts

These pie charts show the total number of sessions by device category (mobile, desktop, and tablet) compared to the total number of sessions by device for the same time period of the previous year. The pie chart on the right shows the most recent 28 days (excluding the current day) and the pie chart on the left shows the same 28 days last year.

Visitors by hour line chart

This line chart shows the total number of pageviews broken up by hour over the last full month, compared to the same month last year.

Audience page context

By understanding when and how constituents access the Mass.gov site, Mass.gov internal teams can ensure that content is performing optimally for the devices constituents are using and that content is available during peak times.

Page 5 - 211 calls & nursing home hotline calls

Daily calls line graph

This graph shows the number of people who have called 211 each day since March 13th.

Total 211 calls scorecard

The sum of people who have called 211 since we began collecting data on March 13th.

Average monthly calls scorecard

The average calls per day over the past 30 days (excluding the current day) compared to the previous 30 days.

Nursing home hotline calls bar chart

The total number of calls each day since May 1st.

211 calls & nursing hotline calls page context

A goal of Mass.gov content managers is to make information easy to find and widely available to constituents so that they do not have to go through the process of calling a number. It’s therefore a sign of success for 211 calls to decline, as this shows that Mass.gov is doing a good job of sharing information through other channels like texts, social media, email, and more.

The dashboard aims to show the number of reports as well as educate the public about the number they can use to report their concerns.

Page 6 - Response team & Buoy symptom checker

• Buoy symptom checker site: https://www.buoyhealth.com/symptom-checker/

Total volunteers bar chart

Shows the total number of unpaid volunteers with and without background checks that have signed up each day over the last 14 days (excluding the current day). When no data is available, we don’t show a bar.

Specialist volunteers to date scorecards

These three scorecards show the maximum number of nurses, doctors, and respiratory therapists who may be deployed.

Total symptom checks to date scorecard

This scorecard shows the total number of interviews conducted via Buoy to date since March 26th, 2020.

Symptom checker bar chart

This chart shows the number of interviews done via Buoy compared to the number of cases escalated for further testing or care from April 13 - May 13, 2020.

Response team & symptom check page context

This helps internal teams at EOTSS assess how useful constituents find services like Buoy by measuring how often they are used. It also helps them judge how successful their campaigns are at driving sign-ups for the volunteer specialist jobs, and measure changes to their campaigning tactics.

Trail Map

• Click through rate by text for each URL