Feedback tagging

Search…
Feedback tagging
Summary
This is an overview of how we analyzed paidleave.mass.gov feedback. The end result of these analyses were the following charts:
After we finished, we passed the results along to Bryan for possible presentation at an Executive Steering Committee meeting. 
Method
To perform the analysis, I first took all PFML feedback between launch and the time of analysis (early February). I divided it into employers and claimants segments.
For each, I read through the associated feedback once quickly, developing a list of common themes I was seeing. Then, I read through again, tagging each item with 0 to 5 themes. I did my best to tag each item at least once, including using tags like compliment  when users said only things like "Simple to use," or wrong portal when the feedback was clearly meant for the other segment (i.e. a comment about the employer process from claimant feedback). When users did not leave a comment, I did not tag the feedback. 
One improvement I made between tagging claimaint feedback (which I did first) and employer feedback  was to begin formulating tags as problem statements--or at least descriptions of the issue(s) users were describing. For example, in the first round of feedback, I included tags such as "unclear benefits," "salaried workers," and "login." However, these tags don't adequately illustrate what trouble users were having. In the second round, I wrote tags such as, "want to partially approve," "want an email confirmation," and "trouble logging in." Including a verb (or at at least an adjective that implies a value judgment), made for clearer results.
Lastly, it was important to keep the authoritative list of tags in a separate, easily accessible document or spreadsheet tab so that you can ensure you don't have multiple tags referring to the same theme, e.g.  "trouble logging in" and  "login trouble." This would cause the topic to appear less frequently in the final analysis, as its count is split across tags. 
Data aggregation
Once all the feedback was tagged, I counted the  occurrences of each tag. (This results in a much larger count of tags than the total count of feedback, since multiple tags could be associated with a single piece of feedback.)
After counting, I had a table comprising 2 columns: tags and count of occurrences. I was asked to visualize this as a dual-axis bar chart, where "count" is on one y axis and the running percent of total is on the other. The x axis was the list of tags.
It's likely that you'll have quite a few tags, and I would recommend taking just the top 15 or 20. You can see how much harder to read the employers visualization is above than the claimants. 
Digital analytics products - Previous
PFML Web Analytics & Feedback Dashboard
Paidleave tasks
Last modified 6mo ago
Copy link
Contents
Summary
Method
Data aggregation