These are valid points from an expert in psychology like you, Sam! Appreciated it, and you’re very right about the MMS to be spelled out for others to understand, yet we need to be comfortable with not having a standardized MMS. This is design, where some degrees of mess is expected, and no two process can be exactly the same.
The problem with this is, “things that are interesting” relies on what the interviewer and observer find relevant and surprising. This differs from one person to another because of factors like personal experiences, cultural biases, and preferences.
Valid. And you just gave yourself the answer. That’s why it’s important to have different people in the team who interview and observe the participants. We need the different minds in order to have the different notes on interesting things.
Naturally, what stays in our working memory after the interviews are things that stand out — information we find surprising and interesting.
Sure, for some interviews, we might go back to the audio or transcript a few times during the debrief. But we do so with the intent to quickly check facts or quotes rather than checking if we have fully represented the participant through our post-it notes.
Valid. That’s why we do the debrief as soon as one participant is done. The key is to get first insights. In design, insights are more important than facts. Note that it’s a bad practice to go back to the audio or transcript during the debrief, because it distracts you from retrieving the working memory. Refer only your quick notes. Do the audio/transcript after all interviews are done, only when you need to provide a thorough analysis. I once used only first insights and shared them with the stakeholders yet it was enough for them to get surprised and immediately prioritize a project. If your stakeholders already heard most of the first insights then do the thorough analysis.
At times, we want to include more people in the synthesis process (people who were not interviewers and observers of the interviews), to get more help in synthesizing.
These rundowns are often rushed because it takes up quite a bit of time. This leads to teams presenting a summary of points for each participant, leaving out a lot of rich context and stories.
Valid. Note that it’s a bad practice to include people in the synthesis process when you don’t process the data first at least with two other people. Your group of three people (from three different teams/roles) would have three different minds and produce three possibly different first insights. Do the first round of synthesis also with these people who already dwell in the richness of the stories. When we include the 4th person and so on, it’s okay for them to get only the summary of points, even the half-baked synthesis result.
When the group is presented a summary of points on the participants, it’s common to note down certain points to better recall the main learnings from the participant.
However, when people are taking notes individually in a group setting, peer pressure can happen.
Valid, however this really depends on organization culture and who’s invited to the subsequent synthesis round. Note that it’s a bad practice to note down those summary of points. The summary is just for the synthesis facilitator to get newly-joining people started. Nothing is worthy of note until these 4th person and so on finish their participation in the synthesis session. It will be an embodied summary of points. Everyone will get the synthesis results anyway after the session (even if the 4th person and so on have to leave earlier due to limited time commitment).
*****
I appreciate your feedback from psychology / cognitive point of view, especially on your concern on not fully representing a participant. This is actually a secondary concern, because what we need to obtain here is whether the insights are valid (what a participant says should match their context i.e. we can all say why they say something, which differs from person to person depending on their context) instead of whether we know everything about a participant. First insights in the debrief should include this contextual information. We can’t say, e.g. “participant dislikes the 1 hour payment time” but e.g. “participant cannot make use of the 1 hour payment time due to lack of m-banking and living far from an ATM/convenience store”.
I must admit that it’s difficult to work with beginners in deriving first insights during debrief session. Depending on how much time we have, we can train them first with examples or use the first 3 sessions for practicing debriefing. Our facilitation skills can only get better!
Lastly, it also depends whether you need to get fast results with stakeholders who never heard from customers or to dive deep into a new population to study with stakeholders who already know a lot about customers. There are many possible combinations of situations, although with #DesignInTech it tends to be the fast and minimum viable insights.
