Sourcing data projects through observation
Taking inspiration from Contextual Inquiry to uncover the highest ROI projects
This post concludes my Learning from Metrics Review series for data leaders, where I discuss using signal from Metrics Reviews to drive your organization’s data culture and data-driven decision-making forward.
For links out to all posts in the series, check out the kickoff post.
I've previously written about the role growth mindset can play in your data-driven culture, helping to strike the right balance between a bias to action and a focus on continuous learning. But what if things are so lopsided on the "action" side that there's little to no hunger for learning? What if the requests for work are drying up?
If you're reading this while on a team that's barely keeping up with requests, this may seem far-fetched. Yet, I see this question come through with some regularity across my data leader networking groups. The responses are often some combination of:
Make sure you're proactively meeting with your business partners.
Don't wait for them to tell you what they want, ask them what they want!
Ask about their pain points to inspire possible projects.
When I chime in on one of these discussions, it's usually in the form of a "Yes, and."
Yes, you should be regularly keeping in touch with your business partners and stakeholders, giving updates about current work and understanding what's top of mind for them beyond that work. It's a critical part of building a strong advisory relationship with your partners.
And, putting the onus on your business partner to articulate the project can still be very limiting. This is especially true when you consider the bigger picture. Short-term, yes, you're trying to find work for your team. But longer-term, the projects you choose will further shape the design of your organization's data-driven culture.
Once we frame this as a design problem, we can take inspiration from Product and Design thought leaders. And if you ask where the best ideas for products come from, it's not from asking users what they want. It's from observing them to learn what they need most, even when they can't articulate it themselves.
When I was studying Human-Computer Interaction, the very first class I stepped into introduced Contextual Inquiry.1 While information is gathered in an "interview" setting, the interview is less focused on conversation and more on observation. The session typically lasts across multiple days. The idea is to focus on observing, taking copious notes, and photographing as much as you can. Once the interview is done, the photos and notes are processed to create 5 conceptual models that represent and document that person's context.
I still remember the week where I worked with 4 other students for hours on end, parsing a transcript to source each box, line, and insight across the five models. The richness of insight that can emerge from this methodology is truly inspiring. Done well, it can spur electric design sessions that frame, and focus, new product work.
But there is a drawback: It's a huge time commitment. And it's more time than a data leader has to spend planning projects — let alone her business partners. Nevertheless, I've often found that just reminding myself of the 5 contextual models uplevels my conversations with business partners. Below, I discuss each model, and how I use it to frame my approach and identify the most impactful work for my team:
(1) Sequence Model: The sequence model details the steps that are needed to accomplish the participant’s important tasks.
Questions for data leaders to consider: What are the important, day-to-day data tasks for each of your business leaders? How — in detail — do they accomplish those tasks? Is anything more involved or complicated than it needs to be?
(2) Artifact Model: One artifact model is created to represent, in sketch form, each critical artifact that the participant leverages in the course of their day.
Questions for data leaders to consider: What happens once data leaves the dashboards or other tools that your team has created? Is the data reproduced? In what tools? For whom is it reproduced, and why? What do the resulting artifacts look like? Are they effective in meeting the goals for which they were created?
(3) Physical Model: The physical model illustrates the participant's physical environment, and how it supports — or hinders — their work.
Questions for data leaders to consider: What components are needed to effectively use data team outputs, and how are your business partners leveraging them? Do they have a data catalog open on one monitor, and a self-serve environment on the other? Do they print anything to help use the data team's dashboards or tools? What do they write on notebooks or sticky notes to facilitate their use of data tools?
(4) Flow Model: The flow model describes collaboration and communication between people in the course of getting their work done.
Questions for data leaders to consider: How many people are involved when one of your business partners has a data-related task to complete? Are there opportunities to streamline that work so fewer people need to be involved?
(5) Culture Model: The culture model articulates elements of culture, policy, and procedure that impact the participant's work.
While data leaders are likely well aware of their company's data governance policies, there is still an opportunity to understand how those policies are understood by your business partners. Are there questions that they aren't asking, because they — perhaps incorrectly — believe that they can't access that data?
At a prior role, my team supported a Customer Success organization. CS team members regularly built presentations (Artifact) to connect with customers from a template. These presentations regularly included data, and that data was sourced from dashboards that the data team had created! So all good, right?
Well, when we dug a little deeper, we found instances where CS team members had to reference multiple data sources (Sequence) to create the presentation. One slide in particular was built by copying and pasting charts from four(!) different locations. In other cases, charts were recreated in Excel to align them to the company's brand guidelines. The building process was so involved, that a senior team member created a template with detailed instructions, so everyone on the team could create their own presentations. Given all this complexity, each presentation often required help or review from senior team members (Flow), since there were so many hidden gotchas.
There was no ask here for the data team. Yes, there was a pain point, but the CS team solved it themselves by creating the template. Nevertheless, we kicked off a project, with a simple goal: To create a dashboard, where each view was a 1:1 map to the template slides, all designed with the company branding.
It ended up being one of our more successful projects, saving tens of hours for each CS team member each quarter. But had we waited for it to be explicitly requested, it never would have happened.
Familiarity with Contextual Inquiry can do much to expand your approach when seeking work for a data team, but at the end of the day, there is no substitute for observation. Above all else, my experience with Contextual Inquiry and similar UX methods pushes me to ask to be in the room when data is discussed. There is no better source of insight on your organization's data culture than observing the successes and breakdowns of data conversations, firsthand.
And so it's fitting to end the series on metrics review meetings with this note: It was never (just) about the review meeting. It was about finding opportunities, as a data leader, to observe your data culture playing out, assessing it with honesty and humility, and seeking continuous improvement. Hopefully this series has sparked some ideas where you can show up differently in your organization, to better understand — and steer — your data culture in a direction that is best for you and your team.
Communication post photo by tito pixel on Unsplash
At the time, we learned from Karen Holtzblatt and Hugh Beyer’s book on Contextual Design. In the years since, they’ve published a few variations of the book. Contextual Design: Evolved is a good introductory option if you’re curious to explore the method further.