how to receive feedback like a product manager

Many of us try our best to give good, timely feedback, and in turn we try to receive feedback graciously and patiently—after all, feedback should be a gift. But, let’s be real, it can be really really hard sometimes. If you’re anything like me, I’m sorry. Kidding! If you’re anything like me, you probably have a hard time separating your emotions from the the feedback that you’re either trying to give, or trying to receive well.

I think we could all improve our feedback giving/receiving skills by approaching feedback a bit more like a product manager. Not to say you should form your feedback in the way that a product manager might, but rather you can approach feedback in a similar way that a product manager approaches a customer problem. There have been a billion things written on how to give feedback well, so for now I’d like to start with how to receive feedback… even if it’s bad.

Let’s get into it!

• Bad feedback

• It’s ok to feel things

• Prepare yourself

• What’s your problem, anyway?

• Solutions are better together

• tl;dr

---

Bad feedback

I don’t mean receiving negative feedback. I don’t mean getting dope-ass feedback, either. I mean getting feedback that is not well-thought-out, not timely, not specific, and—horrors of all horrors—is phrased as YOUR personal failing as a human, rather than a skill or behavior that can be corrected. It’s well enough that you’re trying to give constructive feedback, but we can’t expect that everyone who has feedback for us is quite as, well, thoughtful.

We’ve all been told feedback isn’t supposed to be personal, and even givers of bad feedback will try to make you believe this. Allow me to quote Meg Ryan’s character, Kathleen Kelly, in the Nora Ephron rom-com of my childhood (nay, life), “You’ve Got Mail.”

What is that supposed to mean? I'm so sick of that! All that means is that it wasn't personal to you. But it was personal to me.

It’s ok to feel things

Receiving bad feedback can feel like a punch to the gut. You will feel it physically. Your fight or flight response will kick into high gear. If you’re like me, you might cry. If you’re more of a fighter, you may lash out. Allow yourself to feel those feelings and observe them.

If you need some time to calm down, let your feedback-giver know that you’d like some time to think about this, and schedule a meeting for the next day. There is nothing wrong with letting things sit so that you can calm down, and more importantly so you can get to the bottom of the actual feedback that’s buried deep within poorly executed words.

Prepare yourself

To get to the bottom of the feedback, you may have to throw out a lot of the initial round. The giver of feedback can be awfully similar to a customer. They feel a certain way about something, they believe they’ve diagnosed the “problem,” and they’ve told you how they want you to fix it. Customers are highly unreliable So are inexperienced feedback-givers.

---

DO

take seriously that there is some piece of actionable insight beneath their words

DON’T

believe the feedback at face value

---

The good news is, you no longer need to focus on the claim about yourself. Instead, you’ve got a mystery to solve. And who doesn’t love a good mystery? Your mission, should you choose to accept it, is to discover what your feedback-giver’s ultimate concern / problem is. And it’s going to be tricky!

You may have a story in your head about where this feedback is coming from, or the motivations of the person giving you the feedback. The reality is your story about this person may be completely false. Think about Elizabeth Bennet’s low opinion of Mr. Darcy for the first half of “Pride and Prejudice.” She had a very clear story in her head about not just his behaviors, but what his behaviors meant about his inner character. Of course (spoiler alert!) she eventually finds out she was sorely mistaken, yadda yadda yadda. The point is, don’t assume your story is correct.

Your feedback-giver may not be aware of what is actually bothering them. They may say it’s x, when really it’s y. For example, they might cite your lack of willingness to provide weekly updates as evidence of your laziness, wherein the actual problem might be that there’s a lack of trust in your work. A weekly update will never fix the trust between you, but together you might be able to develop a plan that would.

What’s your problem, anyway?

My favorite question—the real MVP within a product manager’s toolset—is, “What problem are you trying to solve?”

(Yeah, that’s right. I MADE AN MVP JOKE! I’m proud of myself. And like any good joke I will now explain it. I meant Most Valuable Player, whereas in most product contexts one would assume I meant Minimum Viable Product. See what I mean about assumptions?)

Sometimes you can come straight out and ask, “What problem are you/we trying to solve?” Sometimes that’s a bit more awkward, though, especially if your feedback-giver doesn’t realize they haven’t really pinpointed the root problem yet.

If you feel like they’re getting micro-manage-y, try these questions:

• What concerns do you have about [x]?

• How are you planning to use [this data/this information]?

• Is there something in particular you’re worried about?

If you feel like they want you to do things their way:

• Do you have an example of what you’d like to see? Let’s walk through it together. Tell me what you like about it, and why.

• What are the outcomes you’d like to see with this?

• Let’s look at [document x]. What things didn’t work? The structure? Content? Tone? What things did work?

If you feel like you got an answer wrong:

• Can we walk through our different approaches to calculating this? I’d like to better understand how to do this going forward.

• Is there any context I was missing?

• Did I miss a key piece of information?

Don’t feel like you have to stop after one question. In the product manager world, there’s a concept of the Five Whys. The idea is that you take the initial problem and drill down until you get to a root problem. Do you always need exactly 5? No. You might only need 2 or 3.

Example: problem extraction

“I want you to send me a weekly email on the progress of [project a].”

→ “I am worried this project will go over schedule.”

→ → “I don’t trust you to keep the team on track.”

→ → → “The last project you delivered was a quarter late.”

→ → → → “I delivered [project b] late because I was pulled into [project c] and was told to prioritize that.”

Remember, something that “sounds” like an answer to the problem might not be the real problem. You might be given an answer that sure seem like a reason for the behavior, but are more likely red herring phrases that require further digging.

Examples: Dig more

• I just want to know what you’re working on (why? how will you use this information?)

• I want to be able to unblock you (are you worried that I won’t reach out when I need help?)

• I want to make sure I give you context (what prevents you from proactively providing me context?)

Solutions are better together

In product management, we believe that to find the best solution to any problem, it’s wise to have multiple people brainstorming multiple solutions to that problem. We then keep the best solution, no matter where it came from. If a solution was provided as a part of the “feedback,” make sure you evaluate that solution against the root problem. It’s much easier to have your feedback-giver come to the conclusion that it doesn’t solve the problem than it is for you to explicitly try to convince them.

Example: solution ideation

• Will a weekly email help you learn ways of keeping things on schedule? e.g. does what they’re asking for solve the root problem

• What options are there to help you accomplish the goal of keeping things on schedule? (sheer force of will is not an option)

• Is there additional training or coaching that could help prevent this in the future?

• Do you need more tools for prioritizing pieces of the project?

• Do you need your manager to shelter you from external factors that might lead to things slipping? (e.g. other teams eating up your time, conflicting priorities, or scope creep)

tl;dr

• Getting poorly constructed / worded / thought-out / plain old bad feedback is hard

• It’s ok to give yourself time to work through emotions

• Acknowledge that you may have a story in your head about the feedback-giver which may not be correct

• Go through the feedback and extract the problem(s) like a damned scientist

• Ask 5 why’s and other probing questions to get to the heart of the problem

• Brainstorm solutions with the feedback-giver

quick update

Hey folks! Wanted to send out a brief updated on what to expect from me in the coming months.

Cool stuff

• I’m working on translating my recent webinar for ProdPad into a blog post—coming soon!

• I’m going to start offering a few facilitated group discussions—if you’re interested, please fill out the form!

• I’d love to offer a more regular ask eboyle post—please submit questions for that! (small narrowly scoped questions preferred, but ask big ones if you have them)

Shameless plug

As a reminder, I do offer individual one on one coaching. While the majority of my clients tend to be product managers, I also work with program managers, and even a few folks in HR roles. Whether you just need to get a sanity check on how you’re approaching something at work, need an honest take on a situation, or want to work on a soft-skill, I’m here for you.

Reach out to get the full range of options (and pricing) available to you. Happy to discuss a sliding scale if needed—just let me know.

What else?

What else would you like to see from me, given that the world turned upside down? I’m open to ideas of how I can support you all!

I’m out… (metaphorically. literally still in the same 450 sq ft I’ve been in for 3.5 weeks everything is totally fine)
Erin

simple changes to help move to working remote

Look, the world is both scary and weird right now. Folks who usually work in offices—especially open-plan offices—may be struggling to focus, be productive, or even stay sane. It’s important to realize that your employees and team members are humans and likely have a lot on their mind right now, and you are just not going to be quite as productive as business as usual.

Here are some really simple tips to try and keep you and your people in good shape for more long-term remote work.

Make sure bio breaks are built in.

It’s really, really easy to join video conference after video conference and forget that people need time to take a breather and use the restroom. Most people will just take the break and be late to their next meeting, or just hold it. Neither of these are ideal outcomes.

Use speedy meetings

(end half hour meetings 5 minutes early, and hour meetings at least 10 minutes early) and stick to it to ensure people have a chance to do whatever they need to do as humans.

Build physical movement into your meetings.

Look, we have all forgotten to stand up once in awhile when we’re in an office setting. With people working from home, it’s even more likely they’ll sit for hours on end, forgetting to move around a bit. Besides, we’re all going to get more and more stir crazy as this goes on.

Use physical ice breakers

at the starts of recurring meetings. Maybe it’s 5 minutes of easy stretching, doing 20 jumping jacks, having a dance party for one song… whatever you can think of to build movement into your meetings will help everyone. It’ll make you feel better, and remove some of the cobwebs from the brain. Do make sure you keep this inclusive—consider having optional variations for folks who might not be able to do a full-on jumping jack.

Limit context switching as much as possible.

When we’re distracted by external events and worried about the world, it’s hard to find ways of concentrating on work that—let’s be honest—is not always the most important or interesting thing on your mind. One way to make it easier to get into a flow state is to construct your time in a way that limits context-switching and groups like-work together.

If your job requires a lot of 1 on 1 meetings (people managers, product managers, and program managers), considering rearranging your calendar do those all during the same block of time.

If you work on multiple projects, consider assigning different projects to different days of the week.

Now is a really great time to Marie Kondo your meetings.

I’m not saying you should get rid of any meeting that doesn’t spark joy—that would uhh, really limit the productivity of your organization. But you absolutely should look at some elements of your meetings and Goldilocks them a bit.

Evaluate and modify your meetings

• Function - what is the goal of this meeting, and is it fulfilling its purpose?

• Form - what about this meeting is working or not working well remotely?

• Frequency - is this meeting happening too often, not often enough, or just right?

Based on your answers to these elements, you may want to brainstorm some ways to switch things up. Maybe some meetings turn into a quick Slack chat, and maybe some move to a less-frequent basis for now. Some meetings may need to occur for a shorter time but more regularly.

Create space for unscheduled discussions.

Let’s be honest—we kind of hate drive-by conversations when we’re working in an office. They distract us, get us out of our flow state, and can’t be prioritized. However, there is very often a need to chat in an unscheduled way, despite our desire to structure the heck out of our calendars.

Create virtual office hours

and encourage people to literally drop in on you. Hold it the same time each week, with the same meeting link if you can, and plan to share your video even when no one has joined. When someone does stop in, you’ll be ready for it. You might find that this reduces the need for some ad-hoc meetings as well.

This can also help teammates who may be feeling especially isolated and just need to see a friendly face.

Pretend you’re a real YouTuber.

I’m joking! Mostly. Sort of. You’re having a lot more video calls, right? Why not curate something interesting to look at behind you? I am not encouraging anyone to buy anything new—first of all, that’s going to be difficult right now, but more importantly this is a fantastic time to really implement the reduce and reuse portion of reduce, reuse, recycle.

Have a colorful sheet or towel? Hang it up. Have some house plants? Relocate them to be in the frame. Glass jar that once contained pasta sauce that you’ve now consumed? Slap some acrylic paint on it or fill it with knickknacks for some instant pizazz. Finally, play around with your lighting to make sure you’re not creeping people out.

Do this for you, but also do it for the people who now have to stare at your plain white wall for hours each week. Heck, maybe change it up a little bit every week! Fighting monotony in our lives will also fight monotony at work.

benjamin button, doctor who, and seurat walk into a bar

I recently sat down with a former coworker of mine to catch up and talk shop. He’s an engineering leader with a keen interest in product, and we were discussing how to create a product vision that communicates direction yet doesn’t prescribe exactly what to build.

This is a topic I know in my bones, but have historically had trouble finding the words to articulate how to achieve this. Sometimes you learn things that, after awhile, seem obvious… and yet aren’t even remotely obvious to someone who hasn’t done it before.

This is the long way of telling you I had a breakthrough in that moment, sipping coffee together in the window. Product vision is the Benjamin Button of the business world.

Go ahead, laugh. But… it’s just so crazy it might work.

time and detail are inversely correlated

View fullsize

I don’t even want to admit to you how long it took me to figure out how I wanted to draw this. But hey—we got there!

The further in the future your vision depicts, the more detailed, specific, and high quality it can be. As you move closer to that time horizon, the vision becomes fuzzier, less clear, and depicts more of the problems to be solved. At the shortest distance, it may become words instead of images.

Long-term vision

In order to effectively communicate your 5 or even 10-year vision for the product, you’re going to need something a little bit more inspiring than words. You need to literally create images so that your people can see where you’re heading. The beauty of the time horizon being so far out, is that this vision should not impact the product by the time we reach that horizon. You’re not prescribing anything, because no one on your team is staffed to work on this area yet anyway. All you’re doing is demonstrating what could be.

Mid-term vision

You still need some visuals to help communicate a vision that’s 1 or 2 years out, but you’ll want to lower the “quality” of those visuals. In other words, the images will spend more real estate depicting the problems to be solved, and less to what specifically will be implemented. This is where I find it very useful to say “NONE OF WHAT I’M SHOWING YOU WILL BE BUILT! But it’s representative of the types of things we might build to solve these issues.” Also maybe never show this to a sales person…

Short-term vision

You’re a couple months out from starting work on an area, which means your visuals should be of such low quality that they’re actually just... words. At this point you’re not sharing any nods to solutions or interfaces—you’re simply articulating the problems and areas to be prioritized in the coming months.

a vision is never done

The fun thing about product is that it’s a living, evolving, changing thing. What you think it will be in 5-10 years is almost certainly not what it’s going to be, whereas what think it will be in 2 months is going to be a lot closer to the truth. And, as we move forward through time, our 5 year vision becomes our 3 year vision becomes our 1 year vision becomes our next project.

This means your vision is never done.

Think about that for a moment. Your product is constantly changing (hopefully…), your knowledge of your customers and market indicators is constantly changing. That means to some extent your vision is also constantly changing in subtle ways. You may find yourself re-making your long-term product vision every few years, but the mid-term vision is likely shifting once a year. It’s removing things that have already happened, it’s adding things previously nodded to in the long-term vision, and it’s removing specificity.

you are in both the present and the future

And it’s not because you’re either Doctors Strange or Who. As a product manager you will have to learn to flex through time. One minute you may be writing minute stories for a feature you’re implementing in a week, and the next you’ll be mocking up visions for 2 years from now. While to the untrained eye they might seem like completely opposite activities, you should see the trails and traces of threads that connect the two.

You are implementing this feature, because it was a solution to a problem that was presented in the short-term vision. Solving that problem moves the product closer to the “what if” that was painted for the mid-term, which in turn leads to the “someday utopia” presented for the long-term.

case seurat: let’s van gogh with another metaphor

Still not making sense? That’s ok—I’ve got one more metaphor to throw at you. Imagine you’re Cameron Frye, standing in the Art Institute of Chicago, wondering why the worst friend ever, Ferris, is actively destroying your life. You’re staring at a painting. It depicts a scene. Your eyes zoom in. You see a woman’s face. Your eyes keep going. You see nothing but dots.

I’m describing pointillism, an artistic technique in which the artist creates an image from dots of color. It almost certainly describes the saying, “the sum is greater than the whole of its parts.” (Dot matrix printers also come to mind, although that rather dates me.) In addition to all of that, pointillism also the perfectly demonstrates product visions.

View fullsize

At a distance, you can recognize the image. It’s clear, it’s beautiful.

View fullsize

Get a little closer, and you can still make out some detail, but the scope is much smaller.

View fullsize

Even closer? You can’t see the image at all, only the underlying problems.

---

I also cannot believe how hard I just punned

But hopefully it was worth it for you, if not me. Does this track with how you’ve worked on product visions? Are there any other metaphors I should cover in the future? Let me know in the comments!

throw out your spec—meet the problem page

If not a spec, what?

We all know—I don’t love a spec. I don’t like “requirements doc,” and I certainly do not like a waterfall process. That said, we product managers have to do, well, something, right? Probably. One of the key foundational activities of a product manager should be exploring and defending what should be prioritized and why. We can’t just say “no” and expect people to go along with it. That would be much too easy. Instead, we have to justify why certain things are more important than others.

A quick tangential aside on prioritization

One of my pet peeves with some leadership teams is their inability to effectively prioritize. You’ve probably seen this before—a list of 20 company priorities are presented, and 10 of them are P1s, 8 P2s, and 2 P3s. I just want to be clear here… this is not prioritization. This is basically saying “EVERYTHING IS EQUALLY IMPORTANT except for these other things that we will never have enough time to do”

This IS NOT helpful. By definition, you should have more P2s than P1s. P1 is the top of the pyramid. If the top of the pyramid is larger than the bottom, well, first of all, it isn’t much of a pyramid is it? And more importantly, the damned thing is going to topple over. Please don’t crush us with your inability to prioritize, dear leaders.

Enter Problem Pages

Since we know people won’t take our word for it, we need to put pen to proverbial paper and make the case. To do this, I find it very effective to create an internally-facing document for each general set of problems—a problem page. This document should articulate what problem set you’d like to address, for what audience, and why.

While this page should serve up data and facts, its purpose is also to persuade your internal audience that this is, in fact, the thing you should be working on. This is the time to get on your soapbox, to tug at heartstrings. You want a rallying cry that is as persuasive as Daenerys on her best day. You want for someone to read your words and jump up from their desk to CHARGE!

Note: This has never happened to me, but a girl can dream.

Anyway, I love to see this in a wiki format—whether that’s Confluence, Notion, or something similar. It should live in a place that is easy for engineers to find, product people to collaborate on, and internal stakeholders to engage with through comments. This is why I call it a “problem page” and not “problem document that will never change.”

I also want to note that what is covered below only pertains to the first part of the process. This same page will eventually be used to communicate possible solutions, eventual designs, and progress updates when actually in development. But, for the sake of brevity (who, me?) we’ll leave that for a future post.

Before you dig in

By the time you’re writing a problem page, you should already have done a ton of information gathering that led you to the point of creating the page. You shouldn’t need to do explicit research on the what, as that was likely the impetus for prioritizing the problem to begin with.

Before you even start, you should already roughly know the following:

• Who within your customer base is facing this challenge or would benefit from addressing this area

• What the problem is

• How the problem fits into your vision for the product

• How the opportunity fits into your team’s goals

A Word to Newbies

This is one of those things that gets way easier with practice and familiarity with your product area. If you’re new to being a PM or new to this concept, this is gonna suck a little bit. That’s ok. If you’re new to a particularly complex product, this is gonna suck a little bit. I promise you, though, that with time and experience this will be incredibly straight-forward.

If you are new to this concept, I’d highly recommend reading examples that already exist in the world. It might also help to start with a straw man of one of your problem areas, and then pair up with a more experienced PM to flesh it out. Engineers are always pairing up, but it’s just as useful for product managers. Talking these things out have a way of solidifying the reasoning in our heads.

---

So what’s in a Problem Page?

There is a basic, recommended structure to a problem page. The exact headings and sections aren’t worth losing sleep over, but you should make sure the content is there.

---

Overview

Description of Problem or Opportunity

This is an overview of the problem you’d like your team to solve.

It could be:

• A challenge current users are facing

• A capability deficit preventing new customers from adopting your product

• A usability issue preventing customers from finding / using / valuing something in your product

• An opportunity for some tangentially related problem space that isn’t represented in your current product

It should be:

• Phrased in terms of the challenge(s) you hope to tackle

It should NOT be:

• A specific feature

• A description of a solution to the problem

Target Audience

This might seem relatively self-explanatory, but it’s important to make this as broad or as scoped as is needed to really describe who you’re going to solve these problems on behalf of.

Things to consider

• If you have a multi-tiered product, is the problem space most important for a subset of tiers?

• Is there a certain size of customer? A certain captured revenue?

• Is this for existing customers, is it to encourage existing customers to choose a higher tier? Is this to attract new customers?

• Is this meant to impact/help an internal audience? (Very often this might be Support, Fraud, Trust & Safety, Billing, etc) 

Problem Prioritization

To really make a compelling argument about WHY you’re assigning priority to an issue, you may need to go back and gather more detailed information than you  needed when working on the overview section. I highly recommend you have as much explicit data as possible—both qualitative and quantitative.

Customer Impact (Qualitative!)

Get yourself some customer quotes. I’m talking first names of specific people. I want you to be able to write about Andrea who is trying to accomplish this task for her company. It’s a brutally manual process, and she dreads it every month. In the meantime, Wes from this other company is having nearly the exact same challenge.

You’ll want to discuss the FUD (fear, uncertainty, and doubt) of users in this area, and really tug on the heartstrings of your audience.

Supporting Data (Quantitative!)

Collect a variety of quantitative data, which could include any of the following:

• How much this impacts your customer

  • Hours spent

  • Money spent

  • Satisfaction levels

• How this issue impacts growth

  • Blocking deals?

  • Would unlock a market?

  • Contributing to churn?

• How this issue is placing burden on an internal team

  • Billing Team has to do x things manually each month

  • Customer Support is receiving x tickets per week

  • Trust & Safety spends x hours per week addressing this issue

• Don’t forget to quantify qualitative results if you have them!

  • NPS

  • Customer satisfaction

  • Task success

Company Goal Relevance

Time to bring it home. You’ve already tugged on heart strings to show why this is important to your target audience. You’ve queried the data warehouse to prove this is a meaningful pursuit. Now it’s time to relate it back to your team’s goals (which... hopefully roll up to your company’s goals…). How will this move your team toward its goals? How will this help achieve your company’s goals?

Now, this does not have to be a hard company metric type goal. A lot of us get caught up thinking this has to be a number like cohort retention rates or customer acquisition numbers or churn rates, but those are just a subset of the type of goals your team might be pursuing. Your goal might be to increase customer satisfaction, or to increase task success on a particular feature, or even to simply half an answer to a problem area that your competitors already address. (Though, never implement something just because your competitor has it. Perhaps this is a good topic for another post.)

The key here is to make a compelling argument that not only is this problem important to your customers, but it also helps the team move toward its goals.

---

Go forth and write

Let me know if you’ve ever tried something like this, and how it went for you. Do you have an example you’d be willing to share? Any helpful tips to share with your fellow PMs? Comment below! I may pull together some examples for a future post (with permission).

my engineers keep telling me not enough is figured out

How much detail should go in the spec? How much should be figured out before starting dev work? My engineers keep telling me not enough is figured out, so they can't start working on my thing. HALP ME ERIN.

—Krampus

Dear Krampus,

Ahh, this is such a great question, and one that may require several posts to cover. Nevertheless, let me give this a try.

Rage against the spec

I really, honestly, dislike the term and even the concept of a “spec” so much. Typically a spec is structured as a document written entirely by the product manager that lays out all of the product “requirements” for the rest of the team to follow. It walks the engineers through exactly what and how the product should work, and leaves only the implementation details up to the engineering team.

This is a really, really dated concept. It much more closely resembles the typical IT partner process that happens in large companies. I know this because I was that person for 6.5 years. Now, it’s entirely possible IT project management has changed in the 8 years since I left it, but nevertheless is a really helpful comparison for me.

System of a spec

In the spec universe, we ask our customers or business partners what they need. We determine everything that they will need in the foreseeable future, and we write those requirements into a multi-page document. We write out test cases that should cover every possible permutation of actions a user could take. We get THE ONE design to rule them all. And then we hand it over to engineering and ask them how long it’s going to take.

This is what we like to call “waterfall” development. Waterfall has its place—for example, if you’re building aircraft engines, it’s probably for the best that you have long-term plans—but that place is not in a small or midsize software company.

Alternative systems

In the problem universe, we ask our customers what they need. Then we ask them why they need it, and what they’re actually trying to accomplish. We throw out the thing they asked for and instead concentrate on the outcome they desire. We write out those problems and relevant constraints, and we only look at the most important ones. We collaborate with a team of designers and engineers to dream up the best possible solutions to those problems, and we then write those solutions up as user stories.

This is targeted, collaborative, and modular. If something changes, rather than updating a long out-dated document and determining how the change cascades through all of our hard-laid plans, we simply delete one story, and replace it. We simply de-prioritize one story, and move two others above.

The connotation of spec

There are additional reasons to avoid the use of even the word “spec.” Some may call this semantics, but in a world where we are leading and influencing a group of smart, opinionated engineers to pursue a vision, semantics matter. So much of leadership comes down to understanding the nuances is which words you use, and how you use them.

The word spec means, quite literally, specifications. Requirements. Things that should be followed exactly. If you build something to spec, you build something that exactly matches the requirements. There’s typically not a whole lot of creativity involved, or room to come up with something that’s better than the spec. 

Master of solutions

If your engineers are telling you not enough is figured out and you don’t know what needs to be figured out, I’d suggest booking a room for half a day and hashing it out with them. There is definitely an amount of product ideation, story-writing, and technical exploration needed before diving in, and that needs to happen at the team level. The team doesn’t need you to flesh it out yourself, but they might need you to lead them through the process of figuring that out.

Here are some questions to ask yourself:

• Does your team fully understand the problems they’re setting out to solve?

• Have you and your team already drawn up a rough agreed upon solution?

• Has your designer contributed lo-fi designs that support that solution?

• Has your team walked through proposed business logic and identified edge cases?

• Does your team have enough of an idea of the solution to:

  • Research required technologies?

  • Design the database?

  • Design the APIs?

  • Articulate what could be delivered in a phase one?

There’s a good chance that if they’re stuck, you probably haven’t spent enough time with the team addressing these questions. Remember, you don’t have to come up with the solution or edge cases or anything else on your own. But you do have to facilitate the process.

You’re gonna go far, PM

Remember the tools of our trade, and you’re going to be just fine.

• Make sure your tripod is set up

• Get to the heart of the problem & create a problem page

• Have a sketch session with your team to collaborate on a solution

• Have follow-up sessions to flesh out core product logic

• Write great stories

You’ve got this, Krampus.
Erin

the holy tripod

tl;dr

1. At a minimum, a Lead PM, a Lead Designer, and a Lead Engineer should form a tripod that leads the product engineering team

2. The tripod should meet to establish agreed-upon responsibilities and communication patterns, starting from the generally accepted roles

3. No member of the tripod has the final say on everything, they’re like 3 co-equal branches of product government

4. Product is a team sport of solving a Rubik’s cube—you can’t change the constraints, so use prioritization and negotiate with one another in a collaborative manner.

---

Holy Tripod, Batman!

I couldn’t resist. But in all seriousness, the product tripod is a staple in my product belief system (PBS). I believe deeply that every product engineering team should have a tripod at its helm. Hopefully, by the end of this blog post, you’ll agree with me. Although why you wouldn’t just blindly follow me I’ll never understand. 🤷🏻‍♀️

Introducing: The Tripod

What are the key roles?

• Product Manager / Lead PM

• Product Designer / Lead Designer

• Engineering Manager / Team Lead

Why is it so important?

Successful products aren’t typically created from a single person’s mind, but rather through the art of collaboration between a diverse group of people. Even if your organization isn’t super, ahem, diverse (we’ll address that in a different post—I have thoughts y’all), you can achieve a degree of diversity of thought simply by ensuring these three roles are collectively leading the team.

Tripod+

Sometimes a product eng team tripod has even more legs than three (I’ve been known to call it a fivepod) if you’re one of the lucky teams to have access to one (or all) of these roles:

• Data Scientist

• User Researcher

• Program Manager

• QA Lead

It’s fairly common for these roles to either be non-existent in smaller companies, or to be shared among several product engineering teams. Depending on the demands on that person, you may find them being just as involved as the core tripod members, or you may need to prioritize the most important meetings and interactions.

The Tripod and Other Metaphors

Why do we call it a tripod?

What happens when you remove one leg of a tripod? Barring some sort of shenanigans, typically a tripod that becomes a bipod is a… laying-on-its-side-pod. A tripod cannot stand if any of its legs are out of commision. Such is the physics of a product engineering team. It requires a complete tripod to stand on its own three feet and perform to its full potential.

Is it a dictatorship, oligarchy, or democracy? (Dictatorship, right? The PM is clearly the dictator, right?)

Nope.

The tripod is both none of these and all of these. It both listens to / represents its people, but it is not a majority rule system.. It is both ruled by a group of people and gives individuals the power to make final decisions.

If you are working with a PM who’s acting like a dictator, they’re doing it wrong. Please refer them to me. If you are a PM and you’re acting like a dictator, please stop. And also come see me.

But I really, really want a political metaphor...

A much better metaphor is the system of checks and balances most Americans will be familiar with as represented by our three co-equal branches of government. A group of three working toward the greater whole, but with specific, individual areas of responsibility.

Roles & Responsibilities

Why are they doing what is clearly MY job?

The responsibilities of the tripod overlap by design. This can often cause some consternation—someone’s always convinced someone else is usurping their job—but it’s important to understand the primary roles of each member of the tripod as well as the overlapping contributions they may make.

Things I’ve thought or said out loud (to people not in my tripod):

“I just feel like he’s trying to do my job.”

“But I’m supposed to run the sketch sessions!”

“I just need us to get this thing done… AGGGHHHHH”

“But I can’t let them design a solution without my participation!”

Yeah... don’t be me.

What are the definitely locked down and set in stone responsibilities?

Not so fast, rule-follower! While I like to believe stubbornly that these roles are super widely defined and acknowledged, the truth is we are all unique humans. (I’m here for the hot takes.) While there is a general guideline for who is responsible for what, each of us brings certain strengths, interests, and areas of expertise to our tripods.

The lines between each role are bendy… flexible, if you will. They can flex to conform to your unique tripod. The only absolute constant is that you must collectively discuss and define your roles when forming a new tripod. Like any relationship, communication is everything.

What are the more-or-less generally accepted hand-wavey responsibilities?

Product Lead

• The expert in target customer needs/problems/opportunities

• The main representative of company goals back into the team

• The main representative of the tripod out to the rest of the company and customers

• The final decision-maker on overall priorities / and or scoping decisions as a result of collaborating with the other leads

Design Lead

• The expert in user experience personas for the customer base

• The owner of the overall user experience and design aesthetic for the product

• The main representative of company design systems back into the team

• The final decision-maker on implemented designs

Engineering Lead

• The expert in the technical lay-of-the-land for the product

• The main representative of technical capabilities, constraints, company maintenance, and technical debt surrounding the product area

• The final decision-maker on resourcing and structuring implementation within the product engineering team

It’s easy to say the designer has the final decision on designs and the engineering lead has the final decision on timeline and implementation and the lead PM has the final decision on scope, but it’s much more complicated to piece this together in practice.

Tripod Decision Making as a Rubiks Cube (because we needed another metaphor)

Look, this stuff isn’t simple, so I’m just putting it all out there. 

Here’s a common situation that comes up time and time again in product:

-------------------------------------

DESIGN LEAD: Here’s the best possible
user experience

ENG LEAD: Cool, that’ll take us about
6 months.

LEAD PM: Nope. Try again.

DESIGN LEAD: Ok fine we could
cut this piece or phase it in later.

ENG LEAD: That doesn’t really help
us much… PM?

LEAD PM: We absolutely have to have this
piece and this piece, but that piece we could
probably punt on. What do you both think?

[debate for awhile, and comes to shared conclusions]

DESIGN LEAD: Cool here’s a new
design.

ENG LEAD: Would you be open to
changing this design element in this
way? It would save us a ton of effort.

DESIGN LEAD: Ahh, yeah here’s a
revised design.

ENG LEAD: Cool.

LEAD PM: Cool.

DESIGN LEAD: Cool.

ENGINEERS: WAAAAAHHHHHH.
(jk jk)

-------------------------------------

The Rubiks cube represents all of the constraints that the team is working with, and you, as the tripod, have to solve the puzzle together, as a team.

Imagine that your cube has these six sides:

1. Problems to be solved

2. Resource constraints

3. User experience standards

4. Technical constraints

5. Time constraints

6. Prioritization

Now, you may have noticed that “prioritization” seems like a bogus side, and you would be correct. It’s actually a placeholder side. Constraints are unchangeable, but using prioritization we can, in fact, move constraints around to find the optimal product.

Doesn’t fit into a totally important (read: arbitrary) timeline your company has? Ok, turn some rows. Now it fits in the timeline but doesn’t have an acceptable user experience? Great, adjust some columns. Keep working that cube as a team until you’ve got it solved!

The responsibilities of The Tripod

Obviously each member has their own responsibilities within the tripod, but the tripod itself has responsibilities as well.

Ok, what do they do?

• Represent a shared front to the engineers on the team

• Discuss and debate team priorities on a regular basis

• Discuss team performance, concerns, and develop plans for how to address any issues

• Cover for one another in case of illness / emergency / babies coming early / whatever

• When in doubt, unblock the engineers

• Collectively take ownership of the goals and results of the team

• Challenge each other in a constructive way

How do we do that?

You may find that you need to meet as a tripod on a structure basis (once a week, once a sprint, whatever) in order to stay aligned with one another. Alternatively, you may find that simply working next to each other and chatting in ad-hoc ways may work for you. Either way, it should be discussed and explicitly chosen.

your PM Interview take-home is measuring all the wrong things

The infamous “Product Manager Take-Home Assignment” has gotten on my last nerve. I’ve been asked to do them in the past, and my clients now are often tasked with them as well. Every time I look at one, I’m appalled at what I see—because what’s being asked is not remotely what it’s like to be a PM in real life.

Why you should ditch your PM take-home assignment

1. It doesn’t even remotely simulate what it’s like to be a product manager

   Sure, the take-home is testing something, but it definitely isn’t testing your day-to-day product management skills.
   

2. You may inadvertently be measuring how much time the candidate has, not how qualified the candidate is.

   Product managers are high in demand, and often are interviewing at more than one place at a time. With these assignments, interviewing itself can be a full-time job.

   You may miss-out on great candidates simply because they don’t have the time to do homework. Maybe because they already have jobs. Maybe because they have families to take care of. No matter how you look at this practice, it is not inclusive. So stop it already.
   

3. The first step of being a great product manager is to know your customer.

   Without that knowledge, no exercise in the world can make use of a PM’s best asset. Instead, you’re testing a PMs ability to make assumptions about things they have little to no context for.
   

4. Your candidates are not unpaid interns.

   Homework assignments that involve “real-world examples” from your company are nothing more than free work. Have some respect.
   

5. A poorly-structured take-home assignment can actually reflect badly on your product organization.

   Why would I want to work at a place that doesn’t fundamentally understand what my role is?

Determined to have a take-home assignment? Here’s how to have a decent one

Do not make your candidates make assumptions about the customers.

• If you’re going to give a take-home, you’d better include a treasure trove of (anonymized) customer quotes.

• Don’t be stingy—to simulate real life for a PM, they must have access to your knowledge about your customer.

• If you can’t share qualitative or quantitative data for some reason, make it up. Make sure every candidate gets the same packet of info.

Do not make your candidates make assumptions about the goals of your company.

• Contrary to (apparent) popular belief, individual contributor (IC) product managers are not responsible for setting high-level company goals

• While PMs may help define company strategy, it’s critical to give them a little top-down guidance to aim them in the right direction

• If you don’t know what the goals of your company are, perhaps you are not ready to hire an IC product manager.

Do not ask a product manager to develop a go to market

• A go to market strategy is not the sole responsibility of a product manager

• A product manager certainly contributes to this strategy, but relies heavily on other roles (like marketing) to do this

• Also… asking anyone to do a go to market in a take home assignment is absolutely inane. Sounds like a great way to have a candidate spin themselves in circles.

Frame the assignment around identifying key customer problems

• If a PM knows some customer information and understands the high-level goals of a company, they should be able to dig through said information and identify the main problems that both address customer needs and work toward company goals

• They should be able to articulate:

  • The top pain-points of the customer base

  • Their recommended prioritization (without any additional inputs)

  • Next steps for prioritization (e.g. how technical limitations, timelines, etc. might alter their recommendations and prioritization)

• Under no circumstances should the assignment be around designing a solution

Wait, let me say that again.

Under no circumstances should the assignment be around designing a solution

“WHY?” you ask. “I WANT TEH WIREFRAMES,” you say. Frankly, my dear, I don’t give a damn. Product managers should not design solutions in isolation. Ever. Ever ever*.

SAY IT WITH ME: Product managers should not design solutions in isolation.

*Exception: You’re a 10-woman startup and everybody’s gotta be scrappy. Ok fine. Move along. This blog in general is not for you.

But back to not assigning a take-home

Behavioral-based interviewing techniques are a gold-standard for a reason. Past performance indicates future behavior. And so goes product. You’ll get a much better sense of how a product manager works  and thinks by having them walk you through a past project or two.

Stop wasting your time and the candidates’ time—and stop assigning take-homes.

---

Are you trying to hire a product manager? Want some help
structuring your interviews? Reach out for some expert assistance.

---

neapolitan sundae: the three archetypes of product management

Have you ever had a conversation with a product manager where you felt like you were just missing each other completely? That you did product one way, and they did product another way? You both cited articles and sources and were confident you were thinking about product The Right Way™️.

The truth is, while our titles may be the same, fundamentally different types of PMs are needed to keep our product-world running. And they all do product, well, a little differently.

The Three Archetypes

View fullsize

I like to divide the world of product managers into these three basic types:

1. customer oriented

2. consumer oriented

3. optimization oriented

---

FOCUSED ON

needs & challenges of existing customers

PRODUCT ORIENTATION

long-term and strategic

DATA PREFERENCES

qualitative feedback from user research over metrics-driven

---

FOCUSED ON

usage at scale, such as daily active users

PRODUCT ORIENTATION

shorter-term, trend-based

DATA PREFERENCES

quantitative data from a mix of metrics and user research

---

FOCUSED ON

growth of platform / usage / conversion

PRODUCT ORIENTATION

optimizing existing flows & funnels

DATA PREFERENCES

quantitative, metrics-based data over research or qualitative data

---

B2who?

There’s yet another lens to add into the mix: your company’s audience. The style of product management at your company or in your organization is directly correlated to the type of audience you serve. Are your customers using your product for fun, or to get their job done? Is the goal of your product to keep your users around longer and more frequently, or to fill a need on a regular basis?

B2C Archetypes

If your audience is mainly people who use your product (or feature area) for fun, for leisure, or to manage something about their personal lives, you’re probably in a B2C (business to consumer) world. PMs who have a consumer oriented mindset are going to be the most successful in this environment, alongside optimization oriented PMs.

B2B Archetypes

If your audience consists of people trying to do their jobs or trying to run their business, you’re probably in a B2B (business to business) world. PMs who work in a customer oriented mindset will thrive in this type of environment. Optimization oriented PMs are also often needed in a B2B context, but to a lesser extent than those in a B2C context.

Double check your work

Note: You don’t have to provide software to corporations or enterprises to fall into a B2B style of product. When I first started working at Patreon, for example, the generally-accepted categorization was that we were a consumer product. However, Patreon is for creators—creators who are trying to run a business, that is. In actuality, my job in particular fit much more into a B2B world.

When assessing whether you are in B2B or B2C context, don’t use how the brand feels as your barometer. It can lead you astray.

paying off tech debt -OR- my PM is scary

I’m an engineer, and I am told to work on “paying off tech debt”,  but I think if I do that it’s going to make my product manager mad. @eboyle what do I do?

Dear Engineer,

My advice is to get a new PM. Nah, just kidding. (A little. Mostly. Kind of.)

Questions like these are hard, because they’re not directly within your control. I do believe—quite strongly—that no product manager worth their salt will ever flat-out reject talking about the work you need to do to clean up tech debt. A good product manager will ask questions, and work with you (or your engineering manager) to determine how to layer tech debt work into ongoing product work.

Weighing the options

Here are some questions I would ask as a PM:

1. What’s the scope of tech debt you plan to address?

2. How urgent is this debt? If we don’t do something, will we have problems in 2 weeks? 2 months? A year?

3. Does this tech debt directly impact any of the product areas we have plans to touch?

4. How does this stack against other tech debt within our domain?

5. How might this impact our existing goals? Will we not be able to address [problem y] if we pay down [tech debt x]?

6. Can we divide this work up into smaller chunks?

Now, note that I don’t ask any of these questions because I don’t think paying down tech debt in important. But rather, I think it’s just as important as product feature work, and prioritizing feature work goes through a similar gauntlet of questions. The main difference is that you, the engineer, know way more about this problem set than me, the PM. So be a little patient and get me up to speed.

At this point, it really becomes a joint decision between the engineering and product managers. Both are responsible for weighing the trade-offs and coming to a reasonable compromise.

Build tech debt pay-down into your team processes

Every product or product area is unique when it comes to how much time needs to be spent working on features, updating designs ( design debt is real, y’all), or paying down tech debt. A healthy product tripod (engineering manager + product manager + lead designer) should decide on the default weight of tech debt for their ownership area. 

For example, if your product is fairly healthy and has a steady (but not overly aggressive) feature release cycle, a good rule of thumb is to dedicate 20% of engineering resources to paying down tech debt. This might mean that you always have one or two engineers working on tech debt in a rotation, or it might mean everyone is responsible for spending two days a sprint paying down tech debt.

A product area that isn’t seeing a lot of new product development, on the other hand, may have a higher rate of tech debt management. A feature in startup growth mode? Probably not going to be working on much tech debt.

Ok but I have a leaning tower of Pisa-code

Are things… leaning? Does it feel like you’re playing a very technical game of Jenga? Are you worried that working on the next feature will start crumbling the whole damned thing? Cool, 20% is probably not going to work for you. This is a perfect time for the tripod to reassess priorities and put the majority of focus on paying down the debt before it becomes a 4-alarm fire.

Please don’t let it become a 4-alarm fire. Don’t let your PM let it become a 4-alarm fire. But also be careful of how many times you play that card. If you treat everything as urgent, there’s a good chance you’ll start losing credibility with the PM. If it can honestly wait a year, say that. If it could really use an overhaul but really only this one piece needs to be done soon, say that. If it’s going to hell in a hand basket, for heaven’s sake say that too.

What even is tech debt?

Spoiler: almost everything in this post are questions I ask when I’m interviewing engineering manager candidates. I worry less about the answers, and more about how much thought has gone into tech debt as a concept, how much experience has gone into processes, and whether those views are nuanced. Let’s be honest, this is all just a pile of nuance.

So, here’s my take on what counts as tech debt. But again… nuance. Hand wavey. Also some things may fit into more than one of these categories.

• Backend performance issues (think: long-running queries, database design issues, capacity issues)

• Frontend performance issues (think: insanely large images, network chattiness, non-incremental loading)

• Spaghetti code (we didn’t know where this was going to go when we first built it, but now we do and this is only going to get worse)

• “Building feature xyz will take us 10 years and a million bribes in the form of cookies if we don’t redo this” things

• “I-don’t-understand-but-trust-that-you-do” things

• “Starting to address this now will prepare us for that thing we wanna do next year” things

• Bugs. Yes. It’s true. Bugs happen. WE WILL SMASH THEM. But this is not, in the classic sense, tech debt.

You, too, can own a backlog

In my (in)famous guide to writing stories, I decree the following:

Thou shalt have all stories, chores, and bugs related to a product initiative in a single backlog

Exception: Tech debt or tech-only projects may have a separate technical backlog

This is probably a great time to talk about that exception. I strongly advocate for having a backlog dedicated to non-user-facing-bug tech debt. There’s already enough ordered chaos in the main backlog—there’s really no need to complicated matters further by adding in tech debt stories that I (or your product manager) may or may not understand—and I say this as a fairly technical PM.

There are several key advantages to having a separate tech debt backlog, other than saving my eyeballs from confusing stories:

• The engineers can manage this backlog independent from product and design

• Having a single list of tech debt allows the engineering manager and engineering team to prioritize tech debt amongst other tech debt

• Whether everyone spends a percentage of the sprint working on tech debt, or whether there’s a rotating spot on the team, everyone has a single prioritized place to know what to work on next

• Having actual stories can help keep engineers on task / prevent y’all from going down a rabbit hole of code optimization that I know you all fantasize about. I know you. Do not doubt me.

• Having actual stories means engineers can pull them into a sprint, so that there’s no magical work happening that’s not being tracked against your velocity

But Erin, I thought tech debt should not be a thing that’s estimated!

Did I say that? I actually don’t think I said that, although… maybe I should’ve.

Honestly, whether to point or not to point tech debt, that is the question for your whole scrum team. Many teams decide to not point tech debt. Some teams decide to point-only-as-an-effort-box tech debt stories. It kind of doesn’t matter to me. As long as you pick one method and stick with it, it all can work.

No matter the method and whether there are points on tech debt stories, dragging them into your sprint will help your PM understand what you’re working on and where your time is going.

NOT BECAUSE THEY WANT TO MICRO-MANAGE YOU.

Seriously we have enough to do. We have no interest in micro-managing you. But we often have to set expectations and communicate progress to stakeholders and partners within the company. The more we can do that without having to bug you about what you’re doing, the happier we both are.

It’s ok to make your PM a little mad. Sometimes.

A product-engineering team is a lot like a system of checks and balances. It can feel like PMs are the executive, legislative, and judicial branches all in one, but it’s not actually true. Healthy teams should have disagreements. That doesn’t mean you don’t treat each other with respect… you absolutely treat each other as humans. And you also should push each other. Our strength as a team comes from being able to challenge one another because we all have different backgrounds, strengths, and perspectives.

when it comes to Product managers being mad… it’s all relative.

tl;dr

So, that’s what I’d do. Make sure that you, as a team, talk about how to handle ongoing tech debt pay-down vs one-off or project-level pay-down. Be prepared to answer your PM’s questions about what you’re proposing. Prioritize tech debt within a backlog. And, you know… irritate your PM once in awhile. Tell them I gave you permission.

Until next time,
eboyle

---

Curious about something
I haven’t covered?

Just Ask

lightweight QA

You’re a PM at a startup, you’ve got more engineers that you can shake a finger at. The backlog is hangry. It needs more calories story points. Your deploys are continuous and—if you’re lucky—you might have time to test that what’s being shipped meets its acceptance criteria. You receive word that your feature doesn’t work on IE [insert incredibly old version here]. There are questions as to how it got past QA. QA? What QA? You are QA.

Take a breath. Product is a TEAM SPORT. No, really… it is. You might be acting QA right now, but the truth is your whole product engineering team should be acting QA. The good news is, with a little planning, you can have a pretty lightweight QA process that will cover your biggest needs.

---

Oh god do I need a test plan

Anyone who’s ever tried to develop an extensive test plan knows the following:

• It’s exhausting

• It’s not for the faint of heart

• It’s nearly impossible

• You only ever get 80% completed constructing a test plan (Seriously, it’s like infinite divisibility. Zeno’s paradox. That last 20% just keeps going on and on and on and on and…)

• If you’re asked to make a test plan, run away. RUN AWAY

Good news, fam. YOU DON’T NEED an exhaustive test plan. As it turns out, exhaustive test plans pair with waterfall software development rather well (as well as a test plan can go), but just don’t make any sense in our agile development world. Besides, that’s not how Google does it.

---

How does Google do it?

To be honest I have no idea… I’ve never worked at Google. But according to the interwebs—that are definitely always right—they use the ACC Framework. Attributes, components, and capabilities. I was first introduced to this by Heather Wells during my time at Zendesk, and it really resonated with me.

Here are a bunch of people who talk about it:

• https://testing.googleblog.com/2011/09/10-minute-test-plan.html (the blog post written by the OG)

• https://www.amazon.com/gp/product/B007MQLMF2/ref=dbs_a_def_rwt_hsch_vapi_tkin_p1_i0 (the book written by the OG)

• https://99tests.com/blog/log-bugs-google-way-acc-methodology/ 

---

Directed testing using the ACC Framework

Here’s the tl;dr of ACCs.

Attributes (think: adjectives & adverbs)

Attributes are the descriptors  essential to your product’s brand and value promise. Attributes for payment software like Venmo could include accurate, secure, and timely, whereas attributes for social products like Instagram might include easy, engaging, and enticing. Pardon the alliteration.

Different parts of a single product can absolutely have different attributes. If I think back to the Zendesk Support product, I can easily assume that workflow tools and the agent interface will have pretty different attributes, but both might include Zendesk’s mantra of “beautifully simple.”

Components (think: nouns)

Components are the basic building blocks of your product or feature. The amino acids to our product proteins, if you will. (Who would though. Honestly, Erin.) Components are not as small as something that could be described as a single user story, but rather the size of  a group of user stories or even a lower-level epic.

Let’s look at this blog for an example. Yes—the very one you’re reading right now! Here are the components I spot:

• Post body

• Share options

• Comments area

• Comment compose box

• Paginator

• Menu bar [which would itself be broken into its own set of ACCs]

Capabilities (think: verbs)

Finally, capabilities are the things that components should do. What are the things this product or feature must be able to do to fulfill its promise to users? My blog must provide these capabilities:

• A viewer can read a post

• A viewer can react to a post

• A viewer can share a post

• A viewer can navigate to the previous entry

• A viewer can navigate to the next entry

• A viewer can comment on a post

• A viewer can view other comments

• A viewer can like other comments

The ACC matrix

The final step is mapping which capabilities belong to which component and to be tested against which attributes. It makes more sense visually:

---

On directed vs exploratory testing

The ACC framework is a perfect example of directed testing. Directed testing is simply making sure products work in their expected manner in the most standard configurations. Basically, does it do the thing it’s supposed to do when it’s expected to do it?

But there’s a whole other piece of QA that is super easy to miss when we’re feeding the backlog and testing the acceptance criteria and going through the ACCs and pitching the vision to the company and sitting in on a sales call and talking to customers and… you get the point. Exploratory testing is literally undirected testing. Put your product in front of a 3-year old and watch them get it twisted into a knot of unexpected states. The entire point of exploratory testing is to find the things that we didn’t even realize people might do because it’s so not what we meant to happen.

When logging bugs or “wait what’s supposed to happen here”s found during exploratory testing, do make sure you log them against the components you identified in the ACCs.

---

None of this excuses engineers from writing good unit tests

Say it with me: “QA is not a substitute for unit and integration tests.” That’s right. QA is necessary but not sufficient, just as unit tests alone are necessary but not sufficient. Unit tests answer the question, “Does the code do what it’s supposed to do?” whereas QA testing answers the question, “Does the product work as expected and handle unexpected things gracefully?”

---

Other things you should document and use during QA

Common user states

All products have weird quirks that result from the built it fast culture of an earlier startup. Often there are weird dual states from “legacy” features, and there can also a lot of intentionally built-in complexities from various plans and pricing models. Document the ones that come up over and over!

At Zendesk, user states would include each base plan, as well as (potentially) any add-on combinations. For example, an admin on the Enterprise plan with the Collaboration add-on may see some different states than an admin on the Enterprise plan, who might see a different state that an admin on the Team plan.

At Patreon, user states would include the creator’s chosen plan as well as their payment model. For example, a creator on the Lite plan with a per post payment model will see some different states than a creator on the Pro plan with a monthly payment model.

Supported browsers

Yep, it’s true. If you’re dealing with web development in any way, this is a fact of life. Find out what browsers you support, and document those in your test plan as well. You’ll want to test on the major versions.

If you don’t know what browsers you support, perhaps because no one’s ever asked the question and no one’s ever decided it, find a nearby engineering manager and lovingly hand that problem off to them. (Thanks, Derek!)

That’s all I’m gonna say about that.

---

Real talk

I am not an expert on QA, and I’ve never played a QA engineer on tv. So if you’re serious about wanting to go down this path, I’d highly recommend reading some of the resources linked above, going down a Google rabbit hole yourself, or hiring an expert (like Heather). What I do know is it’s damned hard to get it all done when you’re a PM at a startup. You may not have the power to hire a QA engineer, and that’s ok. Do yourself a favor and empower your team to develop ACCs and run QA alongside you.

kaizen, agile, scrum

I started my career as an IT Project Manager at GE Aviation (called Aircraft Engines at the time) in Cincinnati, Ohio. I was thrilled to land such a great job straight out of college, and started in one of their famous leadership programs. Despite this sounding obvious now, GE did an exceptional job at preparing me for my career ahead in ways I’ve only recently come to really appreciate.

A key part of my job at GE was to be a technology partner to various business organizations in the company. I spent time with Sales & Marketing, Services, Engineering, and the Technology Services Group—what I liked to call IT for IT. Our business partners walked us through their incredibly complicated jobs, and we would work with them to identify and implement improvements to the way they worked. It wasn’t always quite problem-first—I didn’t know!—but it was startlingly similar to product management nonetheless.

Learning from the greats

Kaizen

Over the years, well before I arrived, GE spent a lot of time studying Toyota’s methods of operating. If you’ve never heard about Toyota’s famed Kaizen method, you should really spend some time and read up on it. Literally, just google “Toyota kaizen” and a veritable shit-ton of information will pop up. Kaizen simply means continuous improvement.

GE (and, specifically, one-time CEO Jack Welch) saw the incredible benefits Toyota reaped from Kaizen, and folded its concepts inextricably into GE’s cultural fabric. You cannot work at GE without hearing about Kaizen and Lean, just as you cannot work in tech without hearing about disruption. (Oddly, one concept is much more valuable than the other.)

Seeing is believing

I was lucky enough to have an opportunity to tour a Toyota manufacturing floor in Kentucky not once, but twice with other GE employees. It was absolutely fascinating—I loved being on the floor and seeing both the results of Kaizen, and the actual in-process practice of Kaizen. Factory employees are empowered to identify things that are making their job more difficult and be a part of their solutions. I distinctly remember watching a literal robot that carried tools to workers, using a magnet to follow a metal line etched through the floor. That started with the workers. That was Kaizen.

Lean before digitize

While its applications to a manufacturing floor are fairly obvious, I did not work on the floor next to the enormous jet engines. But GE had figured out how to weave the concept into its technology organizations as well.

Lean before digitize

As technology partners, we were often engaged to take a complicated, manual process and build (or buy) a tool to enforce and track that process. As any IT employee knows, we were often challenged to do more with less every year with every new budget. It was therefore our job to eliminate as much waste as possible from our projects. The number one way to eliminate waste? Remove waste from a manual process before building a digital tool to codify it.

Tame that process

I watched a lot of leaders, Six Sigma Blackbelts, and Master Blackbelts facilitate process mapping sessions. They asked critical questions, like… is this approval step necessary? What’s the goal of step x? If we’re gathering that data, what are we doing with it? And, over time, I got pretty good at doing the same. I’d be met with a process, and I’d tear it apart and rebuild it to be the leanest form of itself that still met the business needs.

Get to the point, Erin

I digress. This is not about my ability to tear apart and rebuild a process anew (although I find that stupidly fun). Rather, it’s about learning from old concepts and integrating them into the way we work in this age of technology.

Agile, Scrum, and Kaizen

We, the people of Tech, are enamored with buzzwords like Agile and Scrum. And yet, we fail to see the direct parallels to industries who have been using Kaizen and Lean for decades. Agile is to continuous delivery what Kaizen is to continuous improvement. And Scrum is the join table that binds them together.

Ehh? ehh? I mean… it made sense to me…

Scrum has been widely adopted in software engineering organizations as the process for building products, though ironically it’s often used alongside waterfall methods of product delivery, rather than Agile. Often, though, I walk into organizations that know little about Scrum beyond stand ups, or are worried about adding “too much” process.

Process is not the enemy

But process itself is not the enemy. Stagnant processes—processes that aren’t continuously improving over time—are. Scrum does not prescribe a specific process, rather it gives you the scaffolding of a starting template and the tools to tailor it to your organization.

Just as Toyota empowers its people to call out inefficiencies and solve them, we should be empowering our product engineering teams to do the exact same thing. Are we having a meeting weekly that adds no value? Speak up! Let’s figure out what the original purpose was, and then determine whether it’s still needed. Scrum is just as much about reducing process as it is about adding it.

Designing a process

Let’s say you get it, and your team is ready to embrace process! Where to begin? Well, luckily, process improvement is remarkably similar to product management. Start with some basic questions:

1. What problem is this process (or step) trying to solve?

2. Who is the audience?

3. What’s the current user experience?

4. What are the unintended consequences of the current process / step?

5. What are our constraints?

6. What are possible solutions to the problem this process / step is trying to solve?

7. Which is the best solution for the problem and user experience?

We’re constantly improving our products, and we should be constantly improving our processes. We have the skills, our people have the drive—they just need the freedom.

Unleash continuous improvement. Embrace Kaizen.

your mission

We tend to think of PMs as jacks of all trades… of great product idea people… of having constant meetings and never-ending to-do lists. And yeah… ok so that’s true… but that’s not the most important part of the job.  Even if you’re starting off as a PM and you know all the things you’re supposed to do, you still might be overwhelmed by the sheer volume. How do you know how to prioritize all of these things that seem equally important? It’s a lot.

But, it’s actually pretty simple if you pare it down to the basic things you, as a PM, should be doing.

1. Know your Customer

2. Motivate & Inspire your Team

3. Empower your Team

4. Remove Obstacles for your Team

5. Know your Customer

Know Your Customer

I’m not going to spend a lot of time here. That doesn’t mean, however, that you shouldn’t spend a lot of time doing this.

You must talk to customers.

When you’re talking about a problem you’re trying to solve to, well, pretty much anyone, you should be able to name the humans you’ve spoken to that have experienced that problem. Without any prep. You should be able to say, “Oh, yeah I was just talking about this exact problem with Hillary from [famous person’s] team!” or, “Wes described his experience with [feature y] last week on the forums.”

Good politicians do this all the time. Candidates are always talking about [insert first name] from [insert city or state] and their challenges. They use these stories to show that they are in touch with real people and real problems. They use these stories to inspire their teams to help them canvas. These constituent stories are nothing more that customer stories. You, as a PM, should talk about your customer stories just as effortlessly as a candidate running for office.

Motivate & Inspire Your Team

All good PMs should be systematically trying to work themselves out of a job.

Seriously. Great PMs should use every moment as an opportunity to inspire their team. They should be so passionate about the problem their team needs to solve that the team cannot help but care about the problem. The motivation should spread like a god-damned virus. You should hear engineers talking to each other in the hallways about the problems your users are facing.

What do you think the productivity differential is between an engineer who cares about the problem they’re solving and an engineer that is simply writing code to a spec?

I don’t know and, to be honest, I don’t particularly care, because it’s enough for me that humans are enjoying their jobs. Then again we’re PMs, and no matter how human-focused we want to be, at the end of the day there is a company who needs us to get shit done. It’s a rhetorical question. Obviously motivated engineers are way better for a company, as long as you do not exploit that motivation*.

*An aside about Caring and Exploitation

That last bit is important. You want your engineers to be productive while they’re coding… you don’t necessarily want them coding more or for longer hours. And if you find that that’s happening, go stop it. In general, your engineers are probably only coding for maybe 3 hours a day (depending on the day). You should NOT see engineers coding all day every day, because they should also be doing all of this:

• Thinking deeply about the problem they’ve solving

• Thinking deeply about technical design

• Doing code reviews

• Interviewing to help your company grow

• Cross-pollinating with other engineering teams

• Pairing with other engineers

• Taking a walk or grabbing a coffee to let their brains marinate

• Prioritizing technical debt

• Testing the stuff they build

• Shit I don’t know… I’m not an engineer. But like… they’ve got stuff to do. Respect it and Get out of their way.

Anyway. Thou shalt not exploit caring.

Empower Your Team

Over time, your product-engineering team will develop its own ability to practically PM itself. A team that works together consistently over time on particular problem spaces builds up domain knowledge in those areas.

• Teams become more familiar with the code base for the product areas they work on

• Teams begin to see and understand trends in the problems they solve for their users

• Teams experience user feedback to the products they ship

A team like this has hit their stride, and all of that “slowing down” from being a part of the solution process is paying off. A PM for an empowered team will still prioritize problems and present those problems. But when it comes to defining a solution, they will simply become a resource.

An empowered team doesn’t even need a PM in the room to work on a solution.

The engineers and designers know enough and have enough experience in the domain to make a kickass product.

Remove Obstacles for Your Team

So you’ve lead a team and they’re empowered and cranking out wonderful products. Don’t get too complacent, because your job has now shifted. Your primary responsibilities are:

1. Unblock your team

2. Continue talking to customers

3. Continue iterating on the long-term vision

4. Unblock your damned team

Occasionally you may onboard a new face to your team, and during some dark times you may still need to inspire or give a pep talk or two. Your main, day-to-day responsibilities, however, will mostly be making sure the train stays on the rails.

Often PMs are inundated with emails or Slack notifications or whatever the current hotness in productivity is. No matter the tool, prioritize your inbox/notifications/whatever to focus on unblocking your teams first. Is there a comment or question on a JIRA story or github PR? First order of business. Do all the support tickets assigned to your team have appropriate replication steps? Is there a backlog of at least two weeks’ worth of stories written and prepped?

Unblock your team first thing in the morning before you do anything else.

That’s your mission. Choose to accept it.

That’s it. That’s the job. Know your customer, inspire your team, empower your team, then get (everything) out of their way.

Glamorous? Nah. Satisfying to sit back and watch the team do its thing? You’re god-damned right.

eboyle's guide to writing user stories

Rules:

• Thou shalt have a shared backlog with all engineers on your pod

• Thou shalt have all stories, chores, and bugs related to a product initiative in a single backlog

  • Exception: Tech debt or tech-only projects may have a separate technical backlog

---

STORIES

What does a story look like?

Name

As a [persona] doing [x], I can [y]

Description

Give a bit of context of why this story is important to the persona. If this is a part of a large set of stories, reference where it fits into the larger picture

UX/UI

[link to invision]

embedded image when possible

Acceptance Criteria

1. When x is clicked, y happens

2. Edge case expected behavior

What belongs in Acceptance Criteria?

I mean. This isn’t gospel… but this is the law according to eboyle: Any action or interface element that needs to be present, conditionally present, or change through the course of the story should have at least one acceptance criteria. Basically if an engineer is going to have to know the answer in order to build it, it should probably be in there.

Let’s take a look at some examples:

EXAMPLE 1:

As a Creator viewing PRM, I can filter by charge status (because this helps me do x)

Description

We want to add a new drop-down filter! The filter's label should be "Charge status," and it should list the "friendly" charge status labels (paid, pending, refunded, declined, deleted, fraud, other).

UX/UI

[link to invision]

Acceptance Criteria

1. A new single-choice drop-down filter is present to the right of the existing filters

2. The default option is "All charge statuses"

3. When a charge status is selected, the list should refresh automatically

4. If "Reset" is clicked, the charge status filter should clear

EXAMPLE 2:

As a Creator viewing PRM, I can sort the list by pledge value (because this helps me do x)

Description

By default, the list of Patrons is sorted in alphabetical order. We want to add the option to filter by a few other columns as well. In this story, we'll let creators sort by the current pledge value of patrons.

UX/UI

[link to invision]

Acceptance Criteria

1. When a creator mouses over the "Pledge" column header, the cursor becomes the hand (shows it's clickable)

2. When a creator clicks on the "Pledge" column header, and no previous sort was set on Pledge, the list is sorted by pledge value ascending

3. When a creator clicks on the "Pledge" column header, and the list was already sorted by Pledge asc, the list becomes sorted by pledge value desc

4. When a creator clicks on the "Pledge" column header, and the list was already sorted by Pledge desc, the list becomes sorted by pledge value asc

5. When the list is being sorted by Pledge asc, an upward arrow is displayed to the right of the column header

6. When the list is being sorted by Pledge desc, a downward arrow is displayed to the right of the column header

7. When the list is no longer sorted by Pledge, the arrow is removed

8. Upon sort, you go back to page 1

What does an API story look like?

Pretty similar, but with a few tweaks tailored to the real need. If you don’t feel comfortable writing this as a PM, that’s cool - partner with your Scrum Conductor!

Name

As an API Consumer, I can [x]

Description

Give a bit of context of why this story is important to the persona. If this is a part of a large set of stories, reference where it fits into the larger picture

Permissions

Who should have access to this endpoint? Just the creator? Patrons and creators? Patron only?

Inputs

What parameters should this endpoint accept? An ID? an array of IDs? Which of the inputs are required? Will you allow any side loading?

Outputs

What data should be returned when this endpoint is hit?

Acceptance Criteria

1.

---

EPICS

What’s an epic

Text book definition: 

(https://www.atlassian.com/agile/delivery-vehicles)

An epic is a large body of work that can be broken down into a number of smaller stories. For example, performance-related work in a release. An epic can span more than one project, if multiple projects are included in the board to which the epic belongs.

Erin’s definition:

A way for a PM to keep their stories organized so as to stay relatively sane. Usually they align to some chunk of work that delivers value to a customer. I always prioritize within the epics, rather than try to prioritize the whole damned backlog. It’s just too much.

Do I have to use epics?

Absolutely not. If you’re working with only 2-3 engineers and you really only have a small scope of work, there’s no need for epics. Epics are more useful with larger teams who are working on multiple projects at once.

---

BREAKING IT DOWN

User stories should be granular enough that an engineer can complete the story within a day, generally speaking. Each team probably has their own point system, and each team should also have a threshold for pointing a story. Above that threshold, and you’ll need to split the story apart.

Most full-stack features will likely need some backend stories (database schemas, etc), some API stories (CRUD endpoints, for example), and frontend stories. Generally speaking, it’s great to keep these things separate.

Backend Stories

I’ll sometimes stub out the real technical work here (like creating a schema, etc), but mostly… I leave this up to the backend engineers to write.

API Stories

I like to write these out after getting a better understanding of what’s needed from the front end. What information will the page or state of your page need to be provided? In the case of a table or list feature, this comes in the form of which columns I need to display, or what information I need to be able to create or modify within the table.

Frontend Stories

To me, these are the easiest stories to break down (because I’m a visual person). I like to start with the largest, and least specific detail possible, and work my way down to various elements on a page.

For a typical “list” or “table” feature, I’d usually break it down like this:

• The page exists, with the title

• Data is displaying in an unformatted table (and here are the columns)

• Data is displaying in a formatted table

• Pagination controls are present

• Hovering over a row does x

• Button y is present, and when clicked does z

---

ESTIMATING

Are you an engineer? Great. You should be estimating. Are you a PM? Stop. Go get an engineer.

How should you estimate?

It’s cool for all engineering teams to do pointing in a slightly different way. However, there are definitely some best practices in how to think about estimates. Estimates should not reflect the amount of time it will take you to complete a task, but rather the relatively complexity of one task as compared to another. Fibonacci, as it turns out, is great for this.

Let’s say we have the following 5 stories in the backlog:

1. Change the description on Page x

2. Create a new modal that contains a list of data

3. Add a button on Page x that pops a modal

4. Create a new component to manage sorting for a table

A string change is probably… 1 point. If even. Is creating a new modal that hits and endpoint and displays data more or less complex than the string change? Sounds more complex. Great. Now, when you thinking about adding the button in comparison to the modal, is it less complex, or more complex? Probably a little less. And finally, how does the complexity of a new sort component compare?

At the end, you might say that the tasks, in order from least complex to most complex, are task 1, task 3, task 2, task 4.

To assign actual points, though, you have to go a step further. Is a button TWICE as complex as a string? Then 2 points is fine. Is it actually more than twice as complex? Maybe 3 points is a better estimate. The other thing to keep in mind is unknowns. The less certain you are about what it will take to fulfill any given task, the higher the point value you should give it.

After some time, the team should coalesce around roughly what points makes sense for your team, and have anecdotes like, “well a 3 point story feels like x, but a 5 point story feels like y.”

Why should you estimate?

A couple reasons. First, it should help you understand how much you might be able to accomplish in a week.

As engineers, estimates will inform how you think about milestones over time. At some point, you’ll have to say, “we think we might be done with x chunk of work by z date.” Understanding (over time) your point-based velocity will give you a more accurate prediction of this.

As a PM, estimates will help you understand the relative complexity of what may seem like small features, giving you the ability to prioritize more effectively. Is this 5 point story actually providing much more value than this other 2 point story? If not, maybe it’s time to scrap or de-prioritize that 5-pointer.

What even is an estimate?

It’s just that, an estimate. It is not a deadline, it is not a promise or guarantee, and it will often be wrong. It should also more often be directionally correct, and help us get better at things like milestones over time, because it’s a way for us to start quantifying and grasping something that is largely, let’s face it, unknown.

How much time will it take you to solve this puzzle?

Really hard question to answer, if you’ve never solved the puzzle.

How much time will it take you to solve this puzzle, given that it took you x minutes to solve this other puzzle that was similarly complex?

Easier to answer. But still very inexact.

Also, keep in mind that as a product organization we are AGILE. That means our priorities might change, or we might learn something new, or some unexpected thing will come up that will throw our milestone dates RIGHT out the window, even if we because wicked good at estimating with Fibonacci points to begin with.

tl;dr estimates are a tool, and nothing more

---

BACKLOGS

How much backlog is the right amount?

In general, the hand wavey rule on this (no, not just my rule, it’s a real thing) is about 2 weeks. In other words, you should always have 2 weeks of granular, fleshed-out stories for your team.

Can I have more?

I mean, of course. But, I’d encourage you to leave stories that won’t be addressed for >2 weeks at a higher level, like an epic.

What about all the ideas I don’t want to lose track of?

First of all, skeptical face. Second of all, keep an ideas backlog in some other place for you, as a PM, personally.

Also, I break these rules all the time, and should probably go blow really old stories away, because real talk we’re never going to get to them. And if/when we do we won’t remember there’s already a story.