Lindsay Velecina: 0:03

Welcome to the Scrum.org community podcast, a podcast from the home of Scrum. In this podcast we feature professional scrum trainers and other scrum practitioners sharing their stories and experiences to help learn from the experience of others. We hope you enjoy this episode.

Patricia Kong: 0:20

Hi, everyone. Welcome to the scrum.org community podcast. My name is Patricia Kong and I am your guest host today. So recently professional scrum trainer Todd Miller and I spoke on the scrum.org webcast, actually on March 23 2023, if you're going to look it up, and it was called engaging stakeholders and management with evidence based management EBM. So that was about concepts you can use to develop your own dashboard of information to proactively engage upper management and stakeholders about the value of your product and the capability of your team. And Todd and I were doing that because we were we're actually doing a project together. And a lot of the questions that we're seeing in organizations and teams and management is they're constantly almost in a defensive mode. And so it's fielding questions like how's the team doing? What are you going to do to help your team? When can I have that? What have you achieved? How do you make decisions? Those kinds of questions. So Todd and I are together today. Hello, Todd. Hello, Patricia. Because we actually received almost 100 questions during that webcast. So we're here to circle back on some of the open questions. So 100 Questions time, does that mean that the topic is popular, or we didn't make sense?

Todd Miller: 1:43

I think maybe a combination. I'm hoping that this is a popular topic, because this is really important to you. And I specifically. I mean, you and I have talked about this since the day we met, like seven and a half years ago, right? This is like, I think that first conversation we had then it was like, Hey, how's it going? I'm from Pennsylvania, you're like, oh, for Boston, after like an hour of VBM talk. So hopefully, hopefully, it's interesting.

Patricia Kong: 2:07

I think you were like, I'm so scared of getting beat up as a developer. Alright, so we we we did pull the questions, we obviously won't have time to go through all of them. But we hope that this adds a little bit more color to the conversation. And we're going to continue to do that. So let's start with the first question. Is EBM a mindset to do something? Or is it an actual tool to get any result?

Todd Miller: 2:40

This is a this is a pretty deep question. I'm interested to hear your take on it. And my take it like maybe I'm just I have the scrum framework, you know, thought process going in my head right now where you have these 11 elements. And they're accompanied by the scrum values, which is a combination, like the scrum framework is just that framework. So I guess you could view it as a tool. But without the appropriate mindset. It could be really done. As we talked about zombie Scrum or mechanical Scrum. I think the same thing applies here to EBM, right, I think that with EVM, we have three levels of goals. We have the notion of empiricism embedded in there with the goals and then we have the four KBase key value areas accompanied by a measurement or several measurements under each right, which is the key VMs. So you could do that. But you could do that in a very mechanical way. Right? And so, I think that it is a EBM is a framework. So it is I guess you could view it as a tool. But without the appropriate mindset. You may not get a result that you're that you're expecting. What's your tip, Patricia?

Patricia Kong: 3:53

I agree. So I think all of these things, I mean, they're they're, they're a way to achieve something. So EBM Scrum, I think EBM Scrum, when even when we talk about portfolio management, some of the stuff we're working on, it is very much almost just scrum to me, the thing that EVM adds on to it as to say, and that was actually added to the scrum framework was about how do you know this is working? What's the success? What is that? That inspection and adaptation? That's the evidence that you're looking at? For me, the mindset is a part of everything we do, and I know you were talking about zombie mindset, what it makes me think about is, one you have to have the courage to actually do this, it's gonna get you into trouble. You're gonna have a different kind of conversation. But also I can see that this could maybe be used for evil. Like you could use it to have any conversation essentially, you could do that with any sort of data research and spin it so you know, be be mindful of what you're trying to do. I'm not pushed your way through by, you know, putting, it's like instead of ignoring what's there, it's almost manipulating the information to say what you want it to say, I guess

Todd Miller: 5:14

that's a good point. Because you can really do that with data can't you can, you can really, one thing I thought about quite a bit is how, you know, we all have our own narrative about a product, about life, everything, we all have our own narrative built in our heads. And you can manipulate data to meet your own narrative, rather than change your narrative based off of the data that you're seeing. And so when we're talking about like, objectiveness, and we're talking about measurement, we're talking about how you formulate goals and how people behave. It can be manipulated, which could lead us into a e Ville type scenario, right? Also the interpretation of it, right? Because the EVM guide is not a do it all tell you how to do everything. So it can be interpreted and maybe implemented incorrectly. Maybe unintentionally, but the cause is pretty bad behavior and from a visceral reaction to how it's been implemented. So, like, all those things are there and, you know, I can't help but think of mindset when I think of how you and I have been in situations where we're, you're talking about these things, and you're asking questions, and people get very defensive, right, the transparency that EVM can bring to you can can can sometimes put people in the defensive, which tells me there is mindset, right?

Patricia Kong: 6:41

Yeah. All right. Next question.

Todd Miller: 6:45

Ready to move on? I got this one. So what are the risks associated with implementing EBM? And how can we address them?

Patricia Kong: 6:53

We just talked about how you can manipulate stuff, I think one of the risks two of them. One, you are trying so hard to figure out the perfect measurements and the perfect goals that you never get started. That would, that would be something that would be a risk to me. And that's the same thing with anything in life. But it's just trying to, you know, Oh, are we supposed to put down measurements for every Kba? Or how about these? How can I make the perfect dashboard and all that stuff and, and I've seen this and I've worked with the companies that have done this and and I think in the webinar, we specifically even like we were going down that path when we were talking about our experiences and what we could capture on the dashboards in the webcast. We were said you know what, let's just let's just put a couple of things there. That's enough to have a conversation. So So those those are those are the risks that I think there are there's of course others what about you?

Todd Miller: 8:07

Yeah, I keep thinking that the evolution right the the evolution of like you're saying like this perfect setup, right let's let's implement evidence based management using waterfall, let's get everything right before we look at it. That's that's a that's not a mentality you want to take just enough to get started is what what your what you were mentioning, you were just mentioning, but after you get started the evolution of it right, to continue to work on writing better goals, we know that goal Writing is hard, we see it all the time, we see, we see the the epic struggle of writing outcome oriented goals in every organization, right. But that doesn't mean you shouldn't stop trying. Another thing I've seen too, is just because you measured it at one point in time doesn't mean that it's necessary to measure it in the future. So I guess what I'm really getting at here is that that your first take at the implementation and become set in stone, and you fail to adapt the implementation and the process of evidence based management as it goes on. Right? So there is no built in way for you to say that you're going to continuously improve your implementation of evidence based management. But perhaps that should be something that you consider as you're, as you're building your product or you're applying this to your organization or your portfolio is when do we look at our inspector implementation of evidence based management? When we

Patricia Kong: 9:32

think that does make it a way of thinking doesn't it does make it like because what you were just saying made me really think about how you don't need all this buy in like if you just start thinking and asking good questions, you can get that. Alright, so we have we probably have time for a few more questions. Okay. How do you prioritize your backlog or ideas with EBM to ensure that you aren't missing a higher value opera Trinity, do you rate these on the 4k vas? Key value areas? Friends?

Todd Miller: 10:06

Yeah, this is a, this is a pretty deep question too. And here's that. Here's the thing really, for you to keep in mind on this is that everything that you have on a backlog or every idea you have is just a hypothesis of value. And no matter what I mean, I know you and I talk about this quite a bit, Patricia, that we've had these ideas and have talked to customers about them and gotten really charged up about them. And then we go implement them. And it doesn't bring any value, even though every indicator before told us it would. And I've also implemented things that I'm like, Okay, well, this person asked for it. And I let's just it's a quick thing. It's implemented, and it becomes wildly popular. Everything is a hypothesis. And that's where we can validate those hypotheses. But I don't think that anything is going to tell you exactly how to order your product backlog to KBase. The goals, there is some gut in here, what do you think?

Patricia Kong: 11:09

So I agree with you that. So evidence based management is not database management, bring the information that you know. And I think if you're if someone is trying to think about how can I look at the KBase in terms of decisions, so obviously, have your goal, but you are now I think asking a question about unrealized value, what you should pursue, and you can look at that with current value right where you are. So where you are what you want to pursue, that's going to help you make decisions. And then for me to balance it out. You do want to know, your time to market ability to innovate. And the reason I'm talking super abstractly, is, let's say you're looking at some ideas, like Todd said, you're thinking about these all equally. Now, what your job is, is to try to disprove or prove as quickly as possible. So you think about those outcomes, or now you're thinking about with your ability to innovate. Now you're thinking about how fast can you actually realize value and see what actually starts moving things. And that might help someone think about their different ideas, but they really should understand from their experience and their own gut, what, what might be the right, decision, there are better decision, right? You don't don't undermine your own experience. But also take the information you're receiving from the market. pretty seriously, but but that whip thing is what I'm thinking about. Currently.

Todd Miller: 12:39

Yeah, I think that's a good point. And I mean, tie, like your tie in there to time to market and ability to innovate, because if you if you're if you're not shipping, you're not being effective being shipping. You might not be delivering anything that you probably

Patricia Kong: 12:54

get into though, those are the most important thing we're gonna work on this year.

Todd Miller: 12:58

Right? Yeah. And then suddenly, two weeks later, it's not the most important thing anymore.

Patricia Kong: 13:06

Yeah, or it's still important the next year, and we don't know why. Yeah,

Todd Miller: 13:13

yeah. Next question. Do you think we're good with them? Yeah. How do you persuade C suite, that outcome is the starting point when they just want outputs?

Patricia Kong: 13:27

So so this question is a little bit it's funny, yet it's a little bit loaded, because it's assuming that people in the C suite aren't thinking about the outcome or don't think about goals and visions? And that might be an interesting starting point is do they have? What is the vision of the company? What is the thing that we align on? Right? If they're really just like, give me this, this and this, it's really having that, that conversation with them, really, on a leadership level, and C Suites have to make a lot of decisions. And they have a lot of accountability, sometimes that's financial, there's a lot of risk there. So sometimes, getting them to think about the vision and honestly just making a decision about what's more important to you, will lead you slowly to the why is this important? What how is this going to help us why? Because sometimes that reaction of just show me outputs is show me what we're getting done so that I know we're making progress. But usually if you dig a little bit, there's there's something that they're worried about, or something that they really want to achieve, and that's a little bit probably more coaching situation. So that's a that's a softer way of entry. What about Utah?

Todd Miller: 14:49

This might be one of those where it's a cautionary tale to not maybe forge your way in depending on your position in the organization and go really hard at it out of the gate. If I've made that mistake, that's why

Patricia Kong: 15:02

I gave the slower version you can get the violence.

Todd Miller: 15:06

So, but I think you're writing most C suites got there because they are outcome oriented. You know, I do think that we have issue in a lot of organizations, though where it is thou shalt do this and without any consideration of the of the relevancy of it. So, so if it is an output oriented thing that that executives are demanding, maybe you could get into the conversation by by tying that output into what's actually happening, right, and say, like, Okay, this output produces and try to do it in a really non invasive way. That's because I think if you just if you just go, you know, barking about this, I think that you could get that phrase. And rather, I think, if you say, okay, so and maybe ask the question, so the this output that you're that you're asking for, you know, produce this thing? How does that tie into the customer, maybe just start? I really like the idea of like centering, and just asking customer consumer related questions. And then and then maybe building out from there, start small and then move big.

Patricia Kong: 16:13

Yeah, I like that I actually what I've done in the past is give them the outputs, and then I come up with the outcome. So I just kind of do the work of thinking about the outcome and how I'm relating to it and making sense of it in my head. However, I think, probably because it's the Todd and Patricia conversation, we would say, if you are working in a company where they're just concerned about outputs, consider if you want to be there, you know, what's gonna happen with that company? Is that something that that that interests you? And is that the sort of leadership that you think can guide you? So?

Todd Miller: 16:54

Yeah, I agree with it. I mean, I don't know that we talk about that enough. If if you feel like you're not having any influence into what those outputs are, and you desire having influence, and then then you've tried and you can't get it, maybe it's time is split.

Patricia Kong: 17:07

Hmm. I think that's where we are. So that might lead us into this next conversation. Okay. Todd, who would you recommend be involved with developing what the goals are? Should we involve stakeholders in creating goals?

Todd Miller: 17:29

So I think that this is so contextual, that like, I want to make sure that I don't answer this like this is the exact way to do it. Because right, so in Scrum, we talk about product owner, and a product owner that is responsible for product goals and visioning. Right. And so from a scrum perspective, which, which I think is okay to use here. I would say that the product owner drives this conversation. And as a product owner in the past, I really did like to involve stakeholders, and team members in the creation of these, although I would vote maybe I would, I would be heavily influencing it, because I'm ultimately accountable. So I might show up and be like, here's my, here's my, here's my product goal that I'm thinking about activating after we retire this product goal. So in evidence based management, we consider that maybe a intermediate goal, right, something with a three month to six month time horizon, I think is what we said in the webinar. So I don't think it's a bad idea to have stakeholders involved. But if you're the person that is accountable for creating the goals, and what I would tell you is that make sure you come with your your own path in mind. Now, intermediate tactical goals tend to be as the implementers that you're really heavily influencing. But generally, there's somebody developing these skills that fair patrician now I'm just like, running on about it and thinking about it. What are your thoughts?

Patricia Kong: 19:03

I think? So should we involve people it developed? So it's contextual for me on the level of goal, right? Like you said that probably you couldn't even if it was just a sprint goal or tactical goal, or, you know who's closest to that they're going to know. So they have they have input, right? We have developers, we're going to talk about how we're going to achieve this product goal, the stakeholder the strategic goal, those bigger goals. I mean, if you want to make that by looking into a magic eight ball, go ahead, but typically, you know, it's kind of those ones where this decision is when you have for me, it's about having input learning, and then I make the decision. So that's, that's kind of into decision theory. But I would think about the level of the goal and then what level of information do you feel comfortable with? If you do have someone just said, Who has the authority to be like I made this goal this is it. That's it. answerable. So, but the, the accountability and the learning will be what's important. And I think another thing we should probably note is that goals can change. And so that's, that's it's not something that is set in stone, especially as they're further away. Right. So we talked about that.

Todd Miller: 20:21

Yeah, I think it's really important to emphasize the point that you just made. So I'm going to re emphasize it for California, is that depending on the level of goal, and depending on how you're implementing evidence based management would, would would be how, who you involve and who is involved? Like, I think it's really, that might be a pretty important thing to clarify. Like, if you're implementing evidence based management at the portfolio, from the portfolio perspective, that can be substantially different than if you're implementing it on a product, right? So I, and who is accountable for what might be something interesting for you to explore and make make pretty explicit? Might be Yeah, so I just want to emphasize, I think that's an important conversation.

Patricia Kong: 21:11

All right. Stick up. Couple more.

Todd Miller: 21:14

Next question. It's my turn. Yes. Okay. But here's the best way, I know more about stakeholders, what is the best way to keep stakeholders and managers engaged in actual updates, to keep them interested, especially during busy and pressured duties? So basically, the stakeholders are real, and managers, those are really busy. How do we keep stakeholders and Matt and managers in involved in our evolution and our current implementation evidence based management?

Patricia Kong: 21:50

Have a conversation? I think this is where in the webinar, we kind of used the the two different dashboards. And it was saying, Here's information, here's information radiator, let's use that as a tool to have a conversation. And so that's really where the updates I don't, I don't I don't know if it's like any different than how you would normally like if you're using Scrum at the sprint review. You could choose information you can have a conversation. In it might not? Am I not looking at thinking about this question correctly?

Todd Miller: 22:29

No, I think you definitely are. I don't think there's anything built into evidence based management that tells you exactly how you engage your stakeholders. I think it's so contextual again, right. I think that centering around a time duration, for immediate, intermediate, and strategic goals, like those time horizons, I think might give you indicators.

Patricia Kong: 22:53

I think, I think the thing that I would add is if you have because I've dealt with this, too is if you have someone who's like no, I don't I want to know the velocity of the teams that have that information ready, but have the other information you want to talk about to drive the conversation more, right? We're taking baby steps here. So I probably leave it at that. So I'm sorry, if we didn't answer that well, but that's that's kind of how I'm thinking about it.

Todd Miller: 23:18

I think it also would be important to say here that not all stakeholders, you engage with the same. So there's different types of stakeholders, like their stakeholders that you want to constantly keep informed. And then their stakeholders that it's okay to just occasionally and keep them informed. Right. And so, I would probably start here by understanding who your stakeholders are and see if you can even categorize like, I need to be in this person's office at least once a week. Right. Yeah, you know, that's important,

Patricia Kong: 23:48

like conversation, conversation around stakeholder management and who are stakeholders? I don't think I don't know if do you do that enough? And to say, Who are my stakeholder who, whose to thinks their stakeholder but they're not? They just want to be kept apprised of, of our situation? For sure.

Todd Miller: 24:05

And those stakeholders change? Yes,

Patricia Kong: 24:08

yes. Good point. Okay. Okay. Oh, I like this one for you, because it's different. And maybe we can end on this one. How can I measure the individual performance of team members, all these metrics and values will be team wise, and one team member may not perform well, but they're going to take advantage of a good scrum team. So I think that's saying, you have like someone who's not performing but they're in a team that is good to get not they don't know they're not exposed that they're, they suck.

Todd Miller: 24:45

So, from from my so first of all, I don't think that in I know that in knowledge work that you cannot measure the individual performance of someone you because those metrics that you're using to measure the individual performance of somebody are going to get found out. And then we'll give you know relevancy into what those measures told you to begin with. So I think when it comes to measuring performance of a person, I don't I don't, I think of knowledge work, it's impossible. Now, on a factory line, how quickly somebody is moving a gadget from point A to point B might be a measuring point, so that you can use some lean practices to cut that down. That is not at all applicable to knowledge and creative work. So let's just get the notion out of the way that there's possible to measure people we're big in knowledge work, because we're not robots, and it's creative work. So maybe we could just cut that out. Is that fair? Are you What do you think about that?

Patricia Kong: 25:49

I like, Yeah, I like how you're delineating this from the type of work. So if we're in a type of work that requires a team of people to work together, and that work is somehow going to be influenced conversations? Yes. If you have different type of work, like you're saying, it's a lot easier to see, do they show up on time? Are they doing their job? That kind of thing? Right, like the factory work you were talking about? Or if you're working in something else, or they're just doing an individual job, I think. I think it's another conversation. If you have a team that is pointing out, Hey, we have a team member who is not really stepping up, that's a conversation for the team to point out first, but that's not a I'm going to track your individual performance. So it's a nice little conversation probably that segues into, like some of the Agile leadership stuff, agile management stuff, who has that conversation, but who's telling you that they're a good scrum team and that that person is not performing would be something that I would think about, but that's not really where EVM is gonna shine anything like I can do EVM on a person, but I can, you know, a team using EBM or a team using EBM teams using IBM on a product will be interesting because the thing that you can start to look at, for instance, is a little bit around that ability to innovate, what are they doing? What's lacking? How are they being effective at achieving value? Is the scrum master helping them be effective at achieving value? Or is the scrum master focused on? You know, just setting up little icebreakers all the time for them? I think about I think about like, also, you know, if, if, are people having hard times working with those people, I would think about when a slot opens up on a team, does anybody want to join that team? Like, those are the things I think a different way of thinking about?

Todd Miller: 27:55

Yeah, you know, I've also been in this circumstance where you know, that there's a person that isn't keeping up with the rest of the team. And we're paying people like this is, we're professionals, right. And so if I feel that way, I and I am a developer on the team or on the, you know, person producing something, and I don't think that that person is keeping up, I owe it to myself to have a conversation with that person, right? We can talk to people, we need to have that courage to be able to sit down maybe as an individual one on one and just say, Is there something I can help you with? Because you're hurting this team? Right? And so like, to me, I go back to this professional conversation where we don't need proxies to people, I wouldn't go to my manager without saying something to this person and calling it out. And then if if that person still can't keep up, maybe then it is something that needs to be elevated. But I think we go out. I think you nailed it when you said this goes outside the bounds of evidence based management. And we would really, really caution you around having individual performances, under any caveat. In fact, we wouldn't even caution I would say, don't do it. But this is leadership stuff. This is how this stuff works. This is professionalism stuff, this is going to happen. Right, you need to be equipped for this to happen and use your traditional stuff in HR to cope with it. That's why HR is there.

Patricia Kong: 29:27

For your retrospective.

Todd Miller: 29:30

Yeah, yeah, yeah. But I circle sometimes I hear this stuff. And I and I think to myself, like, are there just people out there that don't want to like go and say, hey, you know, Patricia, I can we have a chat? Or is there something I can help you with? It seems like you're struggling a bit. Right? And I need to have that conversation. I can't just go straight to some authority and say that that's that that to me is is is weak. So I have to take a chance to say that

Patricia Kong: 30:00

All right, well, thank you. So I like how you tie that back into the mindset and some values and professionalism. I think I think that's it. I hope this was helpful for everyone. You know if if this this was and there's more of the questions that we should be answering, let us know. But thank you and thank you, Todd for taking this time today to do this.

Todd Miller: 30:27

Oh, it's great hanging out with you, Patricia.

Patricia Kong: 30:29

Always Okay. Thank you.