Lindsay Velecina: 0:03

Welcome to the scrum.org community podcast, a podcast from the home of Scrum. In this podcast we feature professional scrum trainers and other scrum practitioners sharing their stories and experiences to help learn from the experience of others. We hope you enjoy this episode. Welcome everyone to the scrum.org community podcast. I am Lindsey Velecina from scrum.org and I am stepping in today as a guest host. Today we're going to talk about a topic that is very near and dear to our hearts here@scrum.org That is learning. And today I have two very special guests with me I have the scrum.org product owner over courseware Sabrina love and I have one of our professional scrum trainers Ravi Verma so welcome to both. We are going to have a discussion here around the ways people learn and the different formats that work better for some people than others. And we are going to talk about immersion training specifically as well. So we'll get to learn a little bit more about what we're talking about there. So with that, I'd love to jump in and just let the audience get the chance to know Sabrina and Ravi. So Sabrina want to go ahead and introduce yourself. Hi, everyone.

Sabrina Love: 1:16

My name is Sabrina love. I'm the Product Owner of courseware scrum.org. I've been here for about almost nine years. I've been a product owner for about a year and a half year. So I've had a little bit of a growth journey here which is has been great.

Lindsay Velecina: 1:31

Ravi, would you like to introduce yourself?

Sabrina Love: 1:33

Sure. Thank you. Yeah, so my name is Ravi Verma. I am a professional scrum trainer. I've been a PST for I think, maybe more than 10 years. And I have trained 1000s of students in Scrum and Agile both through scrum.org. And outside. I'm also a certified ROI professional from the ROI Institute. So I I specialize in delivering training, ROI based outcome focused training. I'm also a certified trainer from Sharon Bowman's training from the back of the room. So I integrate ideas from the world of Scrum ROI, and adult learning to deliver learning experiences that are consistent with the outcomes of the scrum guide.

Lindsay Velecina: 2:17

So I would like to give our audience just a little bit of an overview of the current learning landscape. And this will help us set the stage for our discussion. So Sabrina, can you tell us a little bit about what you've been seeing? Sure,

Sabrina Love: 2:34

yeah. So a lot of the training that we've that we've seen, or that we offer, actually also, is, is traditionally comes over consecutive days, you know, pre COVID, it was in person, post COVID known now we get to see live training virtually, which is, which is awesome, and being able to bring people together from across the world. But traditionally, those classes are still delivered over consecutive days, and kind of all at once. So you get almost what I call a fire hose effect, which does work for some people. But in some cases, we've started seeing from at least within our community and a few places outside of our community that people are more interested in a longer, more long gated learning experience that gives them time to absorb something small, and learn from that. And then there's been a little bit more focus also within our community about helping people to practice what they've learned, and not just, you know, having a learning experience. And then you know, have a nice life. See you later. But there's a little bit more focused on engagement, and helping people to apply their learning, which ultimately ties back to our our mission, right about helping people and teams solve complex problems, and how are we actually going to do that? So we've been a little bit more focused on that. But that was in response to what we've been seeing in the current land learning landscape, like you mentioned, is, is this consecutive learning experience, where people kind of get a little bit stuck with there's so many things that you could do, and where do you start? So that was kind of the the catalyst for getting into something like what we're going to talk about today. Yeah,

Lindsay Velecina: 4:18

and Ravi, what have you been experiencing with your students? And with what you've been just seeing in the market?

Sabrina Love: 4:26

Yeah, you know, I've had very diverse experience in my training. So I've had situations where I deliver the traditional learning over two days. And in some cases, when I re engage with my students, either on LinkedIn or I meet them somewhere at a networking event. In some cases, they tell me Robbie, we are still struggling at the same level as we were. There's another example a real world client where actually the training was harmful to them, because I gave them a bunch of jargon. And then they started weaponizing, the jargon to fight against each other by saying, Hey, this is what Ravi said, no, no, this is what he said. So I actually heard, I heard them. And my highest obligation is to do no harm. And I also had the third experience, where companies were transformed because of traditional learning. And in fact, we have a scrum.org case study for one of those instances, so I just want to explain to people, this is not a one size fits all. result, different companies with different cultural and operational contexts can derive radically different outcomes from the exact same training delivery method. salutely. Yeah. Yeah.

Lindsay Velecina: 5:46

And we're gonna be getting into that a little bit further here in our discussion today. But first, I just kind of want to go to the foundation of what we're talking about here. So what problem are we trying to solve when it comes to how people learn?

Sabrina Love: 6:01

Yeah, I think, to go back to what I was mentioning earlier, is is helping people apply. And a little bit to what you were saying, Ravi, about, people are still struggling at that same level that that they were in some cases, right. It's not it's not universally applicable, like Ravi was saying. But in some cases, I think the problem that we're that we're trying to solve is to help people convert their learning into practice into application. And that aligns a little bit with Bloom's Taxonomy, if you're familiar with that, where the first few layers at the bottom, you know, there's a layer of remember and then understand and then apply. And traditionally, I, I have this belief that, that it's, it's almost a lot to expect from a student to be able to reach a level of apply when they're confined to the context of a classroom, because there are no real world consequences in the classroom. So I think that fundamentally, it has something to do with with your actual skills growth is getting outside of a classroom and having real world consequences. And that is the catalyst for for real skills growth. And Lindsey, if I may add to what Sabrina was saying, what are we trying to achieve here? Let's start with the why let's begin with the end in mind. For me, as a practitioner, as a scrum.org trainer, I am trying to help my learners achieve the outcomes listed in the scrum guide. And the scrum guide basically says I'm paraphrasing, that we want to empower organizations to generate adaptive solutions which optimize value, optimize predictability, and control risk. So as a, as someone who is delivering learning experiences, I am looking at the learning landscape from that lens. And I'm looking at the organizational context through that lens and trying to see, okay, we've got a bunch of learning delivery choices. Which choice might be appropriate in this context, such that it helps this company or this context, generate adaptive solutions, optimize value, predictability, and manage risk. So that's how I look at what are we trying to do here? Yeah, it was beautifully, put thank you.

Lindsay Velecina: 8:17

Yeah, that was great. So I would like for our audience to hear a little bit about immersion training, which is one of the learning formats we have here@scrum.org. Can you describe how that works?

Sabrina Love: 8:30

Yeah. Um, so in a compared to a traditional, a scrum.org traditional class, which is over consecutive days, an immersion training actually splits up the learning experience into multiple weeks. So you're, you're learning a little bit at a time. And after you learn something, you get an assignment that is focused on an outcome, to give you some flexibility and some creativity and how you would approach that assignment. And then at your next classroom session, you come back and spend a little bit of time in a facilitated retrospective, almost it's, you know, come back together, what went well, what didn't go well, where do you need help? Kind of conversation. So compared to a traditional class, which is two days consecutively, usually, this experiences over multiple weeks with assignments and reflection built into it.

Lindsay Velecina: 9:25

That sounds great. And this is really just a different approach to taking a class who would benefit more from this approach? Do you think Robbie?

Sabrina Love: 9:39

I think the here's some criteria or questions I would like to give to our listeners as a decision making framework to figure out what context is appropriate. The first question might be, what is the available budget or capacity for learning? So if For example, I've had clients where I'm talking to the CTO, and I say, Look, I'm seeing this pattern where the team is making the same mistake over and over again, it's costing you money, would you be willing to give me two days time, so I can train them. And the CTO responds, Ravi, we have deadlines. I cannot bring my organization to a standstill for two days, because they're off learning. So, so I say, Okay, give me a counteroffer. What are you willing to give? Most people psychologically, are willing to show my lived experience are willing to give 90 minutes per week, or per two weeks. So one question your selendy? I love your question. Well, what what might what might be the context in which people might benefit? The first is what's your learning budget? I would also expand learning budget to see to include not only how much time are you willing to give to your learner's to attend, but also to do what Sabrina said, which is to apply in the real world. So some of my clients, the contract that I have signed with them is, are you and the managers of the learners willing to give 90 minutes per week to learn? And 90 minutes on the next week to apply? So that could be a question that can help guide this selection of this learning format? Yeah, absolutely. I think there's, there's another, at least what I've experienced for myself, in attending classes is I'm a learner that needs to see an example. And then I want to try it myself. Some people can go to a traditional class and, and absorb all of that information and convert it. And they can do that I'm not one of those people. I would love to learn and practice it with somebody who's done it before. And then I want to try it for myself. So that's kind of where where I see some alignment, at least in myself, and maybe some learners out there can relate to that. Yeah. And, Lindsey, if I may, I would like to add two more suggestions or questions. So great. Yeah, one thing I love about the scrum.org offerings is so much flexibility is given as long as we honor the learning outcomes. So let's imagine we go to a client and the client, like I went to my CTO, and I said, Okay, you know what, you can't give me two days, you can give me 90 minutes per week. That's cool. So that means in a typical scrum.org, workshop, let's say product, professional Scrum Product Owner. Let's imagine we teach them 10 practices. And now, if we spread that out, over time, then 10 practices, let's imagine 90 minutes per teaching session, plus 90 minutes per week for application. Now it got elongated, hypothetically, to a 20 week program. So I would ask my client, are you okay, that some of the business impacts that you're trying to address might not get impacted for 20 weeks? And sometimes they're like, Yeah, that's cool. In fact, I'm having this conversation with a client right now. What he said was dhokra, we q4, we just want to do backlog refinement, we need our backlog to be in good shape starting q1. So you know what? I don't mind. Perfect. Some other people might say, no, no, no, I can't wait that long. We need to create measurable results right now. So we can give them a counter offer. How do you feel about a scrum.org offering, where instead of doing work on a hypothetical case study, it's like a bootcamp where you bring your real world into a scrum.org workshop, and all of our workshops are experiential, so you can apply the place to the real world context. And they might say, yeah, that's fantastic. So I do two days, and I come out and a bunch of my problems are solved. However, there are some people who will get overwhelmed if you teach them a technique, and you expect them to apply it in a real world context inside class. And if that's the case, then I would not offer this this offering. But I would say you know what, maybe you need to understand the technique in a case study context. And then when they have mastered the theory, then you go back into the real world, and you apply it into your real world context. So these are some questions I would like our learners and our listeners to consider as they try to match the context to the right learning format.

Lindsay Velecina: 14:36

Okay, and can we talk just a little bit about just the students perspective, and how people learn so in, in both formats, so we, we went in pretty deep, I think, with how Sabrina described, you know, her her learning preference with applying, but what really is the drive for a student to take a traditional class versus the immersive, or

Sabrina Love: 15:06

are the other way around, or the other way around? Yeah, that's a great question. So here's what are the patterns I am noticing in 1000s, as I look back at a dataset of a sample space of 1000s of students, and different human beings prefer to learn in different formats, right. So there were there was this one example of a client I was introduced to through scrum.org. And I delivered the EBM workshop, and I delivered a pspo workshop. And we extracted the 10 plays that are taught in the scrum.org workshop. So, and then what I told the clients, what I told the learner was something that Sharon Bowman talks about that learners prefer choice, so I gave them choice. First, I was like very regimental, like a drill sergeant, you have to do it in the exact same order that I taught you. So you have to start with product vision, and then you have to go to product go. And they're like, No, this is not resonating. And then I said, Oh, my God, I'm violating choice. So I backed off, and I said, you can choose, we taught you 10 plays from the scrambled out playbook. You I'm giving you autonomy, that you choose which play you want to apply in your context. So if you have a context in which students are hungry to learn, they are not going to get overwhelmed with learning 10 plays, and they would love the autonomy to choose which play they want to apply. Perhaps you can give them a constant concentrated dose of learning, and then give them autonomy to apply. But I love what Ken Schubert said, or he often reminds us ask the team. So I think it's best to have this conversation treat the learners, the buyers, the managers, the stakeholders, as adults, responsible adults, model the strong value of respect, laid out lay out the different options with pros and cons and ask them, What do you think might work? So that's how I factored in the learners preferences. Mmm, that's awesome. I mean, and that actually highlights there's a different there are different ways to learn from scrum.org. And from PSTs. You know, we're talking about traditional consecutive classes, we're talking about immersion training, but there's actually a myriad of different ways that you could learn from a PSD. And that's up to your PSDs and their customers to figure out what their needs are and what their choices are. Right, Robbie, and I'm sure you've you've helped your customers learn in many different ways other than traditional or immersion settings, like are there some other ways that they have helped people that they've made choices? Yeah, yeah, this is another reason I love scrum.org, I feel that the heart of Ken Schreiber is shining through scrum.org. And I feel that Ken wanted a sincere hungry learner to be able to learn and the price of a training should not stop them. So what I love about scrum.org is that we have so much high quality curated content in terms of the learning path. So I feel scrum.org offers so many learning options, look, if you are sincere about learning, Scrum, to solve complex problems, but you don't have time or money, but you have desire, okay? You use our curated learning partner, you're trying that. You want some more help, okay? You can have trainer facilitated sessions. You can have traditional, you can have immersion, you can have hybrid, actually, I've done both. So, these are so many options that are available through standard, all I have to do is ask, right?

Lindsay Velecina: 19:01

That's right. So just to go back a little bit into the immersion training. So how is that going for you and your students, Robbie,

Sabrina Love: 19:12

you have for me, I have found emergent training to be the most effective learning format. And what students love is they have deadlines, they have pressure, but most of them if there's a good support from managers, and C suite are okay with 90 minutes per week. So that's one thing that they love, that training is not becoming a source of anxiety. That's point number one. The second thing that my students love about this format is that they get to do real work that they would have done anyway, in an unskilful way. But they get to do it in class and after class. And there is someone who supports them. So that's The second thing I'm going to do this work anyway, might as well do it in a skilled way, there is a third thing that really helps, which is community support. So I use many of the techniques that I learned from scrum.org to provide a kind and safe space where the community can provide feedback to each other. So for example, I learned about the perfection game from Ralph Yocum in a scrum.org, webinar, code protocols. So I use I tried to find kind and effective feedback techniques. So the community learns better from each other than they learn from me, my job is just to create a space and hold space and gradually fade in. And I want to share two more things that have worked for real world clients. One thing is, for example, I told you about the client where I did EVM and product owner training. I taught we taught them the 10 plays, something magical happens. One team, this was a this was 30 people and I complemented live traditional training with immersion program, some teams started making great product vision. So now everybody got a real world example of what a great product vision looks like. Another team made a great business model canvas. So what happened was best of breed solution started emerging and became like a playbook for training. They got samples, it was beautiful. And then here's something that happened over time this company had grown through acquisition. And so they had a hodgepodge of product management practices. But because of this community over a period of one year, there was consistency in product management principles, values, practices, and outcomes. So they got an external auditor to come in and audit them. And the auditor gave them the best marks because of curriculum that was delivered. It's basically scrum.org curriculum, which was delivered in a hybrid format. So I want to shoot tell you real world example of the power of scrum.org immersion programs. That's awesome. That's yeah, the honestly, the, the assignments in the immersion training programs are awesome, the the classroom session has been split up into smaller more consumable chunks is awesome. But I feel like that facilitated reflection, that point of coming back after you've tried something and coming back together is, is kind of the, the most amazing part of all of this is even in a public class, even in a private class, that point where everybody has tried something, and they can come back together and ask each other for help or show people where they found success and, and support each other in that way. That's, that's awesome. I would say that the what we are really teaching them is not the content. We are leaving them with like a meta skill, we are teaching them how to learn. And we are teaching them how to bring kind loving and helpful learning communities. Because if we teach them this meta skill, long after we are gone, they will remember and they because we they learn how to learn in a community environment. I love hearing the real world examples from it. Yes, yeah. This is very powerful stuff. All

Lindsay Velecina: 23:18

right. So thank you for sharing those experiences. Robbie, I think that paints a really nice picture for our audience. So I want to ask this question here to you both. How might we apply the same principles, values and practices to learning management as we do to product management Scrum?

Sabrina Love: 23:40

Yeah, that's so such a powerful question. So in the scrum guide, we talk about ready. The product backlog items should be ready and actionable before sprint planning. I feel that the ecosystem should be ready for learning before no matter what learning format you choose, the ecosystem must be ready. What do I mean by ecosystem? The buyer must be ready the buyer of the training must be ready to articulate what are the measurable outcomes of this learning in terms of business outcomes, value increasing value from X to Y, they must be ready to articulate empirically and transparently. This is the measurable business outcome I want. And by the way, it cannot be passing a certification because certification is not going to increase value. So that's what I mean by ready. The managers of the learners must be ready to create time and space for their direct reports to attend the training and to come back and practice apply the training to achieve the outcome that the sponsor of the training wanted. And the learners must be ready to come with a beginner's mind to not be multitasking in class and to sincerely come and learn some Think new and try something new, even if their prejudices biases might be screaming, no, that's never gonna work. So that's what I mean by ready. So we do we do ready all the time in product management. When we talk about product backlog management, we have to transfer those ideas to the world of learning as well. And I have a few others. But let me pause and see if you folks have any reactions, questions or comments. Yeah, I agree completely. I'm it makes me think of when you were talking about Sharon Bowman's Sharon Bowman, earlier and giving learners choice. So people have choice, I think there's something about what you were saying that makes me think of not just the buyers, being ready to support who they're who they're sponsoring the training for, but also the learners themselves. And you touched on this to Robbie, but the learners having an idea of what they'd like to get out of the training, even if that's something that that maybe isn't their choice, you can still find some kind of value in it. And maybe it is their choice, which is even more awesome to attend the training. But yeah, and Sabrina, that was so powerful that you use the word choice, let learning be a choice, please don't force people who don't want to be there, they suck the oxygen out of the air for others who are desperate to learn. And I bet those people who choose to be there become champions in their organization. Yes, right.

Lindsay Velecina: 26:30

It adds so much more when you actually want to learn, yes, rather than just being forced into a training and not be passionate about it. Yeah.

Sabrina Love: 26:42

So maybe there's a there's an element of ready in the organization and the leadership and the sponsorship and also in the learner, the learner. Yeah. And there is one more, you know, in the instrument, or we talk about evidence based management, we talk about outcome based thinking, we, I also learned from Patricia Kong about hypothesis driven delivery. So hypothesis driven delivery says, Well, if I delivered this PBI, then X days after the PPAP VI, I'm gonna get some metric outcome to move from Y to Z, there is no reason why we cannot transfer that practice, from the world of product management, to the world of learning management. So if we are giving them product, product owner training, and the goal is the goal could be, we want to increase the feature adoption. So we are our hypothesis is if we apply what is taught in the scrum.or pspo class, or the workshop, then 90 days after an item is delivered, feature adoption will increase right now from 10% to 30%. Or if it's an applying professional scrum workshop, we could say, feature cycle time will decrease from 90 days to 30 days, within 90 days of completing the workshop. So there is no reason why we cannot have outcome based acceptance criteria associated with our learning programs, time bound, you know, smart criteria. So that's one technique, I would recommend we transfer from the world of product management to the world of learning management. And that's, that's definitely a different lens on validating your learning than, for example, an assessment, which is still valuable, it's still an individual, like, individually valuable, but how do you validate the value to the organization? That you are not, not wholly you know, they're not mutually exclusive? I guess? Yes, they are. Both are they're all necessary and sufficient. And in fact, we have a case study and a webinar standard or case study and webinar, which suggests five levels of learning metrics. So you could have the assessment which is validating theoretical understanding. So this, all students must pass the PSP or one assessment within one month. That's a level two objective level three is application, all students must apply the learning, let's say, to connect PBIS to the product vision within 30 days, that is an application metric impact impact metric would be all students must apply the learning to collaborate and increase feature usage from 10% to 30%. Within 90 days of the training, that's an impact metric. And if we wanted to, we could go next level, which is ROI, which is probably too advanced for this conversation, but I just wanted to make it real and offer other metrics people can consider besides 100% certified, the certification is necessary but not sufficient to create business value.

Lindsay Velecina: 29:54

Those are some great measures to think about for sure.

Sabrina Love: 29:58

I also want to leave one A technique or a practice that we use in product backlog, backlog management, which might be relevant to immersion learning programs. So when we talk about product backlog items readiness, we recommend that the items should be should get to done within a sprint. And in fact, I recommend to my clients, that the cycle time of an item in a sprint should ideally be three to five business days. And this is where a lot of my clients say, but heavy, our items are so big. And that's when we introduce vertical slicing. Yeah, okay, that's fine. We use vertical slicing, there are plenty of blog blogs. In fact, Lavinia got on one of our PSDs may have recently done a webinar on this topic as well. So we have workers, is that right, Lindsay? Yes. Yeah. So we do vertical slicing all the time for PBIS? Why can we use Oracle slicing for learning, and I think that's where immersion learning comes in. It is a vertical slice of learning. So a vertical slice of learning could be let's say, we look at the scrum.org pspo. Workshop. Yeah, maybe we teach them 10 practices, product vision, Business Model Canvas, and so on and so forth. A vertical slice could be what might be a principle, a value or a practice that might fit within a 90 minute time box of learning. And if we apply that lens, then because oh, you know what, I can teach a product vision and I can my students can apply it in class, and then they can go back to the work and apply it in the next week. Perfect. That's about business. Now we're starting to overlap with learning inference to Oh, man. No, it works. It works though. I, I hadn't thought about it as vertical sliced, vertically slice the learning. You know, I learned something from Gator Alaska. She is one of our PhDs in pole. And one of the things that she used to do was at periodic intervals in her traditional learning, I think she would encourage her class to take an open assessment together. If I were to steal Kate's idea and apply it here. Another possibility is that at regular intervals, we invite the community to take the scrum.org practice exams, and see what questions they get right or wrong, and then feed that into the at the appropriate juncture in the Learning Program. Almost like a learning backlog. I love that idea about vertical slicing of learning, vertically sliced learning.

Lindsay Velecina: 32:43

Right, thank you for those insights there. So I want to close this out with a really big question. So if there was only one thing that you would want our listeners to take away from this conversation, what would that be? For me, I

Sabrina Love: 33:01

think Lindsey, I will steal the words of Ken Schreiber. Maybe I will say two things. So forgive me. One is be focused on the outcomes mentioned in the scrum Guide, which is help the learners generate adaptive solutions that optimize value, optimize predictability and control risk.

Ravi Verma: 33:27

That's point number one,

Sabrina Love: 33:30

with the end in mind, point number two is ask the team, ask them, how might we design a learning experience that helps you generate adaptive solutions that optimize value predictability and control risk? I think that's that's related to mine. Just to let our learners know that there are different ways to learn. We have a couple quote unquote, official ones on on scrum.org website, you know, there's the consecutive traditional classes, there's immersion training, but if you were to ask a PST, that you would engage with, if there's something else that would be more suited to your environment, I'm, I'm more than confident that they'd be willing to work with people on on a learning experience that fits their environment. So there are options and and the answer is always no, if you don't ask, right, so. So if you're thinking of something else, please ask.

Lindsay Velecina: 34:28

Right. Awesome. Well, thank you both so much for joining me today for this conversation. I hope our learners got as much out of it as I did. And I hope that our learners understand that they do have that choice and there are options out there to cater to their learning needs. So thank you, sir. Brina, and Robbie, again for joining today. And thanks for listening.

Ravi Verma: 34:59

Thank you thank you everybody