​Integrating Symptom and Outcome Measurement Into Your Therapy Workflow Our 10 Best Tips

​Integrating Symptom and Outcome Measurement Into Your Therapy Workflow Our 10 Best Tips

Discover 10 tips to incorporate symptom and outcome measurement in therapy

In this webinar, Jill Levitt and Maor Katz focus on including tools to track symptoms and treatment outcomes in therapy. These outcome measurement tools help therapists in measuring progress and improving client results.

Research shows that clinicians who do not use measurements in therapy are often poor predictors of their clients' outcomes. Meanwhile, therapists who use them can improve their therapy outcomes and reduce client dropout. 

Speakers emphasize using measurements, such as brief mood surveys, before and after each session. The pre-session measurement provides therapists with immediate knowledge of the client's current feelings. The post-session assists in evaluating the session's effectiveness. These measurements help in understanding the client's progress and ensure accountability in therapy. 

Feedback forms after each session help in assessing the therapist's empathy and helpfulness. Introducing these measurements to the client as a standard part of CBT therapy is important. In some situations, clients might refuse to engage in measurement activities. The therapist should then address their refusal and emphasize it as a necessary tool for effective therapy. Additionally, tips for suicidality measures are discussed, highlighting it as crucial for client safety.

This session focuses on using measurement tools to address progress in therapy. Watch the full webinar today.

IN THIS VIDEO:

Jill Levitt: Hi everyone. I'm going to, we are going to start the webinar together in just a moment. I'm Jill Levitt, and this is Maor Katz. I'm just going to get our screen set up, so give us a moment to do that. Just a moment. Okay, so at this point, you all should be able to see us and hear us and also see our PowerPoint presentation. So if someone would like to enter in the question box just a yes that indicates to us that our tech is all set and ready to go. Thanks to those of you, we've got a lot of yes now, so that's great. If at any point during the presentation you can't see us or hear us or see our presentation, you can also let us know in the question box. So I'll orient you to a few things about GoToWebinar and then we'll introduce ourselves in the content of our presentation. So on the right side of your screen, you should see the GoToWebinar control panel. About halfway down for most of you there is a little caret that says question next to it. If you have questions that you want to ask us, you can enter them there. We won't be answering questions throughout the entire presentation, but we'll take a break and for sure at the end, we'll have an opportunity to answer questions. We'll also ask you a few questions, and you'll need to enter your answers in the question box. And then finally, the only other thing to let you know is at the bottom there should be a caret that says handouts, and here we've shared with you this PowerPoint presentation. So you should be able to click on FGI measurement webinar, and that way you can download a copy of the presentation. So it's nine o'clock. Should we get started?

Maor Katz: Yeah, let's do it.

Jill Levitt: Okay, so my name is Jill Levitt, and I am the director of training at the Feeling Good Institute. And I'm here with Maor Katz, who is our director and will be presenting to you today our one-hour webinar on integrating symptom measurement into your therapy workflow to improve results, our 10 best tips. I'm going to turn it over to Maor, but I realize one other thing I want to mention before is that if you're here for continuing education credit, you do need to stay for the full hour. We'll be taking attendance, and following the webinar within about 24 hours, you'll get an email from us with the survey that you need to complete so that you can get your continuing education credit, and you'll also receive a copy of a recording of this webinar.

Maor Katz: Thank you, Jill. So I'm really excited to do this webinar today, and I want to mention that this is the first co-creation that we did. Basically, a few weeks ago, or actually yeah, a few weeks ago, we came to or sent this email to our audience and asked to see if there's an interest in a webinar like this, and we had such a great response, and people gave us just so much support, which was wonderful and very useful. Now instead of us imparting some knowledge on what we think people need to know, then people tell us what they need to know, and we can create it. And so though we had hundreds of support emails and dozens of suggestions and requests, and this was compiled into a list of 17 main questions that people had because a lot of the questions, a lot of the desire to learn things around measurement was repetitive. People have similar things, and so we wanted to make sure that we touch at least a little bit on every and each one of people's requests. So thank you for everyone who participated and helped us create this webinar. I want to give some background about Feeling Good Institute and through that also share my conflicts of interest of Jill and mine. Feeling Good Institute is an organization that our mission is to basically prove therapy to help people provide better therapy, and the way we do that in multiple ways. We train therapists in cognitive behavioral therapy tools and techniques with different training opportunities like this webinar, for example, and other courses. We provide certification, a roadmap for people to advance their skills in TEAM-CBT, this particular form of CBT, advanced CBT that we practice based on the work of Dr. David Burns, and this certification helps support and recognize quality therapists. We also have treatment centers. So more specifically, we have a treatment center here in Mountain View, where we're talking to you from, as well as in New York City, and we provide intensive one-on-one therapy in CBT in these centers. We also started online, basically an online center to allow for live video-based treatment in many United States, many states in the States and Canada. And we've also been developing some electronic tools to help provide for better therapy, including an integrated electronic medical record measurement service. And finally, what we do is we basically put together, I would say we've been teaching and training, and all of our work, I want to just make sure that we give credit to our mentor David Burns in teaching TEAM-CBT. So all of us have a lot of experience in this and very proud of this work. So maybe Jill you want to take over.

Jill Levitt: Sure. We hope we're not preaching to the choir here. We are actually interested in knowing how much measurement you use in your practice. So I'm going to run a poll now. Our question for you all is, do you use standardized scales in your therapy? So you should see, I'm going to launch the poll, and on your screen a question should come up: do you use standardized scales in your therapy practice? If you can please select one of the following options: not at all, occasionally, most of the time, or every session with every patient. I'm going to give everyone about a minute to click on one of the four answers, maybe ten more seconds. Okay. So let's see, I'm going to share the results of the poll with all of you so we can take a look together. So five percent of you said not at all, 41% said occasionally, 21% most of the time, and 33% of you are in the same boat as us where you're using measurement at every session with every patient. So from that, we can see that most of you, the majority of you, see the importance of using measurement, but it does look like, you know, almost about 50 percent are using it sort of, you know, occasionally. So let's just go back. So I'm going to talk to you, that's useful to know. It means we probably do have some helpful tidbits to share about how to integrate measurement into your workflow to make it something that you actually do regularly rather than occasionally. I'll share with you a little bit. We won't be spending a lot of time talking about research today, but there's sort of good news and bad news when you look at the research. The data shows there's a number of studies that show that actually without using measurement, without symptom scales and scales of how therapy is going, therapists are actually not good predictors of how their patients are doing. So there are a number of studies that show that the perception of the clinician alone is actually a poor predictor of client outcomes. There's an interesting study where therapists do several hours of evaluation with patients, and then therapists and patients both indicate symptom levels of what they think the patients are experiencing, and they're not highly correlated. So the bottom line is that clinicians are not great at mind reading and also not great at detecting worsening of symptoms over time. So that's sort of the bad news, means without measurement, we're not doing a great job. The good news is that actually there are studies that show that regularly assessing symptoms, so when they do studies where therapists actually are administering measures to their patients at every session, this leads to a decrease in patient dropout, therapists are better able to detect when their patients are doing worse or getting worse. And also this leads in the end to better therapy outcomes. So this bolsters the case for actually integrating measurement into your typical workflow, and this is certainly our experience as well is that if there are sessions where we don't use measurement or in the past, and we didn't use measurement, it turns out or you know, not that great at knowing how our patients are doing. I certainly experienced that.

Maor Katz: So another thing that came out when we were researching for this webinar, we realized that JACHO now created a new standard of care. And for those of you who don't know, JACHO is the Joint Accreditation Commission for Hospital Organizations. It's really the standard bearer of patient care, at least in the United States. And we're thrilled to learn that it's now a required standard of care to be accredited by JACHO in any hospital setting to routinely check, routinely find out from patients outcome measurements of therapy and make sure that they are actually used to inform treatment. So it's something that is regularly done, that's the standard, and that is taken into account and reviewed and helps inform treatment. And for us, this was thrilling to learn because we've been doing this for years, and now starting January 1st, 2018, here we go. This becomes a standard of care just throughout the United States. So this presentation is organized around the questions or the requests of the audience, and we organize it around tips that came to mind or seemed relevant given your given your requests or a tip.

Our first tip is the basics here, and what we're suggesting you do is for those of you, those of you haven't yet, is to use measurements before and after each session. So it's not a given that people would use both before and after session measurement. Some of you, I'm sure, have been using it because some of you are dedicated TEAM-CBT therapists, but I think many of you don't, and this is our first step, and we want to talk about it a little bit. And for those of you who've been using it, I think there's something in it for you as well just to think about how to present it to patients if the question comes up or how to justify etc. So here's why we kind of, here's why we really want to encourage you to test, to check symptoms before each session. So before sessions, if you get this information from the patient, it provides you with a really immediate assessment of your patient's current feelings before the start of each session. So there's no lost time, it happens before the session starts, and you can immediately start the session with it. This helps you dive right back into important, salient issues for the patient's. We're talking about feelings immediately rather than maybe the previous week or something like that. And also prompts the patient to think about how are they doing, what's on their mind, what would they like to work on, how is this progressing, what can they do? So it helps them think about it, it helps both patient and therapist be more accountable. So we like symptom measurement before each session for this reason. And then I think another great advantage, of course, is that those symptom measurements go into your chart, and then you can follow up over time and see long-term track of how therapy is doing and detect whether you're getting stuck, and you can bring that up with your patients as well to get unstuck and figure out a way out of this. Measurements after session are for a different reason. Right, it's a different kind of entity, it's a different kind of practice. You've done the measurements before session, and now we're advocating for it to be session as well. And after session, we measure both symptoms and an evaluation of the therapy session. We'll talk about that a little bit after that, why we like doing it this way and how you could do it comfortably. But the reason that we measure symptoms after each session is that we want each session to be expected to have some effectiveness to it. So it can give us immediate feedback and gives the patient immediate feedback that's helpful, and it sets again sets this accountability. We want our therapy sessions to be meaningful, and we're setting it up to be work that is towards achieving something useful. So that's one area. Then the other part of it is that it creates a link between the previous session and the next session. So if the patient gives the measurements after the last session, you can bring that up in the next session and immediately go back to do that. And that can help this feeling that often I certainly have in my practice where I feel like a new session starts, and we didn't quite finish dealing with everything that was brought up in the previous session with anxiety or depression or whatever we're dealing with. And now this provides a really nice way to just get that integrated and feel like there's a cohesive ongoing treatment. Jill.
 

Jill Levitt: Sure. I wanted to bring a little bit of myself into the equation here and share given that we feel strongly about using measurement, kind of why as a therapist I would really never sort of dream of practicing therapy without using measurement, and I really mean this. There have been occasional times if I'm using the paper and pencil measurement that, you know, we run out of copies, or for some reason I don't have my patient fill it out, and I do authentically feel lost in doing therapy without having that initial check-in and without knowing when the patient leaves the session actually how they're doing. So essentially, in my view, symptom measurement will accurately assess the symptom severity of my patients at every session. So I think of it like you go to a doctor and you are sick, they take your temperature, you're having palpitations, they'll do an EKG or they'll take your blood pressure. So I feel like I need the measure to know how my patient is doing at the beginning of every session as well. I find it incredibly effective in monitoring patients' progress over time. So my patient and I can look and see if they are getting better on the indicators that they want to be getting better, is their depression improving, is their anxiety decreasing, is their relationship satisfaction improving. Also, and this feels really important to me and I gather to all of you, I know at every session whether or not and to what extent my patient is experiencing suicidal ideation. That makes me feel much more comfortable, it makes me feel more responsible as a therapist. And lastly, and we'll talk to you a little bit about this, we're not just looking at symptom measures, we're actually also looking at a measure of essentially the therapeutic alliance or how the patient perceives the therapy session. This is well, I'm not unfortunately able to read my patients' minds, and so there are times where a patient will leave a session, and I'll think it was quite a fantastic session, and I'll get some feedback that's humbling but also incredibly useful. Or other times I think it was sort of a mediocre session, turns out my patient thought it was like the best session ever, and again, there's so much that I can learn from that. So I feel a bit like I'm doing therapy in the dark without measurement. I also wanted to share, I thought about it from a patient perspective, like if you are a patient, why would you only want to see a therapist who uses measurement? And I think from a patient's perspective, if I'm walking in to see a therapist and they're asking me to use these measures at every session, it proves the measures provide me, the patient, as a way to raise issues that I might not feel comfortable raising aloud. I have in fact had this being the case where a patient doesn't tell me. That they're experiencing suicidality impromptu and maybe even sometimes is evasive verbally but will actually indicate it on a paper and pencil or a digital form. And certainly, it appears to me that it's easier for patients to share with me negative feelings about the session, what they liked and didn't like on reflection in the waiting room. And we'll tell you in just a minute how they do that. Also, if I'm a patient, if my therapist uses measurement, it shows me that my therapist cares about being an effective therapist, that they want to provide me with positive outcomes and that they want to hold themselves accountable. And as a patient, I would have a lot of respect for a therapist that showed me that doing effective therapy is really a priority. And then lastly, as Maor touched on this, but using measurement will help me, the patient, to keep us focused on the issues that are most important to me because I'll be able to show my therapist each week what I'm struggling with in a very focused and specific and efficient way. So I'm going to talk to you guys now about how to integrate measurement into your workflow. So we'll talk a little bit about the timeline of events. So essentially, we introduce measurement during the first session that we have with our patients. And we do that using either a paper-and-pencil measurement that we have on a clipboard in the waiting room, we have a bunch of clipboards with the same measure, or and we'll talk to you about this a little bit later, we also have a digital measurement. So we have patients sign up online at Feeling Good Now prior to their first session, and they can complete the measure on their iPhone or iPad or device that we have in the waiting room. So we have the measures in the waiting room in some form or another. The patient actually walks into this, so the waiting room five minutes before the session, they complete the measure and either bring it into session or click submit, and the therapists can then see it. The patient comes into the session, and the therapists already has done this pretreatment measure, it's in their chart at this point. So the next thing that happens is my patient walks in the door, and I have this measurement, so I can actually, as Maor was describing start the session by checking in with my patient in a very specific way. I can look at their symptoms of depression and anxiety. We'll show you what the scales look like in just a minute. The scales that we use. I can also look at how my patient is doing right now as compared to how they were doing last week because I have their post-treatment scores from the last session, and I can also bring up any issues that came up if my patient gave me feedback about what went well, what didn't go well, if that feels important, I can check in on any kind of loose sentence from last week. Then I actually carry on and have my session, a full-length session with my patient, and then at the end of this session, I'll just remind my patient to complete the post-session measure in the waiting room again. This takes, you know, maximum five minutes of their time, not my time. And then that measure will go electronically, automatically into their chart, or if we're doing a paper and pencil, they leave it in the waiting room in a folder with their initials and the date, or you can use a lockbox if you prefer. And that's how I get there sort of post-treatment measure. So we decided that this idea of workflow and how you could actually integrate pre and post measures into practice was so important that we're going to repeat it one more time before getting more into some other specific questions. So I figured I'd have Maor do the second section play by play.

Maor Katz: I won't take it, do it fairly quickly, but the idea is that the reason that we want to bring it up is because a lot of people have their question said, hey, what about time? I don't want it to take more time, I'm so busy as it is, and here's another demand, another thing that I have to do. And what we're saying is there's a way to do this and bring it into your workflow in a way that doesn't take any of your time at all, and in fact, it makes your time more efficient. So what we're saying is that it starts with introducing it to your patient on their first visit or even before your first visit if you're able to use the online application, and then the rest is just in the flow. You introduce it to the patient. They do it before their appointment. By the time they come into your door, into your office door and sit down, you already have the report. You can talk about it. And then when they leave, you give them back that sheet of paper to do the post-session evaluation on the other side, or just remind them to do a post-session evaluation online, and they do it on their smartphone on their own after they leave. And then the next session comes, you have in your chart their pre-session evaluation, the previous session scores, and you also have the current session scores, and so very smoothly can work into your workflow. That's how we've been doing it for many years now. It works really great. Did you want to take it from here?

Maor Katz: Okay. Where's it, my turn?

Jill Levitt: I think it's your turn.

Maor Katz: It’s mine. Okay, so the measure that we use is the Brief Mood Survey that was created by and validated by Dr. David Burns. That's the one that we like. You're welcome to use it, you don't have to use it, there are other measures around. I think what I like about this measure is compared to maybe the BDI or the Hamilton scales, is that this was designed by David or Dr. Burns to be for patients, and it feels that it works very well with any kind of workflow. It's very much, pretty much feels like it's with inpatient terms, it's not, doesn't feel like a research tool, but it feels like a connecting tool. So we use a Brief Mood Survey before each session and we use it after each session. It really takes two or three minutes to complete, and it's been established in research that the scales are valid. Another great advantage of this form, and that's the last time I'll, well probably not the last time I'll toot for it, but it includes not only like for depression or measures for anxiety, it actually very quickly in two to three minutes you get measures for depression, anxiety symptoms, anger, which we find quite important to know when a patient comes in, positive feelings, it assesses, has a nice scale for positive feelings which often can be seen as like a self-esteem level, questions like I feel worthwhile and etc. And also relationship satisfaction. Relations are so important, so much of what we deal with in therapy, it's so important to all of us as human beings, and it's right there on the scale very immediately as well measured.

Jill Levitt: I'll just add, but we're going to show you what that looks like, but I will add that we both feel that using any measure at all is far superior to using no measure. And so this is a measure that we really like, but we also essentially our talk today is just promoting the concept of measuring, using measurement in your practice.

Maor Katz: Yeah, and how to do it, and how for you to feel more comfortable doing it and answering your question. So this is what it looks like on the web application Feeling Good Now. It's the before a session survey. You can see here just a depression, and notice that one thing that I didn't mention the previous slide is that we also take a measure up survey. We'll touch on that too in the coming slides. Now our next tip here is the feedback form after each session. So this is not a trivial thing. What we're saying is don't only use, we're suggesting not only to use symptom measurement before and after each session, but we're also suggesting you introduce after each session, only after each session obviously, a feedback form for the session. And the idea there is that it gives us information and it's another way to communicate with our patient. It assesses the one that we use assesses not only the helpfulness of the session and the satisfaction like in any kind of feedback that you'd expect, but also empathy scores. And also we assess the commitment, the patient's commitment to doing their homework, to doing what it needs, what he'd like to do in order to improve between sessions. So we go through all these all these questions, and again, doesn't take very long to fill out, two to three minutes, and you get all this valuable feedback. And I'll again this other opportunity to connect. I know I'll talk more about that in the coming slides. This is what it looks like, see, not very threatening. It kind of has that as a the one that we used evaluation for therapy sessions for you did by David Burns, it feels very friendly and not intimidating. And the way it helps is that it helps us as therapists become or address and be aware of frequent empathy failures that we do, and it facilitates discussion about them. The patient actually is requested, and it has an opportunity to give us feedback and tell us about difficult moments that they had or things that didn't work for them, and that could be so healing for the therapeutic alliance. And I can share with you maybe a quick something that happened, a story that happened to me just recently. I didn't have a good feeling after a family meeting that I had with a patient. We were, I don't know that I, I was feeling like I was wondering if I got into like a little bit of a power struggle with my patient. My patient was struggling with alcohol over years, and I kind of made it a point for the family to know, of course with his consent, and discuss it. But then he felt kind of on the defense. I didn't exactly know that I was concerned that that might have happened. And I was so grateful that I had that post-session evaluation because he totally expressed it. And he said that my empathy was poor and that he felt uncomfortable during the session and didn't like how much I kind of emphasized the alcohol. And that gave such a great opportunity, an easy opportunity for the relationship to heal. Because I could then bring it up immediately in the next session. I could even, I actually decided to reach out to him between sessions and mention it, and we decided that we'll talk about in the next session. But it gave this immediate opportunity to heal, and I'm not sure that I would still have him as my patient if I didn't have this tool to help us identify it and heal it and get into the conversation and connection around it. I think after that, after the result of it ended up rather than that kind of an empathy breach where it started, and now I feel like there's better and closer kind of connection and better intimacy and therapeutic alliance as a result. So let's move to the next tip that we talked about, which is we want you to think about it as a therapist extender. Is that the measurement allows for the therapy session to continue after it actually ended. It actually also starts it, helps it start before it even started. So without missing any of your time, it actually extends the patient's experience of therapy. And we want you to also think about it as an actual therapeutic intervention in and of itself. It's not only just a required standard, and that's the standard of care that's annoying, but it's actually a therapeutic, a helpful therapeutic intervention that's going to make your therapy more effective, improve your therapeutic alliance, and also an opportunity for different kinds of interventions in itself. I can talk more about that, I think I will in the coming slides as I kind of alluded to before. Think of the measurement part as an opportunity to connect. You start the session, this goes a little bit to the workflow as well. The way that we do and I would recommend doing it as you start the session by reviewing the current symptoms, right? What you have in front of you after you're doing this for at least one session, you have in front of you the previous session evaluation, and you have in front of you at the beginning of a session the current symptoms scale. And what I'm suggesting here is you start always by the current symptom scale for the obvious reason of wanting to connect, wanting to be with what's pertinent now for the patient. There were a number of people that were worried about what do we do, and it feels too, won't it feel really technical? What do we do, and it feels kind of mechanical when you're just reading scales, and it all feels really technical, and CBT already has this bad reputation for being or can be kind of technical and cold, and that could be a problem. So the way that you can solve this problem is use your own casual language to summarize how the patient is feeling right now.

So rather than saying, I see that you feel sad and down kind of three out of ten and I'm feeling kind of discouraged five out of ten, you could go and say, oh, I'm looking at your scores here, and I see that feeling kind of down today and that there's some depression going on and more than anything feeling really discouraged right now, is that right? Is that so? You use your own casual language from the information you got from the skills to make it less mechanical. And this way you get the full advantage of the skills in that it gets you into therapy more quickly. So the session starts, you immediately are talking about feelings, you're merely talking about how they're doing right now and with really no time wasted making it, making your time much more effective. So another example that goes through this kind of warm language of what's not technical or mechanical is saying something like, I can see that your mood is pretty good and that you're not feeling as down and pressed as you have before, and at the same time, you're not feeling very worthwhile and your self-esteem isn't feeling very high right now, is that right? Am I getting you? Is that true? And you can again start the session. This is like seconds into the therapy session, and you are already talking about what's important, feelings, right?

Jill Levitt: So I'm going to talk a little bit about how you introduce it to your patients. For sure, we get this question from people, which is, I don't really know how to, not just integrate it into my workflow, but how do I talk about it with my patients?. I will also say that I don't have data on this, but I would say how you introduce it to your patients probably largely predicts how your patients respond to it. So oftentimes we have questions, which we'll actually go over shortly, like what do you do if a patient doesn't want to complete it or complains about it? I never have that happen basically, and I think part of it is because of the way that I introduce it to patients. So a little reluctant to type this up, but we also thought it would be helpful for people after the fact to be able to review this. So this is a lot, these are a lot of words, but my next slide has kind of the pointers that come from this. So if my, when my patient comes into the first session, the evaluation, and basically the first time that I meet them, I say something along the lines of, you know, I'd like to just show you the measures that we'll be using together before and after all of our sessions. Here is a symptom measure that we'll be using that will help me get a quick and really accurate sense of just how you're doing, how you're feeling when you come to see me each week. I'm going to have you complete it in the waiting room, and that way it won't take up any of our session time, and then you can bring it into your session and we'll review it together. And that way I can see kind of just how you're doing when you come into session. What's really helpful about this measure is not only that it's a quick way of like me taking your temperature or your blood pressure each week, but also we'll be able to track your progress over time. So you and I can both be accountable for doing really good work together, and we can see how things are going and if things are improving in the way that you want to, and we can stay on track together. Also, after session, I'll have you fill the same measure and also have you give me some feedback about how the session went. And I find this invaluable because although I would like to think of myself as really really perceptive, there are surely times where you might experience our session in a way that's different from mine, and this is a way for you to be really honest and open with me and for us to connect about how each session is going, and it'll keep us on track. And I also say something like, you know, I give you my word that I will humbly receive this feedback. I'm actually oftentimes excited to receive criticism because it really helps me to connect with you and to understand better how you're feeling about how our sessions are going. So I think of this as like a quality assurance measure that we'll use to make sure that I'm providing you with really good therapy. So I say something along those lines.

Maor Katz: Do you go through all of it with your patients?

Jill Levitt: I do. I say, show them the measure, and if I'm using a paper measure, I kind of show them the symptoms, I flip it over, I show them the back, and I say this is helpful for this reason, and this is helpful for this reason, and that I'll have you fill it out before and after the session. I might have been more verbose just now, but I do a bit of a description of how the measure is used and why I think it's really helpful.

Maor Katz: That's really cool. I noticed that I probably spend less time introducing it to patients, and I don't find that it takes that it brings up more resistance even so in embrace this. Oh man, it needs to come up if there is some sort of resistance and not necessarily has to all be there. I'm saying this because I'm wondering if some of you feeling overwhelmed.

Jill Levitt: I need to memorize information or so, and we all have our communication styles. I probably err on the side of overly verbose. I also talk with my hands as you can see. But yes, I think you can streamline it a bit. The point is that we bring it up, and I'll move to this slide. The point is that we bring it up in a positive way. I think this is about expectations, right? If I describe in a way where I have positive expectations for how the measure will go and be perceived, then my patient will also likely buy in. I also describe it as you notice, it's just a part of the therapy. So I'm not saying like, oh, there's this thing I need you to do, but rather as a part of our work together, here's what you'll be doing. I also always make sure to make use of it. That means I'm not going to ask my patients just like I don't assign homework and then not to go over it. I'm not going to ask my patient to fill out this measure and then not look at it. Right? I look at it. I'm making use of it. If my patient gives me negative feedback, I'm sure to bring it up and to talk about it in a warm and interested way. And I always require. Right? It's not kind of optional. We use it occasionally. I think the don'ts are as important as the do's here. I don't ever apologize for using measurement. This isn't, oh sorry, there's this thing you have to do before session, right? It's just, I actually say, here's this thing that I find really useful, you know, here's part of therapy. So I don't make it seem optional at all. I actually don't let patients begin sessions without it. So this is one thing that occasionally happens. The patient will be running late to session, we'll just walk in the session, and then I say, oh, did you not grab the clipboard or did you not fill out? I'll just give you a minute here in session to fill it out. Sometimes patients look at you for a minute, but in fact, if you think about it, you're really training them not to be late for session, use your own time pre and post session, and then we won't have to use therapy time. And then the last thing which feels really important here is don't sort of chicken out on reviewing the negative scores. You know, it can be really humbling, especially if this is new to you to receive kind of negative scores meaning patients are saying that they didn't feel that you were warm, they didn't think this session was very helpful. In the end, for me, this is like the most useful aspect of the measure is getting negative scores. As Maor gave you an example, it usually turns kind of a tension into sort of like a breakthrough, a moment of connection and deepens the work that I'm doing with my patients.

Okay, so we're going to actually ask you a question. So we're going to ask you to move over to that sort of like question area of the GoToWebinar and answer this question. So share with us, what do you do, what would you do if your patient refused to complete a measure?. So you're asking them to do this as part of therapy and your patient's like, I don't know that, I don't like this. This measurement takes up too much time. Do I really have to complete these measures? So just type some, type in an answer and share with us what you would do, and then we'll share with you what we would do. So there are also some questions you guys have answered or asked that we might want to look at at the end. So I'll read you a few answers we got. So if someone said explore their refusal, and someone who knows the kind of therapy we do, TEAM-CBT, said use the gentle ultimatum, which will. I'll actually talk a little bit in my response. Let them know the importance of it, and if we can still maintain a working relationship, someone said offer to make the form shorter but insist that some form of measurement needs to be done without getting into a power struggle hopefully. Yeah, that's a good one. Let's see, open it up for discussion and find out the reasons, negotiate with them. Here are some ideas you have: use empathy, share why it feels important. Great. So I see lots of answers here. Should we go ahead and share what what our response, Maor, yeah?

Maor Katz: As we were thinking about this, we're Jill and I were preparing for this webinar, we were saying this is just so rare. This is the number one concern raised by you all when we were doing when we're asking people what they want to learn about integrating measurement into practice, but for us and our practice, it's just so very rare that we get into this issue. So I think the number one when I was thinking why is that, and I think the number one reason it doesn't seem to happen I think is this non-apologetic, this is the system that we work in. And patients expect to have, you know, every therapist have their way of doing things, and they prefer things to be effective, and we meet this problem very rarely, but it does happen. When I was thinking about how good idea to answer this question, I was thinking what, how did I deal with it the last time it occurred in my practice? And I think the last and probably the only time it happened in my practice in and then in my memory is I was seeing this teenager, and teenagers can be a little bit more kind of easy to get into power struggles with. You know, they as they as they go through as we all go through teenage years, we question things, and we want to assert ourselves. And so I remember this teenager, lovely teenage girl that was in my office, and she came in, and first session I, you know, she introduced the the measurement tool. She says, oh, you're doing measure, you're doing these scales again, paperwork, you know, I I've done all of this at Stanford when I was there, and you know, I don't want, I don't want to do this. This was terrible. And I felt like, mmm, it's my first session with her, I don't think that I need to get into this power struggle. And I said, okay, that sounds good, totally understand it, and I empathize with it, so annoying in the scale, and I moved on and reintroduced it a couple sessions later. We already had a rapport, kind of liked each other and respected each other, and I reintroduced it. So and since then, she's been using the measures without any kind of problem. So that my point is that I was able to avoid the power struggle, and I was not forgetting, but this isn't the way I practice, and I let the report on a bill and reintroduce it. That's one way. What do you do?

Jill Levitt: Great. So I figured I would share my thoughts about this as well, although I said, as I mentioned before, I honestly can't think of a time where I've had a patient refuse to use the measure. So I couldn't, I couldn't think of like a real story to share with you. Zero, like zero, right? No, I mean, all of my patients do use the measures and don't complain about it. So, you know, what I do, at the same time, I do a lot of consultation with therapists, and I have that therapist bring this up to me. So first of all, again, I repeat the whole idea of how you introduce it is so important. But the other thing that I teach when I talk about this with other therapists is the idea that I would say to a patient if this did come up was, I would empathize for sure. So we have a bunch of skills for really tuning into how our patients are feeling, so I would empathize with the frustration or the time-consuming nature, how busy they are, if it feels mechanical or things like that. I would also talk to them about it. There may be ways that I can help them to feel more comfortable with it actually. But then essentially, I would let them know that while I totally understand how they're feeling, at the same time, I actually feel I, I would feel kind of concerned and uncomfortable doing therapy without it. I would feel sort of like a surgeon trying to operate without his scalpel or cardiologist working on a heart without, you know, an EKG or without my objective measures, and that I. Wouldn't feel like I could give them, offer them good therapy. I wouldn't feel good about the work and wouldn't feel like I was giving him kind of my best effort. And so I would essentially say, I would feel uncomfortable trying to treat you without it. If you want to work with me, well, you know, I'm going to ask you to complete these measures and expect you to complete these measures. At the same time, if you feel really uncomfortable using these measures, I would be sad to see you go, but I would totally respect that, and if you feel like you need to find a therapist who doesn't require measurement of you, I would be happy to try to find you a really good referral to someone in the community who, you know, doesn't use measurement. So essentially, this is what I think. Someone mentioned that what we call the gentle tomato, I'm sort of explaining my limits, the way that I practice therapy, and at the same time empathizing and saying that maybe not not be what you're looking for, and I would respect that and understand that, although I'd love to be able to work with you. So that would be another another option, right?

Maor Katz: Yeah, you could even make it sometimes we call it the gentle gentle tomato where the it's not that you'd refer them to someone else or you know basically say you can't work with them, you can say I'm willing to go along with it as long as you understand that I'm probably going to be less effective and this is I'm not going to work as well for you. And that's I think that is pretty unlikely that a patient would want. So yeah, so that we can move on to our next kind of tip here, and in this we want to encourage you to not think of yourself as a scientist, but be a therapist. And by that we mean the scales, they're not there, and they're not part of a science experiment that we're trying to collect the most accurate data. That's not our goal. Our actual goal is to be able to connect with our patients better and to be more effective for them. And so we want to use them as a communication and therapy enhancer rather than data collection. So we're not so worried about the numbers that way. We just, we don't really, it's not that important to be exactly correct and exact. And this comes with the reason we're bringing this up is because a lot of the questions that we did receive from the audience when we were creating this is what do we do in there's like these big discrepancies. Like in this brief mood survey, you see that the patient is scoring not at all but every and each symptom, and what do you do? Many of you asked, what do I do if I sense that there's suffering going on, that there's maybe depression or anxiety, their symptoms definitely there, and patients just not reporting anything. Or another thing is when what we often see and that's not uncommon, I definitely see it in my practice as well, when people are kind of just really maybe afraid of conflict or not saying something negative about me about the therapy session and then just say no negative symptoms at all, no negative feelings at all around the evaluation in the therapy session. And it might be small for you to see, but you can see here from this scale that there are no negative feelings, no difficulties with the questions. And notice the bottom two questions are actually, they're kind of small and it might not see them, but they're my favorite in this form which are what did you like least about the session and what did you like most about the session?. And in a way, if that was only these two questions by the way, they were my only evaluation of therapy session measurement, I would probably be okay with that. That would kind of give the answer to like 80% of what I'm looking for on this form. But in this example, what we're saying is say that this particular patient, and that happens not infrequently, says, would you like we do the question, what did you like least about the session? The answer is nothing. What did you like best about the session? Everything. And so we're kind of left with really meaningless data. And so what do we do with this? If you remember my previous tip, it was don't think of yourself as a scientist. Here, this is not about the accuracy of his response. This is about therapy and connection. And so you can use it as an opportunity to connect, use your relationship here as a petri dish.

To communicating abilities and being able to improve the patient's ability to connect with people and bring up some difficult kind of issues with people in their lives starting with you. So we bring up the discrepancy. We kind of said, hey, I noticed your scores show no hint really of depression or anxiety and that surprised me because my sense was a little bit different. Was I just sensing it wrong? You know, one gently asked a question that might bring up the patient to to dare to say something. You can say, I noticed you didn't mention anything you didn't like about the last session. It kind of made me wonder how easy would it be for you to share with me any negative feelings about the session. And again, let the patient think about that. And you use the evaluation of the therapy session form basically as an intervention in and of itself. And very often with many of my patients who have kind of a little bit of conflict avoidance and feeling uncomfortable, and that affects their lives and other relationships, I ask them as actually part of their therapy homework is to make sure to write something negative clearly after each session on the form. And that serves as an exposure practice for them to treat their own conflict avoidance, and I presented as such. And again, an opportunity that these kinds of forms and these measures can help enhance therapy and make make us better therapists and an opportunity.

Jill Levitt: One more procedure which is only for those of you who are what we call scientist practitioners and who would feel concerned about this statement that our goal isn't to collect accurate data. I'm only going to say that our goal isn't only to collect accurate data. And when I say that, I mean because we're using reliable and valid measures, our tool certainly is to get really accurate information. But the idea is that if the patient is giving what looks like inaccurate data, we're not approaching it from the perspective of whom this doesn't look right. I don't think you're filling this format correctly. But Maor's beautiful warm response of. You know, I noticed that your scores are showing this that doesn't totally match my experience. Let's talk about this so I can understand you better. And that often it's really a sign of something clinical that's happening, right, where they're feeling anxious about sharing their feelings with you.

Maor Katz: Yeah, not only a scientist, right. I like that. Our next tip today is about the suicidality measure. So we really want to encourage you to use measurement scales that includes personality assessment. We really like David Burns's assessment tools, and that's what we use both in the paper pencil version as well as the copy here they see of the online version. And that came out also for people in the and the feedback, what do I do if if I don't detect, I do detect their suicidality? I think don't detect it with if it doesn't appear the scales, and if I do detect what do you do with it? So the first thing I want you to ask yourself is do you prefer not to know or to know if your patient is suicidal? Because I noticed that I feel kind of a perception knowing sometimes, but I do know that I want to know. And I want to share with you a case that happened to me maybe two or three years ago now. And this was a patient that came into my office. It was a patient, it was new to me, and he just signed up on my online calendar and showed up for intake appointment. I didn't know much about about the patient, actually didn't knew very very little. And when he came, I just gave him to fill out the before session measurement, which is basically a brief mood survey, David Burns. Yeah, I should really not see all right. So that this patient came in, and this was his personality or so, 99% of my patients come in, their suicidal scores are zero, they do not have any suicidal thoughts, nor they intend to end their lives. But here I knew nothing about this patient, and I was so thankful to David Burns for creating these measures because I saw that he was actually very, very suicidal, and I knew nothing about him but I knew the most important thing about him. And this allowed me during this short intake session that we had only just less than an hour to connect. I made, gave me the opportunity to create an intervention right then and there. And so we immediately started talking about this, I could immediately do a suicidal assessment, I could call his wife in. And it so happened that for him once he didn't feel a connection and he was able to communicate, my assessment was that he with these measures that I took, he didn't need to go to the hospital. I could definitely, you know, I could have if I felt like that was necessary, and I was basically able to spend the session on the important things here. I think the other if I didn't have that measure, I might have learned about it, you know, and I started feeling comfortable asking him these kind of more difficult, more intimate questions about Mommy genders life, and I would have been like 40 minutes into the session and that would have been disastrous potentially. So yeah, do you want to say something more?

Jill Levitt: No, we have a kind of similar, not the same at a similar response on our next slide. And I'm also just noticing that we only have five minutes left in our webinar, so I'll go through this quickly. So the other question that came up for sure. From all of you was what if my patient isn't making much progress and I'm using these measures and I'm reviewing this and I'm showing them this, will the patient be discouraged? So again, our response is similar, like would you rather know if your patient isn't making progress or would you rather not know?. Right? Isn't this something that in fact is useful for you? Would you prefer to sort of be in the dark about it and blindly keep seeing your patient or would you like to know that actually you're not, you know, they're not getting the results that they would like to be getting or even that their symptoms are getting worse? And even if that leads to frustration on the part of the session, essentially we want to use our empathy skills to connect with our patient. And also I use something that we call the kind of positive reframing which is to say that, you know, I have something here, you're right, your depression really hasn't changed very much over the last few sessions and I'm feeling a little disappointed and frustrated too because I'd really love for you to be getting, you know, what you're looking for out of these sessions and for us to be really making a dent in your depression. At the same time, I'm really glad to know this and I feel kind of hopeful that in us looking at this that we can talk about it, we can think about what's working and not working in our work together so that we can try to get back on track. And as a therapist, I also know that all of the thinking in my mind, you know, what are the reasons for this lack of progress? Is the patient not doing homework? Does the patient need longer sessions or more frequent sessions? Are there motivational obstacles? I'll seek supervision if I need to. So the idea is that measuring progress and actually seeing the patients aren't making progress is incredibly useful. We want to quickly touch on measurement with specific populations.

Maor Katz: This is another question that came up. There's a question about how to, how do we do measurement with teenagers? The answer is exactly the same. I think a rule of thumb is kind of twelve year olds, eleven, twelve year olds totally capable of filling out before and after session measurement, actually going to do it pretty honestly and accurately in our experience. Younger kids, we have much less experience with kids who are younger than that. They do have different scales. We definitely, I've talked about it with my dear friend and colleague Jacob Towery’s, a child analyst and psychiatrist and wonderful TEAM-CBT therapist, and he uses scales and sometimes just modifies the kind of skills to be used before and after session with his younger kid population as well. There was a question about what to do when you working in a residential setting, and this is kind of together with intensive treatments. And so what we do where we recommend doing residential setting is kind of thinking that figuring out once a day of what, when would be the right time to do measurement. I don't have to do it necessarily multiple times per day, but sometimes even in residential settings there is maybe a case manager for the case or something like that that can be the meeting with her could be the time to do the pre and post session evaluations. With intensive treatment setting, we do that here in our intensive treatment. So people come in for, you know, it's somewhere between three to six hours per day, and what we do there is we don't check measurements before and after each hour because that would be kind of tedious and redundant and not necessary. So we look for the natural breaks in the day, maybe the morning session, an afternoon session, and we do before and after and before and after for each session like that and just think of it as a block of time kind of way. It was another question that came up about electronic use. We've kind of mentioned that here today, and I'll introduce the electronic measures that we use. We like him a lot. It's basically designed them to work with our workflow, and they become more pertinent now because of also doing a lot of or more of video-based therapy. So ability to communicate the skills as well. So I want to introduce to you the Feeling Good Now application that that we're now making available. It's been used only really for research and kind of in beta use until very recently. They'll today basically almost, and what we came up with, the thing that would be most useful for us in the way that we do therapy is to have an electronic medical records and measurement integrated system. And it's HIPPA-compliant electronic record medical record, and it flows really seamlessly from any patient's device into the therapist's electronic medical record. There's zero setup, it's really convenient, you know, it's very clearly been designed by therapists and not by engineers in the sense that it's not like a complicated remote control, you know, it's a very, very simple design eliminates all paperwork and makes it very easy to track all symptoms. It does include David Burns's scales, so it includes depression and anxiety and suicidality, anger, and the self-esteem measures and the relationship satisfaction measures as well as the therapy evaluation scales that we like using. And so maybe to use this opportunity to thank Dr. Burns for his help with designing it so it can be maximally used, useful and simple. We are, for those who can, who are interested in and getting onto this program of using Feeling Good Now as we're launching it, we are giving some special perks for people who are starting to use it because we really want people to do, you didn't get the benefit of this of this work. We're using it in our practice, and we came up with this kind of early access program that gives some extra benefits, and what we're basically want to do is kind of similar to them through the webinar today, but live, we want to give people that are interested that are signing up to use it clinical support in integrating it into their practice. So one of the I think the nicest perks about the see early access program is that it any anyone who signs up first 200 therapists who use it, we're committed to giving them as much limitless clinical support as well as technical support obviously from our experienced clinicians from Feeling Good Institute on every step in aspect of the use of your patient. If you're starting to use it and you feel unclear what to say, how to say, how to approach what you know, want to rehearse something, want to bring something that didn't work clinically or something that you'll basically have limitless limitless access to a lot of our clinicians for the short sessions to help with that. We're also offering price lock-in because we're trying to expand the we're working on expanding the service to add intake forms and diagnosis and intervention forms and tools, and you can see more about the early access program if you go into feelinggoodnow.com, and we're kind of out of time. We wish, I don't know. You can definitely contact us. My email is there. Feel free please to contact me with any kind of questions or thoughts, and Jill, I'm sure you feel the same.

Jill Levitt: Absolutely. So if people have additional questions, we designed the webinar to answer all the questions we had already gotten, but if you have additional questions, you're welcome to email myself or if you have questions about using measurement, if you have questions about the measure we use, if you want to ask more questions about this tool about feelinggoodnow.com free to reach out to us. We do need to end the webinar now, and so I also just want to remind people that you'll get an email from us in about 24 hours and to please, if you're looking for continuing education credit, complete the survey and submit it, and then you'll receive your continuing education certificate. I also want to say thanks to everyone for your help and support in creating the webinar and attending. Yeah, thanks everybody.