sos-transcript_working-group-on-integrating-technology-10-20-2017

And part of that is understanding how much time is going to need to go into the workflow, right?So, ideally, we're saying that we're saving time, right, but you're going to have to spend

Working Group: Integrating Technology October 20, 2017

DR FARRIS: So, we are missing our two healthcare perspectives here, but I think there are other healthcare perspectives in the room, so I think that we can make sure that is reflected

In my view, the purpose of this discussion is for us to be able to provide some recommendations moving forward about how we can improve integration of mobile health into I'm going to call it, systems So, it might bethe EMR It might be social service systems It might be educational systems

So, while we didn't present an educational system perspective this morning, we had a question about

it Paula is here, who can, I think, help us with that view, as well So, if you guys are comfortable with that question, then what I would like to pose is, from your perspective, given what we presented this morning, what are those next steps in important research questions or best practices questions from your experience, from your view? Everyone understand the question?

DR FAIRMAN: I can get it started One of the things that we looked at doing - because I did have the opportunity to look at the commercialization of

technology, and understanding who the stakeholders are andwho would potentially be buying this One of the biggest things is, well, how much does this cost? So, there's the cost of the technology, itself, but there is the cost of using the technology And part of that is understanding how much time is going to need to go into the workflow, right?

So, ideally, we're saying that we're saving time, right, but you're going to have to spend some time using the technology as well And so, workflow analysis needs to be a big part of that Workflow analysis is tied

to what the cost is of personnel to integrate the

technology into their day And whether that personnel is ateacher, whether that personnel is a physician, a case manager, whoever it might be, that has a dollar amount that goes along with it So, we have to be prepared to answer those questions

DR SMITH: The workflow is really important I know at the University of Michigan, we were trying to implement telemedicine in the physician activities, and they have their existing workflow So, we always tried to figure out, okay, how can we fit this in Initially,

telemedicine tended to take a lot more time because

they're getting used to it And then later on, I know with

tele-epilepsy consultations, they were able to be more efficient, you know, and they were able to actually save time So, ultimately, it comes down to time and money and work If it's more work, they're not going to want to embrace it.

DR KOHLER: I was going to point out just a challenge Because I really appreciated Sarah's comment when Seth said, well, how about getting this in health classes How do we move these things into everyday

practice and education?

And so, there, in working with NIDRR and NIDILRR, then I'm sure you appreciate the challenge

because we are funded to do research, right? Now, they have this technology transfer piece that they have added

to it, which is fantastic, but there's an issue that we continue to deal with funded by the U.S Department of Education, Rehabilitation Services Administration, Office

of Special Education Programs, as well as IES

Back in the administration before the Obama administration, there was a big push on reading Of

course, the No Child Left Behind Act and all of that kind

of stuff came about So, the administration was working with some reading researchers, and they funded all these grants to improve reading in various contexts The problem

was, to get the grant, you had to use this particular reading - commercially available reading curriculum That was a huge conflict of interest These were the people driving the decisions by the administration, funding was attached to it, and there was a huge blowup So, one of the things that we're charged - every time I go to the feds to work on something, we sign - when we are doing research, we sign that we are not proposing any particularcommercially available product.

And then, our other challenge is when we put things on our website and we roll out to this educational community at the state and local level, then, again,

sponsoring commercially available things without the

evidence of working, like that IES proposes, Institute forEducational Science – so, that's a challenge

So, I think when I worked - I was associate vicepresident for research at Western Michigan University before I took this current position and we funded

internally the work of faculty Most universities do this

So, how do you evaluate the proposals that come from

engineering, which they wanted to know whether this works

or not? That's what their project was We're developing this and we want to know whether or not this works

Researchers were proposing a study and theirs was all

based on methodology So, was your methodology going to besound and how were you going to control your variables andall this other stuff So, those are the two worlds that we're talking about here, but they're in this context of moving things into practice within the public or policy arena, commercial We can't support commercial things.

So, that's our challenge as we talk about movingthese into various areas, I think Or that's one of our challenges It’s not the only one Educating educators to use the stuff is another huge challenge

DR FARRIS: So, do we think that NIDILRR does a good, acceptable, whatever the right word is, in terms of

at least incorporating this notion of tech transfer,

allowing us to test commercially available products? Do Ihear that?

DR KOHLER: It's a great that it is there now because that's an improvement, definitely

DR FARRIS: Are you advocating that on occasion

we need to be testing commercially available products, I guess? That is a question for the audience

DR KOHLER: That doesn't necessarily mean randomized control trials because you are never going to get that very often in educational settings because it's just about impossible

DR FAIRMAN: I think the best-case scenario is apartnership with industry So, when it's only coming from the academic side or only coming from the industry side, the conflict of interest often gets in the way So, when you have a partnership, you can have the researcher on theacademic side ensure that there's no sort of moving the technology forward without evidence, right, and at the same time, you still have stakeholders who are able to kind of protect and also make sure that there's a place for this in the market So, if you have that partnership, that's the best possible scenario It's not always easy toachieve, but that’s really what is necessary because thereare often barriers in between academics being able to moveinto commercialization, which is what we spent the last hour or so with the last panel talking about.

DR KOHLER: Another issue is, as Seth said, there's individual education programs So, get it in the IEP Assistive technology is a huge place within IEPs Butthen the question always comes up, who is going to pay forit? So many times, it's putting as a burden on the

parents When you talk about working with parents and thentheir children, then that's hugely important because the parents have got to bring the need for this assistive technology if no one else is doing that, into this IEP

context At the same time, there has to be - the burden ofcost, if we are talking about things that aren’t 99 cent apps, need to be shared or paid by insurance or whomever Because it can't be just placed on the parents

DR LORD: I just wanted to add to the question about testing, testing apps, I think, and getting back to sort of are you testing to see if it works or are you testing, which I think is more helpful, is testing use cases - testing in terms of implementation science and testing using sort of principles of implementation

science, so that you really can understand when you

introduce something, like you - take a couple use cases where you introduce some assistive technology into an IEP,and then really study it with the facilitators and the barriers so that you can come up with some guidelines and some standardization around implementation

That can then help sort of create a foundation Broaden the implementation that ultimately gets to policy folks Hopefully, you're then able to sort of see more of

a system-level impact But you need to be able to measure

it in order to be able to then demonstrate it

DR KOHLER: Our whole center is based on the concept of implementation science It is really important.Yes

DR NEWMAN: Would you guys be able to explain what implementation science is? Maybe everybody else

knows I’m not familiar with that term

DR LORD: It's really sort of the, you know, taking a scientific approach, if you will, or a more

rigorous approach to implementation, to studying how an intervention or a technology, in this case, actually is integrated and used in a system of care

And looking at sort of different, you know - thinking about barriers and facilitators like what helps and what gets in the way, from a different sort of level

of analysis So, what is it about the intervention that maybe it's not a good fit for your client population? Maybe it's not a good fit for your educators Maybe the attitudes of your educators is what is getting in the way

of the system being used? Maybe it's an organizational thing, there's no leadership support And so on, maybe there's no money to pay for it So, it's really just

taking a rigorous sort of scientific approach to studying implementation

DR KOHLER: Yes, so we do intervention research,which everybody here is familiar with So, then the whole concept of implementation research is how do you move these scientifically proven interventions into practice?

With our center, for example, we work with state

departments of education And so, you look at things like doing an infrastructure analysis What is in place? What policies are in place? What do the data say about this particular context? And believe me, the state of Michigan differs very differently from Delaware from California

So, understanding those contexts

Then things like readiness, resources So, if you're going to implement say a full out technology

adaptation within a state, focus on a particular - you know, this intervention that we have proven works, then there are a number of variables that help you understand what you have to do to move that implementation forward And there are some folks at University of North Carolina, Dean Fixsen's group, which have done a lot of work on understanding these various variables We have our own model, but we have shown over and over again that it

works It is not the Fixsen model, but it is

implementation science

DR FARRIS: Mark, we have folks in family medicine and in the VA at the U of M who are quite

accomplished in this area

DR LORD: Laura Damschroeder is the one

DR FARRIS: Yes She is the woman She is the one.

DR NEWMAN: I am familiar with all of the concepts I just hadn’t heard that label before In

software engineering, all of those things are done In business IT, they all have different names for it

DR LORD: It is sort of implicit in your discipline, right?

DR NEWMAN: Well, no Not really But yeah, it

is sort of a different – when you spell it out, I’m like,

oh, yeah, okay, I can see what that level of analysis is

DR FARRIS: It is just getting different views around the table, right? And I think the concepts are the same, but the labels may be different

DR NEWMAN: I like it because it has science Wesay implementation all the time Usually, science isn’t part of it

DR LORD: That’s why it is really important to kind of shift the model So, I really appreciated the earlier comments about shifting a model from sort of a medical model, and it's the same in my field I'm a

psychologist When you think about intervention

development and sort of the treatment development, it would go from kind of your basic science and move through

efficacy trials and then move through sort of

effectiveness trials, and then you're supposed to

implement And by that time, you have such a non-feasible product, really, you've already shot yourself in the foot

So, really thinking about kind of implementation-focused development I think is really interesting and it's more inline with a business model

DR RAFI: Speaking from the ARC, which is one ofthese systems, we're on the other side of the wall waitingfor these wonderful innovations to come from you It's interesting that they're formulated on your side of the wall and then you decide what to do about it later Not always, but that seems to be more of the tendency

On our side, we're just taking whatever works Does this thing do something? Let's try it out That is more of the way the software industry generally works whenyou're trying to sell a consumer product You throw it outthere and see - spend as little money as possible, take aslittle time as possible to see if you can get any

validation that what you're doing is right If you’re wrong, you stop Rather than formulate an entire idea and then gradually get it out there and then the world has changed

So, I think something about like including the whatever we want to call it, implementation science plan, maybe in the way NIDILRR is thinking about -

-DR FARRIS: Development, as well

DR LORD: Yes At the beginning, you are always thinking about it

DR RAFI: I would say put your partners in there So, we’re talking with school systems and they’re

on the - because what you're really getting is you're finding out about the users The users are telling us on the other side of the wall, this is the kind of thing theywant and they're ready to digest it We'll see if they spit it out or if it nourishes them or not later

DR LORD: It really does help, you know, thinking about, you know, involving educators, involving teachers, in the development of an intervention Goes a long way at the end of the day when you're done and you present it and they feel like they've been heard It goes

a long way around then integrating it into their plan of care

DR HURST: One thing I wanted to chime in with

at the risk of introducing more terms and concepts into the discussion, was when we are talking about evaluation, really splitting out accessibility evaluation from

usability evaluation And both fields have a lot of

metrics, and I think one challenge with a lot of

accessibility – or assistive technology evaluations is they only focus on the accessibility pieces and not the usability They’re like, there's a button Someone could totally do it But they don't actually think about the amount of time or training or whether not that is how someone would want to do it So, I would love to see more

of that built in together

DR FARRIS: I think we are focused on a researchframework, but I don't think that that's bad because I think we're trying to move forward with some

recommendations about what those next steps are I do want

to make sure we don't leave out the healthcare

practitioners What thoughts do you have about next steps and/or need and/or whatever the right word is to connect mHealth for people with disability and their health

practitioners or our health system, broadly? Let's talk about it broadly, not just a hospital, but case managers, social workers, am care clinics, long-term care, the wholegamut Thoughts about next steps for that area? And maybe it's the same concepts we've heard for education, but let’s just have a discussion about that if we could

DR LOVE: I'm Clarissa I’m from the University

of Michigan My role is partially – part of my job is to oversee our organizational diversity, equity inclusion, strategic priority and plan for the system, which includesboth our health system and our medical school So, we are student – we service all of our constituents, students, faculty, staff, and patients We are a very outcomes and evaluative driven So, for us, we would like to see how can we embed technology from a standpoint of actually being able to, in the end goal, improve patient outcomes

I can't remember, somebody presented the Triple Aim, but that's really our goal So, how can we achieve that? I think that's so important for people to be

thinking and I'm so glad that Andrea brought up really thinking about the cost analysis, in terms of time and financial So how can we bring and bridge the gap and help improve the patient outcomes? I don't have a clear answer or direction, but that's really kind of like our need at this point

DR FARRIS: Right, but I think you've articulated a different view or outcome that's needed perhaps with different audiences And that is an importantone in our health system for sure The value proposition, which is coming straight out of CMS with the Triple Aim

DR FAIRMAN: One thing I think that might help

us is this new notion of pay for performance And it's notthat new anymore, really, but in thinking about where is the money going to come from and how can we leverage thesehealthcare dollars, because we can change outcomes

Bambang alluded to the fact that we had some great

outcomes with our initial study We were able to save close to $30,000 per person per year That's a lot And spending probably less than $1,000 per person per year

So, those are some pretty significant outcomes

But it's a big investment up front So, we have

to be able to show, one, how they're going to be able to bring in more money to cover that up-front investment, because most organizations aren't willing to do that So,

if they have a business model that works, and they can apply that, that's going to be the sell We can't wait forall the RCTs We can't wait for these really big numbers

Although I would like to say that part of - what

I think we're missing sometimes is that we could do a lot

of this testing with the average population So, we're trying to launch a lot of systems and a lot of unique ways

of managing healthcare, and trying to test it, initially,

on persons with disabilities when we haven't even tried it

on the general population yet And people are people, right?

So, if there are some major components there that don't work for the average person, why do we assume that it is going to work for persons with disabilities?

So, I think that's probably where our first testbed should

be and then move on to the disability populations and the unique ways that we need to sort of finesse the

applications to meet the needs of the population We're missing that sometimes, I think

DR MANN: How can we do this differently? So, I think a couple of pieces One, and I started to allude to

it earlier, is that when we embark on any development project - I think this is really - the divide between sort

of academic and industry and the healthcare business, which is just another industry, just needs to go away Fundamentally, I just don't see the path to –

implementation and dissemination science is a fascinating field to me because we're developing this whole science, when the reality is the implementation/dissemination

pathway in this country are businesses That's our

implementation mechanism So, you need to find a way to get there You're not going to get there - and whether that business is a healthcare business or a nonprofit

business, it is some sort of model that distributes stuff.It's not researchers who actually give things to people.

So, you know, fundamentally, if you're not thinking that way from the start and you're trying to build something that eventually, you want to change lives with, you're kind of handicapping yourself And I think that's what we've been doing for a long time The fact that the whole software industry approaches development differently is strange to me because they're building things that they give to people all the time

They're responsive and flexible and all that kind stuff On the research side, yeah, we get really rigorous results that have high certainty, high

reliability, but unfortunately, that's why implementation science has grown up, which in my mind, when I answer whatimplementation science, it is the same science as non-implementation science, it just prioritizes the process metrics, the things that it takes the facilitators and barriers, rather than prioritizing the reducing – raising

of internal validity, which is essentially what clinical efficacy is about, reducing all threats to the validity ofyour results

So, we get comfortable with some of that loss and what we gain is some understanding of how to actually

make things really work To do that in the real world, when you start a project, you need to either have the people who are going to disseminate your work and that is

a business leader at your institution or a commercial leader, either in your mind and you are empathic to them

or on your team right from the start Because that's the only way you are really going to understand how to develop

a system with that target audience in mind

All the talks were talking about your path to market or whatever, essentially your market analysis and all that kind of stuff That should be part and parcel with doing your lit review You do a lit review before youstart You should also be doing a review of sort of the landscape and what's your business model and who is the target audience

It's not that different and I would love to see more mashing up of those things from the beginning It doesn't mean you have to abandon science It doesn’t mean you have to abandon validity You try to do the RCT as best you can, but you have to kind of give a little So, sometimes the RCT may be shorter or smaller It may be more focused You may take advantage of a natural

experiment going on at the same time because the grant money wasn’t there at the time Maybe the grant you wrote

and what you study in the actual grant you do are a littledifferent and that's okay We all have been doing that anyway You just have to be more flexible.

So, if you have that in your heart at the beginning, I think you're far more likely to end up with aproduct that your consumers want rather than you trying tothen sell it to people

DR RAFI: Can I add to that real quick?

Something analogous to a lit review, like a user review,

to make a concrete construct around that Like have you done your user review of this? Maybe it is, well, I found

an industry that is into it I found a school system or health system or whatever That's one thought

Also, I'm thinking about the way Facebook decides what to integrate into it is it gets data from itsapps and third-party apps to understand all the other appswe're using When it sees a spike in us using some

particular new app, it goes and investigates that company and maybe acquires it and integrates it This is thinking largely about data the government is already gathering Weknow, in theory, everything that is in every IEP in the country, it is being typed into somewhere

We know a kid with these attributes is getting the following educational goals with this app or this