And just see how to make a big.
Okay, talk to Nordstrom.
Do you want to go ahead? I certainly do. Thank you very much today. We're going to have an interesting presentation and I just want to introduce our speaker Dr jaw.
He's an academic radiologist and associate professor radiology at the hospital at the University of Pennsylvania. He got from the University of London.
It can't be United States. Well, according to him, he came to the United States for better training.
More predictable weather and a larger yard at his house. Now, he says he got 2 of the 3, but I don't know which of the 3 of the 2 of those are his scholarly interests include uncertainty decision making.
And the value of diagnostic imaging, he says.
And I'm hoping today that he'll blend all 3 of those together, as he answers for us the question.
Can an algorithm be sued doctor John the, the podium is yours.
Thank you, thank you for the.
I'm going to begin to.
My screen.
Eva.
Tight.
Of the school is.
The 1st thing to say is that this is a speculative area.
Because in radiology.
Has just started we don't know how things are going to go.
What are the adoption is still very minimal.
What we do know that.
In the United States in particular is shaped by.
2 pillars science.
Being 1 of them, or the other is the legal area, which is also shaped by science, but perhaps not as much.
As we believe we will be affected by the regulations.
And tools, and the 2 will feed off each other.
And we do have history on certain legal principals on which we can make.
Some educated guesses, which is correct, it's going to go.
So that overview of low medicine.
As some rules and those rules sprout from case laws.
Precedence is that are set is also the practice of defensive medicine.
In which physicians respond to lawsuits, or the threat of lawsuits, change their practice accordingly.
And then finally, there is standard of care, which kind of friends and everything, which is what, a reasonable position.
Should do that's important to bear in mind because the standard of care doesn't just come out from nowhere.
Have a reasonable for physician practices is the way it is defined, but how does.
Or know what a reasonable position as well.
Some of it comes from literature. What's in the publication? Societal guidelines.
Um, a cynical view would be that, uh, some of cares determined at the time of the lawsuit.
Uh, by the expert witness, standard of care has a Grazie.
There are certain things that almost certainly the way physicians practice at the moment but there was a time when there's some ambiguity and that ambiguity relates to technology adoption. And this is important to understand when we think about.
Because at the moment is in this part of the technology adoption technology, adoption, as a segment shape where there's a slow growth.
And an acceleration, and then that's them taught when things are catching up and there's no more need. Everybody is kind of caught up.
So, to give you an example at the moment, a C. T scan.
It's pretty much standard of care for suspected acuity, aortic dissection. You don't even need to think about. There's no ambiguity that, uh, I think in the seventies.
It wasn't because he was just.
Coming out and then there was a reason period where.
C. T was competing with other modalities. It was just about.
Getting there, but it wasn't quite there and that razor period is a time where you actually have most uncertainty and the most amount of propensity for lawsuits because right now, um.
If somebody suspects, acuity, aortic dissection, they will get a lawsuit isn't going to be you didn't get a lawsuit. It's gonna be you did not suspect acuity, aortic dissection but there's a point where.
The lawsuit was, you got the wrong test for the right suspicion.
So, how does that relate to air? Well, to give you an example, which is going to be quite recurrent? Um, let's say that there is a algorithm for nodule detection characterization.
At the moment, it's not standard of care at the moment. It's very experimental, but if there is a point where, where do you all is left right? And center are using it and it becomes part of society guidelines, then it will become standard of care.
That's 1, way of looking at sanative care and another way I think your standard of care is.
I'm thinking of it as a interaction with the legal system. So standard of care tells.
The more courts when a radiologists or physician deviates and also.
The more courts tell radiologists and physicians what the standard of care or to be and it's kind of an iterative process. Um, and.
This is where the lions get very blurred between defense admits and and safety.
So, to give you an example of that.
Um, at the moment we.
Nobody has actually said this quite explicitly, but in many places after a knee injury.
They get a of the lower extremity.
To look for injury to the archery.
And this has come partly because of what people do, and partly because of lawsuits. And so.
Standard of care is both peer pressure what the reasonable position would do, but it's also aspiration.
It's aspirational because you may be doing a certain thing a certain way, and everybody might be doing it, but I might not be good. And.
Your courts have said more than once that just because a reasonable position.
Um, does that make a certain way? Doesn't mean that the whole community has a.
Oh, doesn't try to improve themselves to 1 particular lawsuit, uh, versus carry.
Was instrumental in that where the plaintiffs food, the.
Um, for failing to diagnose glaucoma mainly just said, well.
The plaintiff is less than 40 years old as a 1 and 40,000 chance of glaucoma very low chance and it's not standard of care to actually test.
All that in this particular group and expert witnesses agreed in the lower Court's also agreed within the.
The Supreme Court, the state Supreme Court overturned that and says.
And cited an earlier decision, I'm actually 1 where all of a wonderful homes came up with a very if you line saying that just because certain things have done a certain way doesn't mean that that ought to be the standard care.
So, you can see the standard of care is very much shaped by.
The locals and that could very well happen with with where you have a adoption of air, and that adoption might be slow.
And and the courts might say, well, can you hurry up? Can you make this more of a, um.
A technology that everybody uses, if it is deemed to be that. So, with that background and standard of care, that's 1 component. What how we want to think about this. Let me.
Take you through certain scenarios and, um, speculate might happen. So radiologist picks up lung obtained for trauma trauma where they get a C. T scan of the head.
Neck chest, pelvis, uh, lots of images to look through and you're looking for fractures looking for things that could actually fill the patient in the next 24 hours. And this is the classic missed area.
Era where so overwhelmed and focused on the acute that you tend to forget the chronic and the radiologist misses a.
It should be misses, not picks up and misses a London audio and the audio goes on to become cancer.
Can radiologist be suited for not using ai so be.
Particular scenario here is you have lots of images.
There's a missed module and a, I.
Algorithms are out there that help radiologists, detect these articles.
Um, we always just make today gets for missing.
The missing the material and that can arise.
Independent or whether or not the radiologist decide users, but as this technology gets more adopted, it could very much.
He said that you had an adjunct.
And you had an agent that could have helped, you pick up this, not you, your colleagues are using it.
You should be using it too so that extra factor.
Of missing a, when you need to.
So, of course, the very forgiving of perceptual areas of human errors.
And There'll be multiple scenarios where.
Courts have said just because you've had a sexual error doesn't mean that you're a bad radiologist because we think that this is part of.
The human nature, the atmosphere and there, but to have the sexual era and to have a tool at your disposal and not to use that.
That could be problematic and not today, but could be problematic as time goes on. And the, the.
Algorithm gets adopted scenario number 2.
So the 1st scenario was radiologist misses this scenario.
The radiologist picks up on Jesse to obtain for college trauma.
And, um, an audio is, let's say 12 millimeters.
It doesn't look completely benign too.
The, I, and the radiologist ends up recommending a biopsy.
A patient under, there's a biopsy. The nodule is benign. It's not malignant patient as a new for access the results of the biopsy.
And let's say that there is an AI algorithm.
Out there that gives much more information about the model much more information about the texture of the audio and whether.
Uh, the probability of its malignancy, malignant nature and benign nature.
Uh, above and beyond, simply what the human I can do, and that not that algorithm confirms its United.
Patient sues unnecessary biopsy. Can radiologist be not a.
Using this scenario is, uh, is another 1 where.
Um, where the radiologist an algorithm.
Can the rigors can be held accountable for not using the algorithm because here the algorithm is not telling you what's wrong, but pushing you away from doing an unnecessary and invasive procedure. So, again, today have not.
5 years from now, when the algorithm gets adopted.
That could very well be the case scenario.
So we think about algorithms as.
Helping us detect generally helping us characterize.
As well, and those 2 are the main stage of.
The way we think about helping radiologists, so.
Saying, hey, you know, did you, did you look at this area that has a customer modules?
Or looking at a renal mass, do you want to look at this a little bit further?
Um, it may be like, so.
Functions the red flags, potential areas.
That could be worrisome.
And flags a potential Newcastle on a city scan, but the radiologist says this has been really troubling me with a lot of false positives. I don't believe it.
I'm going to use my clinical judgment and I'm.
Dismiss it turns out.
Was right hand, radiologist be suited for ignoring recommendations.
And this is the problem. This is not the same problem that we had is scenario 1 and 2.
Scenario 1, to the radiologist simply did not use it when they could have.
So, it didn't get the assistance when they could have had slightly different. Yeah, they got the assistance.
But they chose to ignore it so how would such a lawsuit go? And I think it's speculated.
But it would at least be conceivable.
That it may not go to full trial and may be settled. Because again, the optics of this doesn't look good.
So, what I want to point out to.
In the 1st, sort of half of this.
A legal contract, which is the algorithm.
Use or not use, is that there is a symmetry of risk.
There's always a symmetry at risk, but there is a symmetry of risk and.
In these sort of situations now think of this more broadly.
Why is the asymmetry asymmetry when the radiologist makes a call.
We get more punished under than we do for overpulls. We get more punished for missing.
Than we do for false positives. In fact, we don't get punished false positives unless the false positive itself has a.
Um, obviously detrimental consequence.
Um, and you can see that that plays out in multiple arenas, including, including as, as, as you mentioned in the pandemic where people make projections. And if you.
Project the number of that's it's not as punishing as if you under project, because under projection 2 seems.
More negligent, and that's sort of the way we think about it. So risk is asymmetric. And what happens when you use is you've created another layer of symmetry.
And give you 4 scenarios, and you can see how those 4 scenarios may not play equally.
So, the radiologist using it makes recommendation. Radiologist agrees with it. Of course, that's going to have no consequences.
Is right radiology scenario number 2.
Radiology disagrees with them is right and I think that is the most situation.
Scenario number 3 radiologist disagrees and is wrong.
Um, and that sort of situation again, there are no consequences because so you disagree with the I, but you are right in.
No consequence of that. The 4th situation is also tricky.
Which is when the radiologist degrees.
But is wrong here.
How the law quotes will deal with a full situation depends very much on.
How much radiologists will push autonomy.
Saying that I am an independent entity for.
For me, I my guess is.
That everything will be dominated by the 2nd, 1, the disagreement with the radiologist over time when they start using a.
They will disagree less and less with the on the 2 kind of will converge.
More and more into a single entity where.
It wouldn't be so much around helper, but because you can see that the consequences of.
Disagreement with very well be different if it is right?
This is wrong is wrong. Radiologist say well, you know, I had.
A data point that was hard for me to know, and if it is, right and the radios is disagreed, then it'd be like.
You disagreed with this recommendation and look at the.
So.
These scenarios play out, particularly 2nd and 4th.
Will determine the adoption of the AI and my guess is that.
After the initial adoption, which will be driven largely with.
By efficacy my safety and by.
Ease or ease of use that using the algorithm the next when the next push, but very well come from the lawsuits.
And that would increase the adoption towards.
Radiologist saying, well, if I don't use the, I, I'll get.
I'll get dinged for not using it. Once they start using ai they'll say.
I disagree with and I am wrong that I can get dinged and convert.
This is the radiologist.
Can you.
The next scenario I want to talk to Bill is a.
Trickier 1 that is more hypothetical.
More speculative, um, but also, uh, 1, that.
Will shape the development.
Of algorithms, and the scenario is, let's say the hospital uses an algorithm.
To interpret chest radiographs in fact, that is happening in a lot of parts of the rules, just India, where they use.
Certain algorithms to detect TB to detect it. So, this is not science fiction. This is more a case of.
Temporal, uh, there's a disconnect certain parts of the world they're using it.
United States may be the last place with this.
Usage happens multiple uses an algorithm to interpret chest radiographs.
Radiologist out of the loop that deemed more valuable to look at more complex imaging.
This is an obvious pneumonia and this is 1 of those 2 caustic areas. Sorry? Caustic random errors tickets to cast. It would mean that it's predictable. There's random errors. That's.
7 applicable reason happens every now and then with with algorithms.
Um, and the patient guys from sepsis, since there is no oversight, cause if there's oversight, then that kind of.
Things the radiologist packet look. So who get sued? Well, of course not the radiologists because they're absolutely so, here, the concept to appreciate is that enterprise liability.
The hospital can do whatever it wants.
So, as long as it assumes the risk of putting out.
Other people out of the loop by the radiologist. So it assumes the risk and it would particularly assume the risk.
If the algorithm was with empty.
Hospital the academic center itself in the event that the algorithm is from a vendor.
And the hospital purchased it on whatever basis.
Then it kind of depends what the who the plaintiff wants to sue just painted 1 to see at the hospital.
Or the vendor or or both, and it depends on the pocket.
And this type of enterprise liability is.
If a large hospital center.
Would want to go.
Standard of care. What it would do is you would factor.
The liability component as well that say, right is.
What we may have to pay out if there is a.
If there is a huge lawsuit, but it could be worth, uh, because we would save costs.
So that's.
1 element, enterprise liability that's a major question is.
Can the vendors the ones who developed these algorithms?
Be suit, and here we have to.
Kind of go back into legal history to see how this might play out.
And the thing to appreciate here is a concept and it's preemption.
Preemption is a legal principle whereby.
When you have a legislation or regulation elected.
It's sort of.
Convince the.
Same thing done at a state level.
Kind of sounds a little complex. So, let me give you a particular example if a vendor.
Safety advocacy requirements.
Off the FDA, it doesn't need to do the same thing.
At every state level, now there are exceptions to this.
That's the general idea of preemption FDA approval in a sense transfers risk.
And that.
Transference of risk means that.
There is some shielding from port.
Device manufacturers.
Now, this is slightly complex, because for a start.
Fda approval isn't a 1 size fits all and after the approval.
Is a spectrum of rigor and, um.
From the highest to the lowest rigor, there's substantial difference.
Now, this area believe it or not.
Particular area, medical devices and FDA approval.
Has made it to us.
That's quite surprising given how much has to deal with that because I had to deal with.
3 major pulses in the drug device.
With the briefly.
His lawsuits, Medtronic, incorporated versus law.
Why this is Levine rival vested Medtronic.
Very interesting in their own, right but they have certain principles.
In the Medtronic in corporate law suit, Medtronic, because of failure pacemaker in.
They state court Metronic said, hey, hang on. Hang on. You can't do that.
We went through FDA approval, so eventually.
The lawsuit made itself to the Supreme Court.
And the Supreme Court found in favor of plaintiff.
Not in favor of Medtronic and try and use the preemption argument.
Found a favor painter, et cetera. Medtronic.
Yeah, you did fulfill requirements, but you only fulfill their very weak requirements.
I'm not there more stringent requirements.
So, you can still be sued on the locals.
Same company Medtronic at a different outcome with another plaintiff.
Sued Medtronic, because a balloon angioplasty ruptured in the coronary arteries.
Again, this went to the Supreme Court, but this time the Supreme Court found for the device company saying.
Yes, you're right, you were preempted and the reason you were preemptive was because you went through the most rigorous FDA approval process and therefore.
You can't really have a scenario where the state and do any better than what we did in terms of.
So, whereas with Medtronic.
Supreme Court made exactly the opposite argument saying that.
Now, you pretty much got through this approval.
Versus quite easily, so that it could very well be possible that a state might have a higher level.
With wrangle it came up with the opposite.
Good 1 wife also the.
Supreme Court found for the plaintiff who had to have an amputation because of a drug, which.
Fail to give a particular.
Warning about a side effect.
And let's be Camry and, um.
Said well, it passed the FDA and that was a more worrying thing, but.
That's slightly different, because traditionally regulations that.
Treated frogs devices slightly separately.
So, why is this? It's relevant.
Because FDA approval, as I mentioned as a sliding scale of the highest level of scrutiny.
The Pre market approval at the lowest level scrutiny is justice.
Pathway that through which many algorithms are going through at the moment many algorithms are going through the lowest.
On that's to accelerate the, um.
Arrival an adoption of it is deemed to be 1 of those things that's going to transform.
Healthcare part of the 21st century cures act.
And but it's a problem, a problem because you've got all of these.
Algorithms that are kind of in the same situation.
That the Medtronic was in the 1st sclerosis case I spoke about.
So, there is a logic of the logic here, which is that, um.
For many things is just really an object. Do you really want to do a randomized control trial submitted to the same.
Rigor that, you know, we did with the vaccinations, um.
Only for detecting modules, because.
What we want to use for.
The initial kind of run it. It's okay.
Ready to go through the pathway, but.
Once that changes to so, the moment is detection detection is.
I don't know lying thing, it's not it's not deemed to be cognitively.
Difficult task, but characterization that's a different task and diagnosis.
Actually, making the call making the judgment call is a whole different task.
So, if you think about it from a chest radiograph, it's 1 thing to say, hey, there's a patch that's detection. It's another thing to say there's a pneumonia.
That's making a diagnosis and so a lot of what we lot of what these algorithms are doing, kind of like the lowest line detection.
So essentially how I will play out.
In these in these large, um, kind of.
Big trials will.
Determine the.
Manufacturing option there are 2 other elements that make to devices. The 1st thing is that algorithms evolve.
And you could say that same thing about any technology roles, but every time a city's kinda evolves, there are new features in the standards. Canada are.
You go from a 16 detect that 264 detector is a certain thing that, you know, has changed with algorithms. Just simply get better. We think they get better.
Might not necessarily get better the more data that it's fed to the.
All the more spread, it could very well crew more bias as well or a different type of.
A different type of accuracy may trade a little bit of sensitivity specificity. It's complex.
So, algorithms evolved and so the algorithm that approves very, very different.
Years hence, at which point.
It may not be the same algorithm that was approved. So how no quotes will deal with us is going to be.
Um.
Very arbitrary me, you can use certain principles, but itself is such a unique technology. At least we wanted to be unique.
Um, it it would stretch the, the.
The stress the legal principals a little bit, the fundamental questions at all this.
And it kind of gets down to the root.
Um, of both its adoption.
And, um, it's.
Legal consequences, and also the role of the radiologists is.
What exactly is information how should information that.
Imaging is giving information is just giving you information.
How do we treat information? Should it be treated as a drug a device or should be treated as something? That is just.
Not a driver, so, for example, if.
Somebody has a scan.
And you see a renovation.
That real lesion is a piece of information and if.
Somebody then undergoes a.
Partial defer to me or that ventilation.
And it turns out that religion was going to kill them anyway.
In a sense, you can say that they've suffered a side effect.
Of that information, which they wouldn't have had they not have that information in the 1st place.
This is, of course, very different to a side effect of a truck.
Like, a drug such as, like, a, and.
Status, which is that you still have a choice with information, you have a choice not to pursue.
Not to act on that information, you kind of don't have that choice for the drug side effect will happen, or not happen for that matter.
Once you take so.
Information, how it's dealt with will be areas where.
The low courts will think about it, so, putting it altogether.
What do we have? We have.
Algorithms increasingly going through the approval.
It's almost certain that this can be around the mirror.
And it happens to radiologist.
I certainly don't think that.
Uh, should be held to a higher standard than radiologist, but humans are forgiven.
And algorithms may not be easily.
Um, if there is a whopping big lesion.
Yeah, I misses you could have a scenario where.
There could be a big lawsuit, take payout and that, that that happens.
That could change the dynamics of the industry.
And that could stall the development and adoption of artificial intelligence.
So there are 10 liability issues.
Of a make sure that we don't quite know.
And it's very hard to anticipate, but there are also checks and balances in the system. And I think.
My message here may seem contradictory, but.
What I'm really saying is that 1 can't predict a certainty with which direction it's going to go. So there is the role of risk.
That could be very much like, Metronic issue.
But the other factor is that there is a check and balance where.
Naturally, if we, if we get to kind of stay in its name and only.
Do the job of detection without trying to.
Go too far too fast.
Then its risk may not be that much because then the radiologist will still.
The can so to speak and so.
Then the risk and fall on the radiologist and that's going to be a tricky situation where.
Radiologists will have to balance agreeing with a AI and being.
Um, wrong and disagreeing with the being.
Over time, I think there will be some sort of equilibrium. I think that scenario is more likely. And so overall, I think the 2.
The algorithm and the radiologist will keep each other tenant.
And the algorithm, if it does get to.
The point of thinking on its own will probably say to the radiologist please stay around and don't leave me alone to be.
Mercy of the courts so thank you very much. That's that's an overview. And I'd be happy to take any questions that you might have.
Continue the discussion.
We have questions, or we're waiting for that. Let me just ask a quick 1. what do you see is the role of the radiologist in the future with all of this in mind.
Well, I think that, uh, the radiologist will have to be.
Well, is still going to be here certainly in the next 10 years, but I would even speculate in the next 20, 30 years and their role would be more in terms of information management. Because what we're gonna get from is lots and lots of information.
And that's going to overwhelm the overwhelm the condition.
Certainly over what the patient and that's where the radiologist will come in and.
Sort of reduce that information, but make it more clinically relevant.
Interesting interesting. Any, any questions from the group.
Yes, we have 1 question from David Clooney. Uh, and David, the question is so long. Would you please unmute yourself and ask the question.
David, sorry, it was muted in 2 places. Yeah, we have about a quarter of a century experience with mammo CAD and it's exposure from a.
Legal perspective and memo kind of failed miserably in many respects with too many false positives and negatives.
How does this.
What does this teachers with respect to, which is in a way? Very similar to CAD, although arguably people believe it might perform better.
Yes, I think that, um.
Has.
Kind of distorted the landscape a little bit, because.
As you mentioned, the number of false positives was so much that.
It was essentially ignored, um, with the exception of novice readers, anybody with a modicum experience was just.
Ignore card, and it was more like a monkey on the shoulder than.
Uh, something that actually said anything terribly helpful. I don't think that the next iteration of, um.
Products is going to be.
Like that and you've.
And if it is, then it's going to be dead on arrival because if it actually increases the workload of the radiologist.
Then it won't, it won't be.
You won't be adopted that readily. I can see.
Something like Nadia detecting algorithms.
Being adopted by a busy chest. Radiologist.
If it makes the life easier, and there after the.
As time goes on such an algorithm will start.
Picking up the modules and the radiologist will start getting more and more complacent.
What we're looking at looking for session modules but in essence, what I'm saying is that.
I don't think that the history of card.
I should be projected onto the next generation.
Of algorithms, I think the better and because they'll perform better.
It will be more difficult not to use them particularly.
If you have that scenario where.
Um, if somebody has cancer.
And they've had a previous chasity and you're able to run an algorithm.
Finds modules that were missed, or ignored on the followed up with the same propensity.
Um, enthusiasm.
I think that it's adoption that could be quite rapid at that point.
Related to that, do you see plaintiffs attorneys buying a ai algorithms? So they can study old cities and find things that radiologist missed.
Absolutely, absolutely. My lawyer approach to yes.
Yeah, good.
No, you got it no, I, I think that, that that's going to be.
That's going to happen sooner than widespread adoption by radiologists.
The legal world has been looking at.
Safety and missed errors and they've got a much.
Stronger handle on this than we do because then, I think in in some ways, we're kind of at.
Were kind of an optimal position. We were fully optimized. We can't read any faster. We can't look more fastidiously.
And, uh, and so that, uh, that type of, um.
That type of hope for them would be kind of where.
Be where it breaks the ice and enters the.
Clinical space, my other question was related to the proposed deregulation of software and particularly since you mentioned the federal preemption effect on product liability, if deregulated, then product liability is exposed.
And I'm sure there are other comments you might have on the regulation.
Yeah, I can see the, I can see the.
Deregulation and, um, you know, it really isn't.
Practical to have randomized control trial for every iteration.
And if we're thinking of the eye as being an incremental innovation, which a lot of people are saying, saying, you know, it's just something that's going to help you a little bit. It's like another button on the max. If that's.
Really how we're viewing.
Then, you know, then deregulation is fine because that's really going to be for the for.
Most cognitively bland tasks.
Such as detection, but it but.
It it won't detection and charging, like, charging for stroke.
Where the radiologist still has.
Ma'am of control over the whole decision making and I think, you know, in the end, Rachel is because that.
Sort of guarantees them.
You know, it goes against the whole existential problem with versus radiology, relegating radio to simple task. So, in summary what I'm saying is that.
If the if a AI is used.
And truly yours for the most.
Cognitively planned tasks then then.
Deregulation isn't the problem and in that sort of situation.
Vendors won't have liability won't be expressed liability because the radiologist will still be there.
Carrying the count in fact, that's the best situation for them.
If it's more complex, if deregulation is for complex tasks.
Like, we can see, we can get the radiologist out of the loop.
Then I think that miss then.
But have huge pants, any final questions that we could ask.
Yeah, hi, I guess I'd like to ask question. I was a great talk. This is Daniel. Reuben from Stanford. I wanted to just clarify bland tasks. Are detection tasks. Is that what you're saying?
Yeah, so for example, um, when I'm reading through a format.
Okay, CD is kind of the test. I didn't call this.
I'm looking for big things. Yeah Yeah. Things that can really hurt the patient like.
Panic injury, and at that point, you know, having lung nodules floating around that kind of detection task.
Yeah, it's taxing.
Company to be simple.
So, what do I want to just bring up our concerns about so, because of burn out and the high pressure practice there's a huge risk for 1st of all adding and having that paradigm, which I agree with that.
The radiologist is the ultimate arbitrary decider, but that just adds more time to the radiologist workflow and the high pressure environment. Radiologist. I think there's a greater chance.
The radiologist when using a, I will minimize their involvement much like, drivers, autonomous vehicles, minimize the involvement of time. And to the extent, the algorithm does a good job. That may be. Okay but there is a high risk of automation bias.
And then there's the whole thing of why did you get different vendors of the same product? But they perform differently. So, how will this affect the radio quality of Radiology practice? At the end of the day the algorithms aren't radiologist the radiologist learn.
As they go from following up outcomes, but.
The ai algorithms to my knowledge, there's no methods for post marketing surveillance or continuous learning in place. So I'm just curious what your thoughts are about how that paradigm will evolve, especially because of the high pressure practice. Radiologists are under.
Yeah, so I I don't think that.
Radiologists would adopt it.
Increases their work substantially, or even.
Because we're already kind of, as you mentioned about the burner, we're already kind of.
Working very intensely and.
X has made us more efficient, added more images for us to look at.
So, there's kind of a barrier to adoption based on how.
How, um, user friendly the algorithm is going to be now in the event that it is user friendly.
What would that do? What what, what would that do to the radiologist?
Skill would it create, um, would it create some sort of automation.
Bias issues with anything that if you do.
Usual obviously do less and less of anything not going to be as good as.
Thing that thing, uh, and to give you an example.
Um, we, we are probably not as good at looking at facial radiographs.
So, it's complex rate across now, as we were 20 years ago, because everybody gets a C T scan even when they get official radiograph they got a lot of people wait for the people that read the radiograph. So, yes, we will become de skilled in certain areas.
But what I think is going to happen is that there's going to be skill shift. There'll be new areas, which will require new skills.
That would probably be the, uh, the, the actual.
Interpretation of the information itself.
Lots of information coming from a I'm putting it all together.
On giving much more of a known sponsor is, of course, very hypothetical because no one's really practicing like that at the moment with with but it's.
Oh, I view how I've seen things evolved from plain radiograph.
To and.
On with how things have evolved, and we've had.
Ways that software, and I'll have helped us segment the arteries out. We're doing other things now.
The offering has been segmented and we don't have to segment the entry ourselves.
I suspect 1 area where that skill shift would probably happen is radiologists to learn when can they trust or not trust a I mean,
if you're in an autonomous vehicle and driving over a bridge under the edge of the cliff,
you're probably going to put your hands on the wheel,
and likewise,
radiologist probably could learn when is a critical situation and 1 of this blindly,
except the.
Yeah, that's that's that's sort of a.
Somebody that's going to happen very much at a local level over over the years.
But, yes, radiologist will shape. How is going to be used.
And, um, as they will get used to each other.
Okay, thank you. Thank you. This has been a very interesting view of the other side of and the medical world, and we appreciate that only you want to take over.
We have 2 more questions. Okay. I hope we have the time to answer them.
So 1 question is what percentage of currently practicing radiologists can handle the information into interpretation without special training and is a I,
and Informatics going to be part of radiology,
training,
curriculum,
moving forward.
Yeah, so the 1st question, I can only also from the point of view of.
My own institution where.
I don't think that we are currently using.
And I practice I know that there are institutions where they're using it for free algae stroke.
Um, so it hasn't is really in the slow.
Part of the adoption very early days of the moment.
I am not sure I'm qualified to answer the 2nd question.
Because I'm not a, I don't have a background in Informatics, but my guess is that.
I don't think it is necessary.
For radiologist to lend to code.
Um, as part of their training, although it is necessary to understand.
The biases and limitations.
All so that last part is necessary to understand is sort of like, when we think about.
Physics, you know, not not many people really understand 48 transformation.
I pretend to do, like, I sent me.
Wouldn't be able to give you a mathematical exposition, but I do.
Think that it's important to understand the artifacts.
C. T. artifacts and have some idea how the nuts and bolts work so yes, you would need to have some idea of the nuts and bolts, but I would not.
We expect everybody to understand, um, statistics behind machine learning same degree, but.
Of course, enthusiasts can do, but not everybody.
The last question we have a goes along the same lines.
The comparison was drawn that doctors currently get a red and green good, bad, positive, negative score when they do, for example, blood testing. So there is no skill really necessary.
Very little skill necessary to.
Reviewed our splat tests uh, do you think there will in the future will be something similar for the results?
Well, I hope not, I'll put me out of a job.
I think if you can, that's a number and say.
Greater than 50, whatever diseased and less than 25 not diseased without any.
Involvement of them.
A condition that.
Very futuristic that I would really mean that that'd be really have no role of radiologist. And there will of somebody who has.
Multiple years of medical training I don't think you're gonna get to that. I think what's gonna happen is you're going to have lots of little data points. We already see that with radio radio dynamics. Um, and, um.
It will then be up to somebody who understands that to put it all together and.
Uh, also to understand the appropriateness of testing so knowing which piece of information should be sort so really a sense of.
Information contraction information.
That's the role I think of I can't, I think, you know, diseases are so complex that reducing it down to 1 number.
A very few things can be done that way. You can do that for maybe.
Any reasons, and their risk of rupture me down to 1 number, but even then it's complex, because.
Small aneurysms can rupture if there are other signs of rupture and larger aneurysms that have been around for a while may not rupture the same extent. So, there's, there's.
There's biology is not.
Easily reducible and that's good news for radiologist and clinicians.
You have any more you want to move on.
We can we have no further questions so we can move on.
Um, and the next topic on the agenda is an update.
Thank you, thank you so much. The next topic on the agenda is, uh, an update from.
Hi, uh, this Justin can you all see my screen? Yes.
Okay, um, so, uh, just a couple of quick updates, uh, mostly some new data sets that have come available on the cancer imaging archive this past month. The 1st, 1, is this segmentation of vestibular.
Uh, from magnetic resonance imaging and open annotated dataset and baseline algorithm. This contains a memorize from 242 subjects undergoing Kim and I started tactic radio surgery.
And there was an initial publication of the data set on around the 9th, and then that they, uh, updated the data with the addition of some registration matrices and contours a little bit later in the month.
Then the 2nd update is in March, we.
Publish the medical physics to this.
Virtual issue called data sets, hosted and cancer imaging archive. These are what they call medical physics, data, set articles, and unlike traditional manuscripts, rather than describing novel, scientific findings. These are focused on.
Providing a thorough description of data sets and, you know, in an attempt to make them more usable and understandable to other folks that are that want to take advantage of them. So there's 7 different papers.
About different data sets that were shared on included in the special issue.
And then towards the end of the month, we publish 2 and the clinical trial network.
Data sets the 1st, 1 was our 1st from the children's oncology group. It was a hip 731, which studied the side effects and how well risk based therapy works and treating younger patients with newly diagnosed liver cancer.
And then the other 1 is the GB 5033, which is a randomized phase 3 trial that studied with with 2 different combination chemotherapy regimens to compare advocacy and patients with diffuse large B, cell lymphoma.
And for both of these data sets, the images are hosted in the cancer imaging archive, and the corresponding clinical data, patient, demographics outcomes, et cetera are hosted in the data archive.
And this is a little bit.
New, but we only have 1 other data set that's like this but you, you basically apply for access through the archive.
Uh, website, and then as part of that.
Approval workflow, you'll you'll receive access to the images as well.
That's all I have for this month. Any questions? Thank you, Justin.
Um, and I would like to provide an update on the imaging data comments, the imaging data comments.
Um, is part of the NCI cancer research data comments down here on the bottom corner you see the, your L, for the comments.
The data comments itself consists of repositories of data,
for example,
the genomic data comments,
proteomics,
all make data comments,
uh,
integrated,
canine data comments and,
of course,
the imaging data comments,
uh,
the imaging data comments can be accessed at this.
U. R. L.
We release the minimal viable product in October 2020. sorry? For the typo.
And the product release is scheduled for the fall of this year. So this month in between, uh, give our users a chance for testing.
And for providing feedback, the feedback can be provided 2 different ways in the.
Top menu bar there you see here and enlarged here. There is a link to this course, which is a forum where you can pose your questions and will receive answers.
But you can also find the contact us link in the footer of the home page. And you can reach out to the imaging data comments team, using this means.
We very much appreciate if people go to the imaging data comments, look at the data that's currently available. It's only a subset of the data.
We will eventually host, but please go there test and provide feedback and by doing. So, you will help us improve the product for all imaging researchers.
And that's my update on the imaging data comments. Uh, if it's okay with Dr Nordstrom, I can move on and just make a few announcements. That's fine.
Go ahead, and the invitation to the webinar come now, uh, via different channels to you.
You receive update via email is 1 of the email lists is 1, Google group that we still host, but we also have started a nice collaboration with the communications team.
The information about the webinar also.
Hosted on the wiki, and on the same page, we are going to report, uh, we're going to post the recording of this webinar and you can find also there, the recordings and.
In some cases, the slides of previous.
Our next calls are scheduled for May 3rd, and for June 7th.
Doctor Nordstrom right I'm still looking for the May 3rd speaker. I will let, you know, in a week or so There'll be plenty of time.
For that, and, uh, uh, will handle the June 7th meeting.
Yes.
That's all that I have everybody I thought was a very good meeting. Thank you for participation. Any closing remarks from Lee or from on.
So, no, I'm fine. I agree. That was a great presentation I want to. Thank you. Thank you.
And I think for me.
Ok, then we'll give you back your day.
Thank you for attending. Thank you.