Okay, I see. Uh, people are still dialing in so Dr not Nordstrom. Shall we give it 1 more 1 more minute before we start the meeting? Yeah, I think we can afford to do that.
Okay.
Okay, I only see a few more people dial in, um.
If you would like, you can.
Go ahead. Okay.
Let me thanks.
You know, this is a distinct pleasure for me to be introducing our speaker today. Dr mariellen Geiger is the Pritzker distinguished service professor of Radiology.
Committee on medical physics and the college at the University of Chicago for over 30 years she's conducted research on computer aided diagnosis,
including computer vision,
machine,
learning,
deep learning in the areas of breast cancer,
lung cancer,
prostate cancer,
lupus and bone diseases.
He's only recently extended her medical imaging research to include a analysis of coven 19 on and chest radiograph.
She's a former president of the American Association of businesses and medicine and a former president of the he's also the inaugural editor in chief of the Journal of medical imaging.
She's a member of the National Academy of engineering, and was awarded the William D Coolidge gold metal for American association with physicists and medicine. The highest award given by the.
She was a fellow of many societies and the recipient of several achievement awards.
In 2013, for example, Marianne was named by the International Congress of medical physics as 1 of the 50 medical businesses.
With the most impact in the field in the last 50 years.
She has more than 200 peer reviewed publications, more than 30 patents, and has been mentored over 100 graduate students, residents, medical students and undergraduate students.
Because if this isn't enough, she's also the CO, founder of quantitative insights incorporated, which started through the 2009, 2010 venture capital challenge at the University of Chicago.
The company produced context, which in 2017 became the 1st FDA cleared machine, learning driven system.
To aid in computer diagnostics in 2019 context was named 1 of time magazines.
Inventions of the year, and it was eventually bought by clarity imaging.
I might add on a personal note that Mary Ellen was very productive member of the quantitative imaging network, having served as the chair of the executive committee, and participating in many committees and annual meetings.
It's my distinct pleasure to turn the microphone over to mariellen guyger.
Go ahead, thank you very much. Dr. um, it's my pleasure to be here. Um, let me start.
Okay, can you all see this? And this presentation would not have been possible, except with.
The funding from over the decades. Um, so, let me start here. Um, I'm going to talk on machine analysis, data, science and medical imaging of breast cancer and cobra 19.
I use those 2 as my main examples that I've done other work. Um, how do I make this smaller? Okay here are my conflicts and you see, um, NCI has played a role of major role.
I'm not going to go through this diagram. Probably every speaker puts it in, but what I, I put it up so that my talk involves and medical imaging.
That's encompassing all of this the deep learning machine learning, um, radio mix and others.
And this has been a part of our work in computer aided detection diagnosis for a long time.
Artificial intelligence and medical imaging has been investigated for decades,
both human engineered radio mix and deep learning and involves development the algorithm but also a lot of work and is the evaluation of the end user their performance when being aided by the.
An, at many stages along.
The radio ology imaging chain, it could include, um, development of sources, um, reconstruction methods and others for example, here, this is some of the work we've done with detection of pneumothorax.
We used a ai 1st, to distinguish between frontal laterals to improve the hanging. Then in detecting pneumothorax, and then for visualization for this talk, I'm focusing on image interpretation.
Computer aided detection, diagnosis, risk assessment and all those many tasks.
So the focus of most of my work is on images that are clinically and routinely obtained from the population,
I do have some projects grants where I provide the,
a,
I,
and I work with a collaborator who's developing a new image ecosystem.
So, we could have that feedback and give a comprehensive product, but for this talk, we're focusing those that are clinically and routinely obtained. And the goal is to understand what's seen in the image. So eventually we can detect and diagnose disease early and give the right patient.
The right treatment at the right time in radiology, lot of a AI is going to be used to improve the efficiency and workflow. Interpretation actually is just a small amount.
And if we think about medical imaging interpretation, medical imaging, our meaningless gray scale color scale patterns, unless viewed by an expert, that could be the radiologist, the computer or a combination of both.
So, the task of the human eye brain system is defined or locate a signal. And image characterize and diagnosis signal as disease or not and then do clinical decision making on patient management.
These are very similar to the task of a I.
In medical interpretation tasks, there's many other tasks that can be done. We can use a AI to help and risk assessment, screening, diagnosis, prognosis, treatment, planning and other ones and I'll go through a few.
So, when we think about what happens with the human, and then the computer, we can think about computer aided detection as.
Finding a locating a signal and computer diagnosis as characterizing, or diagnosing the signal as disease or non disease.
This is 1 of my favorite slides. You know, I've worked in and then deep learning came and they said, oh, API is gonna be much better than than C. D. and these are commonly confuse terms. And I also want to note.
That is coming back into voec. And being used to describe, because I think people are recognizing that an algorithm on its own is going to augment the user and not replace it.
So, API is the algorithm that exists within a system is 1 way how it might be used in clinical practice and to do research. You need to understand the image. The algorithm.
Definitely need to know what clinical question you're answering. And then how the end user, for example, in radiology will use this aid that's computer, aided detection, computer, aided diagnosis and that's secondary.
Or as a concurrent reader. I might be used as a partial primary reader, such as in triaging or in rule out of cases. That would not even go to a human observer and maybe autonomous a AI.
Um, but right now medical imaging is very narrow, does a specific task and doesn't encompass everything. So if you're doing research in this, you really have to have a handle on all of these.
I like to look at medical imaging is 3 errors the 80 s to 80 s and 90 s when we 1st working in CD and in the 2000 s with deep learning, we have faster computers, larger data sets, much more advanced algorithms.
And we realize that we can apply it to a lot more of the clinical questions. However, the basic clinical task, still the same there's still a concern for garbage in garbage out potential for misuse.
Same methods for statistical evaluations. Same need for sufficient number of cases.
Not just the number, but the distribution of those cases, just to.
Span the distribution of both disease and normal and recognition that this is a multidisciplinary task, you guys can hear me. Okay.
Because I hear yeah, we hear you. Fine. Okay. Good. Yep. Okay so I'm now going to go, um, into the 2 use cases. Um, my work in breast cancer has been funded over decades by NCI.
Um, especially within the, um, uh.
This is our 994 prototype system for computer aided detection back then I put this in just so folks realized that this effort has been going on for decades and we keep learning both from our successes and our mistakes here,
this was filmed that was digitized.
It was stored, um, we have a computer. We have a print out and all this was done to put out little arrows, focusing on the true positive the false, positive, false negative there as you know, this led to commercialization.
There's multiple companies out there in using this. But.
Um, what did we learn? Well, 1st, off I want to note that by 2016, there was a paper that showed that over 92% of digital facilities and screening mammography were using some kind of aid.
I want to point out that the commerce, the, um, FDA approved system included, deep learning with a convolution of neural network.
Um, I believe this is the 1st, medical imaging article that used and this was in computer aided detection and mammography. This was the 994 and looking for classifications.
We've learned a lot back, then the evaluation for FDA was to see how well did the computer do on MS. lesions. Now the goal is to see how well does the end user.
Perform when they use the aid, but back, then it did demonstrate that patients were ready for a AI. They were interested that existed in a through many anecdote type stories.
But more importantly, we realized that performance needed to be improved, get rid of some of those false positives.
We also recognize more now that radiologists need to learn that a ai system should be used for the task for which they were developed and approved. They're not if they're computer, if they're approved as a secondary reader or concurrent reader, they're not a primary reader.
Um, they should not be used off label and, uh, for other types of breast images. If there's an algorithm that is developed for 1 type of breast image, then it should not. If you put a different type in in the system doesn't work.
Well, because you put the wrong image in, um.
And there's a lot of research going on, and I'm not going to spend any more time on computer aided detection, but it's.
Role not just improving radiologists performance, but also improving efficiency, especially as the image acquisition systems used and screening are using more and more complex images. 3 D40 images.
Next is computer aided diagnosis. Something is found how can we help? Characterize it? Sometimes the are combined together. I've separated here for understanding.
Here again, this is an output from the computer that would in this case, you'll a computer determine likelihood and malignancy with the final decision, left to the patient.
And these are also getting more complex as there are more and more acquisition systems being involved.
I put this up here, the work in my lab spans both human, engineered a ai as well as deep networks for and we focus on specific clinical questions.
Here's an example of human engineered a AI. Um, we use this to present, like a virtual biopsy of the tumor, the computer automatically segments the lesion extracts out lesion features.
These are mathematical descriptors more recently called radio makes in the past 10 or plus years. Um, they are then merged by a classifier to use a signature. Now if it's trained and.
Cancer versus non cancer, it gives the likelihood of malignancy. If it's trained on different subtypes of cancer, it might give a likelihood of aggressiveness.
11 of our features that has done quite well in various clinical task is we, we are here looking at breast dynamic contrast enhancing, um, it's done in 4 D. there's 1 slice here for just for the slide.
And if you look at all those boxes, and you look at the uptake, this, this is dynamic contrast enhance summarized. So, there's an uptake in a wash out for each voxel over time.
And you notice that there's a lot of heterogeneity in that with this is a cancerous tumor.
And often, but not always wrap it up, take and wash out indicates cancer. So we did besides performing fuzzy means clustering to segment the tumor.
We did a 2nd, round of fuzzy means to isolate the different groups of uptake and what's shown over here.
Uh, these red is where, if you have the most up taking in those boxes, and we looked at this 2 ways 1, and you see this later.
If you look at the volume of the uptake, um, that was, we found useful for looking at risk of recurrence. But if you look at this texture, those were more heterogeneity. And this is, for example, if, you know, just looking at the single time point here.
Um, this is useful in classifying between malignant, benign because benign tend to be more like this.
Um, we also look at deep learning in our computer aided diagnosis work.
Because of limited a number of cases with good annotation, we use a lot of transfer deep learning with this. You don't have to segment deletion. There's no extraction of features.
The task is still distinguishing between cancers and non cancers, but we look at.
Network sets train for other things, even cats and dogs or other types of medical image, and we extract out the features.
Do various data reduction methods, and then use a classical classifier in doing this and this allows us to get good results without the needing them millions of cases.
This slide here is showing and this is for cancers and red and green are benign. Each dot is a patient.
Um, here we have the likelihood of being cancer as determined by radio mix of the human engineered features that we extract. This is from our deep transfer learning what we found and these are examples of those images.
Again, on dynamic contrast, enhance them. Right right. Indicates this is a cancer. Green is benign. A perfect world would have all reds here and all greens. Here. Couple things to notice is sometimes.
The human engineer day, I did better than the deep learning, and the other way around. But 1 thing to note is, if you use both of these, you probably are doing better to to, um, distinguish them. So we do this across 3 modalities.
This is us also this from 2017, our databases are much larger now. Um, this is the area under curve for human engineered and deep learning and we found that when we fused these, we did statistically better.
So, to me, I encouraged to don't give up on human engineered. They are trained in a sense and millions of images because they are develop based on what the human brain system of the radiologist is doing.
What features do they look for. Um, so I encourage to do both.
We along the way I mentioned how the reader does is very important. So we've done that across modality.
This is just 1 study from 2011 where we showed that the readers, um, using the computer, uh, statistically better in the task of distinguishing between malignant a benign.
Interestingly, here's the computer, but you see, it has trouble here in the lower sensitivity area.
But radiologist did improve that encouraged us to translate because while we are doing research to create new knowledge, we also believe it's very important to translate that knowledge for the public. Good.
And this is a system, um, which went through a reader study, similar to that's done in laboratories and many other FDA studies, showing that when radiologists, this is the radiologist performance without the aid.
And this is with the aid and they all improved.
But how do you approve these algorithms? Just checking on the time? Oh, okay. I've got 15 minutes. Um, you can use.
Different methods, but also more effective inputs. I'm going to skip this. I'm going to just tell you to think like human when you design deep learning. Um.
1 of the things is, we have radiologist look at different, uh, images, different protocols so should not the deep learning.
So,
what we've done is looked at ways of how do you merge the different,
um,
multi parametric images so,
for radio mix for each 1,
we pass through the segmentation we pulled out features and then we looked at do we fuse the features at that point into our classifier or do we have them each go to their own classifier and then fuse outputs and all of these
we did do that and both fusion methods significantly outperform the a single parametric.
Um.
Dx, um, at that, in this task of malignant versus benign, so radiologists are looking at all that information. Why not have the computer.
We did the same thing with deep learning, but in this case, besides merging the classifier outputs or the features, we also looked at. Can we merge the images as input to the deep learning?
And in that case, they did improve, but the feature fusion 1 improved at a statistically significant level. So, look at all the information that's going to the radiologist.
I put this up because what we work at is we 1st work at this discovery time at diagnostic imaging. There might be biopsy results. And at that point, we can conduct discovery.
We can then translate, let's say we find a a signature on diagnostic imaging. That relates to cancer, can we see that signature on screening or in assessing risk of recurrence?
These were would be like virtual biopsies, but keep in mind. We're doing this when actual biopsy is not possible.
Here's an example, I mentioned this when I were looking at those, most enhancing this is the most enhancing tumor volume that's picked off on the image. Very quickly.
We applied it to the ice by 1 data and it showed, um, that it could predict recurrence free survival. Very early on using the 1st, or before the 1st data.
And this was a major effort between the,
and the in the breast mapping,
um,
project we're here,
we related the radio mix features that we've pulled off to various cancer pathways.
And, oops, and here we found that transcriptional activities of various genetic pathways were positively associated with tumor size, blurred tumor margin and irregular tumor shaped.
And that micro expressions were associated with tumor radio, make phenotypes of size and this tumor heterogeneity that I spoke of earlier. And so this may mediate the growth of tumor in the head of, of angiogenesis and tumor.
This was a multi disciplinary effort of, um, uh, many, uh, institutions that was greatly facilitated by the, and the Q. I. N.
So, let's go through cobit quickly in the last 10 minutes. Um, here we, um, there was a need for civilians died, uh, prognosis predicting, and between the post cobit patient.
So, what was the role for imaging and a AI and so in my lab, um, we, 1st looked at algorithms to distinguish covert positive from cobin, negative on chest radiographs. However, that was, you know, there are a lot of tests out there.
So that, while we learned a lot is not that much medical use, but we use that knowledge to then create algorithms for prognosis to assess severity and to do that.
We needed Sarah, get markers of severity, so on covet positive cases.
We perform prognosis by using the intensive care as.
They're the surrogate marker if 1 group of patients were either incubator, or admitted to the ICU versus another group of positive patients that were not and interesting with Kobe.
While our work in cancer spans decades where there's different treatments, different imaging arriving and going.
The same thing happened with cobit, but pushed into 2 years. So this change in data over the 2 years is something we have to worry about as we go forward.
We use deep transfer learning where we took the weights from,
um,
the image net we then trained with the chest X Ray,
14 from the,
the numerous pneumonia images from the are sending pneumonia detection challenge.
And after that, then did the last set of transfer learning using coven 19 patients from the University of Chicago we also looked at some visualization. This is a severe patient. This is not so severe.
Um, the probability that was output was the need of intensive care or not, and we achieved point 77. we now have tripled the status that are validating edit the University of Chicago.
This is another 1 here is with our colleagues, um, in China and Mohan. Um, this is again looking at severity but now our surrogate market was patients who were put on steroids versus those.
That were not this is also using, uh, deep transfer learning.
Where it's done by slice and then combine.
Always evaluation by patient, um, and this showed again promising results, um, in distinguishing between severe and not so severe patients, but we also used it to check out.
How could it be as a marker.
So this is these are patients and red are ones who receive steroids, and we follow them over time and we saw this severity mark or decrease. Whereas those that did not receive steroids kind of stay the same. We've just now.
We had around 50 cases here now we have 900 patients, so we're validating this right now this all led to metric and, um, I hope you've all heard of it by now.
I think, um, this is where we all work together, this is funded by the is established August of 2020. it's led by the, um, and, um, an 80 PM.
It's hosted at the University of Chicago. The goal was for an open discovery data repository and also jumpstart some API work. Currently we have 24 institutions from academia, community, practice and FDA over 100 investigators.
Please see our website for more information.
Oops, what happened okay.
Um, as we develop this, a lot of the work is done in these 4 data subcommittees and I put this up because it's not just collecting cases but it's what are the data standards?
What are the policies for contributing and user and which I know um, others have worked in, but this was kind of switched into right now a year and a half hour.
Uh, what kind of how many harmonization and data checks are needed, and what should be put on the data search and user portal.
Um,
so metric takes and data through,
and it then undergoes various curation,
um,
harmonization and ultimately,
notation,
this incorporates expertise in assessing image,
quality and evaluation.
It's then output on a gen, 3 platform, which is housed at the University of Chicago. This is the data website where you can build cohorts and download data. 80% of it goes to the open access public comments.
Making it ready data 20% roughly and these 2 are match goes to sequester, which I would talk about more.
This has no access and the goal here is to expedite translation through re, regulatory through the public. So Here's an example of cases coming in all marked with checks and here, you see a knee and obviously that's not a chest.
And so, data curation is a very important. The other part is, how do you harmonize study descriptions?
Um,
because ultimately,
yes,
you can download that and figure it out yourself,
but when you're searching for cases,
if you want all of the chest well,
what kind are there and,
or do different,
um,
institutions use different wording in the dotcom header.
So that's a big project. That's led by Paul Cunningham. Um, but he has a very active committee of probably some folks around this call now.
And there's a lot of challenges, but they've been learning a lot. And this is good collaboration, um, with, um, investigators from many Institutes included in on this effort.
And, um, they're getting to a point where there will be using the link mapping. So, to enable, um, searching and cohort building.
Right now, as of March 17th we've ingested 85,000 imaging studies. 55,000 of those are undergoing quality checks and there's almost 30,000 release those that that's on the public. We're very sensitive. Um.
That we have diverse data, this is from May 2021 when we compare it to the U s2020 census and this is our current 1. um, we, we, we'll keep looking as we intake cases currently. We have institutions from 23 States in the pipeline either.
Site working on signing the contributing agreement, prepping the data, transferring the data having a curated or having it post it.
I mentioned the sequestered comments.
20% is sequestered and when we do that, where we can match them, and this is to serve as a large test set.
Where samples can be drawn to test out their algorithms and we do this in a sequestered way, uh, which I'll go through in a 2nd.
So here, this this is what comes in it is then separated into 8,020 by matching various attributes. This is just 1 showing matching of race.
So you can see the distributions are approximately the same and we don't, they don't need to be exactly the same. But, um, close enough, you don't want to have people training to the test.
For the performance evaluation using the request is set then.
We have our, we need to draw test cases from the sequestered set. So, based on the claim, the clinical question, the type of population request that we can then extract those using.
Randomly pulling from the sequestered set based on that.
Distribution and then we can test the algorithm and give the amount the performance back. We're not going to say how well this case did or that case there did.
And that preserves the integrity of the test set,
if,
uh,
the investigator,
and ultimately,
if a company comes back in a year,
we can re,
pull from our sequestered Pi cases representing that distribution of the type of cases that algorithm would work on and give them another score and within metric we have FDA involved,
so we hope this will expedite regulatory to get to the public.
So, please visit our website, both of them and, uh, for information and then for the data.
We have very seminars once a month,
and we have a newsletter you can quickly sign up and I,
we also have 12 research projects ongoing and these folks are linked,
sometimes through various scientific work groups,
such as the grand challenge work group and the bias and diversity work where we're looking at how do we assess bias?
How do we mitigate it?
1 of the important is that we are medical imaging experts or that the data comes as a medical images with some clinical. But ultimately we want to link in the cloud with the non imaging data.
So we have a pilot going with to link the imaging with the data we have another pilot with for the bio data catalyst from pedal and we also recognize that some cases will be centralized
such as a matrix some will be at other institutions.
It'd be a combination of centralized and Federated, so we have a pilot funded pilot going in with our gun. Um, this is a pilot grant.
And why are we doing this? And this holds for but for everything we work on is there's been a lot of publications, not many of them.
Good, well, none of them really have gotten translated to for imaging. I'm saying for a routine clinical use and part of this is when everyone wants to help, which is great, but then some images.
Some there was a data comments here, image repository here, a database here, and sometimes the same image guide in multiple ones. So you had leakage between training and testing um, there was no external validation. How do you, um, check the truth?
The quality of the images, and there was also sometimes, um, uh, folks working in a very narrow area and.
All of this is made possible by the multidisciplinary nature of it where we have both experts in medical, biomedical imaging experts and of course bias we are have to be very sensitive to bias.
Um, I've talked about data sets before where.
You know, some, um, I have a database, it's just an breast, it's built collected for the task and malignant versus benign. If I throw it into a general pot, I kind of skew that general pot. I think it's very important to have.
Entities and,
of course,
I grew up with has helped my research along the way where there's various different types focused,
um,
data sets for this 1 I guess you can say we have a focus on or positive and negative.
Um, but our goal is to, um, collect, for example, go to hospital and say, um, you know, could we have all your chest pelvic abdomen images?
And then the, a, I, of course, we have to have the appropriate data elements could slice and dice it after that trying to get rid of any potential bias and bias can be due to, um, at, um, ethnicity and race.
It could be due to the acquisition system could be due, um, to what the, how the data was collected. So, uh, it's a big task and all involved is great.
So, we hope to Duke is a national resource, um, working, of course, with and all.
Um, right now we're coven money. We're focused on Cupid. Um.
And we're a community, um, I spend half my time on zoom for.
We're open, so we want to lower the area of access and with our real quantification of algorithm performance on the request data set, we hope we translate this.
So, I'm going to end my talk keeping in mind. There's a lot of challenges. It seems like they're growing each year, but we're chipping away.
It's for algorithms, do you have to understand the biological clinical task? You have to know how ultimately it will be used.
By the end user to understand garbage in garbage out robots testing and appropriate data sets are needed.
So, for all of this, um, our aspects of data science and so I will end thinking my lab I have fantastic graduate students.
Um, amazing, um, senior researchers and lab stack staff and, of course, the folks that, uh, Matrix now.
Thank you. Well, thank you. And I don't know how to stop zooming how to stop sharing, but I know. Oh early figured it out before for me.
Um, wherever there, I can simply grab screen share from, you.
Okay is a little bit medium this way. No, that's okay. Thank you. Magic. What magic? Um, on that note, um, let's open this up some questions. I'm sure there'll be some.
Topic this, this is the, uh, thank you for the great talk.
So, I got a question about this explainability.
You talked about in the in the case of covet.
I'm using AI to you were able to.
Um, we can, uh, severity of the disease contradiction.
I was wondering if you are able to actually.
Face back to some of the comorbidities of the patients, for example, even to.
You know, generic disposition if they are data like that. Would you be able to.
Try to go back and see if any of the features I generate the features.
Can be somewhat explained all the places of more.
You know, basic, fundamental understanding of the patient's condition.
About a particular biology, for example.
Yes, I, that is a great question. So, a couple of things we're doing now, we started that work with both. We only had a couple 1000 cases at University of Chicago in about 500 cases with multiple time point scans from, uh, Mohan.
We now have.
Many more cases, uh, of course, they've acquired over the years, so we have to be careful over the year and a half span. Um, but.
Both on the coded cases, examples I showed you were deep learning, but we also have and in fact, we're currently.
Working on a paper where we're relating it to comorbidities where we use radio mix. So on a limited number of cases, we had a radiologist draw cobit involvement. So we've then trained.
To segment out the coban environment, and we look at that area volume relative to the full lung as 1 of our radio mic markers as well as others. So stay tuned.
Um, we're just now downloading the additional data.
Um, it's pretty organized at Chicago, um, but we have to go through the image and yet it's great. You have to go through all that other data to pull out the comobidity data.
So, yes, we're looking at it both in radio mix and deep learning. Um, and.
I believe that when you get you mentioned, explain a bill explainable stuff at the beginning of your question. If a ai works, it has a very high.
Accuracy rate, and I don't accuracy. I'm using very general.
People won't worry about too much explainability. It's like people know how a STC and kind of works. They don't understand all the upgrades, but they see it's a great image. And they use it without question.
I may may reach that stage for a specific task. Lot of the.
Um, uh, heat maps.
For our for explainable, if you want to kind of understand what's going on as you develop the algorithm. But I think it's the association of the deep learning with the radio mix to pull out characteristics that will enhance the interpretability part.
That's the end user would use. So, I think you and I are on the same page, but I can't give you the data yet. Um, because we right now are in the midst of analyzing, um, um.
Thousands of cases on this relating to comorbidities, but thank you.
Thank you. Great. Um.
Anything anything more from the audience.
I have a question, how do Institutes contribute to metric? So when you show the map I saw there was.
A plain empty space in the plain States and we all know, there were a lot of cases in particular in the Dakotas. So, how could.
Institutions in these states contribute to the collection.
Yes, um, I think we have more there now because we have 23 of the 50 states covered, um, which is a different map than the 1 in the beginning, which are the investigators map. Um, what we're doing right now.
Is we have funding that.
If a site wants to contribute, we have on our Web site. Yes, I would like to contribute. Here's my data work with me.
We then find out their situation and we will, you know, if they're, you know, they will either go through our or we have additional funding because.
People may want to give, but it costs money to pull down their images and to organize the data.
So we have, um, initial funding that could pay maybe 30,000 or so, for a staff member at that institution to help organize the data.
And then we have staff people at who work with those folks to help in the translation. Um.
As we look at the diversity, and we, if we see this is the 2nd part to your question, if we see that, we're not making it at 1 of the.
Uh, we're lacking in diversity for we will do diversity directed acquisitions where we go to particular places, but we're only a year and a half old.
So, right now we have in our pipeline.
Between 200 well, over 200,000 cases in the pipeline. If we count those in the negotiation stage. So we're waiting to see where that diversity falls.
Um, and then make sure um, for, you know, we're not saying we're going to do it relative to us census.
It may, because there might be a very small group of people that need more cases just to do, but we will then do diversity directed, but we recognize that it costs money to donate.
Okay, thank you. And may I ask a follow up question speaking of money? Uh, I know metric is quite a young resource and you just mentioned this, but do you have already a.
An average number of.
How much it costs to bring in images and data for 1 patient.
Oh,
for 1 patient or some kind of measure that you could see yeah it's 1 of those things where,
whether you collect 100 case or or 1000 cases or 10,000 cases is that initial setup of how do I download and how do I package it?
So, I can get it into our so, it's a difficult question that way. Um, but we, we could go back and try to calculate it.
Um, you could say we work at roughly a burn rate of about 7Million a year but we did that because we.
Just started full, you know, we were building the airplane as we're.
Of flying it, um, and we have.
Currently in the pipeline too, I guess you could do a back of the envelope calculation that way.
But what we did estimate was that this is just for coban and if we had to ramp up again,
let's say,
for a new disease type,
where you need a sufficient case is sufficient number of institutions for the diversity,
new data elements.
Because we've gotten we did a lot of on how do you create the data element? But we now.
No,
no,
if we did a new disease,
of course,
we're not yet we're moving into long covet but we hope to this is this infrastructure that we build can benefit other,
um,
institutes and disease sites that.
There's a, a shirt certain, um.
Knowhow that's already been done a certain infrastructure already created that we have, and then to maintain and to expand. We would have to create new data elements. Well, how do we do that? How do we change the data.
Dictionary the search technique engine 3.
So, I will go do back of the envelope, try to figure out per patient. That's an interesting question. Being over only a year and a half old. We're still in the ramp up. We literally just started in August of 2020.
um, maybe once we meet steady state, that would be a more I can answer that question but right now it includes a lot of.
You know, the, the 1st time is a lot more costly than the 2nd time.
Thank you have you found that, um.
Well, how should I say that it's all covered the same or you find differences with the delta variant normal Conrad other.
Imaging and and differences. Mm. Hmm. There's.
It's a really good question. So we do have an algorithm in my lab that is on deep learning and is used to separate the presentation of code into 6 different.
Presentation based on the type of appearance in the lung.
We have found that a lot of folks.
Maybe, they didn't keep track of it in their, or we haven't been able to get it.
Is was the person vaccinated?
And what was the type of, um, strain the type of, uh, variant of that's data hasn't always been kept because.
We didn't, we weren't vaccinated for the 1st year. Right? And then the 2nd, year some people were, some people weren't and then parts of the country got the Omnicom before the other part. So it's really a multi dimensional problem where, um.
You know, in cancer, you have what decades that you can do that with. And so we need a lot of data collected over these 2 years of covet to do that. But those questions are on our radar.
Um, we in our collaboration for HR and to receipts and collect, um, variant type, and they're trying to go back and get it.
You know, just didn't think about it, but it went so fast though.
Thank you interesting. Do you have any other questions wrap up?
Yeah, this is a big spectrum. Uh, yeah, so to go back to breast cancer.
As, you know, 1 of the big problems is over treatment of.
And specifically I'm radiation oncologist, and we know that only 5 of patients really need radiation, but we treat 100% because we don't know which those 5 are.
Uh, do you see any progress in that regard in terms of identifying patients back to me?
What the prognosis is going to be after.
You know, I'm a big proponent of everyone working together. I think folks who worked with us for the know that. And I think it's been shown in, um, Matrix too. 1 of the things. I, I.
You know, is that over diagnosis or over treatment? It's a, it's a big question and I've really.
Um, I, my views have really changed and grown over the past 2 years that.
You know, I was at a data science meeting and someone there in radiation oncology.
Was telling me how, you know, they were doing this with the patient this and they go off. We could only see the images. I was like, what you're at the same place.
And part of it is the radiology packs, and then there's a radiation oncology packs, and they don't always talk to each other. And I was.
It I was like, you have to be kidding. Um, I, I think 1 thing, we're, we're.
Learning and especially with metric is you can do so much what your data set,
but you need a lot more infinite information to do a lot more,
especially like the question you're answering and we,
um,
we have related radio mix 2.
um, I, I think we need a sufficiently large data set. We, we have the methods to answer your question. We just don't have the data to answer it. A lot of questions out of the data sets that I have are invasive.
But if someone has that, I'd happy to collaborate and run our analysis on it.
I mean, for is there's lots of, you know, probably millions of mammograms not so many is.
Correct. So right. Right, right. We've looked at more of breast memorize in the past decade and not the mammograms. Um.
Right now our latest data set.
We have about 500 cancers and about a 1000 normals.
Are coming from an screening program.
Um, there we also, as we download those cases, we're downloading all the mammograms and ultrasounds. So hopefully that could help, but that's kind of a pressing.
Um, data set now, which 1, we're going to use to validate our other work our prior work, and then go into multi modality. We did a lot of multi modality breastfeeding in the past ultrasound and mammograms.
But they weren't doing the same patient. Now we have this was very clean truth. We paint it's painstaking. 1 person. That's all she does every day is keep Claire, creating the status set.
I should, uh, right it's, um, it's 1 of those.
Um, necessary tasks.
That, um, uh, 1 has to do, but, you know.
We just keep plugging along that. We do that with our time to the Nicholson.
Okay, we have time maybe 1 more question.
This is just a short question about metric. Are you guys collecting also a normal's?
And the proportion of those compared to the disease that are available.
Yeah, so we have it where they were, um, tests at normal or other tests normal.
Negatives so their, their verified negative of course. Um.
Some of those patients will have come back and test positive.
So,
when we collect from a site,
we get all the cases all the imaging studies and with that we also get if they have say their tests and we can relate it,
you know,
5 days,
20 days how many days you want it?
Um, but that's a tricky question, because things move fast people have different quality images and different quality organizations of the clinical studies but yes.
Simple answer is yes. To your question. Thank you.
Okay, well, I think we're just about out of the time for this part of our.
Uh, program Maryann, thank you so very much for the presentation. Thank you for having me and, um.
I just learned about this group, so maybe I can join. I don't know, it's an internal. It looks like nevermind.
Oh, no okay. It keeps all the people listed usually. Okay. No, I see other people, Andres here. Yeah.
Yeah, yeah, no sorry about mistakes that we're, we're multi disciplinary here. Um.
So we can, I'll turn it over to you. Okay. Thank you.
I will give a brief update on the imaging data comments and.
Uh, as was just mentioned, Andre, the is also here.
So, I just moved to the next slide.
I wanted to remind everybody that the imaging data comments is, uh, doing the monthly data releases and you can find the release notes on this page, which I'm going to copy into the chat.
Um, the next release.
We'll come out hopefully very soon, and very soon being this week, the team also has been busy. Uh.
Too in collaborating with the human tumor Atlas network teams, uh, we are preparing the ingestion of multiplex, microscopy images.
They will be converted into images and then be made available to the imaging data comments users and.
The images all the images on the imaging data comments.
Can be visualized using using either the oh, if you will or the slim viewer for.
The radiology and the slide microscopy images again, I will put the link to the release notes in the chat as well as the link to the imaging data comments portal itself.
Feel free to come and check it out and I think this.
I'll hand over the Mike to Justin or John.
whoever's doing the update for.
Absolutely, um, so our updates this month or old data set related.
So, we have, uh.
This 1st, 1 is a data set for his to history pathological whole slide images for classification of treatment effectiveness to a variant cancer.
Uh, also known as a very in benefits as a matter response.
Um, this 1 has 288 whole slides, um, and clinical information from 78 patients, and should be useful to help people explore and develop methods to predict therapeutic effect of patients with the services. Ma'am.
Then we also have new radiology data set, categorize digital database for low energy and subtracted contrast enhance spectrum mammography images.
Uh, this is about 2000 images that were from, around 326 patients with manual segmentations of the abnormal findings. And also some clinical reports.
And then we had updates to 3 different data sets that have been previously released the, uh, Akron, contralateral, breast M. R. dataset. Otherwise known as accurate.
6667 was previously embargoed available only to quantitative imaging network members that has now expired and it's not publicly available to everyone.
Then we also added 45 new subjects to the ultrasound data of a variety of liver masses otherwise known as B, mode and C CPU s liver.
This 1 enables researchers to use contrast enhance ultrasound to characterize indeterminant, liver, lesions and monitor treatment treatment response.
And then we also had an update to the and projection collection.
This 1 was taking care of a technical problem that was identified after it was released and reported to the to address a fractional shift in pixel positions between the full dose and logos, Siemens cohort of that collection.
So, if you've been using that 1, please head back to that page.
That's all I have. Thank you, Justin and this. Uh, I hope you can see my PowerPoint slides again.
This just leaves the announcements, um, as always invitation to the webinar go out via the Google group as well as, uh, mailing lists.
And I'll just put this in a chat for you.
The, um, the recording of the meeting and the information about the meeting.
About upcoming meetings are available on the wiki and I'm putting this in the chat.
And the next, uh, meetings are scheduled for a 2nd and June 6, and we will announce the Speaker's, uh, in due time.
And these are all the announcements I had.
Back to you. Fine. Um, and I think it's time that we, uh, leave and and do something else. So thank you so much. Marianne, thank you for everything.
And, uh, we'll meet again next month.
Thank you. Bye. Bye. Bye. Now.