In fact, we were able to discover differences in the ISIC dataset between sites on how concepts correlate with benign or malignant categories. This kind of data auditing at scale is not usually possible. We also show that MONET allows us to audit models and automatically identify data features that could be affecting performance!

New paper just dropped!

What if you could audit medical image AI algorithms using generative models partnered with human experts? We dissected 5 dermatology AI algorithms and found that models relied on both reassuring and concerning clinical features!
medrxiv.org/content/10.110… Using a modified version of explanation by progressive exaggeration, we were able to generate thousands of counterfactuals for each model. You can see one example here:

ChatGPT is all the rage in medicine (recently heard that people are asking it all sorts of questions related to patient care), so this is a thread of the things you need to watch out for and what can go wrong! Number 1: DO NOT EVER PUT ANY PRIVATE PATIENT INFORMATION INTO ChatGPT (or any similar model). The data is NOT secure, it is shared with the company. Recently, there was even a privacy bug: proactiveinvestors.com/companies/news…

I’ve had the opportunity to give advice to several AI in dermatology startups at this point. Here’s a short thread of the key questions I ask every startup (also more generally applicable to AI in healthcare): 1) What’s the problem you want to address? Defining the problem and why AI is the potential solution is incredibly important. Sometimes companies are trying to address problems which aren’t the actual problems clinicians and patients care about.

In the past two years, there has been a deluge of papers published in the AI in Dermatology space. In our @JAMADerm paper with @Dr_Vron and @james_y_zou, we audited the transparency of the datasets and models used. #AI #DermTwitter #MedTwitter jamanetwork.com/journals/jamad… This work was inspired by @timnitGebru's famous paper "Datasheets for Datasets", which discusses the importance of documenting datasets with information around how they were created, potential biases, and recommended use cases. arxiv.org/abs/1803.09010