Learning outcomes: Discuss an AI application for medical imaging

• Understand clinic use
• Know dataset quality and diversity
• Explore performance metrics
• Know validation strategies
• Find clinic evaluation
• Apply ethical and regulatory requirements
• Deploy an application in the real world

Requirements

• Each student will select one AI application for medical imaging
• Evaluate and summarize the application based on the 7 key points in the previous slide
• If the information of a key point is not available yet, you should discuss how it can be done with references to relevant AI applications.
• Word limit: max 1500 words; figures, graphs, tables, illustrations, in-text citations, references, etc. are not counted for the limit; penalty applies if exceeding the limit

Requirements

• uAI recommended – demo access is provided
• Other AI applications acceptable
• Address the 7 key points given in the guidelines
• Rephrase essential concepts to show your understanding, e.g. performance metrics
• Use section titles as recommended: The scope of application, Training datasets, Performance metrics, validations, Clinical evaluation outcome, Ethical and regulatory, Deployment and improvement, Reference

1. The scope of application

• Imaging modality, e.g. CT, MRI, X-ray, Angio etc.
• Patient population, pediatric vs adult, inpatient vs outpatient etc.
• Use conditions: e.g. lesion detection, scoliosis analysis, bone age assessment, fractures, pneumonia, PE, ICH, etc.

2. Dataset quality and diversity

• Number of image, age-range, race-range, to represent the target population
• Annotation quality, e.g. ground truth was established by experts (radiologists)
• Bias assessment, e.g. demographic variation, equipment differences

3. Performance metrics

4. Validation strategy

• Internal validation: the same dataset was used for the training and validation
• External validation: the training dataset is different from the validation dataset, e.g. from different institutions or populations
• Cross-validation: the dataset was split into multiple parts and training was done multiple times

5. Clinical evaluation

• Expert studies: AI performance vs the experts (radiologists)
• Workflow integration: Evaluate how AI fits the clinical workflows
• Decision support: Assess and quantify what AI improves/brings to the clinical
  practices, e.g., reduce workload, improve workflow, provide better confidence, etc.

6. Regulatory and ethical considerations

• Compliance: HAS (Singapore), FDA (USA), NMPA (China) CE Mark (European), Number of image, age-range, race-range, to represent the target population
• Transparency: How are the AI decisions understood?
• Data privacy: compliance with PDDA, HIPPA, GDPR, PIPL, etc

7. Real-world deployment

• Pilot studies: deploy the application in a limited clinical setting
• Monitoring: Track performance over time and across different settings
• Feedback loop: Incorporate clinician feedback for continuous improvements

8. References

www.mendeley.com/download-reference-manager/windows

• APA 7th Edition
• In-text citations whenever appropriate to avoid plagiarism
• A list of references at the end
• The words of references and in-text citations are excluded from the word limit
• Recommendation: Use Mendeley reference manager or Endnote to help you manage references; Mendeley reference manager is free.

Grading Rubric – healthcare Informatic Assignment