AI Models DINO and SAM Revolutionize Medical Triage at University of Pennsylvania

In a groundbreaking initiative, the University of Pennsylvania is leveraging advanced AI models to revolutionize emergency medical triage, according to Meta AI. The initiative, which utilizes AI models such as DINO and SAM, aims to enhance automation and effectiveness in emergency response scenarios.

Evolution of Triage Practices

Triage, originating from the French verb “trier,” meaning “to sort,” has evolved from its Napoleonic roots into a crucial component of emergency medical response. Traditionally, triage relies on standardized protocols to guide first responders through critical decision-making processes. However, in chaotic mass casualty incidents (MCIs), adhering to these protocols can be challenging.

Recent advancements in computer vision, robotics, and machine learning are reshaping this landscape. These technologies are pushing the boundaries of medical AI, particularly in military settings by improving rapid and effective triage operations.

DARPA’s Challenge and PRONTO’s Role

The United States Defense Advanced Research Projects Agency (DARPA) has initiated a three-year challenge to drive innovation in this field. The challenge involves using stand-off sensors on autonomous systems to detect physiological signatures that indicate critical illness levels, even in environments with limited connectivity.

The Penn Robotic Non-contact Triage and Observation (PRONTO) team, which includes experts from Penn Medicine and Penn Engineering, is at the forefront of this challenge. By integrating cutting-edge robotics with Meta’s SAM and DINO models, PRONTO is developing autonomous systems for rapid injury detection and assessment in disaster scenarios.

Technical Innovations and Deployment

In the initial phase of the challenge, PRONTO deployed a system using drones and ground robots to survey disaster scenes, capturing vital data for injury assessment. This data is processed through Meta’s Segment Anything Model 2 (SAM 2), which efficiently segments objects in images and videos.

Moreover, the use of DINO, which doesn’t require labeled data, enhances the system’s efficiency and scalability. This model generalizes across diverse domains, including medical imagery, enabling the extraction of visual features for identifying injuries through a specialized neural network.

By collaborating with SAM and the Grounding DINO model, PRONTO utilizes text prompts like “wound?” and “blood?” to detect injury-related features, thus providing a comprehensive triage solution.

For more detailed insights, visit the Meta AI blog.

Image source: Shutterstock