AI Security for SaMD: Fortifying AI Systems against Risks and Attacks in Healthcare Technologies

• Digital Health Market Surge: The global demand for healthcare solutions drives a surge in digital health adoption, aiming for enhanced convenience, accessibility, and improved patient outcomes.
• The Rise of AI Risk Management in Healthcare: Healthcare organizations embrace cutting-edge AI-powered medical technologies to capitalize on numerous benefits. However, the rapid adoption of digital health brings security challenges that must be effectively addressed.
• Insights from Risk Management Summit: To fortify AI systems against potential attacks in medical technologies, insights from the 2023 Risk Management True Quality Summit Series are invaluable. Manpreet Dash from AIShield shared intriguing aspects, shedding light on crucial strategies for mitigating security risks in AI-driven healthcare solutions.

The 2023 Risk Management True Quality Summit Series delved into the critical aspects of Digital Health Cybersecurity, exploring the intersection of AI Security, regulatory compliance, and protection against evolving cyber threats.

The session by Manpreet Dash from AIShield covered a few intriguing points on AI risk management in healthcare.

• The progress of AI and ML in healthcare, alongside practical use cases
• Cybersecurity considerations in digital health, emphasizing regulatory checkpoints for Software as a Medical Device (SaMD)
• The potential impact of AI failures on patients
• The specific focus of regulatory bodies such as the FDA on AI-enabled SaMD
• Strategies for managing and mitigating AI cybersecurity risks
• The implementation of AI security solutions

The ongoing digital revolution is revolutionizing the landscape of healthcare. With the massive adoption of digital tools, the focus extends to prognosis, treatment, diagnosis, and clinical workflows. AI-enabled Software as a Medical Device (SaMD) is at the forefront, leveraging deep learning applications in radiology, pathology, dermatology, retinopathy, and ophthalmology.

Today, healthcare organizations leverage AI and ML technologies in several areas:

• Identifying diseases and diagnosis
• Clinical trial and research
• Smart health records and healthcare operation management Medical imaging diagnosis
• Radiology, ophthalmology, dermatology, neurology and pathology
• Robotic process automation (RPA)
• Drug discovery / new medicine development
• Virtual health assistants, preventive healthcare, and chatbots
• Anesthesiology, cardiovascular, clinical chemistry, hematology

The transformative integration of AI in medical domains yields promising outcomes. Such outcomes include enhanced population health management, informed clinical decision-making, and improved healthcare accessibility and efficiency. As digital technologies evolve, the healthcare sector is experiencing a paradigm shift towards more effective, precise, and patient-centric approaches.

However, Al/ML assets and SaMD are susceptible to cyber threats, resulting in data breaches, patient harm, inoperability of medical devices, and significant penalties. Hence, the key considerations include the following:

• The impact of AI failures on patient well-being.
• Regulatory expectations for cybersecurity compliance.
• Lifecycle and regulatory approval for AI/ML in healthcare.

Healthcare organizations are entrusted with extensive sensitive patient data. Further, SaMD devices, while advancing healthcare technology, also amplify potential security breaches. Attacks on these may lead to privacy breaches, patient harm through misdiagnosis or mistreatment, medical devices, hospital network inoperability, and massive penalties and recalls. Hence, balancing technological innovation with stringent AI/ML safety measures is crucial for maintaining patient safety and confidentiality.

Ensuring cybersecurity compliance involves:

• Designing secure devices
• Maintaining thorough security documentation
• Managing cyber risks
• Conducting verification and validation testing
• Implementing vigilant surveillance for the detection of potential threats

Every step is crucial in fortifying cybersecurity measures and fostering a resilient defense against evolving threats in the digital landscape.

AI risk management in healthcare involves examining various facets spanning datasets, AI/ML model performance, and clinical evaluation throughout the deployment. This comprises a detailed scrutiny of input data and features used to generate corresponding outputs, including the source, size, and attribution of training, validation, and test datasets. Furthermore, the model undergoes rigorous selection, evaluation, verification, and validation processes specific to Software as a Medical Device (SaMD) AI/ML.

Assessing the SaMD AI/ML performance, such as diagnostic sensitivity, specificity, and reproducibility, becomes paramount. Additionally, the compliance protocol delves into establishing the clinical association between SaMD AI/ML output and the targeted clinical condition, ensuring a comprehensive approach to cybersecurity within the healthcare landscape. Deployment involves defining the intended workflow and specifying the interval for updating training data.

Maintaining cybersecurity compliance is imperative to safeguard the functionality of Software as a Medical Device (SaMD) AI/ML and the well-being of patients. Manufacturers must diligently assess the risks associated with cybersecurity hazards, including adversarial AI threats, to ensure the safety, quality, performance, and presentation of SaMD AI/ML. The susceptibility of AI models to unique threats, particularly adversarial AI attacks, underscores the critical need for AI security. These attacks involve manipulating data, retraining models, and embedding secret features without influencing the training process or accessing the deployed model. Adversarial AI threats pose a risk of damage, evasion, or loss of integrity to the AI system, making comprehensive security measures essential in the healthcare landscape.

The session covered several case studies that answered some thought-provoking questions like - Can AI attacks lead to patient safety risk? Can AI attacks lead to financial damage and IP loss? Can AI failure affect patient well-being? And more.

The FDA is proactively adapting to the evolving landscape of technology, particularly AI. With initiatives such as the Proposed Regulatory Framework for AI SaMD (April 2019), the Action Plan (Jan 2021), and the Draft Guidance (April 2023), as well as the upcoming PATCH Act, the FDA aims to enhance regulation and monitoring of AI/ML-based Software as a Medical Device (SaMD). A comprehensive regulatory pathway with AI security involves embracing "Security by Design" throughout the development lifecycle, ensuring vigilance against potential cyber threats. Various global organizations and regions, including APAC, the US, the EU, and China, have proposed regulations and guidelines to fortify digital health against cyber threats, reinforcing the importance of readiness reviews and preventive measures against potential attacks.

Organizations face a significant threat from AI attacks, with 2 in 5 experiencing privacy breaches or security incidents, including 1 in 4 malicious attacks. Data compromise by internal parties and malicious attacks on AI assets are prevalent issues. According to a Gartner survey, 41% faced insider attacks, 60% had data compromised, and 27% dealt with malicious attacks on AI infrastructure. The conventional controls need to be revised to combat these evolving threats. AI risk management in healthcare is crucial for business success and maintaining regulatory compliance, ensuring better outcomes in the digital landscape.

AIShield, an enterprise-ready API-based AI Security product, demonstrates a next-gen capability to secure AI systems against adversarial ML attacks. The product addresses challenges such as AI misclassification, cancerous cell model compromise, and the overall impact of AI attacks on patient safety. Healthcare organizations will benefit from AIShield by gaining:

• Trustworthy and secure AI adoption – implementing security by design.
• Brand and IP protection – safeguarding intellectual property and brand reputation.
• Regulatory compliance – meeting evolving regulatory requirements in premarket and post-market activities.
• Real-time protection – vigilance against AI attacks post-deployment.
• Global competitive advantage – staying ahead with cutting-edge AI Security solutions.

The 2023 Risk Management True Quality Summit Series promises to be a transformative event for stakeholders in the healthcare and AI-driven industries. With AIShield at the forefront, healthcare organizations can be equipped to address AI risk management in healthcare, ensuring patient safety and meeting regulatory expectations.

Contact us for more information.