Introduction

In this document we present a catalog of AI risks and harms in the health domain that mirrors the risks of a variety of different AI applications listed in the AIAAIC Repository. The majority of AI systems informing our catalog of risks is applied in the US context. It is thus important to keep the regulatory differences in mind when investigating the parallels between the AI systems being applied in the US vs. their application in the EU context. The catalog focuses on AI systems applied in both public health and healthcare contexts. We chose a wider scope for selecting the use cases informing the taxonomy since many AI applications may be developed for personal care contexts but are designed with the intention to promote public health by either making processes in healthcare more cost-efficient, allowing for better triaging in clinical settings or by providing the data for health research with high public health relevance. The AI applications we chose were developed for the following purposes: (Prediction-based) diagnosis, clinical care, health research, drug development, health promotion, disease prevention, outbreak response, and health system management and planning. We found several use cases with health relevance related to workplace well-being or advertising that we excluded, since we focus on the professional health context.

Our catalog can be used for ethical and legal risk assessments of AI systems as well as the basis for interdisciplinary work on the risks of AI. As will become evident from our discussion of the current state of debate in the ethical, legal, and social science literature, it is often not clear how the addressed risks are conceptualized. We thus aim for a clear understanding of risks that include a risk source, the actual risk, as well as the risk impact. By establishing this notion of risk, we hope to contribute to a better understanding of ethical and legal risk assessment that allows comparison between different methodologies, and facilitates communication in interdisciplinary groups conducting ethical and legal risk assessments of AI systems.

The following figure illustrate an overview of AI concepts within HART, namely AITechnique, AICapability, AIProvider, AIUser, and Purpose.

: An overview of AI concepts within HART

Concepts related to AI risks within HART, specifically Risk, RiskSource, Consequence, Impact, ImpactArea, and AISubject, are depicted in the following figure.

: An overview of Risk concepts within HART

In the following sections we provide the competency questions which we believe our taxonomy could answer.

Ontology Requirement Specification Document

Below you can see the Ontology Requirement Specification Document (ORSD) table.

hart (AI Risk in the health domain Ontology)
1. Purpose
The purpose of this taxonomy is to support representing the specifications of AI systems in the health sector, which then would be helpful in understanding the risks of AI systems. Eventually, the taxonomy is useful in understanding, identifying, evaluating and mitigating harms of AI in the health sector.
2. Scope
The scope of this ontology is limited to the main risks and AI concepts mentioned in AIRO, which is an ontology for expressing risks associated with AI systems, and then expands the concepts using instances identified from AIAAIC.
3. Implementation Language
OWL
4. Intended End-Users
1. Data controllers collecting personal data, or in particular, Data Protection Officers (DPO), which want to document or/and represent the risk-related specifications of AI systems in health sector. 2. Data controllers or DPOs which want to assess the potential risks of using AI systems for different purposes in health sector.
5. Intended Uses
1. Representing/Documentation of the risk-related specifications of AI systems in health domain.
2. Understanding what are the risks of an AI Systems based on its specifications (e.g., utilised technology, purpose, data subjects etc.)
6. Ontology Requirements
a. Non-Functional Requirements
NFR 1. The ontology should be published online, following the FAIR principle[].
b. Functional Requirements: Competency Questions
Type of AI System
CQ1. What types of AI applications are available in the health sector?
Purpose of AI System
CQ2. What are the purposes of using AI in the healthcare sector?
Technology/Techniques utilised in AI System
CQ3. Which AI techniques are utilised in AI systems used in the health domain?
Source of Risk in the Health domain
CQ4. What types of events can be the source of risk in the health domain?
Type of Risks in Diagnostic Apps
CQ5. What types of risks are associated with diagnostic applications?
Type of Risks with use of ML in Diagnistic Apps
CQ6. What types of risks are associated with use of machine learning (ML) in diagnostic applications?
Potential Negative Impacts
CQ7. What are the potential negative impacts of AI in the health sector?
Potential Impacted Entities
CQ8.Who can be impacted by AI systems utilised in the health sector?

Namespaces

Prefix Namespace
hart http://wwww3id.org/hart#
AIRO http://www.w3id.org/AIRO#
rdfs http://www.w3.org/2000/01/rdf-schema#
owl http://www.w3.org/2002/07/owl#
dct http://purl.org/dc/terms/
ns1 http://purl.org/vocab/vann/
xsd http://www.w3.org/2001/XMLSchema#
dpv http://www.w3.org/ns/dpv#

Specification

In this section, the classes and properties of the taxonomy are listed. .

AI System

Definition: Represents a software that is developed with one or more of the AI Approaches and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with.
Term: AIRO: AISystem
Source: AI Act, Art 3(1),

AI Capability

Definition: The capability of AI for which it is developed.
Term: hart: AICapability

Facial Recognition

Term: hart: FacialRecognition
SubClass of: hart: AICapability

Text Analysis

Term: hart:TextAnalysis
SubClass of: hart: AICapability

Voice Recognition

Definition: Property to associate processing activity to its frequency.
Term: hart: VoiceRecognition
SubClass of: hart: AICapability

Actor

Term: AIRO:Actor

AIProvider

Definition: A natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark, whether for payment or free of charge.
Term: AIRO:AIProvider
SubClass of: AIRO: Actor
Source: AI Act,

Government Agency

Definition: Indicates agencies linked to ministeries, municipalities, etc.
Term: hart:GovernmentAgency
SubClass of: AIRO: AIProvider

Private Company

Definition: Indicates a private company who develop or deploy the AI system for financial profit.
Term: hart:PrivateCompany
SubClass of: AIRO: AIProvider

Public Organization

Definition: Indicates public health organisations such as public health agencies, Public bodies, and associations.
Term: hart:PublicOrganization
SubClass of: AIRO: AIProvider

Public Health Organisation

Term: hart: PublicHealthOrganisation
SubClass of: hart: PublicOrganisation

Research Facility

Definition: Indicates educational and research facilities such as universities, research insitutes, whether they are private or public.
Term: hart:ResearchFacility
SubClass of: hart: AIProvider

AI Technique

Definition: Represents the main approaches and techniques used for development of AI systems.
Term: AIRO:AITechnique

Computer Vision

Definition: Refers to acquiring, processing, and interpreting visual data.
Term: hart:ComputerVision
SubClass:of AIRO: AITechnique

Machine Learning

Definition: A process by which a functional unit improves its performance by acquiring new knowledge or skills or by reorganizing existing knowledge or skills.
Term: hart:MachineLearning
SubClass:of AIRO: AITechnique

Deep Learning

Definition: An approach to creating rich hierarchical representations through the training of neural networks with many hidden layers.
Term: hart:DeepLearning
SubClass:of hart: MachineLearning

Natural Language Processing

Definition: Refers to information processing based upon natural-language understanding and natural language generation.
Term: hart:NaturalLanguageProcessing
SubClass:of AIRO: AITechnique

Robotics

Definition: Science and practice of designing, manufacturing, and applying robots.
Term: hart:Robotics
SubClass:of AIRO: AITechnique

Area Of Impact

Definition: Represents areas that AI might have impact on.
Term: AIRO:AreaOfImpact

Financial Status

Term: hart:FinancialStatus
SubClass:of AIRO: AreaOfImpact

Justice

Term: hart:Justice
SubClass:of AIRO: AreaOfImpact

Public Health

Term: hart:PublicHealth
SubClass:of AIRO: AreaOfImpact

Right to Data Protection

Term: hart:RightToDataProtection
SubClass:of hart: Right

Right To Non-Discrimination

Term: hart:RightToNon-Discrimination
SubClass:of hart: Right

Well-Being

Term: hart:Well-Being
SubClass:of AIRO: AreaOfImpact

Autonomy

Term: hart:Autonomy
SubClass:of hart: Well-Being

Health

Term: AIRO:Health
SubClass:of hart: Well-Being

PhysicalHealth

Definition: Represents physical health
Term: AIRO:PhysicalHealth
SubClass:of AIRO: Health

PsychologicalHealth

Definition: Represents psychological health.
Term: AIRO:PsychologicalHealth
SubClass:of AIRO: Health

Security

Term: hart:Security
SubClass:of hart: Well-Being

Event

Term: AIRO:Event

Consequence

Definition: Outcome of an event affecting objectives.
Term: AIRO:Consequence
SubClass of: AIRO:Event
Source: ISO 31000, 3.6

Impact

Definition: Represents outcomes of a consequence on persons, groups, facilities, environment, etc.
Term: AIRO:Impact

Fundamental Rights Infringement

Term: hart:FundamentalRightsInfringement
SubClass of: AIRO: Impact

Unequal Access To Health care

Term: hart:UnequalAccessToHealthcare
SubClass of: AIRO:Impact

Exclusion Of Patients

Term: hart:ExclusionOfPatients
SubClass of: hart:UnequalAccessToHealthcare

Unequal Distribution Of Health Resources

Term: hart:UnequalDistributionOfHealthResources
SubClass of:: AIRO:Impact

Wellbeing Impact

Definition: Represents impact of AI on persons' health and wellbeing.
Term: AIRO:WellbeingImpact

Death

Definition: Represents death of a person caused by AI.
Term: AIRO:Death
SubClass of: AIRO:WellbeingImpact

Injury

Definition: Represents physical injuries caused by AI.
Term: AIRO:Injury
SubClass of: AIRO: WellbeingImpact

Hemorrhage

Term: hart:Hemorrhage
SubClass of: AIRO: Injury

Psychological Harm

Definition: Represents psychological injuries caused by AI.
Term: AIRO:PsychologicalHarm
SubClass of: AIRO: WellbeingImpact

Feeling of Constant Surveillance

Term: hart:FeelingOfConstantSurveillance
SubClass of: AIRO:PsychologicalHarm

Increased Anxiety Level

Term: hart:IncreasedAnxietyLevel
SubClass of: AIRO:PsychologicalHarm

Inaccurate Outcome

Term: hart:InaccurateOutcome
SubClass of: AIRO:Consequence

Misdiagnosis

Term: hart:Misdiagnosis
SubClass of: hart:InaccurateOutcome

Overdiagnosis

Term: hart:Overdiagnosis
SubClass of: hart:InaccurateOutcome

Mismanagement of Health Resources

Term: hart:MismanagementOfHealthResources
SubClass of: AIRO:Consequence

Privacy Violation

Term: hart:PrivacyViolation
SubClass of: AIRO:Consequence

Surgery Complication

Term: hart:SurgeryComplication
SubClass of: AIRO:Consequence

Unfair Outcome

Term: hart:UnfairOutcome
SubClass of: AIRO:Consequence

Risk

Definition: Indicates a risk (of harm) associated with an AI system.
Term: AIRO:Risk
SubClass of: AIRO:Event

Bias

Term: hart:Bias
SubClass of: AIRO:Risk

Gender Bias

Term: hart:GenderBias
SubClass of: hart:Bias

Racial Bias

Term: hart:Racial Bias
SubClass of: hart:Bias

Data De-identification

Term: hart:DataDe-identification
SubClass of: AIRO:Risk

Hardware Malfunctioning

Term: hart:HardwareMalfunctioning
SubClass of: AIRO:Risk

Low Accuracy

Term: hart:LowAccuracy
SubClass of: AIRO:Risk

Sensitive Data Exposure

Term: hart:SensitiveDataExposure
SubClass of: AIRO:Risk

Unlawful Use of Sensitive Data

Term: hart:UnlawfulUseOfSensitiveData
SubClass of: AIRO:Risk

Risk Source

Definition" Indicates an element that has the potential give rise to a risk.
Term: hart:RiskSource
SubClass of: AIRO:Event

Algorithm-Related Risk Source

Term: hart:Algorithm-RelatedRiskSource
SubClass of: AIRO:RiskSource

DataSet-Related Risk Source

Term: hart:DataSet-RelatedRiskSource
SubClass of: AIRO:RiskSource

Hardware-Related Risk Source

Term: hart:Hardware-RelatedRiskSource
SubClass of: AIRO:RiskSource

UseOf Wrong Features

Term: hart:UseOfWrongFeatures
SubClass of: hart:Algorithm-RelatedRiskSource

Insufficient Training Data

Term: hart:InsufficientTrainingData
SubClass of: hart:DataSet-RelatedRiskSource

Sensitive Data-Related RiskSource

Term: hart:SensitiveData-RelatedRiskSource
SubClass of: hart:DataSet-RelatedRiskSource

Not Anonymising Sensitive Data

Term: hart:NotAnonymisingSensitiveData
SubClass of: hart:SensitiveData-RelatedRiskSource

Training Data Risk Source

Term: hart:TrainingDataRiskSource
SubClass of: hart:DataSet-RelatedRiskSource

Unrepresentative Training DataSet

Term: hart:UnrepresentativeTrainingDataSet
SubClass of: hart:TrainingDataRiskSource

Device Malfunctioning

Term: hart:DeviceMalfunctioning
SubClass of: hart:Hardware-RelatedRiskSource

Technical Failure

Term: hart:TechnicalFailure
SubClass of: hart:Hardware-RelatedRiskSource

Purpose

Definition: Refers to the use for which an AI system is intended by the provider, including the specific context and conditions of use, as specified in the information supplied by the provider in the instructions for use, promotional or sales materials and statements, as well as in the technical documentation.
Term: hart:Purpose
Source: AI Act, Art 3(12)

Advertising

Definition: Refers to the use of health-related data or information for promoting consumer products.
Term: hart:Advertising
SubClass of: AIRO:Purpose

Clinical Care

Definition: Refers to use of AI for identification of risk patients, aid in treatment decisions, triage.
Term: hart:ClinicalCare
SubClass of: AIRO:Purpose

Diagnosis

Definition: Refers to use of AI systems for classification and identification of symptoms, anticipation of symptoms.
Term: hart:Diagnosis
SubClass of: AIRO:Purpose

Disease Prevention

Definition: Refers to use of AI for identification of public health risks.
Term: hart:DiseasePrevention
SubClass of: AIRO:Purpose

Drug Development

Definition: Refers to use od AI for simplified or accelerated development of medications.
Term: hart:DrugDevelopment
SubClass of: AIRO:Purpose

Health Promotion

Definition: Refers to use of AI for identification of risk populations to target with public health measures, e.g. information campaigns and behavior change.
Term: hart:HealthPromotion
SubClass of: AIRO:Purpose

Health Research

Definition: Refers to health research using data from electronic health records.
Term: hart:HealthResearch
SubClass of: AIRO:Purpose

Health System Management and Planning

Definition: Refers to use of AI for resource allocation, optimization of processes, support for decision-making, scheduling, and identification of staffing requirements.
Term: hart:HealthSystemManagementAndPlanning
SubClass of: AIRO:Purpose

Outbreak Response

Definition: Refers to use of AI for transmission tracking, detection of outbreaks, and enforcement of restriction measures.
Term: hart:OutbreakResponse
SubClass of: AIRO:Purpose

Workplac eWell-being

Definition: Refers to the use of AI for promotion of well-being at the workplace.
Term: hart:WorkplaceWell-being
SubClass of: AIRO:Purpose