In this document we present a catalog of AI risks and harms in the health domain that mirrors the risks of a variety of different AI applications listed in the AIAAIC Repository. The majority of AI systems informing our catalog of risks is applied in the US context. It is thus important to keep the regulatory differences in mind when investigating the parallels between the AI systems being applied in the US vs. their application in the EU context. The catalog focuses on AI systems applied in both public health and healthcare contexts. We chose a wider scope for selecting the use cases informing the taxonomy since many AI applications may be developed for personal care contexts but are designed with the intention to promote public health by either making processes in healthcare more cost-efficient, allowing for better triaging in clinical settings or by providing the data for health research with high public health relevance. The AI applications we chose were developed for the following purposes: (Prediction-based) diagnosis, clinical care, health research, drug development, health promotion, disease prevention, outbreak response, and health system management and planning. We found several use cases with health relevance related to workplace well-being or advertising that we excluded, since we focus on the professional health context.
Our catalog can be used for ethical and legal risk assessments of AI systems as well as the basis for interdisciplinary work on the risks of AI. As will become evident from our discussion of the current state of debate in the ethical, legal, and social science literature, it is often not clear how the addressed risks are conceptualized. We thus aim for a clear understanding of risks that include a risk source, the actual risk, as well as the risk impact. By establishing this notion of risk, we hope to contribute to a better understanding of ethical and legal risk assessment that allows comparison between different methodologies, and facilitates communication in interdisciplinary groups conducting ethical and legal risk assessments of AI systems.
The following figure illustrate an overview of AI concepts within HART, namely AITechnique, AICapability, AIProvider, AIUser, and Purpose.
Concepts related to AI risks within HART, specifically Risk, RiskSource, Consequence, Impact, ImpactArea, and AISubject, are depicted in the following figure.
In the following sections we provide the competency questions which we believe our taxonomy could answer.
Below you can see the Ontology Requirement Specification Document (ORSD) table.
hart (AI Risk in the health domain Ontology) | |
---|---|
1. Purpose | |
The purpose of this taxonomy is to support representing the specifications of AI systems in the health sector, which then would be helpful in understanding the risks of AI systems. Eventually, the taxonomy is useful in understanding, identifying, evaluating and mitigating harms of AI in the health sector. | |
2. Scope | |
The scope of this ontology is limited to the main risks and AI concepts mentioned in AIRO, which is an ontology for expressing risks associated with AI systems, and then expands the concepts using instances identified from AIAAIC. | |
3. Implementation Language | |
OWL | |
4. Intended End-Users | |
1. Data controllers collecting personal data, or in particular, Data Protection Officers (DPO), which want to document or/and represent the risk-related specifications of AI systems in health sector. 2. Data controllers or DPOs which want to assess the potential risks of using AI systems for different purposes in health sector. | |
5. Intended Uses | |
1. Representing/Documentation of the risk-related specifications of AI systems in health domain. 2. Understanding what are the risks of an AI Systems based on its specifications (e.g., utilised technology, purpose, data subjects etc.) |
|
6. Ontology Requirements | |
a. Non-Functional Requirements | |
NFR 1. The ontology should be published online, following the FAIR principle[]. | |
b. Functional Requirements: Competency Questions | |
Type of AI System | |
CQ1. What types of AI applications are available in the health sector? |
|
Purpose of AI System | |
CQ2. What are the purposes of using AI in the healthcare sector? |
|
Technology/Techniques utilised in AI System | |
CQ3. Which AI techniques are utilised in AI systems used in the health domain? |
|
Source of Risk in the Health domain | |
CQ4. What types of events can be the source of risk in the health domain? |
|
Type of Risks in Diagnostic Apps | |
CQ5. What types of risks are associated with diagnostic applications? |
|
Type of Risks with use of ML in Diagnistic Apps | |
CQ6. What types of risks are associated with use of machine learning (ML) in diagnostic applications? |
|
Potential Negative Impacts | |
CQ7. What are the potential negative impacts of AI in the health sector? |
|
Potential Impacted Entities | |
CQ8.Who can be impacted by AI systems utilised in the health sector? |
Prefix | Namespace |
---|---|
hart | http://wwww3id.org/hart# |
AIRO | http://www.w3id.org/AIRO# |
rdfs | http://www.w3.org/2000/01/rdf-schema# |
owl | http://www.w3.org/2002/07/owl# |
dct | http://purl.org/dc/terms/ |
ns1 | http://purl.org/vocab/vann/ |
xsd | http://www.w3.org/2001/XMLSchema# | dpv | http://www.w3.org/ns/dpv# |
In this section, the classes and properties of the taxonomy are listed. .
Definition: | Represents a software that is developed with one or more of the AI Approaches and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with. |
---|---|
Term: | AIRO: AISystem |
Source: | AI Act, Art 3(1), |
Definition: | The capability of AI for which it is developed. |
---|---|
Term: | hart: AICapability |
Term: | hart: FacialRecognition |
---|---|
SubClass of: | hart: AICapability |
Term: | hart:TextAnalysis |
---|---|
SubClass of: | hart: AICapability |
Definition: | Property to associate processing activity to its frequency. |
---|---|
Term: | hart: VoiceRecognition |
SubClass of: | hart: AICapability |
Term: | AIRO:Actor |
---|
Definition: | A natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark, whether for payment or free of charge. |
---|---|
Term: | AIRO:AIProvider |
SubClass of: | AIRO: Actor |
Source: | AI Act, |
Definition: | Indicates agencies linked to ministeries, municipalities, etc. |
---|---|
Term: | hart:GovernmentAgency |
SubClass of: | AIRO: AIProvider |
Definition: | Indicates a private company who develop or deploy the AI system for financial profit. |
---|---|
Term: | hart:PrivateCompany |
SubClass of: | AIRO: AIProvider |
Definition: | Indicates public health organisations such as public health agencies, Public bodies, and associations. |
---|---|
Term: | hart:PublicOrganization |
SubClass of: | AIRO: AIProvider |
Term: | hart: PublicHealthOrganisation |
---|---|
SubClass of: | hart: PublicOrganisation |
Definition: | Indicates educational and research facilities such as universities, research insitutes, whether they are private or public. |
---|---|
Term: | hart:ResearchFacility |
SubClass of: | hart: AIProvider |
Definition: | Represents the main approaches and techniques used for development of AI systems. |
---|---|
Term: | AIRO:AITechnique |
Definition: | Refers to acquiring, processing, and interpreting visual data. |
---|---|
Term: | hart:ComputerVision |
SubClass:of | AIRO: AITechnique |
Definition: | A process by which a functional unit improves its performance by acquiring new knowledge or skills or by reorganizing existing knowledge or skills. |
---|---|
Term: | hart:MachineLearning |
SubClass:of | AIRO: AITechnique |
Definition: | An approach to creating rich hierarchical representations through the training of neural networks with many hidden layers. |
---|---|
Term: | hart:DeepLearning |
SubClass:of | hart: MachineLearning |
Definition: | Refers to information processing based upon natural-language understanding and natural language generation. |
---|---|
Term: | hart:NaturalLanguageProcessing |
SubClass:of | AIRO: AITechnique |
Definition: | Science and practice of designing, manufacturing, and applying robots. |
---|---|
Term: | hart:Robotics |
SubClass:of | AIRO: AITechnique |
Definition: | Represents areas that AI might have impact on. |
---|---|
Term: | AIRO:AreaOfImpact |
Term: | hart:FinancialStatus |
---|---|
SubClass:of | AIRO: AreaOfImpact |
Term: | hart:Justice |
---|---|
SubClass:of | AIRO: AreaOfImpact |
Term: | hart:PublicHealth |
---|---|
SubClass:of | AIRO: AreaOfImpact |
Term: | hart:Right |
---|---|
SubClass:of | AIRO: AreaOfImpact |
Term: | hart:RightToDataProtection |
---|---|
SubClass:of | hart: Right |
Term: | hart:RightToNon-Discrimination |
---|---|
SubClass:of | hart: Right |
Term: | hart:Well-Being |
---|---|
SubClass:of | AIRO: AreaOfImpact |
Term: | hart:Autonomy |
---|---|
SubClass:of | hart: Well-Being |
Term: | AIRO:Health |
---|---|
SubClass:of | hart: Well-Being |
Definition: | Represents physical health |
---|---|
Term: | AIRO:PhysicalHealth |
SubClass:of | AIRO: Health |
Definition: | Represents psychological health. |
---|---|
Term: | AIRO:PsychologicalHealth |
SubClass:of | AIRO: Health |
Term: | hart:Security |
---|---|
SubClass:of | hart: Well-Being |
Term: | AIRO:Event |
---|
Definition: | Outcome of an event affecting objectives. |
---|---|
Term: | AIRO:Consequence |
SubClass of: | AIRO:Event |
Source: | ISO 31000, 3.6 |
Definition: | Represents outcomes of a consequence on persons, groups, facilities, environment, etc. |
---|---|
Term: | AIRO:Impact |
Term: | hart:FundamentalRightsInfringement |
---|---|
SubClass of: | AIRO: Impact |
Term: | hart:UnequalAccessToHealthcare |
---|---|
SubClass of: | AIRO:Impact |
Term: | hart:ExclusionOfPatients |
---|---|
SubClass of: | hart:UnequalAccessToHealthcare |
Term: | hart:UnequalDistributionOfHealthResources |
---|---|
SubClass of:: | AIRO:Impact |
Definition: | Represents impact of AI on persons' health and wellbeing. |
---|---|
Term: | AIRO:WellbeingImpact |
Definition: | Represents death of a person caused by AI. |
---|---|
Term: | AIRO:Death |
SubClass of: | AIRO:WellbeingImpact |
Definition: | Represents physical injuries caused by AI. |
---|---|
Term: | AIRO:Injury |
SubClass of: | AIRO: WellbeingImpact |
Term: | hart:Hemorrhage |
---|---|
SubClass of: | AIRO: Injury |
Definition: | Represents psychological injuries caused by AI. |
---|---|
Term: | AIRO:PsychologicalHarm |
SubClass of: | AIRO: WellbeingImpact |
Term: | hart:FeelingOfConstantSurveillance |
---|---|
SubClass of: | AIRO:PsychologicalHarm |
Term: | hart:IncreasedAnxietyLevel |
---|---|
SubClass of: | AIRO:PsychologicalHarm |
Term: | hart:InaccurateOutcome |
---|---|
SubClass of: | AIRO:Consequence |
Term: | hart:Misdiagnosis |
---|---|
SubClass of: | hart:InaccurateOutcome |
Term: | hart:Overdiagnosis |
---|---|
SubClass of: | hart:InaccurateOutcome |
Term: | hart:MismanagementOfHealthResources |
---|---|
SubClass of: | AIRO:Consequence |
Term: | hart:PrivacyViolation |
---|---|
SubClass of: | AIRO:Consequence |
Term: | hart:SurgeryComplication |
---|---|
SubClass of: | AIRO:Consequence |
Term: | hart:UnfairOutcome |
---|---|
SubClass of: | AIRO:Consequence |
Definition: | Indicates a risk (of harm) associated with an AI system. |
---|---|
Term: | AIRO:Risk |
SubClass of: | AIRO:Event |
Term: | hart:Bias |
---|---|
SubClass of: | AIRO:Risk |
Term: | hart:GenderBias |
---|---|
SubClass of: | hart:Bias |
Term: | hart:Racial Bias |
---|---|
SubClass of: | hart:Bias |
Term: | hart:DataDe-identification |
---|---|
SubClass of: | AIRO:Risk |
Term: | hart:HardwareMalfunctioning |
---|---|
SubClass of: | AIRO:Risk |
Term: | hart:LowAccuracy |
---|---|
SubClass of: | AIRO:Risk |
Term: | hart:SensitiveDataExposure |
---|---|
SubClass of: | AIRO:Risk |
Term: | hart:UnlawfulUseOfSensitiveData |
---|---|
SubClass of: | AIRO:Risk |
Definition" | Indicates an element that has the potential give rise to a risk. |
---|---|
Term: | hart:RiskSource |
SubClass of: | AIRO:Event |
Term: | hart:Algorithm-RelatedRiskSource |
---|---|
SubClass of: | AIRO:RiskSource |
Term: | hart:DataSet-RelatedRiskSource |
---|---|
SubClass of: | AIRO:RiskSource |
Term: | hart:Hardware-RelatedRiskSource |
---|---|
SubClass of: | AIRO:RiskSource |
Term: | hart:UseOfWrongFeatures |
---|---|
SubClass of: | hart:Algorithm-RelatedRiskSource |
Term: | hart:InsufficientTrainingData |
---|---|
SubClass of: | hart:DataSet-RelatedRiskSource |
Term: | hart:SensitiveData-RelatedRiskSource |
---|---|
SubClass of: | hart:DataSet-RelatedRiskSource |
Term: | hart:NotAnonymisingSensitiveData |
---|---|
SubClass of: | hart:SensitiveData-RelatedRiskSource |
Term: | hart:TrainingDataRiskSource |
---|---|
SubClass of: | hart:DataSet-RelatedRiskSource |
Term: | hart:UnrepresentativeTrainingDataSet |
---|---|
SubClass of: | hart:TrainingDataRiskSource |
Term: | hart:DeviceMalfunctioning |
---|---|
SubClass of: | hart:Hardware-RelatedRiskSource |
Term: | hart:TechnicalFailure |
---|---|
SubClass of: | hart:Hardware-RelatedRiskSource |
Definition: | Refers to the use for which an AI system is intended by the provider, including the specific context and conditions of use, as specified in the information supplied by the provider in the instructions for use, promotional or sales materials and statements, as well as in the technical documentation. |
---|---|
Term: | hart:Purpose |
Source: | AI Act, Art 3(12) |
Definition: | Refers to the use of health-related data or information for promoting consumer products. |
---|---|
Term: | hart:Advertising |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use of AI for identification of risk patients, aid in treatment decisions, triage. |
---|---|
Term: | hart:ClinicalCare |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use of AI systems for classification and identification of symptoms, anticipation of symptoms. |
---|---|
Term: | hart:Diagnosis |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use of AI for identification of public health risks. |
---|---|
Term: | hart:DiseasePrevention |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use od AI for simplified or accelerated development of medications. |
---|---|
Term: | hart:DrugDevelopment |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use of AI for identification of risk populations to target with public health measures, e.g. information campaigns and behavior change. |
---|---|
Term: | hart:HealthPromotion |
SubClass of: | AIRO:Purpose |
Definition: | Refers to health research using data from electronic health records. |
---|---|
Term: | hart:HealthResearch |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use of AI for resource allocation, optimization of processes, support for decision-making, scheduling, and identification of staffing requirements. |
---|---|
Term: | hart:HealthSystemManagementAndPlanning |
SubClass of: | AIRO:Purpose |
Definition: | Refers to use of AI for transmission tracking, detection of outbreaks, and enforcement of restriction measures. |
---|---|
Term: | hart:OutbreakResponse |
SubClass of: | AIRO:Purpose |
Definition: | Refers to the use of AI for promotion of well-being at the workplace. |
---|---|
Term: | hart:WorkplaceWell-being |
SubClass of: | AIRO:Purpose |