Generative AI for Genomic Data Sharing

Work Stream / Subgroup:

Regulatory & Ethics Work Stream

Date (Day/Month):  Sep 17, 2024

Time (AEST): 11:15 AM - 12:45 PM

Session presenter(s): Prof. Ma’n Zawati, Yuan Y. Stevens, Marc Fiume

Session chair(s) will be present: in person

Recording: https://us02web.zoom.us/rec/share/OToCUS6027vHEhKCVeTOGAQyLARLQsy9KppGZfb936X__aJg9s7FKupOmSeiM3he.JGz6CxIexOgpDaQw?startTime=1726533391000

Passcode: d#d0Db2l

Zoom meeting ID: https://us02web.zoom.us/j/83727954499?pwd=oxyGfHca8dW0LKf0OCh8mniYlG0dJ5.1 

Link to slides: GA4GH Connect September Slides - Generative AI  - Updated.pptx

Link to guidance points: GA4GH Generative AI - Guidance Points August 2024.docx 

Aim of meeting:

• Share the revised Generative AI group’s guidance points with the broader GA4GH community
• Brainstorm improvements to the group’s guidance points on trustworthy AI

Session description:

With the current rise of generative AI tools, promising opportunities for the streamlining of access procedures and data querying are foreseeable. Conversational/generative AI capabilities based on large language models have also the potential to facilitate large-scale genomic data-sharing from multiple dataset providers. If this technology is to be rolled out in a responsible manner, however, guidance on the way these technologies can better account for attribution, privacy, bias, and accountability will be necessary. This meeting will bring together members of REWS and the GA4GH community to provide feedback and ideas for improvements to the REWS Generative AI group’s guidance points on more trustworthy AI tools.

Agenda:

Attendees:

Meeting minutes:

MZ welcomed all attendees to the session and outlined the aims of the group and the meeting to brainstorm and discuss any ideas for improvement.

YS then opened the session by highlighting the suggested themes on the group’s guidelines for responsible use of AI. The themes and comments raised outlined below:

1. Promote Human Well-being, Do No Harm, and Proportionality

1. Key points were around: limits of GenAI

1. Proportionality- Where does data minimisation come from when models are being trained on large datasets

• Medicine not deterministic but GenAI is developed as if it is, how do we balance these needs
• There is a need for use cases for GenAI for Genomics
• Conversational and GenAI- embedding models also to be included in scope

2. Recognise the Purposes and Limits of conversational and Generative AI Systems

• There are questions around the practical usage of GenAI in practical examples such as drafting clinical notes, analysing patterns in data for gene expression analysis and informing clinical decision making and understanding where the limits are for this technology and the need for human intervention such as with challenges like hallucinations, privacy, data breaches, epistemic authority of human experts vs automation bias, bias within AI, accountability and lack of empathy of AI.

• Biases can be introduced in tweaks to tools rather than just from raw training data. We may not be aware of the risks due to ‘black box’ AI.
• GDPR- purposes, what are the legal basis for doing AI work with a purpose.
• Human Rights case for AI i.e. equitable access etc.

3. Ensure Legal Authority and Permission for the Collection and Processing of Data

• Legal/ technical challenges of data scraping
• Reddit signed with Google about data use here
• Do you use open source software or create from scratch

4. Consult Indigenous Communities Regarding Data Processing and System Design

• Challenge-Indigenous data currently not encompassed in training and concerns around data mining purely to benefit technical development rather than bringing benefits back to the community. (Data colonialism)
• Beyond just Indigenous communities wider minority communities also need to be included.
• Indigenous governance and self determination should be included.
• Indigenous groups should decide to be included rather than from an external perspective.
• How to implement this in practise- onus on the developer of the models to be transparent around the biases that already exist within the data
• Methods for accessing your system; speak with any communities who may be harmed.

5. Facilitate Technical Robustness, Safety, Accuracy and Data Security

• How do you test AI tools for the above, sometimes red tape testing used by AI organisations to find room for improvement.

• Development of relevant skills for stakeholders to accurately use the tools, not just understanding the outcomes but the process for creating outcomes.
• Data security and privacy should be under technical theme too.
• Develop ways to access these models before putting them into practice.
• Train an AI similarly to how you would train a human to become an Dr/ Clinician
• How do we align with the growth of AI risk
• Certification for AI- could this framework be leveraged for usage in medical scenarios.
• Accuracy- accuracy from a statistical perspective or from other perspectives?
• Period of rapid innovation- set up the context, there are many unknowns. We should encourage experimentation in a responsible manner.
• Current regulation is limited, currently risk based on one understanding of risk.

6. Protect Privacy and Ensure Responsible Data Governance

• It was flagged that there is currently a tool development arms race against organisations with already established resources such as Google and OpenAI
• Benchmarks are becoming less relevant as models become more sophisticated
• Differentiate between building a model and usage of a model. (Do we want to use GenAI)

7. Foster Inclusiveness, Diversity, Fairness, Non-discrimination and Equity

• It was commented that transparency and accountability are in different sections but go together
• What if the tool fails these guidance points but is better than limited human resources? Liabilities around using tools.

8. Tension in the role of oversight, what happens when tools surpass those who are training them.

• It was raised that people are using tools anyway and that they have already exceeded expert levels.
• It was asked if there's a framework we can quantify and deploy to scan current AI tools- how much genetic information has already been included in AI tools.

9. Protect Human Agency, User and Patient Autonomy and Oversight

• Epistemic authority, no encouragement on clinicians decision making and no impact on care.
• Clinicians to check outputs, encouragement for clinicians to seek out additional information not just using the tools.

• As more biological information is incorporated into AI systems…
• Potential for GenAI to upskill practitioners, risk based decisions.

• Limits of disclaimers

• Transparency and explainability - explain to users about how they came to outcomes.
• Models, tools, system- an AI tool is a combination of things: software, data sets- each of these components have their own policy implications. (Do we need a diagram to explain all these elements)

10. Ensure Transparency, Explainability and Intelligibility

• Transparency- a description of how the GenAI came to a certain output. Important for accountability and informed consent.

11. Facilitate Consent Provenance and Traceability
12. Facilitate Responsibility and Accountability
13. Allow for Incident Reporting and for the Filing and Handling of Complaints

• If the model or tool created a chain of thought that the user could engage with.
• Explainability- language that is easily understood may not be true to the reality of the model.
• AI supply chain- may be different biases and accountability issues when you are using multiple systems that co-exist with one another.
• Can problems be looped back so that can be integrated into the tool
• Whenever you interact with the models you are contributing to training models- any regulations about data you contribute when you interact.
• Thousands of models interacting with one another- how does this scale
• How do you ensure software encompasses contexts of certain questions i.e. warfare.

Themes being workshopped still include:

• Promote Environmental Wellbeing and Sustainability
• Ensure Proper Attribution and Benefit Sharing

Helpful people to engage with as suggested by those participating in this session:

• UK- ICO
• Switzerland University of Freiburg Anna
• WHO- AI and Healthcare
• It was encouraged that data contributors (both who are aware and unaware) to be engaged with
• AI Safety Institute

MF presented on DNA Stack:

• For guidance points- some should be implemented by technical changes and some are for policy changes

Key takeaways:

AI legislation is context specific; there is a need for refining guidance points around responsible usage with feedback from real world implementation and development.

Next steps:

The group is in the process of drafting a publication. Comments on the draft guidance points are still being accepted.