AGENDA

REWS Diversity in Datasets Meeting 10
29th March 2023 @ 11am UTC

Zoom Recording

In preparation for the meeting please read the following:

• Overall Policy Outline

Attendees:

1. Anna Lewis
2. Lindsay Smith
3. Graciela Moya
4. Ariana Gonzalez
5. Beatrice Kaiser
6. Sasha Henriques
7. Alessia Costa
8. Alham Saadat
9. Hayley Clissold

Minutes:

AL: Going to give an overview of the doc, and then flag a few things for discussion.

A big thing we don’t do is define what DiD means. Equity is going to be one of the main concepts drawn upon, only define in very general terms and also going to be context specific.

We mention that there are frequent calls for DiD, and there are benefits to the diversity in datasets. Ongoing and historical background as to why we lack diversity in datasets, also stress the importance of the sharing landscape.

There are some well motivated calls for DiD, but there is a lack of specificity in these calls, not clear what is meant. Often means ancestral diversity, but that is ambiguous.  Even if appropriately thinking about genetic ancestry, there are limitations because of the different dimensions.

We propose a way to go about identifying what types of differences are relevant and why. Equity might often be the most important concept, but not the only. Diversity is not necessarily about representativeness, and we should think about the different stages of a data lifecycle.

We say that researchers should ask the following three questions: 1) what are the goals, 2) given the goals, what types of data are important, and 3) what are the implications for choices made throughout the lifecycle. Two examples for this. First is minimizing reporting false positive pathogenic variants. Evidence that a variant is at or above some frequency cutoff, in any suitable defined group, is evidence its benign for anyone who harvests it. Let’s imagine a projects that aims to end in equity, or equal likelihood for a variant being pathogenic or not. Need to define the groups, and the thresholds. Would need work to process and analyze the data, need to create the data to make sure that there is sufficient coverage to define the groups.

AC: Are we tying the idea of goals to project specifically?

AL: As written, the goals are project-specific. Those project-specific goals are going to involve equity, they need to spell out what that means in that case.

AC: More interested in the difference between a goal to help equity vs the goal to identify pathogenic variants. The goal seems tight to the concept of equity, but could there be projects that don’t see that as relevant to themselves.

AL: Asking the group to connect that to higher order ends. Need to tie this in better, and to Yann’s point, think about what are suitable ends. Most projects should have public benefit motivating them.

AG: It would be interesting to introduce the sense of community and the work, some kind of ownership.

AS: Building on whats been said and thinking about how to conceptualize a goal, worth thinking about different types of goals, like hypothesis-driven vs more abstract.

AL: What we’d need to do is use the language of higher-order goals to think about the eventual goods.

AC: Agree, might be a way to move projects to consider the questions of harm. For the second step, if we maybe need to add something that speaks about the possible implications or applications.

AL: Roll that into #1?
AC: Yes something in step 1 to encourage link between scientific goals and the ethical implications of those goals.

GM: Equity is a requirement for each test we perform in clinical settings. Need to introduce this concept in research. I think it’s not a goal by itself but a way to work, a characteristic.

AL: Expanding this first 1.

GM: Yes, we work with people who are quite different.

AL: Confident that topics of justice and equity are always relevant, but wonder whether we should decide there are core principle to call out.

SH: Started with the language that we are using in defining these terms, it’s language and concepts we’re familiar with. Going towards that list of principles. Having these principles is a way of calling that out. Also, is there is added bit about what the develop of that principle is in the context of a project.

HC: The end goal in the sense of when it’s been achieved, or diverse enough for that project.

AL: Something I like about our two examples is that we make it quantitative.

AS: Principles or values and have people reference as a grounding bit.

AC: Listing a set of principles and asking researchers to connect to them. Also think about additional principles that might be important to their particular project.

AG: Need to give some horizon or explanation.

AL: Some notion of justice and respect. Question is do we go on and enumerate other things. One could easily justify other sets of principles.

SH: Just putting the principles, I agree. Worries me that they can just sit there, so leaning towards b or c, that it needs to prompt this horizon development.

AS: One other relevant factor is being cognizant of the history context of genetics its operating in, and thats been called out in the document. There are different models for this in communities that have been harmed and how do we think about that.

AL: Going to think about point 1 and being more prescriptive.

Example 2

Given the goals of achieving comparable predictive performance of PGS, what are the goals. Should be assessed across different dimensions of difference. Genetic ancestry are driving differences but that’s not the only thing that matters.

AC: When we say comparable predictability or progress towards equity. How comparable is comparable and what progress is enough? How to make assumptions about the dimensions. Could there be a dimension of diversity that is important.

AL: Can think of some types of ways this could be tighter. One wouldn't’ expect these predictive abilities to be better or worse. Curious about how detailed it would be helpful for us to be.

AC: Maybe be prescriptive on what we know

AG: We’re trying to validate PGS for cancer, and we do not have that data for all of our patients.

AL: For anyone overlooking a research portfolio, who need to be thinking a bit more broadly about the projects.

GM: Ethics committees have a big roles here.

AL: Have something different aimed at them or at least stressing to them what researchers should be considering.

AS: Like that a lot. Maybe making a specific recommendation that you should seek consultation from an ethicist. Calling to seek this out.