AGENDA

REWS Diversity in Datasets Meeting 10
28th March 2023 @ 8pm UTC

Zoom Recording

In preparation for the meeting please read the following:

• Overall Policy Outline

Attendees:

1. Anna Lewis
2. Lindsay Smith
3. Yasmeen Kurdi
4. Hanshi Liu
5. Henriette Raventos
6. Sue Chen
7. Sarah Nelson
8. Aime Lumaka

Minutes:

GA4GH April Connect Registration

AL: Goal is to get input on the v0.1 policy draft that was circulated a week or so ago. Will have a session at GA4GH Connect in a few weeks. Planning to a get a final draft to all of REWS closely following the meeting.

Something that characterises this policy is what we don’t attempt to do. Don’t attempt to do what diversity in data means or exactly entails. We don’t define exactly the concept of equity. It’s not that prescriptive. Instead we make a case and outline how researchers should think through what diversity in datasets means for their particular project or endeavour. This is not a very rules based approach.

Are there places where we should be more prescriptive?

We have a section on the need for diversity in datasets, the fact that there are frequent calls for this, complex reasons for lack of diversity. Also thinking about the importance of data sharing, because this puts restrictions on things.

Then we have a critical section about many of the calls for diversity, there is a lack of specificity for these calls, abstract. Often it’s in terms of ancestral diversity and in the document we talk about the problems of framing it this way.

Equity is often the most important concept in defining these ends. This definition also emphasises that diversity is not all about representation. We probably need a new subsection of this based on Yann’s comment, about what the ends should look like.

Framework for researchers: what are that goals, what are the implications of choices. Thinking about the data lifecycle, some points may be specific to certain projects but we lay it out in a table.

Then we show a few examples of what it might look like to apply this framework. The first example is about avoiding false positives. In this project, the goal is to help the achievement of equity. This would be telling someone like gnomad to optimize the groups. About having the individuals represented in a set of phenotypes to represent groups across humanity.

HL: In the beginning I know we said we wouldn't provide a definition of diversity, but in this instance, it seems like there is a specific type of diversity.

AL: It’s more specific than that, by itself that doesn’t mean that much. Point 2 is trying to do that, for this use case, defining the types of diversity important. Need to finesse this example a bit.

I feel like gnomad things they have an use case they’re defining for which is interpretation of variants, but they need to be more specific.

ALu: Isn’t some of that already being addressed by ClinGen? Filtering frequency. The ACMG guidelines, we have specifications being designed by different curation expert panels, with the frequency in particular communities taken into account.

AL: Yes. The question is frequency in the population, but what population. How to define the denominator. This is making a point about how and why the denominator should be defined. Once you’ve defined those frequencies, is then define the thresholds that apply to those frequencies.

Second example is the clinical impact of differential predictive performance of polygenic scores. Often framed in terms of ancestral or continental categories. If our goals is to make progress towards predictive accuracy of PGS, understood as comparable proportion of total variance of the trait/outcome explained.

We close the paper on a bit of a copout, and I think we can beef up this section. Just a short paragraph and saying that across the projects in portfolios, the higher order goals such as justice and fairness should be the focus. Particular interest in ideas about concerns of not being prescriptive enough? How should we address it?

HR: I like it not being too prescriptive, don’t want things to be taken the wrong way. These documents should be recommendations. Looking at it more from a bioethical view.

SN: I agree about not being too prescriptive. A different take is to make sure it’s actionable, which I think it is. Focussing on actionability that prescriptiveness can ensure it has the desired impact while being flexible enough.

SC: I agree with the idea of a fine line of being too prescriptive. We want not just actionability, but that changes over time. I don’t know how much of a living document we want to make this.

AL: To the point on time, sometimes policy docs do get revisited, just need people to pick it up.

AL: Yann’s concern is that we’re not solving the problems unless we set some guidance on what goals are acceptable ones.  My thought is to add a subsection asking researchers to appeal to higher order principles, and appeal to community engagement. Just a bit more prescriptive as to what suitable goals are.