Driving Innovation in Genomic Data Science through Ethical and Efficient Data Access

REWS/DURI Joint Workshop

Date: September 21, 2022

Time: 13:00 - 15:00 CEST

Meeting Chair(s): Vasiliki Rahimzadeh, Jonathan Lawson

Objective: The DURI and REWS Work Streams will be hosting a joint workshop on emerging opportunities and challenges of genomic data access. The workshop will kick off with a Research presentation reporting results from an NIH-funded global survey of DAC members affiliated with public and private genomic data repositories regarding scope of committee work, procedural standards and implementation factors mediating adoption of automated decision support tools for quality and effective DAC review. This will be followed by a facilitated discussion on the relevance of survey findings and its implications for the broadening roles and responsibilities of DACs and a panel discussion to inform future tool development, standards/guidance as well as challenges and opportunities facing DACs.

Zoom Information:

Please contact lindsay.smith@ga4gh.org for details.

Relevant Material:

Slides

Meeting Recording

Minutes:

Welcome to DACReS—1 PM

VR: Developers of the Du and DUOS. talking primarily about data access committees (DAC) as stewards of genomic health data at publicly funded repositories

3 primarily responsibilities - verifying, assessing requests (e.g. data use terms and consents), complying with regulatory oversight and laws

NIH bioethics supplement - studying DACs and their scope of work and how they are evolving as genomics develops globally

2 aims;

Creating a policy on DACs

Interested in exploring automatable elements of data access review via survey and interviews)

Representative worldwide (not Antarctica!)

Main deliverables: DACRES policy with guiding principles and procedural standards, similar to the one developed on ethics review. For new and existing DACs.

Within the policy we have terms of reference, progress reporting, incident management and living and learning institutions, for example

Beginning to publish work: (1) proof of concept study and (2) automated ethics governance

Call to action: if other people are interested that you know of, we welcome anyone to join us!

DAC Review and Meeting the Genomic Data Demand—1:15 PM

JL: QR link for info about our work and our data

Results of interviewing DACRes members over last few years

Were trying to figure out problems of data access by interviewing and surveying DACARres members globally (what is the problem, what is the current state of data access review, what can we do to solve it)

What we learnt: the problem is there is a growing number of genomic datasets and interested researchers creating greater number of data access requests, placed on manual process with consent forms across different institutions

Problems of efficiency and scale, inability to scale will call problems limiting scientific discovery

What are we trying to do specifically is pilot DACRES toolkit and with implementation see how to more towards automated workflows

Results [see slides for more data]:

• Demographics, how they work (operations and workflows), thoughts on proposed potential solutions

Involvement aligns with GA4GH involvement (US, European, Australian etc)

Near 50% gender representation in survey,

What type of institutions are involved? Mostly non-profit or academic, not many for profit entities

Were surprised that data types were genomes, exomes, human samples

How many datasets does DAC manage - some lower, some over 1000

How do DACs work?

• Overall prettys standardized: most have standard operating procedure, very few had nothing to this effect - there is structure that lends well to conversions about potential facilitated structures

Why did DACs develop SOPS

• Consistency, efficiency, transparency, among others [see slide].

70% of respondents not formally designed with DACs work in their job description - individuals likely in place where space to establish policies but individuals not formally established to be in these roles

SD: at Broad have people on it as not part of their job to ensure people not reporting to her institutionally so no pressure to vote based on who they report to - want people to operate independently

JL: Would the DAC chair be formal and members not?

Another example is that we have heard of people who want to deposit data in EGA and find themselves becoming a DAC

Elettra: What is the average size of DAC surveyed?

JL: Cannot recall - we can revisit.

AB: I've seen creators of DAC terms of reference carving our roles, minimum number of members, wanting people outside of institution, did this come out in your results?

JL: The formality of the roles we asked about level of education ut non-intentionality in structuring of DAC in my experience quite hard since not job description//paid role more committee of the willing that institutions will accept so not a lot of room for that I think

YJ:  A little of both, has to do with scope of DAC or project, larger projects more formal structures with designed roles, smaller projects often PIs of project will contact acquaintances who will be right (for good reason or not), depends on size of project

JL: When depending on project are you thinking of specific time and scope project

YJ: When you put data out there not supposed to be limited amount of time unless you want to put in another repository after certain time

What I've seen is sometimes burst of interest for data in first 4-5 yrs of project, then demands for data go down and DAC becomes semi-dormant

JL: What we've seen that in our experience, also the question of long term storage and costs associated with access

Other question, as GA4GH standard ore success ful and data more available for methods development data that would otherwise be dormant may be popular again and if DACs dissipated hard to determine what will happen

Back to results:

For those who said yes formally a DAC member

Or 1 person full time DACs member, very small thing for people to be full time

Who participates in reviewing Data Access Requests (DAR)s - a lot of DACs only use subset of committee at any given time - may be a small committee

How does DAC make a final decision on a DAR - a lot of majority votes (simple), 41% use unanimous consent, less use chair with ultimate authority

(audience member)I: what happens when there is no unanimous consensus?

YJ: It depends on the DAC - I’ve seen the chair has a lot of power if disagreement, often the chair will take a decision, or sometimes ask for more information from person supplying

ED: having a subset of DAC members do review often have to do with how (copy from zoom chat)

JL: If split people will ask for more info or try to assuage one another to majority position

Or if it is mixed and the chair closes it - often people read DARs differently

On volume of DARs there were less being received by DACs than expected, overall most receive less than 10 requests per month, could be because large number of DACs small # datasets

Who are DARs coming from, more large gov data holders handling large number of DARs

More DACs reviewing less than 20 DARs a month

Say increase in researchers and datasets increasing requests, saw matching increasing, but did not see DACs reporting a decrease - have seen more stability in demand than would have guessed

DAR review time: 58% minimum of 2 weeks - if you were a researcher applying would be slowed down by the slowest DAC. 27% take at least a month.

SD: Do you think the bar is low because people are used to waiting a month (at least) for REB review? We should aspire to better but expectations might be low

SB: It depends how you do DA process - NIH counts after you apply but you've already signed your side of the reply, Sanger does differently - depends on where the start is, take this with a grain of salt

We don't allow negotiation, the signing happens after the whole application has been reviewed, NIH you sign then submit

JL: We looked at this in a paper and having the signing process prior still added a day, can be quite variable

SD: Opaque to the person asking for the data (in US you do not know where the data is sitting, however, can be less than 24 hours turnaround)

AB: Not only can contracting signing be long, also can have PIs or research staff who won't provide enough information for the DAC to perform a review - would be interesting to get granular information on which stages of process take time and how to mitigate that, streamline process (e.g. standardizing Data Access Agreements etc.)

JL: We don't have those details on broken down aspect of project, DACs do it differently

You can turn around a DAR really quickly by saying you didn't give us enough information - number will look good on turn around but aren't helping researchers get information

YJ: Most of people on DAC often doing for no salary or included in other work, if look at private IRB, can get fast assessment compared to institutional ones - would it change with monetary figure - not saying encouraging this, but is a factor

JL: I think there is a correlation

SD: It is the prioritization

JL: Causes of delays - missing or incomplete information - spending effort reviewing DAR to realize don’t have the information, verifying legitimacy of a researchers and their institution thinking of DURIs, passports and visa play great role here

Different interpretations of data sharing term

1-4 have direct tools addressing these

Audience member (indonesia) : How do they check legitimacy?

JL: hardest part is verifying legitimate institution and signing authority is really that person, then can trust that’s who the researcher is

Things like a federated identity standard like the passports meant to solve this

Anna [from zoom chat]: Any further comments on ambiguities in data uses?

JL: We didn't dive deep into data uses - we did a through review at the board i.e. “diseases such as smoking” which is not a disease - things like these create problems

Could be other problems i.e. “diseases that affect the brain” - fuzzy categories most difficult

AB: From my experience - when have broad consent to research use will go on fishing trip for data or they are re-applying for a commercial use for data

Issues with identification

Where a researcher will have clear affiliation but not possible to determine if can hold PI role and need to go back and forth with institution to determine if hold PI role

Or when related to institution or can't sign for part of institution PI is from

Hold ups in institutions where lots of back and forth needed, or isn't on public facing documentation of institution online

Audience member (Francis): Do each of the DACs have all the roles, or do they have some of these roles (are they stronger in some areas over others)

JL: not sure that we aligned it that way but could follow up

AN: we at Sage haven't tied those things together i.e. identification does occur at same time as request - would be interesting to dive in further

SD: Do they all have these responsibilities, I know a lot of DACs don't delve into scientific feasibility

On institutional person have problem of people singing things they're not supposed to be signing, have added something about being the authority to sign the document

JL: Once you find the first person to trust as signing official things get a lot easier

Having passport makes things easier

Breakdown of how much time does the DAC actually look at if what researcher proposing matches the consent - wasn't the majority of the process in most cases, was a lot of verifying institutions, identities

Is problem DUO trying to solve most of DAC time, in this case no

Found that over 50% of DACs looked directly at consent forms

Proposed solutions (e.g automation, semi-automation, others?):

What are the main benefits of using software - efficient, consistent, compliant - what has echoed throughout

Interoperability from larger academic DACs

Are surveying GA4GH associated people a lot so may be bias you don't see in smaller DACs

Benefits for standardized format [see slide]

Potential challenges of software  - concerns of complexity of it, people doing in spare time may not be interest in learning burden of new software

Security concerns around accidentally approve something this way

Interest in system like DUO

Majority interest in a system that check compatibility in data use terms and displays to DAC to aid decision

SD: Do you know info on the level of interest if you have how many requests they're getting? If you were getting very few requests maybe less interest

JL: Could follow up

AS: What about under-resourced countries (not machine readable, limited resources and perhaps limited data)

JL: The fed gov required paperwork for legitimacy of institution to do business with federal government - would have to start with something like this

YJ: Will be a big challenge if we are hoping to implement something like that rapidly in LMIC - that kind of certification process needs to come from them, if imposed will create more difficulties than it will solve. We may need to accept were working in diff system than they are for a while

JL: Thinking of mutual trust model - one DAC approves, and another DAC comes along and acknowledges other DACs have worked with them

AB: Easy to implement DAR portal - clarifying that you want statement of institutional affiliation with some documentation - may not solve problem but alleviate the process - once a DAC clearly sees an institution and link, won’t necessarily go much further than that in certainty they require

JL: Some DACs have similar procedure where ask for formal letter

AN: Assumption being made that researcher have an ID that qualifies them, if we expand definition of who researchers are

Margo: Have you, or are you planning to, capture DAC decisions in machine actionable form - was the benefit for reduced ambiguity for using standards for machines

JL: DUO system does that capture in machine readable format, main benefit of that - advocate that aligning with DUO way to facilitate

VR: What is embedded in DACRES policy has been embedded in …(research committees??) Having a report system at all, there's no process for that writ large

JL: We didn't look at progress report piece, but review is one thing but research shifts, maybe changes and researchers forget what they originally said they would do

Interest in the software: efficiency, consistency, expedite process, help researchers access data

I think DACs very well aligned with us for making data sharing easy

Why not interested: may be less compliant if do this and may be slower

As a DAC would you let a software system decide on data access

• New to idea, good number would allow GRU (General research use) datasets to be handled this way, some not interested, some more interested in fully automated system
• Humans should be involved

What did we learn?

• Challenges - efficiency, consistency, fairness
• Challenges: problems of scale, resource constrained (humans are costly), variation (even with humans)

VR: This is first empirical data on DACs scope of work

Looking Back, Looking Forward to The Future Of DAC Work—1:45 PM

Q&A

Audience member (Elena): Did you ask or hear about DACs (within their scope) to look for ethical issues within a request, or research that might stigmatize a population. How much time do they see these?

VR: Not specifically in the survey. Within the working group before ehte survey - the extent to which DACs take up group harm ethical investigation (e.g. risks, benefits, benefit sharing, community). What is the division between IRBs and DACs. WHere does that responsibility lie is controversial, so we did not include

Audience member: Is there a way to categorize what the DACs focal point is? What is DAC's primary purpose? Scientific use case or ethical review or both? Plays into who and how will use particular product

SB: Issue with ethics review is challenging. Sanger is a lot more ethical, but we are not qualified as an ethical committee. We have standardized decision making framework, however, if they fall out with IRBs.

JL: When writing the standard, there was a split in opinion on whether DACs should look at ethical issues, beyond whether research goes outwith the research/data sharing requirements.

Everyone saying looking at science not their area.

Majority focused on DACs looking at proposed research within bounds of data use limitations. If anything ethically questionable, they would investigate but they would not seek it out

YJ: Odd - for me, historically, when we look at why DACs were created it was because of data re-identification issues. Primary role to protect privacy of participants. Privacy is both legal and ethical issue. Before we even look at term of use (conditions that perhaps should not be in there due to complex data access), primary role to verify that sharing the data would not make a threat to data privacy and security. Group stigmatisation and benefit sharing are different. Group stigmatisation links directly to data identification. If we can’t tell where data comes from (the group) this is not an issue. So this mean that there is good reason for DAC to look into it. Whereas, there may not be for benefit sharing (perhaps more for IRBs).

AB: DACs can be hands off on ethics side of things. Concern that they don’t want to step into the bioethical frameworks on use of information, and institutional norms. If a DAC is at an institutions (e.g. global north - formalist bioethics) this could prejudice applicants that are geographically located elsewhere if DAC is involved in ethics review. Where DACs have been more activist in group privacy is where potential for research to create inferences about population group that may have not been caught by IRB or ethics committee because comes from technical nature, rather than research outcome. Very open question - and not yet an answer in the literature

SD: If I am a research subject and I give informed consent ot general research use, who is DAC to say what my data can or cannot be used for. US institutions can be risk adverse, and pull back because they don't want to be in the media. However, if you write good consent forms, where people understand what is being done, then likely to have less problems. In the US IRBs put under HBU rather than GRU to avoid behavioral things related to genetics. If people give informed consent, then a committee should not override a decision that an individual makes in their consent.

JL: Participant selection of DUO the ultimate source of use.

YJ: Still need to be legally compliant. Can consent outwith the law which should not happen.

F: Role of DAC: sounds like right now DAC is static concept. May be an opportunity to make things more dynamic, dependent on use, profile of requester, type of use then we know that the role of the DAC could change depending on this context. If data is public and open then DAC is not even involved. Security may not need to be looked at when in secure storage for example.

JL: If GRU data needs less time, however, more time when DAC members have differing opinions.

Anna Lewis: with regards to all the proposals, puts even more on the consent form. Many issues with consent forms. What can we think about when it comes to stigmatising research?

Developing the DACReS Toolkit—2 PM (Panel)

AN: Our DAC at SAGE - manages a variety of data to registered, restricted tier data and controlled access. How we manage data influences how DAC operates, could be clickwrap for data use, for others will restrict access via all the methods discussed today, use our DAC so the review is as objective as possible, have standard procedure for work, is maintained by members of our team, written into their job description

JC: Ontario Brain institute manages and funds research in neuroscience research activities, we don't do the research ourselves, we have built an informatics platform that allows us to collect and share the data with researchers as part of our projects, have access agreements with all those institutions, and we operate on a consent model.

Divide roles very clearly - We dont let DAC get into ethics review, done separately, Admin work done within our organization by our admin team responsible for the platform, resulting in them making sure credentials of researchers elgit, does not come to DAC, narrow role of DAC to 2 things to verify info from the researcher consistent with analysis we've done to make sure being done consistent with consent and to give us advice on whether the science is unique or different from research work already being done with that data - recommendations to steering committee

DACs made up of researchers who had participated in our research, select members of the group to to individual reviewers have not done will DAC review of any request

As we become more consistent with consent language questioning role of DACs in future

SD: Chair the Broad’s DAC, DAC to different sets broad investigators have brought to us for safekeeping - NIH running out of storage so if you want to publish in “one namers” i.e. Nature has to make data publicly available, and has also been asked by numerous RD associations to hold their data. DAC of necessity on request of others.

Anyone who wouldn't be interested in DUOs surprises me - can't imagine job without it, streamline review, frees up time to do things that require brain power and people intervention - best thing about DUOs - 90% of this is not valuable use of time to be reviewing it - Broad DAC has found DUO system very useful, could be used on volume as well

Also on EGA DAC but last time they put in charge of DAC didn't know about consent forms so this is what you can get when you have small DACs - if you want ethical sharing use something like DUO system so can look at things that are important

VR: Is this DUOs moment to be implemented in DACs? Why or why not (sli.do) - what is the future of DAC work then - compliance body, only issues like privacy and reidentification? What is the scope of their work? Who consists? Are there membership expectations similar to IRB?

JC: As we evolve over time, change, have to figure out what role is missing in admin role, the ethics review for the IRBs etc that understand the ethics, and if you improve your consent processes and languages so clear what data can be used for you don't have a lot left for DAC to be engaged in - if you want them to look at certain thing shave to make sure has skill set to do that - may come from various places

SD: When we wrote the white paper for GA4GH and started thinking about DUOs was thinking ahead to offering researchers the datasets they can used, as opposed to system of going in and asking - these researchers shouldnt have to hunt through 1000s of datasets, should be able to provide cohorts at their disposable - wants DACs to be less needed

AN: One of the things that was highlighted is that data and availability of data is growing. With moonshot and another initiative will see data available at scale we can't maintain with individual decks - there are tools we can use to allow more data request - dont think DAC will go away completely, aspect of how research is done that will require human element - can provide due diligence around - research data sets tools for identification so complex so when we think of DAC and privacy still more work to be done

SD: Let's do that work rather than 15 requests for cancer data when consent has been given for any and all cancer research - lets do that and not this

SB: Strikes me that we keep talking about the DACs and everyone means something different - is what we're talking about governance and organizational level DACs some of decisions are not in power of the DACs - who makes the decisions - do we need to think more broadly than DACs

Audience member: agree with SD - Wasting time over nuances of broader research not the goal, our DAC is the decision making body (for Moonshot), we've gotten to point where GRU is flow through - transparency and accountability are required so we need to not just allow in case of misuse

Love idea of defining the principles of data access and who should be decision maker - saying does your institution or entity have people or processes in place to do Y X Z

JC: In our case is a governance framework established when platform built - put in place before any technical discussions so privacy and security top of the line - not necessarily about role of the committee but about process related to it

VR: Where do we go from here? Where can the DACReS GA4GH do with respect to developing these tools, merging governance entities, what tools do we prioritize developing moving forward, where are there gaps?

AN: From Sage’s perspective looking to implement DUO, are stuck on how to tie the ways we gate-keep data to those terms so can enable automation of that

Need to apply DUO to collections of data - tackling how to do that

Tackling the reporting piece - don’t have precedent for but so important for transparency and accountability

Tools of benefit - that allow people to take request to scale - having common consent language, aligning with DUO to enable controls on data

SD: I agree

DACs roles can be narrower but role needs to exist somewhere - if not at DAC then at IRB

Okay for people to do that differently - so many countries and cultures of doing this, have mind towards enabling the sharing

Audience member: As we don more research with international data - definition of DAC or neutral data stewards - how do you address that they want to be part of DAC

SD: Broad has approached that data is open source, we don't cherish the notion of being other people’s DACs, is thankless job - I don't believe EGA model for every PI have own DAC is a great one - is okay for very small datasets but other than that doesn't work well

But if are RD foundation and want to be their own DAC can use Broad tools

AN: Idea of privacy and as we group different datasets together so re-identification factor increases, if people have tools for figuring that out I would love to talk to you

Next steps: DACReS calls will begin again. If you think of useful tools please contact us at rews-coordinator@ga4gh.org or the DACReS leads.