GA4GH Encrypted Format

Minutes and Actions 2022

These are the minutes for the Encrypted Format meetings, a subgroup of the GA4GH Large Scale Genomics Work Stream. For further information, please visit the GA4GH Work Stream page.

Table of Contents

Meeting Protocols

Agendas, Minutes and Actions

2022-10-18: Crypt4GH and Cloud Integration

2022-09-20: Website,Github & Connect meeting

2022-08-23: [Meeting Canceled]

2022-07-26: Test data and Cloud

2022-05-31: V1.1 next steps

2022-05-03:

2022-04-05:Landing Page

2022-03-08: Connect meeting & Data files

2022-02-08

2022-01-11: First meeting of 2022

2022-MM-DD: Template

Previous Minutes

Meeting Protocols

• Please note that by participating in meetings, attendees agree to adhere to the GA4GH Standards of Professional Conduct.
• Meetings may be recorded for note-taking purposes. Recordings will be deleted within three months of the meeting taking place.
• Dates should be specified in the international format yyyy-mm-dd

Agendas, Minutes and Actions

2022-12-13:

Chair: Robert Davies

Attendees “Name (Affiliation)”: Andrew Ponomarev, Andres Silva

Meeting Recording:

Meeting minutes:

Rmd will try to look into htsget implementations.

Andrew: Proof of concept for enclaves and attestation on AWS.

2022-10-18: Crypt4GH and Cloud Integration

Chair: Alexander Senf,

Attendees “Name (Affiliation)”: Frédéric Haziza (CRG-EGA),Andrew Ponomarev,Reggan Thomas

Meeting recording:

https://us02web.zoom.us/rec/play/2IBihaDDHiqJ1ymnaCx-XvBn4h1Cn-36YxW-jO2Wr30tPMZecVpo3r0yHHMWNsOJrlyNzJK6KKbDUKLM.tmnObbkrkTkQWwzI?autoplay=true&startTime=1666101497000

Meeting minutes:

https://docs.google.com/document/d/16HM-rmT739pqJuTJJJyLU2k6tAp6d6GxikN5sv_wQCs/edit#heading=h.4pork227dqas

https://miro.medium.com/max/1400/1*iMWdekorSXEC5hs8xJiiDw.png

Full meeting transcript can be found here - https://otter.ai/u/dQUEJ3pvbejbUz-ZvHzHBVwpoyg?tab=summary&utm_source=va_chat_link_1

2022-09-20: Website,Github & Connect meeting

Chair: Alexander Senf,

Attendees “Name (Affiliation)”: Rob Davies, Andreas Silva,Andreas Ponomarev, Frédéric Haziza (CRG/EGA)

(*) The deadline for GA4GH website content has been extended to September 25 for Crypt4GH

Meeting recording:https://us02web.zoom.us/rec/share/9HT7OHKm-qIZNaKJnSyfmfiSpHbRQjrh0skypSlgoaRtKpuN9i283D8WYXV6DxIk.CNdcu6TwdN5Qyjet

Meeting minutes :

Discussed about GA4GH new website content review

Discussed about Crypt4GH 1.1 issue and test data

Discussed about Github repository location

Discussed about the topics for the connect meeting and interaction with other workstreams

2022-08-23: [Meeting Canceled]

Chair: Alexander Senf,

Attendees “Name (Affiliation)”:

Meeting recording:

Meeting minutes :

2022-07-26: Test data and Cloud

Chair: Alexander Senf

Attendees “Name (Affiliation)”: Rob Davies (Sanger),Gerrit Botha (H3 Africa),Andrew Ponomarev,Frédéric Haziza (EGA-CRG),Reggan Thomas (GA4GH)

Meeting recording : https://us02web.zoom.us/rec/share/uUZBZuJtDm_JF_iWZMBKFBTrDeB6ZG6Zp49lFJmNfTvSYc_pfR9jYWXSAyicuE18.BXvHNrks7SmghYiB?startTime=1658843830000

Meeting minutes :

RD - to Gerrit - Looking for test data. How is it going?

GB: Just to test the different kind of implementations but eventually settled on the Python implementation too

GB - Yet to do the testing. EGA doesn’t support the Crypt4GH files yet.The team then hoped to do all our encryption of our data sets and store it on disk, and it's been encrypted with a public key of Egi. But it seems to not be the case. So that's something we can follow up with.

RD - CRG is hosting all EGA websites

FH - The plan is Crypto GH for submissions to the EGA.

FH - our plan is very simple. You send unencrypted data over an encrypted channel. So HTTPS, SSH  when it arrives on our server, we encrypt so it's worrying. So chunk by chunk arrives, we encrypted. And then we see the received name somewhere in a database so that you can show it in a website and the website continues with the submission with the data metadata submission. So when it lands on this, it is critical data encrypted.

The advantage is that if we want to change the key anytime we want, we can say something the user doesn't have, like the public key of the EGA and  we could also change the algorithm if we wanted to.

FH - if you have your data already in crypto CGH, on your site, then I just dump it on disk

RD - people could give you a Crypt4GH file with a public key and can decrypt it.

FH - just built a fuse layer that gets started automatically when you try to log in. With NSS and Pam, you can do that. So on login, boot diffuse if it was not booted, a bit of problem with ordering things, but then the fuse keeps running and behaves like a server so that one makes a connection to a database instead of each SFTP connection making a database connection. So it's a little bit like a file server. That's what fuses and you dumped a file in there. It does the right thing. Even if the file is already encrypted, just copies. So we did that. And distribution is reversed. You have your login, you mount the fuse, and you can log in with as per you could well as per doesn't have a login that's better. Uses SSH for logging but asking us the SSH handshake is done. It switches to the UDP one and keeps going. Well it keeps coming from disk and the disk is a fuse layer that does the reverse it talks to it takes the payload talks to the database to say so only that one could talk to the database, not every connection. And when that one says can you give me the header re-encrypted header for this particular user? We re-encrypt the headers and the headers and the payload we did well.

RD - How soon will it be available?

FH - we will be ready for prod but other people have to look at it. So that they say okay, go ahead. But this is the plan. And it's fully leveraging GA4GH everywhere because we get the data we encrypt. We put the payload somewhere in the header somewhere and we split the header. The header goes into a database, the payload goes on this and normally that database doesn't have access to those discs. So that header and payload are far away from each other. And you have permissions to access some files. And if you have permission, then we re-encrypt the header and give you the header. That's the only thing we re-encrypt. We dump things on disk and we read things from this scan. We don't touch anything else apart from the header

Question AS to  - Andrew -  when you access crypto GH files in cloud environments, do you have to create your own versions of services?

AP - Short answer is no.

2022-06-28:  V 1.1 & Test data

Chair: Alexander Senf

Attendees “Name (Affiliation)”: Rob Davies (Sanger), Andrew Ponomarev (Genex), Reggan Thomas (GA4GH)

Meeting recordings:https://us02web.zoom.us/rec/share/fSt-SLF_8hgu2y5KFdI-wRuFQ96W2dECE7mLfG7NBksEv3Mva33bG-RVdyC5Dj6L.UmLa3TWfIW1bAtKx?startTime=1656424764000

Meeting minutes:

AS : Discussed about implementing Crypt4gh 1.1

Added an issue on crypt4gh. And the idea here is that I had at least we can write down what it is that we want to do. And then we can work it from there into the actual specifications.

Next steps : next step would be to write down what it is that we want to achieve. And then we can discuss it head on. And then we can actually start updating the specs based on that.

RD : In refget discussion,they actually write decision records as well, which is quite good.

AS : Decision log will be a great idea

AS : Test data - Garrett is still working on that. There is a test data repo.There is a test data location already.

Change of repo - we did talk about moving away from hts specs into a its own repository

Recommendation - to raise that as a question on the TASC repo and see what the community thinks about that.

If we could put the pros/cons that can lead to a wider discussion.

Cloud API - Integration - We have a demo in place.

talked about in the past is to add support for crypto GH into the cloud API's yet into the Wes trs for example.

AP : Genex is interested in in integration and cloud API

We have some data described in Drs.

And then we have a new enclave and during the Enclave, so Tom regenerate a new key pair. So we need to add them into his header. And then this data can be used in this enclave.

It will be a connection between TRS or Wes, DRS and crypt, it will be integration with the TRS because we need to know what would be used on specific DRS data. So we inquire if it's called at the station mechanism. So we need to ensure that a researcher requested for example, sem tools and SEM tools has this specific hash, and it will be it stated using this hash in the enclave and data in DRS will be like these, as I said these public key from a new enclave will be added in a header. So the enclave can read this data. So yeah, this is our use case.

AS : Is that user specific LAN or is it specific for just the Enclave?

AP : We really think about this combination. So it will be for example a stamp interface and a researcher A requests to use dataset B and on A to C and once he has approval, you replace header with a data set. And all data came to engrave and execute so it's combination user data set to an enclave

We startup enclave enclave generates a key pair and we need to use an enclave public key.

we need to establish this mechanism for a key exchange

AP : current implementation of DRS, it will be easy to do like a gap, replace header and put it back so it will be a new entity or I mean, didn't check it yet. How to easily replace a header?

RD : what would the TASC server be responsible for?  setting up the enclave and doing all the attestation stuff?

It's it's all done on an enclave side

So the test or Wes real just submitted tasks. So it's Trusted Execution Environment is just on our back end for a test

Who's responsible for supplying the code that runs in the Enclave?

presumably the supplier of the data would want to be the one that trusts the code running in the Enclave

RD : presume you you basically have pre packaged jobs people can run

AP : That's why we plan to use trs

 By containers or docstore

AS: Need to bring it up in next connect meeting

2022-05-31: V1.1 next steps

Chair: Alexander Senf

Attendees “Name (Affiliation)”: Frédéric Haziza (EGA-CRG), Rob Davies, Vaso, Reggan Thomas

Meeting recordings:https://us02web.zoom.us/rec/share/l_od0PWl97c7KbByCMFwjaymDc-MSC-DdscjS7bHgiJ1EtGO8Dxl6Gy3FlaKb4Im.6gBiTdZgAwcjDCMx?startTime=1654005400000

Meeting minutes:

Fred - https://github.com/EGA-archive/crypt4gh/tree/master/tests

Fred - https://crypt4gh.readthedocs.io

Fred - Current standard running each flow through an encrypted file but does not really when the blocks are in the correct order, or even missing. Or even extra added for each escape using the HDS it doesn't prevent unused block from being included

If person wants to check the center integrity of the entire file would have a checksum system that we don't we don't,

RD : If we can check that the file is in the right order and no bits in the middle are missing and no bits around the pockets are missing. Then we have a very strong guarantee of consistency.

FH: DGA at the moment is either people download the entire file anyway. But they there is also the other case where they just jump in the middle of the file, read something

FH: An internal checksum for us to for data curation in case we move with white or in case we recruit a new key or something Yes

2022-05-03:

Chair: Alexander Senf,

Attendees “Name (Affiliation)”: Rob Davies(Sanger), Vaso (Stanford univ)

Can’t make it: Fred (fever/sick)

Meeting recordings:https://us02web.zoom.us/rec/share/OZjlIlY3ZvuNslz1JG-vVcx5v19twKyDD98caxYVSE0lFHN01RwitI3pAnHKPSY1.aGwPamaHyjvwGBox?startTime=1651586421000

Meeting minutes: No particular agenda items were discussed since Alex and most of the folks were not there for this meeting.

2022-04-05:Landing Page

Chair: Alexander Senf,

Attendees “Name (Affiliation)”: Fredéric Haziza (EGA-CRG), Rob Davies (sanger), Vaso

Meeting recordings:

https://us02web.zoom.us/rec/share/9Fk6L3-txxrmCI6VgyOxcANNcypyUOuDg6aAmSW7AIBc4iFaZEEFF07YDW_X1X8X.ca3Z9XLJeZcM0qqZ?startTime=1649166706000

Meeting minutes:

FH

Encrypted version 2 in the header file defining new packet types.

The encryption method is only the encryption method of the header. So to retrieve the session key, you use the encryption method. The packet was the payload, the payload encryption is fixed.

RD

At the moment, it doesn't use any ID, however, we can introduce a new encrypted data sanction.Section 341 introduces the current method we use. You could then have another sanction, which introduces this new method.

RD

I don't think you need v2 in the header. As long as your code is correctly, Jason checking all the packet types the result is you don't decrypt the file with the old code. Yeah. But if a newer one is made,find using the original encryption method, when you run the old code to be able to read it.

FH

if I put v2, I would say that my new code just understands v2. It doesn't even understand v1.

RD

I'm kind of expecting you to understand both. You may want to say, you know, your implementation says I, I don't want to read this old format, which is fine

RD

That's,the difference between the specification and the implementation. You're what you're allowed to implement, which bits you do and don't want.

2022-03-08: Connect meeting & Data files

Chair: Rob Davies

Attendees “Name (Affiliation)”: Reggan Thomas (GA4GH), Frédéric Haziza (EGA-CRG)

Meeting minutes - https://us02web.zoom.us/rec/share/E9uJLSZsvDcvNZJDXy3TESSvSLQFABnHgcwLOVlhoy4dVnUDmk7IbbrhjLSl6WPa.epjFIqd0O7k5FQ3U?startTime=1646751562000

2022-02-08

Chair: Alexander Senf

Attendees “Name (Affiliation)” : Frédéric Haziza (EGA-CRG), Rob Davies (Sanger), Gerrit Botha (H3Africa), Reggan Thomas (GA4GH)

Meeting recording -https://us02web.zoom.us/rec/share/GRbM9HP1KVri4vI30-2sNqR8h2vv1sNRu4-cG2oTsn5-NwGIUiQ-vWeFams4-6yG.0Ft6hnOz3QoKY46Q?startTime=1644332382000

2022-01-11: First meeting of 2022

Chair: Alexander Senf

Attendees “Name (Affiliation)”:  Rob Davies, Fredric Haziza, Reggan Thomas

Meeting Notes:

Demo of EGA-QuickView (remote file system over SFTP, with Crypt4GH decrypting layer)

Meeting recording - https://us02web.zoom.us/rec/share/EIRWkuLcWS3gq6aRUMtY782gQ1fOiE-A0ASb8RfekFJoelRLw1wicL-End5GMJkK.CKbscDMYcYhHrsRA?startTime=1641913092000

2022-MM-DD: Template

Chair: Alexander Senf, Rob Davies

Attendees “Name (Affiliation)”:

Previous Minutes

2021 Meeting Minutes

2020 Meeting Minutes

2019 Meeting Minutes

2018 Meeting Minutes