GA4GH htsget

Minutes and Actions 2021& 2022 & 2023 & 2024 & 2025 &2026

These are the minutes for the htsget meetings, a subgroup of the GA4GH Large Scale Genomics Work Stream. For further information, please visit the GA4GH Work Stream page (will link to the page on the website when it is ready) at https://ga4gh.org.

Table of Contents

Meeting Protocols

List of Meetings

Minutes/Agendas

2025-03-12:

2025-02-11: Cancelled

2024-10-22: Cancelled

2024-09-25: Cancelled

2024-08-27: Cancelled

2024-07-31:Cancelled

2024-05-07:Cancelled

2024-04-10: Cancelled

2024-03-12: Cancelled

2024-02-14: Cancelled

2024-01-16: Cancelled (Due to unavailability of the leads)

2023-12-26:Cancelled due to holidays

2023-10-25:

2023-09-26: CANCELLED

2023-08-30: Canceled (due to the unavailability of the leads)

2023-08-01: CANCELED (Due to the unavailability of the leads)

2023-07-05: CANCELED (Due to unavailability of the leads)

2023-06-06: CANCELED (Due to unavailability of the leads)

2023-05-10: CANCELED (Due to unavailability of the leads)

2023-04-11: CANCELLED

2023-03-15: CANCELLED

2023-02-14:

2023-01-18: CANCELED (Due to unavailability of the leads)

2022-11-23: CANCELLED (Due to unavailability of the leads)

2022-10-26: CANCELLED (Due to unavailability of the leads)

2022-09-28: CANCELED (due to unavailability of the leads)

2022-08-30:

2022-08-03:

2022-07-05:CANCELLED

2022-06-08: CANCELLED

2022-05-10

2022-01-18

2021-10-27

2021-09-01

2021-07-07

2021-06-08

2021-05-12

2021-04-13: post-Connect

https://czi.zoom.us/j/91758414570

2021-02-16: First Meeting 2021

2021-MM-DD: Template

Meeting Protocols

• Please note that by participating in meetings, attendees agree to adhere to the GA4GH Standards of Professional Conduct.
• Meetings may be recorded for note-taking purposes. Recordings will be deleted within three months of the meeting taking place.
• Dates should be specified in the international format yyyy-mm-dd

List of Meetings

Also available:

Previous Minutes 2020

Previous Minutes 2019

Previous Meeting Minutes (2017-2018)

Minutes/Agendas

2026-10-20:

Chair: John Marshall

Attendees “Name (Affiliation)”: Rob Davies, Marki Malenic, Reggan Thomas

Apologies: Venkat Malladi

Meeting recording:  https://us02web.zoom.us/rec/share/C3hKJwYy68qna_0swGw7ga5-t5leQLN5fYNTHTki_zEbO_HtS6Ru0Qd4lXF1Bega.tTRpNP7Su7pUOS4h

Meeting minutes:

Notes

HTTP Query Method

• HTTP Query Adoption Expected Slowly (03:20)

• No evidence of active implementation or usage yet
• Slow uptake considered normal for low-level specification changes
• No change to direction; method viewed as reasonable to add now

• Post And Query Semantics Aligned (03:20)

• Query and post treated as equivalent request forms rather than compatibility hierarchy
• Shared URL structure confirmed for GET and POST requests
• POST examples need to show request body alongside URL

• Query Specification Update Planned (07:17)

• Add query support to specification through a POST/query request section
• Reuse shared parameter descriptions where practical
• Explain query terminology because readers may not know it

Specification Consistency

• VCFText Documentation Gap Identified (11:08)

• Query table updated without corresponding update to related POST information
• Most duplicated fields appear consistent; VCFText identified as known missing information
• Separate tables retained because URL parameters and JSON body fields use different syntax

• Documentation Duplication Requires Follow-Up (11:08)

• Potential approaches include shared Jekyll expansion, Perl-assisted generation, or explicit cross-references
• Pure Markdown does not provide an obvious single-source mechanism
• HTML comments could warn maintainers to keep paired tables synchronized

• Issue 875 Created For Review (27:40)

• Review missing query and POST information
• Itemize any additional gaps beyond VCFText
• Contributors may comment on issue or submit a pull request when time permits

Encryption Pull Request

• Encryption PR Update Started (19:37)

• Work began on pull request 808
• Branch requires rebasing against 1.3.1 because of VCF-related changes
• Existing PR contains valuable discussion that should be preserved if possible

• Branch Permissions Blocked Direct Push (19:37)

• Branch owned by Rob could not be force-pushed by current contributor
• Organization-owned fork settings did not expose expected edit access
• Closing and reopening PR considered undesirable because of discussion history

• Branch Transfer Workaround Selected (25:01)

• Updated work to be pushed to origin as a separate branch
• Rob to pull branch, fast-forward PR branch, and push it
• Update and PR comment to be completed before next meeting

Repository Follow-Up

• Windows Checkout Failure Identified (31:31)

• Marco unable to check out repository on Windows
• Git reports invalid paths across SAM test files
• Issue may relate to Windows/NTFS path handling rather than repository content

• Path Naming Cause Under Investigation (33:15)

• Possible causes include special characters, trailing periods, reserved names, or path patterns beginning with orcs
• Failure reproduced with standard Git, not only IDE
• Marco to determine exact offending paths and underlying cause

• Windows Contribution Blocker Escalated (37:44)

• Rob to consult James Bonfield about path behavior
• WSL or Linux remains available as workaround
• Removing unnecessary Windows checkout blockage considered worthwhile for potential contributors

• Next Meeting Tentatively Scheduled (27:40)

• Next meeting penciled in for 15th of December
• Meeting may be cancelled if pull requests progress offline
• Standing item on suppressed data regions remains without substantive progress due to limited time and lack of pressing work

Action items

• Rebase pull request 808 (19:37)
• Push rebased branch (25:01)
• Update pull request 808 (25:01)
• Investigate Windows checkout errors (37:44)

• Review issue 875 (27:40)

2026-08-25:

Chair: John Marshall

Attendees “Name (Affiliation)”:

Apologies: Reggan

Meeting recording:

Meeting minutes:

2026-06-30:

Chair: John Marshall

Attendees “Name (Affiliation)”: Marko, Reagan, Venkat

Apologies:

Meeting recording: https://us02web.zoom.us/rec/share/tInZy9QXmPw1YOLpffEz8TGvp01bHXYPiUjHaDSOeXUtOUQ9lM0llsCeertwh4gh.oXzSnzPH7pv7hNNR

Meeting minutes:

Summary:

The meeting focused on reviewing progress and updates for the HTSGET protocol working group, which restarted approximately a year ago after a period of inactivity. John presented a proposed update to the VCF text format specification to clarify that plain text VCF should be deprecated in favor of compressed formats, introducing a new "VCFText" keyword for plain text while transitioning toward compressed data as the default. The group discussed a second issue regarding data access permissions, where Marko proposed adding fields to specify allowed data regions, though Venkat and others raised concerns about client complexity and suggested alternative approaches like returning blank data for restricted regions. The team also reviewed ongoing work on updating the encryption scheme parameters, with additional suggestions to include public key information and better define how CryptoGH encryption works with HTSGET. Reggan provided an update about the PDAP version 2.0 process, noting terminology changes and the availability of feedback forms for contributors.

HTSGET Protocol Stability Review

John discussed the stability of the HTSGET protocol, noting that it hasn't seen major changes since its expansion from reads to variants around 5 years ago. He reviewed the original advantages of HTSGET over direct HTTP access, including database abstraction, parallelism capabilities, and server-side flexibility. John expressed that while HTSGET remains stable and functional, its value proposition may not be as significant as initially anticipated, given the existing HTTP access methods.

VCF Text Proposal Updates

The working group discussed updates to a VCF text proposal, focusing on clarifying the specification regarding compressed versus uncompressed VCF files. They decided to introduce a new "VCFText" keyword to unambiguously represent plain text, VCF files, while indicating that compressed VCF should be the default. The team also agreed to update the version number to 1.31 and include a deprecation notice for non-compressed VCF, with the change taking effect in version 1.5.

VCF Text Functionality Merge Decision

The team discussed merging new VCF text functionality and agreed to release it as version 1.3.1 rather than 1.4, treating it as a clarification rather than a significant change. Marko confirmed their internal HTSCITRS system already implements this behavior, supporting the decision to proceed with the merge. The team also briefly discussed the potential use of content transfer encoding with VCF, though they determined this would likely require clients to handle the encoding themselves and agreed to investigate this further as a lower priority item.

HTSGET Server Data Access Proposal

Marko presented a proposal for the HTSGET server to include fields specifying allowed and restricted data regions for users, but raised concerns about client complexity and potential performance issues due to varying access permissions. Venkat and Rob suggested alternative approaches, including using missing values or suppressing data entirely in the response, which would reduce server complexity and potentially be more practical for data producers. The group agreed that further discussion with the data security team and data producers would be necessary to determine the best implementation approach.

Data Censorship Implementation Discussion

The team discussed implementing data censorship capabilities, with Rob suggesting it would be easier to implement in HGSGAT than in DRS due to interposing capabilities. They explored adding an encryption scheme query parameter to allow clients to request encrypted data, with Marko confirming implementation feasibility and Rob suggesting the addition of a public key parameter for encryption. The group also noted the recent ratification of the HTTP query method as RFC 10008, which aligns with their current use of POST requests and could be formally supported. Reggan announced the availability of PDAP version 2.0 for review, with feedback collection ongoing through a feedback form.

Next steps:

1. John: Merge the VCF text proposal pull request after the meeting, updating the spec version to 1.3.1 and publishing the changes.

2. Marko: Discuss the proposed solution for handling suppressed data regions with the data security group and the DRS team to get their input.

3. John: Update the encryption scheme pull request to include the ability for the server to indicate it will only provide encrypted data, and to allow the client to specify a public key for encryption.

4. John: Create an issue or note to consider adopting the new HTTP QUERY method for HTSGET, as an alternative to POST for idempotent requests.

5. All attendees: Review the new PDAP 2.0 document and provide feedback via the available form.

2026-05-19:

Chair: John Marshall

Attendees “Name (Affiliation)”: Marko Malenic, Reggan Thomas, Rob Davies

Apologies:

Meeting recording:   https://us02web.zoom.us/rec/share/E3zgnRkn8ZBuUwGVaWy3FdwzOa8IG_N1R3JAnCQlk2YUE4y0_3Eu10PxA9gVODND.7rIDpD52_1DvnXvW

Meeting minutes: (AI Generated notes)

• VCF Compression Clarified: gzip compression standardized; bgzip allowed as subtype for indexing, backward compatibility maintained.
• Versioning and Deprecation: patch release 1.3.1 for clarification; gzip compression mandatory around versions 1.4 or 1.5.
• Sensitive Data Masking: masking sensitive genomic regions prioritized for privacy, clinical use; detailed implementation deferred.
• Encryption Support: optional query parameter planned for Crypt4GH encryption; explicit server errors for unsupported requests.
• Organizational Changes: HTS get moves to cloud workstream; potential headcount cuts and governance shifts anticipated.
• Meeting Cadence: meetings every two months to focus on small issues; next meeting June 30 under cloud group.
• Community Events: 2024 GA4GH plenary in Singapore Sept 28–Oct 2; registration closes Aug 14; travel costs impact attendance.

Notes

VCF Format Specification and Versioning

The team made clear progress on defining the meaning of "VCF" format, focusing on compression standards and versioning for clarity and future-proofing.

• Clarified VCF compression requirements with a pull request formalizing gzip compression as standard, with bgzip as a subtype (04:31)

• The specification now states that files should be gzip compressed, allowing bgzip-compressed files since bgzip is a valid gzip stream.
• This decision supports indexing and range requests but avoids forcing bgzip compression due to lack of index retrieval support for subsetted results.
• The clarification was motivated by practical use cases and minimizes unnecessary constraints on servers.
• This approach aligns with existing HTS get mechanisms and supports backward compatibility.

• Decided to manage the clarification as a patch release in version 1.3.1, with consideration for a minor version bump to 1.4 (08:09)

• Version 1.3.1 is viewed as appropriate since the change is a clarification without adding new features.
• Using version 1.4 was discussed as a possible alternative to gate behavior changes by version but was deemed less necessary.
• The team noted that real-world clients rarely differentiate behavior by version number, making the choice flexible.
• This versioning decision balances clarity with practical implementation ease.

• Planned deprecation of the older format to require mandatory gzip compression around versions 1.4 or 1.5 (11:35)

• Deprecation is expected one or two minor versions after the patch release, allowing time for adaptation.
• The timeline depends on the pace of other protocol changes and version increments.
• The team acknowledged uncertainty about reaching version 1.5 but agreed to keep it as a target for deprecation.
• This phased approach ensures backward compatibility and gradual adoption of stricter compression requirements.

• Confirmed naming conventions and textual clarity by endorsing “VCF text” for the human-readable format (11:35)

• The phrase clearly distinguishes text from binary VCF files, aiding user understanding.
• This naming supports documentation and implementation clarity without ambiguity.
• The choice reflects community familiarity and aligns with existing terminology.

HTS Get Protocol Enhancements and Masking Use Case

Discussions highlighted the need to further explore sensitive region masking and encryption scheme options for HTS get responses.

• Identified the masking of sensitive genomic regions as a key feature to develop, recognizing its clinical importance (17:22)

• The team agreed that sensitive regions should be masked in returned data to protect privacy in clinical settings.
• Specific implementation details need further refinement before progressing.
• This feature supports compliance with data protection requirements and improves utility for sensitive data handling.
• The team deferred detailed discussion to future meetings to allow for thorough consideration and homework completion.

• Explored adding an optional encryption scheme query parameter to support encrypted data requests, focusing on Crypt4GH (25:28)

• The parameter would allow clients to explicitly request encrypted data formats like Crypt4GH.
• Naming conventions for encryption schemes were discussed, favoring clear, spelled-out identifiers over abbreviations for clarity.
• This feature empowers clients to specify security preferences and enables servers to communicate supported encryption methods.
• The team considered that leaving the parameter off implies unencrypted data, simplifying client-server interaction.

• Debated server behavior and error handling when requested encryption is unsupported (29:01)

• It was proposed that servers return explicit error messages if they cannot support the requested encryption.
• This approach improves client clarity and error handling robustness.
• The team acknowledged that security concerns from revealing encryption support status are minimal since transport encryption is expected.
• The final error handling approach will balance clarity and security without overcomplicating the protocol.

• Discussed technical challenges of applying Crypt4GH to different file formats, noting BAM's complexity compared to CRAM (31:54)

• Crypt4GH integrates more naturally with CRAM due to its block structure, easing encryption boundaries.
• BAM requires tracking multiple positions within compressed blocks, increasing implementation complexity.
• Ongoing conversations about replacing BGZF compression with modern alternatives could influence future encryption handling.
• These technical insights guide prioritization and roadmap decisions for encryption support.

Governance and Organizational Changes Impacting htsget The team addressed ongoing structural changes within the broader organization and anticipated impacts on governance and collaboration.

• Learned that the LSG group is being reorganized with products like HTS get moving under a cloud workstream (19:24)

• Sasha announced splitting LSG products into different workstreams, with HTS get joining the cloud group alongside RNA get and others.
• This reorganization aims to streamline management and may lead to headcount reductions by eliminating a workstream lead.
• The governance document co-led by Brian Wenger and Regan is being developed to clarify product ownership and status.
• No immediate practical changes are expected, but the team will monitor governance evolution.

• Expressed cautious optimism that existing expertise and responsibilities will remain despite organizational shifts (21:33)

• Concern was voiced about new cloud personnel potentially taking over established file format committees.
• The team hopes continuity and default responsibility will preserve progress and domain knowledge.
• Future interactions with cloud teams may open collaboration opportunities or require adaptation of workflows.
• The impact on committee membership and meeting participation, especially regarding time zones, remains to be seen.

• Acknowledged the uncertain effects on working group dynamics and decision-making speed (21:33)

• The reorganization could bring new members with different priorities or cause delays as roles adjust.
• The team intends to maintain momentum on technical issues despite potential governance disruptions.
• Communication and coordination strategies may need refinement to accommodate new structures.
• Members agreed to continue meetings and projects as planned while adapting to changes.

Meeting Cadence and Progress Management

The group reflected on meeting scheduling, homework completion, and incremental progress on long-standing issues.

• Confirmed a flexible meeting schedule approximately every two months to allow focused progress on smaller issues (15:38)

• The approach follows a pattern of tackling individual problems with homework assigned between meetings.
• Recent meetings have seen limited attendance beyond the core group, emphasizing the need for efficient agendas.
• The team emphasized making small, steady improvements rather than large disruptive changes.
• This cadence balances workload with the pace of change in the mostly stable HTS get protocol.

• Highlighted successful completion of homework on clarifying VCF compression by Marco, with others catching up (04:31)

• Homework completion directly enabled formalizing the pull request and moving towards merging changes.
• The emphasis on preparation improves meeting effectiveness and decision quality.
• The team plans to assign and track homework items to maintain progress.
• This disciplined approach reduces meeting time spent on rehashing prior discussions.

• Scheduled the next meeting for June 30 under the new cloud group banner, signaling transition from LSG umbrella (40:34)

• The meeting will continue the work with updated governance and participation.
• A six-week timeline sets a clear deadline for outstanding homework and preparation.
• This timing supports maintaining momentum amid organizational changes.
• The transition to cloud meetings reflects broader strategic alignment.

• Acknowledged the last meeting under LSG, expressing readiness to adapt and continue under new organizational formats (40:34)

• The group recognized the end of an era with LSG and welcomed new structures.
• Members committed to ongoing collaboration despite shifting group identities.
• This mindset supports continuity and resilience in evolving governance landscapes.

Community Engagement and Upcoming Events

The team shared information on upcoming community events relevant to ongoing collaboration and engagement.

• Announced the 2024 GA4GH plenary event scheduled in Singapore from September 28 to October 2, with registration closing August 14 (36:51)

• The event includes a NIFT day, plenary sessions, and a connect event across several days.
• Members considered travel logistics and costs, with some leaning toward online participation.
• The plenary offers opportunities to present progress and engage with broader community initiatives.
• Awareness of the event supports planning for participation and contribution.

• Discussed external factors like global fuel supply and travel costs impacting in-person attendance decisions (38:44)

• Price volatility in fuel and transportation affects members' willingness to travel.
• Remote participation remains a practical alternative under current conditions.
• These considerations influence community presence and networking opportunities.
• The team remains flexible on attendance based on evolving circumstances.

• Shared informal reflections on travel history and challenges to contextualize current event planning (38:44)

• Members recounted historical travel times to highlight the convenience of modern options.
• This perspective fosters appreciation for easier access to global meetings today.
• The discussion underscored the value of in-person engagement despite logistical hurdles.
• Such cultural sharing strengthens team camaraderie.

• Encouraged awareness of registration deadlines to ensure timely participation decisions (36:51)

• Early registration supports event planning and potential travel arrangements.
• Highlighting deadlines helps motivate members to consider attendance seriously.
• This proactive communication aids both organizers and participants.
• The team aims to maintain active community ties through event involvement.

Action items

John

• Update the pull request clarifying VCF compression with version number bumps throughout the specification and circulate for team approval (23:41)
• Push additional commits to the encryption scheme pull request incorporating discussed refinements before next meeting (34:07)

Reggan Thomas

• Share draft governance document detailing HTS GET status and integration into cloud workstreams with the team (19:24)
• Ensure agenda and homework items, including masking sensitive regions and encryption scheme discussion, are tracked for next meeting (23:41)
• Coordinate follow-up on upcoming cloud meeting on June 30 and communicate related logistics to participants (40:34)

All Participants

• Review and provide feedback or approvals on the VCF compression pull request once updated (23:41)
• Continue preparations and homework for next meeting agenda, especially concerning masking of sensitive genomic data and encryption scheme implementation (25:28)

2026-05-19:

Chair: John Marshall

Attendees “Name (Affiliation)”: Marko Malenic, Reggan Thomas, Rob Davies

Apologies:

Meeting recording:

Meeting minutes:

2026-03-10:

Chair: John Marshall

Attendees “Name (Affiliation)”: Marko Malenic, Reggan Thomas

Apologies:

Meeting recording:

https://us02web.zoom.us/rec/share/O3bIGk_ndyJX41eb-wuKPobbTFiLPHnq4vJnYgFruZ0fmu-QtEe_MsAcDk2HC83T._wHD1jTfReae4_eh

Meeting minutes:

Htsget Protocol Status: Mature but low activity; minor cleanups and broader usage needed despite interest from the UK NHS.

Test Server Availability: GA4GH domain hosts a test server, providing essential infrastructure for developers to test implementations.

VCF Format Handling: Default to compressed output for VCF; plans to clarify with new keyword for uncompressed text requests.

Pull Request Management: Draft pull requests emerging; ongoing discussions on experimental features like masking and encryption.

Community Outreach: Plans for GA4GH events to showcase Htsget, aiming to raise awareness and encourage broader participation.

Notes

Htsget Protocol Status and Usage

The Htsget protocol is mature but currently has low activity, mainly focused on tidying up minor issues and encouraging broader use.

Current momentum and leadership status highlight a lull in activity partly due to the absence of the nominal leader, Mike Lynn, and the protocol having been largely stable for years.

The protocol is seen as simple and mostly "done," but minor cleanups remain that could be addressed in spare time.

Usage is mostly internal within curator groups, with some interest from external parties like the UK NHS, though often via ad hoc implementations.

The simplicity of Htsget allows quick partial implementations, supporting internal use cases without needing full server setups.

Test server availability under GA4GH domain provides an official endpoint for testing Htsget implementations.

The server is coordinated with GA4GH implementer groups and hosted via an Amazon account, ensuring stable test infrastructure.

This setup helps maintain protocol traction by enabling developers to test easily against a recommended endpoint.

Continued use and visibility of this test server are seen as critical to keeping the protocol relevant.

Internal proxy solutions for curator access to CRAM files help simplify access without fully relying on Htsget.

The current approach uses a proxy allowing direct indexed access to files stored in GCP buckets, bypassing protocol complexity.

This reflects a pragmatic choice favoring simplicity and authentication layers over full Htsget protocol implementation.

The protocol’s design permits such flexible adaptations, increasing its utility despite low external adoption.

VCF Format Handling and Specification Clarification

The group agreed on clarifying the VCF format handling to favor compressed (bgzip) output by default, with plans for a new keyword to explicitly request uncompressed text.

Current format query parameter behavior allows clients to specify "format=vcf" or "format=bcf," with default behaviors varying between implementations.

RGV and IGV clients explicitly request "VCF," and servers support returning either compressed or plain text formats.

The working group sees value in defaulting to compressed formats to align with typical server and client expectations.

Proposal to add explicit keywords like "VCF text" for uncompressed data aims to remove ambiguity and simplify client-server interactions. This approach would migrate "VCF" to mean compressed by default, with explicit keywords for the plain text variant.

The group prefers avoiding multiple confusing keywords and favors a simple, phased transition to compressed defaults.

This change represents a technically breaking change but is considered manageable as clients already handle both formats.

Drafting specification text and pull request preparation to clarify format handling and introduce the new keyword is underway.

The text will specify server behavior when clients request uncompressed or compressed formats, including error handling for unsupported requests.

The group discussed the difference between gzip and bgzip formats, favoring bgzip as the practical standard for indexing.

Contributors offered to help draft and review spec text promptly to maintain momentum on this issue.

Versioning considerations and deprecation were discussed as part of the transition strategy for format keywords.

Clients specify the request version, allowing backward compatibility and soft deprecation of older keywords.

The group recognized that deprecations still occur despite versioning, so clear spec language will guide implementers.

This approach balances technical progress with the need to support existing software ecosystems.

Process Improvements and Pull Request Management

The team is advancing by converting discussions into draft pull requests and clarifying open issues to maintain steady progress on protocol improvements.

Experimental features like masking and encryption remain open for discussion with multiple implementation options considered.

The Htsget IRS server includes an experimental clipping feature, with plans to document its behavior for transparency.

Assignments were made to add comments explaining current server implementations to inform future work.

These efforts aim to bring clarity and consensus on open protocol features through collaborative code reviews.

Leadership and contributor changes impact continuity but have not slowed progress.

Roman, a key contributor, has moved to Sweden and is no longer active on the project, with others picking up responsibility.

The team expressed appreciation for past contributions and adapted to new personnel dynamics.

This ensures ongoing stewardship and momentum despite staff changes.

Naming and specification refinement tasks are being actively addressed through planned pull requests.

Naming conventions for new keywords and features are being debated to ensure clarity and usability.

One participant volunteered to draft pull requests to formalize these proposals.

This reflects a practical approach focused on incremental improvements and shared ownership.

GA4GH Engagement and Community Outreach

Plans are in place to showcase Htsget and related groups at upcoming GA4GH events to raise awareness and encourage broader participation.

Showcase sessions at GA4GH Connect aim to present roadmaps, priorities, and engagement opportunities for each working group.

Htsget will be represented remotely due to travel constraints, providing visibility despite low group activity.

The core message for Htsget emphasizes its simplicity, maturity, and usefulness as a small, handy protocol, especially for internal use.

Organizers are compiling presentations to help newcomers understand each group's focus and how to get involved.

Community building remains a priority to attract more participants to Htsget meetings and activities.

Encouraging attendance and contributions is a common theme across GA4GH groups.

Remote participation options and direct outreach will help maintain connection with the wider community.

This outreach supports the long-term viability and relevance of Htsget within the GA4GH ecosystem.

Action items

Follow up on discussions regarding the possibility of using ga4gh.org domain names for official HTS get endpoints to possibly improve credibility (14:30)

Draft and prepare a pull request to clarify the VCF format behavior in the HTS get specification, including the introduction of 'VCF text' keyword and preference for compressed VCF as the default (27:30)

Add a comment to the GitHub issue describing the experimental masking feature currently in the HTS get IRS server implementation as an example of existing practice (39:30)

Coordinate with the GA4GH team about presenting or facilitating HTS get content during the Connect showcase sessions remotely (46:30)

Unassigned

Consider naming conventions for VCF format keywords and further discuss to finalize the terminology used in the specification updates (20:00)

Review and potentially update the specification text to handle versioning and deprecation strategies around format keywords and ensure compatibility (32:00)

Continue tidying up open issues, including masking and encryption related pull requests, documenting them better for future reference (37:00)

2026-01-14: Cancelled

Chair: John Marshall

Attendees “Name (Affiliation)”: ,

Apologies:

2025-11-18:

Chair: John Marshall

Attendees “Name (Affiliation)”: ,Rob Davies  ,Reggan Thomas

Apologies: Marko Malenic

Meeting recording: https://us02web.zoom.us/rec/share/IyIoyb2kwb2K5ioSRt3bDiB5II_GbosheVm4usfNsGvSznipfabo3fE4smxSoMnM.4xFFiKroijr6FVUB

Meeting minutes:

https://github.com/samtools/hts-specs/pull/808

Summary:

Rob Davies and John Marshall discussed the implementation of improved encryption for Crypto for GH, including potential quantum resistance. They reviewed the impact of daylight saving time on meeting times and the loss of notes from a previous meeting. John mentioned a pull request to add an encryption scheme query parameter. They also discussed the need for servers to handle encrypted data requests and the potential for clients to request specific data formats. Additionally, they touched on the challenges of using VCF and BCF formats, the potential for replacing zlib with Zstd, and the implications of moving to Cram 3.1. The conversation concluded with updates on the S3 cost-saving rewrite and the impact of a fire incident on data storage

Action Items

• Add a comment to the relevant issue about the expectation that the server should be responsible for deciding whether to return encrypted data.
• Clarify the expected behavior when requesting VCF format, whether that should return plain text VCF or compressed BCF.
• Provide more details in the issue about the use case for allowing servers to selectively mask out sensitive genomic regions.

Discussion on Crypto for GH and Encryption Improvements

• Rob Davies talks about a meeting earlier in the day about Crypto for GH and plans for version two with improved encryption.    

• Rob Davies mentions the possibility of making the encryption quantum resistant due to advancements in the rest of the world.    

• John Marshall and Rob Davies discuss the complexity of interoperating with HGS get and the feature of dropping out bits of data.    

• Rob Davies explains the reasoning behind adding a bit to the Crypto for GH layer to handle data blocks that don't end in the same place.  

Issue 808 and Query Parameters 

• John Marshall brings up issue 808, which involves clients requesting encrypted data and rejecting unencrypted data.    

• Rob Davies and John Marshall discuss the server's role in deciding whether to send encrypted data.    • John Marshall suggests adding a comment to the pull request to clarify the server's behavior.    

• Rob Davies mentions the idea of splitting out the header of the file and providing a URL for the rest of the data.  

Challenges with BAM and Cram Files 

• Rob Davies explains the complications of splitting BAM files and the need to redo bits of it.    

• John Marshall and Rob Davies discuss the importance of records beginning and ending at block boundaries.    

• John Marshall shares his screen to show a pull request related to query parameters for encryption schemes.    

• Rob Davies and John Marshall discuss the implications of the pull request and the need for word smithing.  

VCF and BCF File Formats

• John Marshall brings up the issue of whether VCF and BCF should return plain text or compressed data.    

• Rob Davies and John Marshall discuss the historical context and the lack of clear decisions in the past.    

• John Marshall suggests changing the default to return compressed data to avoid insanity.    

• Rob Davies mentions the potential for replacing zlib with Zstd for significant reductions in VCF file size.  

Deployment and Compatibility Issues

• John Marshall discusses the challenges of deploying new versions of the HTS JDK and Picard.    

• Rob Davies mentions the issue with EBI using an outdated version of the HTS JDK for file validation.    

• John Marshall notes the lack of activity in the HTS JDK team and the need for new approvers.    

• Rob Davies and John Marshall discuss the potential for causing chaos with the new release and the need for bug fixes.  

Clinical Data Masking and Sensitivity

• John Marshall explains the concept of masking clinically sensitive regions in data.    

• Rob Davies recalls an incident where a region was masked and later imputed, leading to incorrect conclusions.    

• John Marshall suggests adding a paragraph to the spec to mention the possibility of masking and informing clients.

• Rob Davies and John Marshall discuss the importance of clear communication to avoid misinterpretation of data.  

Bespoke URL Schemes and Indexing

• John Marshall proposes a bespoke URL scheme to signal the use of HTS get for specific requests.    

• Rob Davies discusses the challenges of making indexes work properly with HTS get.    

• John Marshall suggests using the index to figure out the regions needed and then reading the regions from the file.    

• Rob Davies mentions the potential for doing away with separate indexes and using binary searches in the data.  

S3 and Cloud Storage Challenges

• Rob Davies talks about rewriting the S3 component in HTS lib to improve retrying requests and avoid overcharging from Amazon.    

• John Marshall and Rob Davies discuss the importance of not using open-ended range requests on buckets.    

• Rob Davies mentions the need to solve granularity issues and the potential for similar improvements on GCP.    

• John Marshall shares his plans to improve the authentication story for reading local metadata servers.  

EBI Server and Reference Cache Incident

• Rob Davies discusses the decision to move away from the EBI server due to performance issues and self-DDoS.    

• John Marshall mentions the lack of practicality of using the EBI reference server for whole genome downloads.

• Rob Davies shares a story about a fire incident that destroyed several petabytes of hard drives and affected the ref cache data.    

• John Marshall and Rob Davies discuss the importance of keeping data on persistent servers to avoid such incidents.  

Meeting Conclusion and Next Steps

• John Marshall thanks everyone for attending and summarizes the key points discussed in the meeting.    

• Rob Davies mentions the need to inform Fred about the interest in using HTS get for storing and serving files.    

• John Marshall and Rob Davies discuss the importance of continuing to work on the issues and pull requests.    

• The meeting concludes with plans to reconvene in the new year to make further progress on the discussed topics.  

2025-09-23:

Chair: John Marshall

Attendees “Name (Affiliation)”: ,Marko  ,Reggan

Meeting recording: https://us02web.zoom.us/rec/share/XGef2TotPw25NrXLddG72kuagitjkrXJeYNcU6_BnEv97H5Z4cxwyDwlNpH43ASb.QPL71z6UKyL_DdW-

Meeting minutes:

Meeting minutes:

John Marshall and colleagues discussed the scheduling of their meetings, noting the inconvenience for participants across different time zones. They reviewed the implementation of encryption in data requests, agreeing that clients should signal their encryption preferences. They also debated the default format for VCF files, concluding that compressed VCF should be the default unless specified otherwise. Additionally, they discussed the need for clearer specifications in the HTS GET protocol regarding the return of subset data due to authorization constraints. John will draft PRs for these changes and investigate client behavior to ensure

Action Items

• Draft a PR to clarify that the default response for the variance endpoint should be compressed VCF.
• Investigate how existing clients like IGV are currently calling the variance endpoint and what format they expect.
• Reopen the discussion on Slack/mailing list about finding a more convenient meeting time for all participants.

Encryption Implementation and Client Requests

      • John Marshall and Marko discuss the implementation of encryption and the need for a parameter to distinguish between encrypted and non-encrypted files.    

• Marko explains the necessity of the parameter to allow clients to signal their preferences.    • John Marshall suggests that clients should interpret responses based on self-identifying information.    

• Marko agrees but notes the need for flexibility in distinguishing between encrypted and non-encrypted files.    • John Marshall and Marko discuss the implications of not specifying the encryption scheme and the default behavior of servers.  

HTTP Content Negotiation and Error Handling

              • John Marshall and Marko discuss the use of HTTP content negotiation headers and the need for a reserved value for plain text data.    

• Marko suggests using HTTP headers like Accept to signal the type of data being requested.    

• John Marshall questions the interplay between the ticket and the encryption scheme, and the need for error handling when servers only have plain text data.

    • Marko agrees that error messages should indicate whether the requested data is available in the specified format.    

• John Marshall and Marko discuss the potential security implications of disclosing information in error messages.  

Variance Endpoint and Data Compression

               • John Marshall and Marko discuss the variance endpoint and the need to specify whether VCF should mean plain text or compressed VCF.    

• Marko suggests changing the default to compressed VCF and ignoring uncompressed VCF requests.    

• John Marshall agrees but notes the potential impact on existing servers and clients.    

• Marko mentions the use of IGV for client requests and the need to ensure compatibility with compressed VCF.    

• John Marshall decides to investigate the current practice of clients and servers to determine the best course of action.  

Meeting Conclusion and Next Steps

               • John Marshall thanks everyone for their participation and notes the productive conversation.    

• Reggan asks about the poster for the plenary and John Marshall confirms he will edit it and send it to the comms team.    

• John Marshall mentions the next meeting is scheduled for late November and suggests discussing the issues offline before the next meeting.    

• Marko notes they might be traveling on the next meeting date and suggests adjusting the time if necessary.    

• John Marshall concludes the meeting, noting the importance of addressing the discussed issues and improving the meeting schedule.  

2025-07-29: Cancelled

Chair: John Marshall

Attendees “Name (Affiliation)”: ,  ,

2025-06-03: PR 808

Chair: John Marshall

Attendees “Name (Affiliation)”: Rob Davies, Roman Valls Guimera ,Reggan Thomas  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recordings:

Meeting minutes:

The meeting discussed the outdated SDKs used by Condor, specifically G lip c2 point 17, and proposed replacing troublesome macros with memcon. The team reviewed open issues in HDS get, including standardizing on scheme and addressing cross-referencing. Proposal 808 from Roman was the main focus, debating whether to add a parameter to request encrypted responses in HTS get. The team agreed to specify server options for handling encryption requests and to document the rationale for the proposed changes. The next meeting is tentatively scheduled for two months hence, considering time zone challenges.

Actions:

Provide a written explanation on the PR for why the proposed approach is better than the current implementation.

Clarify in the PR the server's options when the client requests encryption (e.g., return encrypted, return plain text, return error).

Cancel the UK-based meeting series and tentatively schedule the next HTS-GET meeting in 2 months at the current time.

Discussion on Python Bug Fix Release and Ancient SDKs

                          • John Marshall congratulates Rob on getting ACS libertail out the door and mentions racing to get a Python bug fix release out.  

      •Rob discusses the use of ancient SDKs by Condor, specifically G lip c2 point 17, and the challenges it poses.  

      • John Marshall and Speaker 1 discuss the trade-offs of different workarounds, including replacing the troublesome macro with memcon.  

      • John Marshall mentions the difficulty of obtaining old SDKs for Mac OS and the use of scurrilous GitHub repositories.

Condor's Licensing Policy and Sanger's Restrictions

•Rob mentions Condor's licensing policy and the blocking of their website from Sanger to prevent accidental downloads.  

      • John Marshall discusses the challenges of working with Condor and the need to pull up a Condor environment to grab the required SDK.  

      • John Marshall notes that Condor is not a flavor of the month due to their licensing policy.  

      • The conversation shifts to the first HDS meeting in quite some time, with John Marshall mentioning the absence of the main leader, Michael, and the outstanding issues and pull requests.

Review of Open Issues and Pull Requests in HtsGet

                          • John Marshall shares his screen to review the open issues and pull requests in HDS Get, noting that there are nine open issues.

        • John Marshall mentions the standardization of HDS Get on scheme and the potential deployment of cross-referencing.  

      • The main focus of the meeting is to discuss proposal number 808 from Roman, which is the closest to an active proposal.  

      • John Marshall suggests going through old issues and pull requests to check their status and decide whether they are stale or need further consideration.

Discussion on Issue 386: Field Parameter in Reads Endpoint

 • John Marshall brings up issue 386, which involves the field parameter in the reads endpoint and the need to specify how to handle it in HTTP and VCF.  

      • Rob suggests dropping all genotype fields to simplify the handling of the field parameter.  

      • John Marshall proposes reopening the issue and cross-referencing it to the samples issue for future consideration.  

      • The conversation touches on the possibility of specifying which samples to include in VCF and the implications for data handling.

Encryption Scheme in HTS Get and Proposal for Encrypt4GH

                          • Rob raises a question about the encryption scheme in HTS Get and the need to document the reasons for the proposed approach.  

      • John Marshall explains the fundamental nature of the web, where requests and responses are encrypted, and the benefits of specifying encrypted responses.  

      • The discussion includes the possibility of adding a field to the request URL to indicate the need for encrypted responses.  

      • Rob and John Marshall discuss the challenges of implementing encryption for different file formats like BAM and VCF.

Alternatives to crypt4GH and Meeting Conclusion

                          • John Marshall and Rob discuss the alternative of using HTTPS for transferring encrypted files and the potential inefficiencies.

        • The conversation explores the possibility of adding encryption information to the ticket or individual URLs in the ticket.  

      • John Marshall suggests specifying the server's options for handling encrypted requests and the client's options for dealing with unencrypted responses.  

      • The meeting concludes with John Marshall volunteering to make comments on the pull request and scheduling the next meeting for two months hence.

2025-04-08: Cancelled

Chair: John Marshall

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

2025-02-11: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-10-22: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-09-25: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-08-27: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-07-31:Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-05-07:Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-04-10: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording;

Meeting minutes;

2024-03-12: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-02-14: Cancelled

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2024-01-16: Cancelled (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-12-26:Cancelled due to holidays

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-10-25:

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-09-26: CANCELLED

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-08-30: Canceled (due to the unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-08-01: CANCELED (Due to the unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-07-05: CANCELED (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes:

2023-06-06: CANCELED (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2023-05-10: CANCELED (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2023-04-11: CANCELLED

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2023-03-15: CANCELLED

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2023-02-14:

Chair: Mike Lin

Attendees “Name (Affiliation)”: Rob Davies, Reggan Thomas, John Marshall, Roman Valls, Marko Malenic

Meeting recording:https://us02web.zoom.us/rec/share/4g8yyYZnCPwUtE5EtFgVdMCppoZpD6d1X294pz2GAgoTm5AMJcksfTDxVEYJ5A9e.YB-LAMD8jU5n8WvH 

Meeting minutes :

2023-01-18: CANCELED (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2022-11-23: CANCELLED (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2022-10-26: CANCELLED (Due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2022-09-28: CANCELED (due to unavailability of the leads)

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2022-08-30:

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:https://us02web.zoom.us/rec/share/BGkskCznj5ML866hmjiXVM90WFrXFfAUag5Ge0HKNPFAyogSIVoEHHHHALrr2P5i.ugUjuDxPW95VHrqy?startTime=1661893105000

Meeting minutes :

Discussed about htsget-re for UMCCR challenges

2022-08-03:

Chair: Mike Lin

Attendees “Name (Affiliation)”: Rob Davies (Sanger), David Jackson (Sanger), John Marshall (University of Glasgow), Andrew Ponomarev, Shaikh Rashid ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

https://us02web.zoom.us/rec/share/9RWzIl-UMYxfCUuH93fB5lGAaL1immVfkD2FofawwBxcJbf4z24GBU-ChRjqX0-5.SbRa22tCrniIsziq?startTime=1659542382000

Meeting minutes :

https://us02web.zoom.us/rec/share/1WTZu6VkP1Kjh9U0iL0yfMP43Eh6eYtzwU9ITz4x57HN092mdD4yYZNoV4jokX19.bQujsb-2jOWt8nCg?startTime=1659542382000

2022-07-05:CANCELLED

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recording:

Meeting minutes :

2022-06-08: CANCELLED

Chair: Mike Lin

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting recordings :

Meeting minutes :

2022-05-10

Chair: Mike Lin

Attendees “Name (Affiliation)”:John Marshall,Roman Valls,Shaikh Rashid,Reggan Thomas(GA4GH)

Meeting recordings:https://us02web.zoom.us/rec/share/sgkxBIQYM1Olseehy0-4svVJKQztt9pnJXXN-T7HUwgkTxPD2RRoE1S_eoLh3MFc.keTrwMOqZTyorql4?startTime=1652216243000

Meeting minutes :

2022-01-18

Chair: Mike Lin

Attendees “Name (Affiliation)”:

(Issues / PRs)

Meeting recording - https://us02web.zoom.us/rec/share/LzYCxqcbHs_XsCEM4JZCIkKhHv97JZcq3EpkU76gVD1WOo_fQwoEplXRaxnKNDPw.HK2ADalAXaVekXLh?startTime=1642542841000

2021-10-27

Chair: Mike Lin

Attendees “Name (Affiliation)”:

(Issues / PRs)

2021-09-01

Chair: Mike Lin

Attendees “Name (Affiliation)”: Rob Davies, John Marshall, David Jackson, Shaikh Rashid

(Issues / PRs)

2021-07-07

Chair: Mike Lin

Attendees “Name (Affiliation)”: Rob Davies, John Marshall, Shaikh Rashid, Neerjah Skantharajah

(Issues / PRs)

2021-06-08

Chair: Mike Lin

Attendees “Name (Affiliation)”: Rob Davies, Roman Valls Guimera, John Marshall, Neerjah Skantharajah

(Issues / PRs)

2021-05-12

Chair: Mike Lin

Attendees “Name (Affiliation)”:

(Issues / PRs)

2021-04-13: post-Connect

Chair: Mike Lin

Attendees “Name (Affiliation)”: Roman Valls Guimera (UMCCR), Rob Davies (Sanger), John Marshall (Glasgow), Shaikh Rashid

Meeting transcript

Meeting Transcript

https://czi.zoom.us/j/91758414570

(Issues / PRs)

2021-02-16: First Meeting 2021

Chair: Mike Lin

Attendees “Name (Affiliation)”: Roman Valls Guimera (UMCCR),  Rob Davies (Sanger),  David Jackson (Sanger), John Marshall (Glasgow), Shaikh Rashid (CanDIG - UHN),  ,  ,  ,  ,  , Jeremy Adams (GA4GH)

(Issues / PRs)

2021-MM-DD: Template

Chair:

Attendees “Name (Affiliation)”: ,  ,  ,  ,  ,  ,  ,  ,  ,  ,

Meeting transcript

(Issues / PRs)