Product Development and Approval Process

All GA4GH standards, frameworks, and tools must pass through an approval process before being adopted as an official GA4GH product.

Download this page as a PDF

1. Introduction

At the GA4GH strategic planning meeting in Hinxton, UK, in May 2017, the community identified a need for a best practices guide to help steer the development and adoption of GA4GH products. This document establishes several best practices that, when adhered to, would allow products to be adopted into GA4GH’s ecosystem. It does not prescribe an in-depth and strict set of dos and don’ts. Each product comes with its own community and working practices. To enforce the same working practice over all products would be counterproductive and only increase the administrative burden of initiating new work or bringing external products into GA4GH.

2. Product life cycle

A product can exist in one of the following states:

  • proposed — GA4GH has been notified of the intention to develop this product;
  • submitted for approval — a product has been submitted for approval to GA4GH. It will be undergoing review by GA4GH committees outlined below. The product may possibly undergo refinements during this phase;
  • approved — a product is accepted into GA4GH;
  • retired — a product is no longer deemed suitable for GA4GH.

3. Product proposal

Products should be proposed and developed in response to the needs of the GA4GH Driver Projects or in response to a known need within the broader community for standards.

The initiation of new products is likely to occur when new Driver Projects are introduced to the Work Stream and work on underway products is completed. The need for a new product may also coincide with a documented community requirement. Initial prototyping and possible exploratory work may take place wherever those developing feel is appropriate.

The Work Stream Leads should remember that the aim will be to produce a small number of new specifications, with wide adoption, each year. Priorities should be reflected by requests for the desired “Product” from multiple sources, be they Driver Projects or others active in the Work Stream activities. The Driver Projects who have explicitly requested a product will be the source Driver Projects for that Product.

Suppose the Work Stream Leads would like to formally turn an initial prototype into a product to develop to completion. In that case, they can ask the Work Stream Manager to create a custom copy of the Product Proposal Template. This set of slides will then be used to inform the GA4GH Steering Committee of the new Product. This will:

  1. notify the other Technical Work Stream leads of the nature of the Product being developed and identify potential areas of overlap;
  2. notify GA4GH staff to prepare a copy of the Data Security Questionnaire for the proposed Product (the relevant Work Stream members should fill this in; once filled in, this can be passed on to the Data Security Work Stream, which can review it);
  3. allow the Regulatory and Ethics Work Stream to be notified;
  4. notify GA4GH staff of resource requirements;
  5. allow the Product to be incorporated into the GA4GH Roadmap.

There may be some amendments to the initial Product proposal. These could be in response to issues raised by the foundational Data Security Work Stream or by other Work Stream leads identifying a possible area of cooperation or scope enhancements.

4. Product approval process

The Product will be submitted for approval by arranging with the GA4GH staff team/relevant programme manager to make a copy of the GA4GH Product Approval Submission Form and then emailing GA4GH staff at info [at] ga4gh.org to notify them that the form has been completed.

The following bodies will review the submitted product:

  • the foundational Data Security Work Stream;
  • the foundational Regulatory and Ethics Foundational Work Stream (REWS);
  • a specially convened Product Review Committee (PRC).

The GA4GH staff team will notify the Standards Steering Committee that the product has been submitted.

A questionnaire will be provided featuring questions set by REWS. The GA4GH staff team will provide this questionnaire which should be filled in and passed as a link in the GA4GH Product Approval Submission Form.

The Product Review Committee will consist of three members, as detailed below, nominated by the submitter:

  • a Work Stream leader from one other technical Work Streams;
  • a member of a third, different technical Work Stream;
  • a representative of one of the source Driver Projects for the Product, who has been involved with the product development.

The choice of members will be approved by the Engineering Group, subject to any changes suggested by the GA4GH staff team. The review committee nominates a representative to communicate back to the GA4GH staff team (this is the technical Work Stream leader should no nominations be made). The review committee may give a response of “Accept,” “Reject,” or “Changes Requested.” All three members must agree unanimously for the committee to give a positive assessment. The committee should provide this response one month after the specification submission to be reviewed. PRC members must follow guidelines.

It is recommended that the relevant foundational Work Streams are contacted with any security, regulatory, or ethical concerns they may have during the development process. The reviews made at this stage are in addition to those performed during the proposed phase.

A positive assessment is confirmed when the GA4GH staff team receives confirmation from the representative of each reviewing body that the review body passes the Product. Requested upgrades will be communicated to the submitting Work Stream Managers if a review body does not pass the Product. An upgraded Product can be sent to the reviewers directly. This cycle can be repeated until the Product passes the review body requirements.

Once all three bodies have made positive assessments, the Product will be sent to the GA4GH Steering Committee for Approval. This must be done two weeks prior to the Steering Committee meeting at which it is to be assessed.

At the meeting itself, one Work Stream Lead will present the Product to the Steering Committee. The Work Stream lead in charge of the PRC may also be called upon to explain the PRC processes. If the Steering Committee votes to approve the Product, it will be deemed “Approved.” If the Steering Committee rejects the Product, it will indicate the reasons for the rejection and if the Product will require a complete resubmission through the approval process or if the issues are minor enough to allow the Product to be reconsidered in a single expedited review.

4.1 Versioning

Specifications should follow the semantic versioning pattern of MAJOR.MINOR.PATCH, e.g. 2.1.1, as detailed at https://semver.org/. This is to communicate the severity of changes in a product to downstream users. MAJOR changes are considered breaking, e.g. changing a URL or renaming a field in a schema. A significantly enhanced feature set may also qualify as a major change. MINOR changes are considered as changes in functionality but not breaking, e.g. adding a new field to a schema. PATCH changes should be reserved for changes that do not change the contracts built within a product, e.g. expanding the size of a field in a schema.

Exceptions may be made where the product intentionally fits into an alternative versioning scheme dictated by the products’ target community. In this case, a clear schema for the versioning must be available, a link to the versioning schema provided in the Product Approval Form, and, if possible, the specification itself.

4.2 Approving new versions

Minor and Patch updates to the Product may take place without the Product Review Committee being reconvened.

Approving a new major version requires a new product approval submission, again using the GA4GH Product Proposal Template. Once this process starts, the Product will be reviewed by the same bodies as for a first-time release.

4.3 Expedited review

In exceptional circumstances, an expedited review may take place for what should be a Major new version. The Standards Steering Committee will approve a Product directly. Consultation with the original PRC should take place for this to occur. The Standards Steering Committee must have no votes against the proposal for this approval to be granted.

4.4 Publish location

The Product should be available in a public configuration management system that tracks requests for changes, authorship, and history. Submitting the specification to GitHub for tracking suffices. The repository should have at least three people assigned by Work Stream Leads capable of resolving product change requests subscribed and watching. Any issue or pull request should expect a response within a short time frame. Each person should be a GA4GH contributor, who agrees to the GA4GH IP policy (once finalised), and GA4GH Code of Conduct.

If a GitHub repository is used as the publish location, the GitHub user ga4gh-vc should be added as an owner or manager of the repository in question. This user is managed by the GA4GH staff team and will help ensure long-term oversight of the repository in question. A GA4GH staff-managed user with similar rights should be set up if a different version control system is used.

4.5 GA4GH paladins

GA4GH paladins are a set of nominated people within the GA4GH community who ensure the smooth running of the Product according to the above best practices. Sets of paladins will be assigned a product/designation according to their background and knowledge. They will also be given commit ability to the product. This will only be used in extreme circumstances if product owners cannot be contacted. These people will be selected from the Engineering Committee members.

4.6 Review

Products will be subject to a review by the Standards Steering Committee after the period of one year, if required. Typically the Standards Steering Committee will review a specification after five years.

5. Endorsed specifications

GA4GH may endorse a specification developed by an external body if it deems it relevant. The criteria and process for this are in development.

6. Retirement process

Work Stream Leads wishing to retire a product from GA4GH can submit a request using the GA4GH Product Retirement Form detailing the reasons why. Once agreed, the product may be withdrawn. Any attribution or mention of GA4GH will be removed if appropriate. Products will be updated to point to their replacement if they have been superseded by a newer GA4GH Approved Product. GA4GH may choose to fork a product if the GA4GH staff team deem a need to.

7. Existing standards

The following GA4GH specifications existed before these Approval Procedures were implemented, so they may deviate from the requirements in this section:

  • CRAM file format;
  • SAM/BAM file format;
  • VCF/BCF file format;
  • Genomics API;
  • htsget.

8. Specification best practice recommendations

Below are the recommendations made by GA4GH to ensure a robust specification is developed.

8.1 Document structure

Each specification should be laid out in a document detailing the following:

  • clear author list of the document (may be within the document or handled by another mechanism);
  • date of publication and a version;
  • executive summary summarising the major points of the document (if applicable);
  • introduction setting the document’s scope;
  • list of standards incorporated;
  • detailed layout of the specification including schemas, APIs, data formats employed;
  • description of any domain-specific terms within the document;
  • brief version history for documents that have received substantial updates.

8.2 Document format

The document should be presented in a human-readable format without the need to install a paid for third-party application or viewing outside of the browser. HTML output and PDF are acceptable formats so long as the source markup format used to generate the document is available e.g. Markdown, RST, HTML, LaTeX. In the case of Markdown and RST, GitHub provides automatic HTML generation features. We recommend using GitHub flavored Markdown (https://github.github.com/gfm/) and limiting the use of dynamic code within documentation.

8.3 Standard endpoints

GA4GH API specifications should use the following common end-points to convey information:

/service-info
The precise nature is to be determined

/error
The precise nature is to be determined

8.4 Implementations

Each specification should have at least two associated implementations. These do not need to be officially maintained reference implementations. For a client-server model, there should be at least two servers implementing successfully with two clients. These are viewed as best efforts and viewed as a way to ensure a robust product is developed.

These implementations are not expected to be written or managed by Work Streams themselves. They are to be developed by the Driver Projects and the community. Ideally they would operate in the real-world environment, with real-world data, in which the developed standards aim to be used. Work Streams would facilitate interoperability testing between these implementations. This might result in updating specifications during the development process if the implementations highlight unforeseen circumstances.

8.5 Requirement level indications

GA4GH specifications should meet RFC 2119 on the use of keywords to indicate requirement levels.

8.6 User requirements

GA4GH specifications should meet the requirements of the source Driver Projects which requested them, or have had some feedback from the community in which it is intended to be used. RFC 7282 is recommended for development teams to consider to guide the decision making process.

8.7 Citable

The submission should include a plan for publication to a journal or similar entity to allow for a citable reference to the product. Journals used for publication must be open access and not behind a firewall. In addition, GitHub repositories can be registered with Zenodo to mint DOIs (https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content). This is not a requirement of GA4GH to do for all standards but is a way to generate a citable entity.

8.8 Interoperability

If the specification is required to interact with areas covered by other GA4GH standards, it must comply with those standards.

8.9 Security contact

To enable a long-term contact for security flaws, an email address of security-notification [at] ga4gh.org has been set up. This email will be monitored by GA4GH staff and security members to allow for an incoming response to be directed to appropriate parties. Please feel free to use this email address in your specifications or websites if required.