Report from the Genetic Testing Theme, from the 4th International Dog Health Workshop

Introduction

Participants reflected a broad segment of stakeholders in canine health, including veterinarians, genetic test providers (GTPs), and researchers, as well as dog owners and/or breeders, kennel/breed club representatives and canine health campaigners. The 4th IDHW was organized to immediately follow the 10th ICCFGG meeting in Bern.  This ensured broader representation of the research community at the IDHW and presentation of an IPFD poster at the Bern meeting.  These various efforts and a plenary talk by Claire Wade was instrumental in ensuring that challenges along the entire spectrum of test discovery, commercialization and application were considered (see Fig 1 below).

Priorities and challenges identified in the pre-meeting survey, plenary talks and previous discussions resulted in the following areas of emphasis to be addressed in the breakout sessions, under a broad title:

How do we know what tests to use? How can we trust the test results?

• Validation of tests: this term continues to be vaguely defined and differently so by different stakeholder groups.  But the is consensus that many tests in use have not been fully evaluated in (all) breeds for which they are offered.  Lack of a clear definition of or strategy for validation poses a significant challenge for recommendations, counseling and interpretation of findings and a source of confusion and misunderstanding for consumers.
•  Even in the absence of regulations, is there a need for better definition of laboratory "quality" and best practices - e.g. what makes a "good" laboratory, and what standards (formal or not) should be in place or aimed for in robust testing?
• The IPFD Harmonization of Genetic Testing initiative has made great steps forward and provides an effective platform.  What are needed improvements, future directions and ongoing challenges and what is needed for continued development and support of this important resource?
• Breed-Specific test information - e.g. recommendations or priorities for breeds, perhaps by country/population are needed.  This related to concerns about validation, but goes beyond just genetic testing to include the ‘big picture’ of conditions in breeds.
• Improved access to genetic testing advice and counseling. As in every previous IDHW discussion, these supports continue to lag behind uptake of tests.  Discussion of the Expert Panel resource being developed by IPFD should focus on its potential to address some of these challenges.
• Veterinary community engagement – in the era of increasing Direct to Consumer (DTC) testing how o veterinarians make sure there role is key and what tools do they need?
• As plans from the previous IDHW did not come to fruition, how can we move forward on Proficiency Testing as a measure of test performance quality across labs?
• As outlined in Claire Wade’s plenary, and discussions at the Bern 10th ICCFGG, challenges for researchers involved in discovery may be underlying other issues, e.g. lack of publication, limited validation, etc.  Addressing some of these concerns may support many of the challenges listed above.
• All discussions needed to reflect the fast evolving and changing landscape of genetics/genomics research and commercial genetic testing.

During the workshop and discussions, it was recognized that the specific concerns around genetic testing (GT) – meaning both in promoting (good, beneficial) use of GTs and also how to reduce risks of poor-quality testing (either because of the test, or the process of testing, or its application) – were strongly inter-related. Therefore, the actions moving forward will likely impact multiple specific GT concerns. A key action/project was interrogating the concept of “validation” – which pulled together many specific GT issues. It was decided that creating a model for addressing Validation in GT would be the best use of time for the workshop. This was effective in guiding discussion to identify specific actions/projects moving forward. This is summarized below:

Validation- IPFD/IDHW Validation Matrix

In starting a discussion around validation, it was apparent that there were different types of validation that were important to genetic testing – from validation of the test being correctly correlated with the breed/type(s) and clinical disease, to technical validation of test performance and results delivery, and impact on the wider dog populations. In meeting “optimal” validation, a test would be considered from many of the discussion points: is the correct gene and mutation(s) identified that is causative or indicates risk for the disease of interest? Is the research repeatable/make sense? Should/Is a test being offered publicly? Is the test being technically performed correctly? Is the test important for: individual dog, breeding plans, population strategies? Etc.

From the discussion the IPFD/IDHW Validation Matrix was created:

Each of the sections (Discovery, Peer review, etc.) were work-shopped to identify specific details and needs for supporting good validation within each section. For example: Discovery included:

• Peer-review publication: details of the mutation/variant in at least 1 breed; robust evidence associated with phene/clinical association; as referring to that initial breed/population only
• What is the benchmark for validation for a test we're willing to accept? What happens when the publication is questionable or in doubt?
• Expert Panel - could be used as a "pre" publication review possibility, as an alternative to peer-review for early test release? Time-limited non-disclosure agreement (NDA)?

Each section garnered a lot of discussion, and confirmed that many of the discussion topics were inter-related.

Expert Panels

The need for a system that would allow experts to weigh in on various GT concerns was raised in principle at the previous IDHW meetings. This was taken forward by IPFD, and the development of an Expert Panel (EP) structure (scope being funding dependent) is anticipated. IPFD’s Dogwellnet has the capability to host moderated discussion groups, including an anonymity provision. In brief, the expectation would be that a body of international experts could weigh in on various subjects, with their findings and comments summarized by an independent moderator. This could be as simple as yes/no polls or as complex as “expert papers” encapsulating complex commentary and discussion.

Within the workshop, the need for multiple Expert Panels with specific focus was identified, as well as some specific topic areas – including those that would relate back to the Validation system. Specific topic areas identified:

• EP to weigh in on individual tests, and their application on a breed-by-breed basis
• Clinical/Population validation EP
• Pre-publication “peer-review” to assess new DNA tests coming to market pre-formal publication

Aspects to make the EP successful and purposeful included:

• list of questions should be developed into an archive that is searchable and updateable; collate and catalogue questions and responses, as well as updating references etc. 
• Experts will be determined by need to address question(s), but could include geneticists, vets, breed experts, etc.
• Support for structure: i.e. anonymous private discussion and then public output

The potential uses for an EP are extensive, but should start with a mini-panel beta test.

Lab Quality – standards and technical proficiency

At the moment, there are 2 challenges to determining laboratory (performance) quality – standards of practice, and technical proficiency. It is recognized that the only near-future options for standards assessment is likely to be a self-assessment, and volunteer reporting. There was concern that GTPs may misrepresent their quality, which is a risk, but that a working group could provide a first template of what a standards process might be and how this might be presented on HGTD. Additionally, how to represent the meaning of standards to the public, as well as sharing this information, and any vetting that could be possible. Proficiency testing was addressed as an independent topic.

Proficiency Testing

Proficiency testing was a topic raised at previous meetings, with some development being taken forward by the Animal Health Trust, who unfortunately were unable to continue progression due to lack of resources. In the discussion section a working group agreed to undertake discussion with ISAG to see if they would have the capacity to take proficiency testing for DNA tests forward.

UPDATE: Since the 4th IDHW meeting, the working party has reported back that a system is being developed, with ISAG collaborating with Laboklin to move this forward.

NEXT STEPS: how to add to/incorporated into HGTD proficiency reporting and perhaps collaboration, prioritizing wish list/plan for particular DNA tests – this could also be linked to EP discussions.

HGTD

HGTD was recognized as a valuable resource. The current catalogues include searchable databases of international GTP’s and their “quality measures”, and Phenes, including a plethora of information on all currently available DNA tests by breed/type. The phenes information includes clinical information, genes and mutations, original research, application commentary, etc. Both databases have significant flexibility and the potential for expansion to include any future metrics/data – such as proficiency testing results, and many layers of Validation. Challenges for the HGTD were primarily participation from both GTPs and particularly researchers and research-associated test providers, and sustainability. Funding and resources for the project desired by stakeholders, and hosted by IPFD or HGTD (such as Validation, EP, etc.) are a limiting factor. Questions raised included possible sources of EP funding, how to reduce any participation challenges (either financial or other), how other stakeholders – Kennel Clubs, etc. could do more to support and promote HGTD, and if breed clubs, individuals, GTPs, academics etc. can do more to support and promote HGTD.

In summary, a number of specific actions and projects were identified to address the concerns in the rapidly changing landscape of genetic testing. These mainly focused on supporting a structure that would enable dog owners, breeders, and veterinary professionals to be able to identify good quality, robust genetic testing that is applied appropriately in their dog/breed for the maximum health benefit to the animals. All of these projects require the collaboration and cooperation across genetic test providers, researchers, veterinary scientists and owner/breeders, in order to be successful.