Validation Framework for Assay Processing Pipelines

Ellis Hughes

August 23, 2019

Ellis Hughes
  • Statistical Programmer
     Fred Hutch
  • R user for 5 years
  • Seattle UseR Organizer
  • Cascadia R Conf Committee
  • Enjoys R Package development
  • @ellis_hughes


Package Development & Validation Framework

Lessons Learned

Fred Hutch and SCHARP
Icon depicting blood stem cell transplantation and immunotherapy

Blood Stem Cell Transplantation
and Immunotherapy

Cancer Risk Factors, and Causes,
Prevention and Outcomes

Vaccine Development
and Virus-Associated Cancers

Molecular Underpinnings of Cancer

Tumor-Specific Translational Research

  • Established 1992

  • Worldwide-impact in the fight
    against cancer, HIV/AIDS and
    other infectious diseases.

Assays and Correlates of Protection
Cutting edge Research in HIV/AIDS prevention and Vaccine Development all over the world

Assays are under constant development

- Antigens could be added to the processing plan
- Exploratory endpoints
- Processing techniques

We need a validated pipeline that is rigid enough to provide form and consistency between studies, but flexible enough to handle potential changes

Package Development and Validation

Develop functions as steps in a work flow

Capture all documentation that is required in a validation

Automation tools to support code development

The roxygen2 package allows for self documentation and manual generation

Roxygen2 -RStudio (

Documenting of the validation process in a vignette:

  Capture the reason behind creating the package
  Goals that the package should achieve
  Capture the process level specifications
  Capture the function level specifications
Risk Assessment
  Capture the process level risks and mitigations in a table
  Capture the function level risks and mitigations in a table

Lessons Learned & Observations
Designing explicit specifications is important

Function naming schemes

Function argument formats

Package Dependencies



Easy to accidentally gloss over important functionality

  Record features explicitly required and double check often

Scope creep is inevitable

  • Unforeseen functions
  • Additional functionality

Co-Authors and Support

  • Anthony Williams
  • Jimmy Fulp
  • Bharathi Lakshminarayanan
  • Alicia Sato
  • Shannon Grant
  • Paul Stutzman
  • Kate Ostbye

Fred Hutch is Hiring!


Thank You