ZifoZifo RnD Solutions

Close

Implementation of Instrument Raw File
Management Solution

Context Before Zifo's Involvement

Customer Overview

Our customer was one of the top global Bio-pharmaceutical companies researching and developing therapies in the areas of cancer, HIV/AIDS, cardiovascular disease, diabetes, hepatitis, rheumatoid arthritis and psychiatric disorders.

Problem Statement

Customer's Translational Medicine function operated in 5 different sites across the globe. Many of the labs supporting Translational Research were also situated at different locations and operating at varying levels of maturity. There were significant differences in the SOPs, policies, workflows, applications and instruments followed by every lab. This resulted in data (such as analytical data) critical to Translational Research getting fragmented across different storage systems limiting accessibility and increasing the risk in data quality. This prompted the Customer to develop a large file management solution built on enterprise IT data standards and using leading storage technologies that satisfied the below needs:

  • Ability to store all type of data
  • Ability to store large files up to 50 GB
  • Ability to secure data using IAM
  • 99.999999% durability
  • Adopt a platform-based approach for LIMS and raw file management solution
  • Flexible cost options

Moreover, the Customer was in the process of acquiring another large Bio-pharmaceutical company. This meant the existing systems and processes had to become structured and extensible to accommodate new laboratories and ways of working.

Objectives

Key value drivers to be delivered by a unified process for raw data management:

Increased collaboration

Improved data quality, availablity and accessibility

Cost reduction with applications retirement and licenses reduction

Zifo's Methodology

Zifo's Scientific Informatics Consultants performed a threefold analysis focusing on Process, Data and Technology to understand the current state challenges and define futurestate capabilities.

The analysis focused a number or critical factors related to Large file management, notably:

  • Data security
  • Accessibility and Scalability
  • Cost options
  • Existing infrastructure of Customer
  • Access and security restrictions of Customer
methodology venn diagram

Future-state solution design was driven based on the following foundational principles::

  • Solution should leverage key enterprise IT components (LIMS, sample management, project/requests management, lab instrument files and analysis data)
  • The solution should respond faster to changing landscape and environment
  • The solution should facilitate access and metadata management by leveraging well-defined governance processes and policies
  • The solution should provide comprehensive audit trails, versioning and traceability

Once the future-state design was approved by the Customer, Zifo Consultants developed a tailored solution for large file management leveraging Industry Best Practices and leading technology solutions.

Solution

A single solution to configure and manage lab workflows

A platform to manage raw files (metadata, file access and archiving)

A low-cost, multi-tier storage system for Raw Files generated during sample analysis

A tool to automatically copy the files physically to AWS in the background

Sapio Sciences Exemplar LIMS

iRODS as a middleware to integrate with Data Lake

Amazon S3

SmartSync

Highly and quickly configurable, works with both SQL and Oracle, is secure and flexible with respect to hosting

Comprehensive solution to policy-based data management across geographically separated users

Globally (and Customer's data policies recommended) means of storage and exchange. Also, Exemplar is hosted by Sapio on AWS

Full-featured solution to back up and synchronize data to AWS

Need

Proposed Solution

Reason

A single solution to configure and manage lab workflows

Sapio Sciences Exemplar LIMS

Highly and quickly configurable, works with both SQL and Oracle, is secure and flexible with respect to hosting

A platform to manage raw files (metadata, file access and archiving)

iRODS as a middleware to integrate with Data Lake

Comprehensive solution to policy-based data management across geographically separated users

A low-cost, multi-tier storage system for Raw Files generated during sample analysis

Amazon S3

Globally (and Customer's data policies recommended) means of storage and exchange. Also, Exemplar is hosted by Sapio on AWS

A tool to automatically copy the files physically to AWS in the background

SmartSync

Full-featured solution to back up and synchronize data to AWS

irods cycle
  1. Analyst performs experiment in Exemplar and gets assigned unique Experiment ID
  2. Analyst performs experiment to process and analyze biosamples through analytical techniques such as FACS, single cell RNA Sequencing, ELISA, LCMS
  3. Raw data from instrument stored in pre-defined folder in Instrument Workstation; tagged with unique Exp. ID
  4. SmartSync moves files from Workstation to appropriate storage hierarchy in S3 bucket
  5. iRODS monitors all updates to S3 hierarchy
  6. iRODS creates Audit Log of all updates
  7. Exemplar LIMS performs the following: Retrieves iRODS log file, parses AWS S3 File Movement log file, parses unique Experiment ID and location of Raw Data files from iRODS Audit Log file

Implementation Highlights and Benefits

Implementation Highlights:

  • Zifo's solution design was approved by the Customer Data Architects, Data Governance and Security teams
  • Automated management of approved result data through a new UI for results uploaded into Exemplar LIMS and transfer to S3 after approval
  • The solution was developed and deployed for user testing and feedback in under 5 months
  • Some facts:
    • Number of sites involved: 3
    • Number of instruments integrated: 60
    • Number of Workflows which supported the integration: 18

Benefits:

One of the most significant benefits to the Customer was the end result of a standardized, global process for large file management which provided:

  • a solution built on enterprise IT data standards
  • a secure and reliable storage space for large files of any data type
  • means to collaborate easily
  • improved data quality and accessibility

Other benefits included reduced human error and effort and reduced cost incurred through procurement of licences of multiple applications.

Let's Build Together

Contact our expert to explore how we could help you

author image

Raj is a Scientific Application Analyst at ZIFO RnD Solutions and is based out of USA.His focus is around AWS, Exemplar, Java, SQL and Solution Designing.

He has helped our customers with:

  • Data migration from existing DB to Exemplar-recommended data model
  • In-House(Customer's) Exemplar hosting on Unix server
  • External integration with Exemplar with AWS S3 using Java for samples, raw data and analysis results
  • External integration with Exemplar with repository that provides REST API services via JSON using Java for samples

He can be reached at rajarshi.s@zifornd.com

Get in Touch

US

500 Lake Cook Road,
Suite No. LL7, Deerfield,
Illinois 60015

T : +1 203 826 8897

F : +1 888 589 0524

T : +1 479 459 2825

UK

Office 7, 37-39 Shakespeare Street
Southport
Merseyside
PR8 5AB

T : +44 1223 969605

FRANCE

24, rue René Boulanger,
75010 Paris,
France

GERMANY

c/o Beil Staehle Milde,
Tengstraße 27,
80798 München

T : +49 69 9451 5840

SINGAPORE

10 Anson Road,
27-15 International Plaza,
Singapore 079903

JAPAN

Humax Ebisu building,
7th floor, Ebisu Minami 1-1-1, Shibuya-Ku,
Tokyo 150-0022

T : +81 3 4405 4140

INDIA

21A, Anna Salai
Littlemount, Saidapet
Chennai 600015

T : + 91 44 43114002

F : + 91 44 42012423

CHINA

Room 201, Building A, NOVASIOT
58 Xiangke Road, Zhangjiang
Shanghai 200120
China

T : +86 21 5077 2711

SWITZERLAND

St Alban-Anlage 46
4052 Basel
Switzerland

CANADA

Suite 302,
7111 Syntex Drive,
Mississauga,
Ontario L5N 8C3.

Close
Zifo

Want to chat? I'm here to help you find your way.

Tail
Chat Green Chat White

ZifoZifo RnD Solutions

Close