A MapReduce Framework for DNA Sequencing Data Processing
DOI:
https://doi.org/10.3991/ijes.v4i4.6537Abstract
Genomics and Next Generation Sequencers (NGS) like Illumina Hiseq produce data in the order of 200 billion base pairs in a single one-week run for a 60x human genome coverage, which requires modern high-throughput experimental technologies that can only be tackled with high performance computing (HPC) and specialized software algorithms called “short read aligners”. This paper focuses on the implementation of the DNA sequencing as a set of MapReduce programs that will accept a DNA data set as a FASTQ file and finally generate a VCF (variant call format) file, which has variants for a given DNA data set. In this paper MapReduce/Hadoop along with Burrows-Wheeler Aligner (BWA), Sequence Alignment/Map (SAM) tools, are fully utilized to provide various utilities for manipulating alignments, including sorting, merging, indexing, and generating alignments. The Map-Sort-Reduce process is designed to be suited for a Hadoop framework in which each cluster is a traditional N-node Hadoop cluster to utilize all of the Hadoop features like HDFS, program management and fault tolerance. The Map step performs multiple instances of the short read alignment algorithm (BoWTie) that run in parallel in Hadoop. The ordered list of the sequence reads are used as input tuples and the output tuples are the alignments of the short reads. In the Reduce step many parallel instances of the Short Oligonucleotide Analysis Package for SNP (SOAPsnp) algorithm run in the cluster. Input tuples are sorted alignments for a partition and the output tuples are SNP calls. Results are stored via HDFS, and then archived in SOAPsnp format. The proposed framework enables extremely fast discovering somatic mutations, inferring population genetical parameters, and performing association tests directly based on sequencing data without explicit genotyping or linkage-based imputation. It also demonstrate that this method achieves comparable accuracy to alternative methods for sequencing data processing.
Downloads
Published
How to Cite
Issue
Section
License
The submitting author warrants that the submission is original and that she/he is the author of the submission together with the named co-authors; to the extend the submission incorporates text passages, figures, data or other material from the work of others, the submitting author has obtained any necessary permission.
Articles in this journal are published under the Creative Commons Attribution Licence (CC-BY What does this mean?). This is to get more legal certainty about what readers can do with published articles, and thus a wider dissemination and archiving, which in turn makes publishing with this journal more valuable for you, the authors.
By submitting an article the author grants to this journal the non-exclusive right to publish it. The author retains the copyright and the publishing rights for his article without any restrictions.
This journal has been awarded the SPARC Europe Seal for Open Access Journals (What's this?)
