biomformat/DESCRIPTION0000644000175400017540000000273613001263737015457 0ustar00biocbuildbiocbuildPackage: biomformat Version: 1.2.0 Date: 2016-04-16 Maintainer: Paul J. McMurdie License: GPL-2 Title: An interface package for the BIOM file format Type: Package Author: Paul J. McMurdie and Joseph N Paulson BugReports: https://github.com/joey711/biomformat/issues URL: https://github.com/joey711/biomformat/, http://biom-format.org/ Description: This is an R package for interfacing with the BIOM format. This package includes basic tools for reading biom-format files, accessing and subsetting data tables from a biom object (which is more complex than a single table), as well as limited support for writing a biom-object back to a biom-format file. The design of this API is intended to match the python API and other tools included with the biom-format project, but with a decidedly "R flavor" that should be familiar to R users. This includes S4 classes and methods, as well as extensions of common core functions/methods. Imports: plyr (>= 1.8), jsonlite (>= 0.9.16), Matrix (>= 1.2), rhdf5 Depends: R (>= 3.2), methods Suggests: testthat (>= 0.10), knitr (>= 1.10), BiocStyle (>= 1.6), rmarkdown (>= 0.7) VignetteBuilder: knitr Collate: 'allClasses.R' 'allPackage.R' 'IO-methods.R' 'BIOM-class.R' 'validity-methods.R' biocViews: DataImport, Metagenomics, Microbiome NeedsCompilation: no Packaged: 2016-10-18 00:07:59 UTC; biocbuild biomformat/NAMESPACE0000644000175400017540000000117313001231115015143 0ustar00biocbuildbiocbuild# Generated by roxygen2 (4.1.1): do not edit by hand export(biom) export(biom_data) export(biom_shape) export(header) export(make_biom) export(matrix_element_type) export(observation_metadata) export(read_biom) export(read_hdf5_biom) export(sample_metadata) export(write_biom) exportClasses(biom) exportMethods(colnames) exportMethods(ncol) exportMethods(nrow) exportMethods(rownames) exportMethods(show) import(Matrix) import(methods) importFrom(jsonlite,fromJSON) importFrom(jsonlite,toJSON) importFrom(plyr,alply) importFrom(plyr,d_ply) importFrom(plyr,laply) importFrom(plyr,ldply) importFrom(plyr,llply) importFrom(rhdf5,h5read) biomformat/NEWS0000644000175400017540000000357613001231113014432 0ustar00biocbuildbiocbuildCHANGES IN VERSION 0.3.13 ------------------------- USER-VISIBLE CHANGES - Added make_biom function. Creates biom object from standard R data table(s). CHANGES IN VERSION 0.3.12 ------------------------- USER-VISIBLE CHANGES - No user-visible changes. All future compatibility changes. BUG FIXES - Unit test changes to work with upcoming R release and new testthat version. - This solves Issue 4: https://github.com/joey711/biom/issues/4 CHANGES IN VERSION 0.3.11 ------------------------- USER-VISIBLE CHANGES - No user-visible changes. All future CRAN compatibility changes. BUG FIXES - Clarified license and project in the README.md - Added TODO.html, README.html, and TODO.md to .Rbuildignore (requested by CRAN) - Moved `biom-demo.Rmd` to `vignettes/` - Updated `inst/NEWS` (this) file to official format - Removed pre-built vignette HTML so that it is re-built during package build. This updates things like the build-date in the vignette, but also ensures that the user sees in the vignette the results of code that just worked with their copy of the package. CHANGES IN VERSION 0.3.10 ------------------------- USER-VISIBLE CHANGES - These changes should not affect any package behavior. - Some of the top-level documentation has been changed to reflect new development location on GitHub. BUG FIXES - Minor fixes for CRAN compatibility - This addresses Issue 1: https://github.com/joey711/biom/issues/1 CHANGES IN VERSION 0.3.9 ------------------------- SIGNIFICANT USER-VISIBLE CHANGES - speed improvement for sparse matrices NEW FEATURES - The `biom_data` parsing function now uses a vectorized (matrix-indexed) assignment while parsing sparse matrices. - Unofficial benchmarks estimate a few 100X speedup. CHANGES IN VERSION 0.3.8 ------------------------- SIGNIFICANT USER-VISIBLE CHANGES - First release version released on CRAN biomformat/R/0000755000175400017540000000000013001231115014123 5ustar00biocbuildbiocbuildbiomformat/R/BIOM-class.R0000644000175400017540000010030513001231115016076 0ustar00biocbuildbiocbuild################################################################################ #' Build and return an instance of the biom-class. #' #' This is for instantiating a biom object within R (\code{\link{biom-class}}), #' and assumes relevant data is already available in R. #' This is different than reading a biom file into R. #' If you are instead interested in importing a biom file into R, #' you should use the \code{\link{read_biom}} function. #' This function is made available (exported) so that #' advanced-users/developers #' can easily represent analogous data in this structure if needed. #' However, most users are expected to instead rely on the #' \code{\link{read_biom}} function for data import, followed by #' accessor functions that extract R-friendly #' subsets of the data stored in the biom-format derived list. #' #' \code{biom()} is a constructor method. This is the main method #' suggested for constructing an experiment-level (\code{\link{biom-class}}) #' object from its component data. #' #' @param x (REQUIRED). A named list conforming to conventions arising from #' the \code{\link{fromJSON}} function reading a biom-format file with #' default settings. See \code{\link{read_biom}} for more details about #' data import and #' \code{\link{biom-class}} for more details about accessor functions #' that extract R-friendly #' subsets of the data and metadata stored in \code{x}. #' #' @return An instance of the \code{\link{biom-class}}. #' #' @seealso #' #' Function to create a biom object from R data, #' \code{\link{make_biom}}. #' #' Definition of the #' \code{\link{biom-class}}. #' #' The \code{\link{read_biom}} import function. #' #' Function to write a biom format file from a biom object, #' \code{\link{write_biom}} #' #' Accessor functions like \code{\link{header}}. #' #' @rdname biom-methods #' @export #' @examples # #' # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' x = read_biom(biom_file) #' show(x) #' print(x) #' header(x) #' biom_data(x) #' biom_shape(x) #' nrow(x) #' ncol(x) #' observation_metadata(x) #' sample_metadata(x) setGeneric("biom", function(x) standardGeneric("biom")) #' @rdname biom-methods setMethod("biom", c("list"), function(x){ # Some instantiation checks chould go here, # or wrap them in validity methods. # Depends on how strict the check should be. biom = new("biom", x) return(biom) }) ################################################################################ #' Create a \code{\link{biom-class}} #' from \code{\link{matrix-class}} #' or \code{\link{data.frame}}. #' #' This function creates a valid instance of the \code{\link{biom-class}} #' from standard base-R objects like #' \code{\link{matrix-class}} or \code{\link{data.frame}}. #' This makes it possible to export any contingency table data #' represented in R to #' \href{http://biom-format.org/documentation/biom_format.html}{the biom-format}, #' regardless of its source. #' The object returned by this function is appropriate for writing to #' a \code{.biom} file using the \code{\link{write_biom}} function. #' The sparse biom-format is not (yet) supported. #' #' The BIOM file format (canonically pronounced biome) is designed to be #' a general-use format for representing biological sample by observation #' contingency tables. BIOM is a recognized standard for the #' \href{http://www.earthmicrobiome.org/}{Earth Microbiome Project} #' and is a \href{http://gensc.org/}{Genomics Standards Consortium} #' candidate project. Please see #' \href{http://biom-format.org/}{the biom-format home page} #' for more details. #' #' @param data (Required). #' \code{\link{matrix-class}} or \code{\link{data.frame}}. #' A contingency table. #' Observations / features / OTUs / species are rows, #' samples / sites / libraries are columns. #' #' @param sample_metadata (Optional). #' A \code{\link{matrix-class}} or \code{\link{data.frame}} #' with the number of rows equal to the number of samples in \code{data}. #' Sample covariates associated with the count data. #' This should look like the table returned by #' \code{\link{sample_metadata}} on a valid instance #' of the \code{\link{biom-class}}. #' #' @param observation_metadata (Optional). #' A \code{\link{matrix-class}} or \code{\link{data.frame}} #' with the number of rows equal to the number of #' features / species / OTUs / genes in \code{data}. #' This should look like the table returned by #' \code{\link{observation_metadata}} on a valid instance #' of the \code{\link{biom-class}}. #' #' @param id (Optional). Character string. Identifier for the project. #' #' @param matrix_element_type (Optional). Character string. Either 'int' or 'float' #' #' @return An object of \code{\link{biom-class}}. #' #' @references \url{http://biom-format.org/} #' #' @seealso #' #' \code{\link{write_biom}} #' #' \code{\link{biom-class}} #' #' \code{\link{read_biom}} #' #' @importFrom plyr alply #' #' @export #' #' @examples #' # import with default parameters, specify a file #' biomfile = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") #' x = read_biom(biomfile) #' data = biom_data(x) #' data #' smd = sample_metadata(x) #' smd #' omd = observation_metadata(x) #' omd #' # Make a new biom object from component data #' y = make_biom(data, smd, omd) #' # Won't be identical to x because of header info. #' identical(x, y) #' # The data components should be, though. #' identical(observation_metadata(x), observation_metadata(y)) #' identical(sample_metadata(x), sample_metadata(y)) #' identical(biom_data(x), biom_data(y)) #' ## Quickly show that writing and reading still identical. #' # Define a temporary directory to write .biom files #' tempdir = tempdir() #' write_biom(x, biom_file=file.path(tempdir, "x.biom")) #' write_biom(y, biom_file=file.path(tempdir, "y.biom")) #' x1 = read_biom(file.path(tempdir, "x.biom")) #' y1 = read_biom(file.path(tempdir, "y.biom")) #' identical(observation_metadata(x1), observation_metadata(y1)) #' identical(sample_metadata(x1), sample_metadata(y1)) #' identical(biom_data(x1), biom_data(y1)) make_biom <- function(data, sample_metadata=NULL, observation_metadata=NULL, id=NULL, matrix_element_type="int"){ # The observations / features / OTUs / rows "meta" data table if(!is.null(observation_metadata)){ rows = mapply(list, SIMPLIFY=FALSE, id=as.list(rownames(data)), metadata=alply(as.matrix(observation_metadata), 1, .expand=FALSE, .dims=TRUE)) } else { rows = mapply(list, id=as.list(rownames(data)), metadata=NA, SIMPLIFY=FALSE) } # The samples / sites / columns "meta" data table if(!is.null(sample_metadata)){ columns = mapply(list, SIMPLIFY=FALSE, id=as.list(colnames(data)), metadata=alply(as.matrix(sample_metadata), 1, .expand=FALSE, .dims=TRUE)) } else { columns = mapply(list, id=as.list(colnames(data)), metadata=NA, SIMPLIFY=FALSE) } # Convert the contingency table to a list datalist = as.list(as.data.frame(as(t(data), "matrix"))) names(datalist) <- NULL # Define the list, instantiate as biom-format, and return # (Might eventually expose some of these list elements as function arguments) format_url = "http://biom-format.org/documentation/format_versions/biom-1.0.html" return(biom(list(id=id, format = "Biological Observation Matrix 1.0.0-dev", format_url = format_url, type = "OTU table", generated_by = sprintf("biomformat %s", packageVersion("biomformat")), date = as.character(Sys.time()), matrix_type = "dense", matrix_element_type = matrix_element_type, shape = dim(data), rows = rows, columns = columns, data = datalist) )) } ################################################################################ #' Method extensions to show for biom objects. #' #' See the general documentation of \code{\link[methods]{show}} method for #' expected behavior. #' #' @seealso \code{\link[methods]{show}} #' #' @param object biom-class object #' #' @export #' #' @rdname show-methods #' #' @examples #' # # # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' show(x) setMethod("show", "biom", function(object){ cat("biom object. \n") cat("type:", object$type, "\n") cat("matrix_type:", object$matrix_type, "\n") cat(biom_shape(object)[1], "rows and", biom_shape(object)[2], "columns \n") }) ################################################################################ #' Extract the header from a \code{\link{biom-class}} object as a list. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @return A list containing the header data. #' That is, all the required elements that are not #' the main data or index metadata. #' #' @docType methods #' @rdname header-methods #' @export #' @examples #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' x = read_biom(biom_file) #' header(x) setGeneric("header", function(x) standardGeneric("header")) #' @rdname header-methods setMethod("header", c("biom"), function(x){ biomheadkeys = c("id", "format", "format_url", "type", "generated_by", "date", "matrix_type", "matrix_element_type", "shape") return(x[biomheadkeys]) }) ################################################################################ #' The matrix dimensions #' of a \code{\link{biom-class}} object. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @return A length two \code{\link{integer-class}} vector #' indicating the \code{\link{nrow}} and \code{\link{ncol}} #' of the main data matrix stored in \code{x}. #' #' @seealso #' #' \code{\link{biom-class}} #' #' @export #' @docType methods #' @rdname biom_shape-methods #' @examples #' # # # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' biom_shape(x) setGeneric("biom_shape", function(x) standardGeneric("biom_shape")) #' @rdname biom_shape-methods setMethod("biom_shape", c("biom"), function(x){ return(as(c(nrow=x$shape[1], ncol=x$shape[2]), "integer")) }) ################################################################################ #' Access class of data in the matrix elements #' of a \code{\link{biom-class}} object #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @return A \code{\link{character-class}} string indicating #' the class of the data stored in the main observation matrix of \code{x}, #' with expected values \code{"int"}, \code{"float"}, \code{"unicode"}. #' #' @seealso #' #' \code{\link{biom-class}} #' #' @export #' @docType methods #' @rdname matrix_element_type-methods #' @examples #' # # # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' matrix_element_type(x) setGeneric("matrix_element_type", function(x) standardGeneric("matrix_element_type")) #' @rdname matrix_element_type-methods setMethod("matrix_element_type", c("biom"), function(x){ return(x$matrix_element_type) }) ################################################################################ #' Method extensions to \code{\link[base]{nrow}} #' for \code{\link{biom-class}} objects. #' #' See the general documentation of \code{\link[base]{nrow}} method for #' expected behavior. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @return The number of rows in \code{x}. #' A length 1 \code{\link{integer-class}}. #' #' @seealso #' #' \code{\link{ncol}} #' #' \code{\link[base]{nrow}} #' #' \code{\link{biom_shape}} #' #' @export #' @docType methods #' @rdname nrow-methods #' @examples #' # # # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' nrow(x) setMethod("nrow", c("biom"), function(x){ return( biom_shape(x)["nrow"] ) }) ################################################################################ #' Method extensions to \code{\link[base]{ncol}} #' for \code{\link{biom-class}} objects. #' #' See the general documentation of \code{\link[base]{ncol}} method for #' expected behavior. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @return The number of columns in \code{x}. #' A length 1 \code{\link{integer-class}}. #' #' @seealso #' #' \code{\link{nrow}} #' #' \code{\link[base]{ncol}} #' #' \code{\link{biom_shape}} #' #' @export #' @docType methods #' @rdname ncol-methods #' @examples #' # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' ncol(x) setMethod("ncol", c("biom"), function(x){ return( biom_shape(x)["ncol"] ) }) ################################################################################ #' Method extensions to \code{\link[base]{rownames}} #' for \code{\link{biom-class}} objects. #' #' See the general documentation of \code{\link[base]{rownames}} method for #' expected behavior. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' @return The number of columns in \code{x}. #' A length 1 \code{\link{integer-class}}. #' #' @seealso #' #' \code{\link{nrow}} #' #' \code{\link[base]{rownames}} #' #' \code{\link{biom_shape}} #' #' @export #' @docType methods #' @rdname rownames-methods #' @examples #' # # # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' rownames(x) setMethod("rownames", c("biom"), function(x){ sapply(x$rows, function(i) i$id) }) ################################################################################ #' Method extensions to \code{\link[base]{colnames}} #' for \code{\link{biom-class}} objects. #' #' See the general documentation of \code{\link[base]{colnames}} method for #' expected behavior. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' @return The number of columns in \code{x}. #' A length 1 \code{\link{integer-class}}. #' #' @seealso #' #' \code{\link{nrow}} #' #' \code{\link[base]{colnames}} #' #' \code{\link{biom_shape}} #' #' @export #' @docType methods #' @rdname colnames-methods #' @examples #' # # # import with default parameters, specify a file #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' (x = read_biom(biom_file) ) #' colnames(x) setMethod("colnames", c("biom"), function(x){ sapply(x$columns, function(i) i$id) }) ################################################################################ #' Access main data observation matrix data from \code{\link{biom-class}}. #' #' Retrieve and organize main data from \code{\link{biom-class}}, #' represented as a matrix with index names. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' @param rows (Optional). The subset of row indices described in the #' returned object. For large datasets, specifying the row subset here, #' rather than after creating the whole matrix first, #' can improve speed/efficiency. #' Can be vector of index numbers (\code{\link{numeric-class}}) or #' index names (\code{\link{character-class}}). #' @param columns (Optional). The subset of column indices described in the #' returned object. For large datasets, specifying the column subset here, #' rather than after creating the whole matrix first, #' can improve speed/efficiency. #' Can be vector of index numbers (\code{\link{numeric-class}}) or #' index names (\code{\link{character-class}}). #' @param parallel (Optional). Logical. Whether to perform the accession parsing #' using a parallel-computing backend supported by the \code{\link{plyr-package}} #' via the \code{\link[foreach]{foreach-package}}. Note: At the moment, the header #' accessor does not need nor does it support parallel-computed parsing. #' #' @return A matrix containing the main observation data, with index names. #' The type of data (numeric or character) #' will depend on the results of \code{\link{matrix_element_type}(x)}. #' The class of the matrix returned will depend on the sparsity of the data, #' and whether it has numeric or character data. #' For now, only numeric data can be stored in a \code{\link{Matrix-class}}, #' which will be stored sparsely, if possible. #' Character data will be returned as a vanilla \code{\link{matrix-class}}. #' #' @rdname biom_data-methods #' @export #' @examples #' min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") #' min_sparse_file = system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") #' rich_dense_file = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") #' rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") #' rich_dense_char = system.file("extdata", "rich_dense_char.biom", package = "biomformat") #' rich_sparse_char = system.file("extdata", "rich_sparse_char.biom", package = "biomformat") #' # Read the biom-format files #' x1 = read_biom(min_dense_file) #' x2 = read_biom(min_sparse_file) #' x3 = read_biom(rich_dense_file) #' x4 = read_biom(rich_sparse_file) #' x5 = read_biom(rich_dense_char) #' x6 = read_biom(rich_sparse_char) #' # Extract the data matrices #' biom_data(x1) #' biom_data(x2) #' biom_data(x3) #' biom_data(x4) #' biom_data(x5) #' biom_data(x6) setGeneric("biom_data", function(x, rows, columns, parallel=FALSE){ standardGeneric("biom_data") }) # All methods funnel toward signature biom,numeric,numeric #' @rdname biom_data-methods setMethod("biom_data", c("biom", "missing", "missing"), function(x, rows, columns, parallel){ # Dispatch with full rows and cols biom_data(x, 1:nrow(x), 1:ncol(x), parallel) }) #' @rdname biom_data-methods setMethod("biom_data", c("biom", "character", "ANY"), function(x, rows, columns, parallel){ rows = which(rownames(x) %in% rows) # Dispatch with specified numeric rows and pass cols biom_data(x, rows, columns) }) #' @rdname biom_data-methods setMethod("biom_data", c("biom", "ANY", "character"), function(x, rows, columns, parallel){ columns = which(colnames(x) %in% columns) # Dispatch with specified numeric columns and pass rows biom_data(x, rows, columns) }) #' @rdname biom_data-methods setMethod("biom_data", c("biom", "numeric", "missing"), function(x, rows, columns, parallel){ # Dispatch with specified rows and full cols biom_data(x, rows, 1:ncol(x), parallel) }) #' @rdname biom_data-methods setMethod("biom_data", c("biom", "missing", "numeric"), function(x, rows, columns, parallel){ # Dispatch with full rows and specified cols biom_data(x, 1:nrow(x), columns, parallel) }) #' @rdname biom_data-methods #' @import Matrix #' @importFrom plyr d_ply #' @importFrom plyr ldply #' @importFrom plyr laply setMethod("biom_data", c("biom", "numeric", "numeric"), function(x, rows, columns, parallel){ if( identical(length(rows), 0) ){ stop("argument `rows` must have non-zero length.") } if( identical(length(columns), 0) ){ stop("argument `columns` must have non-zero length.") } # Begin matrix section if( identical(x$matrix_type, "dense") ){ # Begin dense section # If matrix is stored as dense, create "vanilla" R matrix, m m = laply(x$data[rows], function(i) i[columns], .parallel=parallel) if( length(rows) > 1L & length(columns) > 1L & matrix_element_type(x) %in% c("int", "float") ){ # If either dimension is length-one, don't call coerce to "Matrix" # Note that laply() does still work in this case. # If both dimension lengths > 1 & data is numeric, # attempt to coerce to Matrix-inherited class, # Mainly because it might still be sparse and this is a good way # to handle it in R. m = Matrix(m) } } else { # Begin sparse section ## Initialize sparse matrix as either Matrix or matrix, depending on data class biom_shape = biom_shape(x) if(matrix_element_type(x) %in% c("int", "float")){ # If data is numeric, initialize with Matrix (numeric data only) m = Matrix(0, nrow=nrow(x), ncol=ncol(x), sparse=TRUE) # Create an assignment data.frame adf = ldply(x$data) } else { # Else, matrix_element_type must be "unicode" for a unicode string. # Use a standard R character matrix m = matrix(NA_character_, nrow(x), ncol(x)) # Create an assignment data.frame. # Is slightly more complicated for sparse JSON w/ character values adf = ldply(x$data, function(x){ data.frame(r=x[[1]], c=x[[2]], data=x[[3]], stringsAsFactors=FALSE) }) } colnames(adf) <- c("r", "c", "data") # indices start at 0 in biom sparse format, # and are first two columns adf[, c("r", "c")] <- cbind(r = as.integer(adf$r) + 1L, c = as.integer(adf$c) + 1L) # Subset to just indices that are in both arguments `rows` and `columns` adf <- adf[(adf$r %in% rows & adf$c %in% columns), ] # Fill in data values in matrix, m. # Vectorized for speed using matrix indexing. # See help("Extract") for details about matrix indexing. Diff than 2-vec index. m[as(adf[, 1:2], "matrix")] <- adf$data # Subset this biggest-size m to just `rows` and `columns` m = m[rows, columns] # End sparse section } # Add row and column names if( identical(length(rows), 1L) | identical(length(columns), 1L) ){ # If either dimension is length-one # Try naming by colnames first, then rownames if( identical(length(rows), 1L) ){ names(m) <- sapply(x$columns[columns], function(i) i$id ) } else { names(m) <- sapply(x$rows[rows], function(i) i$id ) } } else { # Else, both dimensions are longer than 1, # can assume is a matrix and assign names to both dimensions rownames(m) <- sapply(x$rows[rows], function(i) i$id ) colnames(m) <- sapply(x$columns[columns], function(i) i$id ) } return(m) }) ################################################################################ #' Access meta data from \code{\link{biom-class}}. #' #' Retrieve and organize meta data from \code{\link{biom-class}}, #' represented as a \code{\link{data.frame}} (if possible, or a list) #' with proper index names. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @param columns (Optional). The subset of column indices described in the #' returned object. For large datasets, specifying the column subset here, #' rather than after creating the whole matrix first, #' can improve speed/efficiency. #' Can be vector of index numbers (\code{\link{numeric-class}}) or #' index names (\code{\link{character-class}}). #' #' @param parallel (Optional). Logical. Whether to perform the accession parsing #' using a parallel-computing backend supported by the \code{\link{plyr-package}} #' via the \code{\link[foreach]{foreach-package}}. #' #' @return A \code{\link{data.frame}} or \code{\link{list}} containing #' the meta data, with index names. The precise form of the object returned #' depends on the metadata stored in \code{x}. A \code{data.frame} is #' created if possible. #' #' @rdname sample_metadata-methods #' @export #' @examples #' min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") #' min_sparse_file = system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") #' rich_dense_file = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") #' rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") #' rich_dense_char = system.file("extdata", "rich_dense_char.biom", package = "biomformat") #' rich_sparse_char = system.file("extdata", "rich_sparse_char.biom", package = "biomformat") #' # Read the biom-format files #' x1 = read_biom(min_dense_file) #' x2 = read_biom(min_sparse_file) #' x3 = read_biom(rich_dense_file) #' x4 = read_biom(rich_sparse_file) #' x5 = read_biom(rich_dense_char) #' x6 = read_biom(rich_sparse_char) #' # Extract metadata #' sample_metadata(x1) #' sample_metadata(x2) #' sample_metadata(x3) #' sample_metadata(x3, 1:4) #' sample_metadata(x4) #' sample_metadata(x5) #' sample_metadata(x6) setGeneric("sample_metadata", function(x, columns, parallel=FALSE){ standardGeneric("sample_metadata") }) # All methods funnel toward signature biom,numeric #' @rdname sample_metadata-methods setMethod("sample_metadata", c("biom", "missing"), function(x, columns, parallel=FALSE){ # Dispatch with full rows and cols sample_metadata(x, 1:ncol(x), parallel) }) #' @rdname sample_metadata-methods setMethod("sample_metadata", c("biom", "character"), function(x, columns, parallel=FALSE){ columns = which(sapply(x$columns, function(j) j$id) %in% columns) if( length(columns)==0 ){ stop("The column ID names you provided do not match the column IDs in x") } # Dispatch with specified numeric columns sample_metadata(x, columns, parallel) }) #' @rdname sample_metadata-methods setMethod("sample_metadata", c("biom", "numeric"), function(x, columns, parallel=FALSE){ if( any(columns > ncol(x)) ){ warning(paste0("column indices ", paste0(columns[columns > ncol(x)], collapse=" "), " are greater than available columns in data. They were ignored.")) columns = columns[columns <= ncol(x)] } return(extract_metadata(x, "columns", columns, parallel)) }) ################################################################################ #' Access observation (row) meta data from \code{\link{biom-class}}. #' #' Retrieve and organize meta data from \code{\link{biom-class}}, #' represented as a \code{\link{data.frame}} (if possible) #' or a list, with proper index names. #' #' @param x (Required). An instance of the \code{\link{biom-class}}. #' #' @param rows (Optional). The subset of row indices described in the #' returned object. For large datasets, specifying the row subset here, #' -- rather than first creating the complete data object -- #' can improve speed/efficiency. #' This parameter can be vector of index numbers (\code{\link{numeric-class}}) or #' index names (\code{\link{character-class}}). #' #' @param parallel (Optional). Logical. Whether to perform the accession parsing #' using a parallel-computing backend supported by the \code{\link{plyr-package}} #' via the \code{\link[foreach]{foreach-package}}. #' #' @return A \code{\link{data.frame}} or \code{\link{list}} containing #' the meta data, with index names. The precise form of the object returned #' depends on the metadata stored in \code{x}. A \code{data.frame} is #' created if possible. #' #' @rdname observation_metadata-methods #' @export #' @examples #' min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") #' min_sparse_file = system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") #' rich_dense_file = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") #' rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") #' rich_dense_char = system.file("extdata", "rich_dense_char.biom", package = "biomformat") #' rich_sparse_char = system.file("extdata", "rich_sparse_char.biom", package = "biomformat") #' # Read the biom-format files #' x1 = read_biom(min_dense_file) #' x2 = read_biom(min_sparse_file) #' x3 = read_biom(rich_dense_file) #' x4 = read_biom(rich_sparse_file) #' x5 = read_biom(rich_dense_char) #' x6 = read_biom(rich_sparse_char) #' # Extract metadata #' observation_metadata(x1) #' observation_metadata(x2) #' observation_metadata(x3) #' observation_metadata(x3, 2:4) #' observation_metadata(x3, 2) #' observation_metadata(x3, c("GG_OTU_3", "GG_OTU_4", "whoops")) #' observation_metadata(x4) #' observation_metadata(x5) #' observation_metadata(x6) setGeneric("observation_metadata", function(x, rows, parallel=FALSE){ standardGeneric("observation_metadata") }) # All methods funnel toward signature biom,numeric #' @rdname observation_metadata-methods setMethod("observation_metadata", c("biom", "missing"), function(x, rows, parallel=FALSE){ # Dispatch with full rows and cols observation_metadata(x, 1:nrow(x), parallel) }) #' @rdname observation_metadata-methods setMethod("observation_metadata", c("biom", "character"), function(x, rows, parallel=FALSE){ rows = which(sapply(x$rows, function(j) j$id) %in% rows) if( length(rows)==0 ){ stop("The row ID names you provided do not match the row IDs in x") } # Dispatch with specified numeric rows observation_metadata(x, rows, parallel) }) #' @rdname observation_metadata-methods setMethod("observation_metadata", c("biom", "numeric"), function(x, rows, parallel=FALSE){ if( any(rows > nrow(x)) ){ warning(paste0("Row indices ", paste0(rows[rows > nrow(x)], collapse=" "), " are greater than available rows in data. They were ignored.")) rows = rows[rows <= nrow(x)] } return(extract_metadata(x, "rows", rows, parallel)) }) ################################################################################ # Generic internal function for extracting metadata from either rows or columns #' @importFrom plyr ldply #' @importFrom plyr llply #' @keywords internal extract_metadata = function(x, indextype, indices, parallel=FALSE){ # Immediately extract just those index indicated by `index` argument metalist = x[[indextype]][indices] # Extract metadata elements as a list, for checking dimensions, NULL, etc. rx = llply(metalist, function(i) unlist(i$metadata), .parallel=parallel) if( all(sapply(rx, is.null)) ){ # If there is no metadata (all NULL), # then set metadata to NULL, representing empty. metadata = NULL } else { # Else, extract names and metadata (both required) # Extract names metaids = sapply(metalist, function(i) i$id) # Test if length of metadata entries is same for all indices. rxlengths = sapply(rx, length) if( all( rxlengths == rxlengths[1]) ){ # If so, can parse it as data.frame with ldply # return a data.frame with colnames metadata = ldply(rx, .parallel=parallel) rownames(metadata) <- metaids } else { # Else, should keep it as a list. But should name the entries metadata = rx names(metadata) <- metaids } } return(metadata) } ################################################################################ # Generic internal function for generating the count matrix. #' @keywords internal generate_matrix <- function(x){ indptr = x$sample$matrix$indptr+1 indices = x$sample$matrix$indices+1 data = x$sample$matrix$data nr = length(x$observation$ids) counts = sapply(2:length(indptr),function(i){ x = rep(0,nr) seq = indptr[i-1]:(indptr[i]-1) x[indices[seq]] = data[seq] x }) rownames(counts) = x$observation$ids colnames(counts) = x$sample$ids # I wish this next line wasn't necessary lapply(1:nrow(counts),function(i){ counts[i,] }) } ################################################################################ # Generic internal function for generating the metadata. #' @keywords internal generate_metadata <- function(x){ metadata = x$metadata metadata = lapply(1:length(x$ids),function(i){ id_metadata = lapply(metadata,function(j){ if(length(dim(j))>1){ as.vector(j[,i,drop=FALSE]) } else{ j[i] } }) list(id = x$ids[i],metadata=id_metadata) }) return(metadata) } ################################################################################ # Generic internal function for generating a named list. #' @keywords internal namedList <- function(...) { L <- list(...) snm <- sapply(substitute(list(...)),deparse)[-1] if (is.null(nm <- names(L))) nm <- snm if (any(nonames <- nm=="")) nm[nonames] <- snm[nonames] setNames(L,nm) } ################################################################################ biomformat/R/IO-methods.R0000644000175400017540000001612713001231113016223 0ustar00biocbuildbiocbuild################################################################################ ################################################################################ #' Read a biom-format file, returning a \code{biom-class}. #' #' Import the data from a biom-format file into R, represented as an instance #' of the \code{\link{biom-class}}; essentially a \code{\link{list}} with #' special constraints that map to \href{http://biom-format.org/documentation/biom_format.html}{the biom-format definition}. #' #' The BIOM file format (canonically pronounced biome) is designed to be a general-use format for representing biological sample by observation contingency tables. BIOM is a recognized standard for the \href{http://www.earthmicrobiome.org/}{Earth Microbiome Project} and is a \href{http://gensc.org/}{Genomics Standards Consortium} candidate project. Please see \href{http://biom-format.org/}{the biom-format home page} for more details. #' #' It is tempting to include an argument identifying the #' biom-format version number of the data file being imported. #' However, the biom-format version number is a required #' field in the biom-format definition. #' Rather than duplicate this formal specification #' and allow the possibility of a conflict, the version #' number of the biom format will be referred to only by #' the "format" field in the biom formatted data, #' or its representation in R. #' #' @usage read_biom(biom_file) #' #' @param biom_file (Required). A character string indicating the #' file location of the biom formatted file. This is a HDF5 or JSON formatted file #' specific to biological datasets. #' The format is formally defined at \href{http://biom-format.org/documentation/biom_format.html}{the biom-format definition} #' and depends on the versioning. #' #' @return An instance of the \code{biom-class}. #' #' @seealso #' #' Function to create a biom object from R data, #' \code{\link{make_biom}}. #' #' Definition of the #' \code{\link{biom-class}}. #' #' Function to write a biom format file from a biom object, #' \code{\link{write_biom}} #' #' Accessor functions like \code{\link{header}}. #' #' @references \url{http://biom-format.org/} #' #' @importFrom jsonlite fromJSON #' @export #' @examples #' # # # import with default parameters, specify a file #' biom_file <- system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' biom_file #' read_biom(biom_file) #' biom_file <- system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") #' biom_file #' read_biom(biom_file) #' ## The previous examples use system.file() because of constraints in specifying a fixed #' ## path within a reproducible example in a package. #' ## In practice, however, you can simply provide "hard-link" #' ## character string path to your file: #' # mybiomfile <- "path/to/my/biomfile.biom" #' # read_biom(mybiomfile) read_biom <- function(biom_file){ # Read the file, storing as a list # generated by jsonlite w/ default JSON parsing params trash = try(silent=TRUE, expr = { x <- fromJSON(biom_file, simplifyDataFrame = FALSE, simplifyMatrix = FALSE) }) if(inherits(trash, "try-error")){ # If JSON interpretation attempt failed, try HDF5 trash = try(silent=TRUE, expr = {x <- read_hdf5_biom(biom_file)}) } if(inherits(trash, "try-error")){ # If still bad, throw helpful error. stop("Both attempts to read input file:\n", biom_file, "\n", "either as JSON (BIOM-v1) or HDF5 (BIOM-v2).\n", "Check file path, file name, file itself, then try again.") } # Use the biom() constructor function to # instantiate a biom-class, perform validity checks. Return. return( biom(x) ) } ################################################################################ #' Write a biom-format v1 file, returning a \code{biom-class}. #' #' @param x (Required). A biom object that is going to be written to file #' as a proper biom formatted file, adhering to #' \href{http://biom-format.org/documentation/biom_format.html}{the biom-format definition}. #' #' @param biom_file (Required). A character string indicating the #' file location of the biom formatted file. This is a JSON formatted file #' specific to biological datasets. #' The format is formally defined at #' \href{http://biom-format.org/documentation/biom_format.html}{the biom-format definition} #' #' @return Nothing. The first argument, \code{x}, is written to a file. #' #' @seealso #' #' Function to create a biom object from R data, #' \code{\link{make_biom}}. #' #' Definition of the #' \code{\link{biom-class}}. #' #' The \code{\link{read_biom}} import function. #' #' Accessor functions like \code{\link{header}}. #' #' @references \url{http://biom-format.org/} #' #' @export #' @importFrom jsonlite toJSON #' @examples #' biom_file <- system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' x = read_biom(biom_file) #' outfile = tempfile() #' write_biom(x, outfile) #' y = read_biom(outfile) #' identical(x, y) write_biom <- function(x, biom_file){ cat(toJSON(x), file=biom_file) } ################################################################################ #' Read in a biom-format vs 2 file, returning a \code{list}. #' #' This function is meant only to be used if the user knows the file is #' a particular version / hdf5 format. Otherwise, the `read_biom` file should be used. #' #' @param biom_file (Required). A biom object that is going to be written to file #' as a proper biom formatted file, adhering to #' \href{http://biom-format.org/documentation/biom_format.html}{the biom-format definition}. #' #' @return Nothing. The first argument, \code{x}, is written to a file. #' #' @seealso #' #' Function to create a biom object from R data, #' \code{\link{make_biom}}. #' #' Definition of the #' \code{\link{biom-class}}. #' #' The \code{\link{read_hdf5_biom}} import function. #' #' Accessor functions like \code{\link{header}}. #' #' @references \url{http://biom-format.org/} #' #' @export #' @importFrom rhdf5 h5read #' @examples #' biom_file <- system.file("extdata", "rich_sparse_otu_table_hdf5.biom", package = "biomformat") #' x = read_hdf5_biom(biom_file) #' x = biom(x) #' outfile = tempfile() #' write_biom(x, outfile) #' y = read_biom(outfile) #' identical(observation_metadata(x),observation_metadata(y)) #' identical(sample_metadata(x),sample_metadata(y)) #' identical(biom_data(x), biom_data(y)) read_hdf5_biom<-function(biom_file){ x = h5read(biom_file,"/",read.attributes = TRUE) data = generate_matrix(x) rows = generate_metadata(x$observation) columns = generate_metadata(x$sample) shape = c(length(data),length(data[[1]])) id = attr(x,"id") vs = attr(x,"format-version") format = sprintf("Biological Observation Matrix %s.%s",vs[1],vs[2]) format_url = attr(x,"format-url") type = "OTU table" generated_by = attr(x,"generated-by") date = attr(x,"creation-date") matrix_type = "dense" matrix_element_type = "int" namedList(id,format,format_url,type,generated_by,date,matrix_type,matrix_element_type, rows,columns,shape,data) } ################################################################################biomformat/R/allClasses.R0000644000175400017540000000334013001231113016332 0ustar00biocbuildbiocbuild################################################################################ #' The biom format data class. #' #' This class inherits from the \code{\link{list-class}}, #' with validity checks specific to the definition to the biom-format. #' Effectively this means the list must have certain index names, #' some elements of which must have a specific structure or value. #' For further details see #' \href{http://biom-format.org/documentation/biom_format.html}{the biom-format definition}. #' Importantly, this means other special properties of lists, #' like operations with \code{$} and single- or double-square-braces #' are also supported; as-is the \code{apply}-family function #' that can operate on lists. #' Note that some features of the biom-format can be essentially empty, #' represented by the string \code{"null"} in the file. #' These fields are returned as \code{\link{NULL}} when accessed #' by an accessor function. #' #' @seealso #' The constructor, \code{\link{biom}} #' #' Accessor functions: #' #' \code{\link{header}}, #' \code{\link{biom_shape}}, #' \code{\link{nrow}}, #' \code{\link{ncol}}, #' \code{\link{matrix_element_type}}, #' \code{\link{biom_data}}, #' \code{\link{observation_metadata}}, #' \code{\link{sample_metadata}} #' #' @name biom-class #' @rdname biom-class #' @exportClass biom #' #' @examples #' biom_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") #' x = read_biom(biom_file) #' header(x) #' biom_shape(x) #' nrow(x) #' ncol(x) #' rownames(x) #' colnames(x) #' matrix_element_type(x) #' biom_data(x) #' observation_metadata(x) #' sample_metadata(x) setClass("biom", contains="list") ################################################################################ biomformat/R/allPackage.R0000644000175400017540000000121613001231115016272 0ustar00biocbuildbiocbuild############################################### #' This is an R package for interfacing with the biom format. #' #' This is an R package for interfacing with the biom-format. #' It includes basic utilities for reading and writing BIOM format using R, #' and native R methods for interacting with biom-data #' that maps to functionality in other languages #' that support biom-format, like python. #' #' @import methods #' @name biom-package #' @author Paul J. McMurdie II \email{mcmurdie@@stanford.edu} #' @references #' \url{http://www.biom-format.org/} #' #' @docType package #' @keywords package NA ############################################### biomformat/R/validity-methods.R0000644000175400017540000001000713001231113017530 0ustar00biocbuildbiocbuild################################################################################ # Validity methods: # # These are delicate, because they are effectively at the S4 infrastructure # level, in between "new" and the constructor. Some of the issues that might # otherwise go here can be also be dealt with in the constructors, if needed. ################################################################################ validbiom = function(object){ # All the required top-level keys are present biomreqdkeys = c("id", "format", "format_url", "type", "generated_by", "date", "rows", "columns", "matrix_type", "matrix_element_type", "shape", "data") if( !all(biomreqdkeys %in% names(object)) ){ missingkeysmessage = paste("", "Not all required top-level keys are present in biom-object.", "Required keys are:", paste0(biomreqdkeys, collapse="\n"), sep="\n") return(missingkeysmessage) } met = matrix_element_type(object) if(!identical(length(met), 1L)){ return("biom matrix_element_type field should have only 1 element") } if( !met %in% c("int", "float", "unicode") ){ return("biom matrix_element_type value is unsupported.") } # Basic biom_shape value validity. bshape = biom_shape(object) if(!inherits(bshape, "integer")){ return("problem with biom shape value type") } if(!identical(length(bshape), 2L)){ return("problem with biom shape value length") } if(any(bshape < 0)){ return("problem with biom shape value: negative value") } # Matrix shape and number of row/col elements matches if( ncol(object) > length(object$columns) ){ return("shape field specifies more cols than are present in metadata") } if( nrow(object) > length(object$rows) ){ return("shape field specifies more rows than are present in metadata") } if( ncol(object) < length(object$columns) ){ return("more metadata cols than specified by shape field") } if( nrow(object) < length(object$rows) ){ return("more metadata rows than specified by shape field") } # The type field has an acceptable value biomtypevals = c("OTU table", "Pathway table", "Function table", "Ortholog table", "Gene table", "Metabolite table", "Taxon table" ) if( !object$type %in% biomtypevals ){ return("type field has unsupported value") } # The matrix_type has an acceptable value biommatrixtypes = c("sparse", "dense") if( !object$matrix_type %in% biommatrixtypes ){ return("matrix_type field has unsupported value") } # The matrix_element_type has an acceptable value biommatrixelemtypes = c("int", "float", "unicode") if( !object$matrix_element_type %in% biommatrixelemtypes ){ return("matrix_element_type field has unsupported value") } # If sparse, all data fields should have length of 3 (row, col, val) if( identical(object$matrix_type, "sparse") ){ if( !all(sapply(object$data, length)==3L) ){ return("Some data fields for this sparse biom format do not have 3 elements") } } # If dense, data fields should have length equal to ncol if( identical(object$matrix_type, "dense") ){ if( !all(sapply(object$data, length)==ncol(object)) ){ return(paste("Some data fields for this dense biom format", "do not have the expected number columns:", ncol(object), sep=" ")) } } # metadata ids -- rownames and colnames -- must be unique dupr = duplicated(rownames(object)) dupc = duplicated(colnames(object)) if( any(dupr) ){ return(paste("The following row ids were duplicated: \n", paste0(rownames(object)[dupr], collapse="\n"), sep="")) } if( any(dupc) ){ return(paste("The following col ids were duplicated: \n", paste0(colnames(object)[dupc], collapse="\n"), sep="")) } # If we get all the way here without returning a string, # then consider it a valid object. # In which case should return TRUE. return(TRUE) } ################################################################################ ## assign the function as the validity method for the biom-class setValidity("biom", validbiom) ################################################################################ biomformat/README.md0000644000175400017540000000452213001231113015202 0ustar00biocbuildbiocbuild # The biomformat Package for R [![Travis-CI Build Status](https://travis-ci.org/joey711/biomformat.svg?branch=master)](https://travis-ci.org/joey711/biomformat) # About This is an R package for interfacing with both the JSON and HDF5 versions of the [biom file format](http://biom-format.org/). This package includes basic tools for reading biom-format files, accessing and subsetting data tables from a biom object, as well as limited support for writing a biom-object back to a biom-format file. # Design The design of this API is intended to match the python API where appropriate, while maintaining a decidedly "R flavor" that should be familiar to R users. This includes S4 classes and methods, as well as extensions of common core functions/methods. # Installation To install the latest stable release of the biomformat package enter the following in an R session (after official release). ```S source("http://bioconductor.org/biocLite.R") biocLite("biomformat") ``` Before official release, or to install the latest development version, enter the following into an R session. ```S install.packages("devtools") # if not already installed devtools::install_github("joey711/biomformat") ``` # Support * Please post feature or support requests and bugs at the [Issues Tracker for the biomformat package](https://github.com/joey711/biomformat/issues) on GitHub. Issues related to the format itself and not the R interface should be posted on the [issues tracker for the biom-format team](https://github.com/biocore/biom-format/issues). * Note that this is a separate (but friendly!) project from the biom-format team, and the software license is different between this package and much of the rest of the biom-format software, which has switched to BSD. * This package is scheduled for inclusion in the next stable release of the Bioconductor project (BioC). BioC is also where [the rhdf5 package](http://bioconductor.org/packages/release/bioc/html/rhdf5.html) lives. * The original release of this package was [made available through CRAN](http://cran.r-project.org/web/packages/biom/index.html), as the "biom" package, supporting version 1.x (JSON) only. The current plan is to let "biom" remain on CRAN in maintenance-only mode until "biomformat" is released on Bioconductor. At which point "biom" will be considered deprecated, and eventually removed. biomformat/build/0000755000175400017540000000000013001263737015040 5ustar00biocbuildbiocbuildbiomformat/build/vignette.rds0000644000175400017540000000033213001263737017375 0ustar00biocbuildbiocbuildmQ0  Q/+/BS%2 旋EAaڤ]Ӟf'JRLAGߢ+D#sZ8C d+>qBߌ%gRy9ҬWAv3`B y=?V,pkjL0zѼ4anυ\O_ߨvXuqТɋ^2Vˋ!̉*w Jbiomformat/inst/0000755000175400017540000000000013001263737014716 5ustar00biocbuildbiocbuildbiomformat/inst/doc/0000755000175400017540000000000013001263737015463 5ustar00biocbuildbiocbuildbiomformat/inst/doc/biomformat.R0000644000175400017540000000446713001263737017760 0ustar00biocbuildbiocbuild## ----packages------------------------------------------------------------ library("biomformat"); packageVersion("biomformat") ## ----read-biom-examples-------------------------------------------------- min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") min_sparse_file = system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") rich_dense_file = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") rich_dense_char = system.file("extdata", "rich_dense_char.biom", package = "biomformat") rich_sparse_char = system.file("extdata", "rich_sparse_char.biom", package = "biomformat") x1 = read_biom(min_dense_file) x2 = read_biom(min_sparse_file) x3 = read_biom(rich_dense_file) x4 = read_biom(rich_sparse_file) x5 = read_biom(rich_dense_char) x6 = read_biom(rich_sparse_char) x1 ## ----accessor-examples-table--------------------------------------------- biom_data(x1) biom_data(x2) ## ----matrix-coercion----------------------------------------------------- as(biom_data(x2), "matrix") ## ----observ-meta--------------------------------------------------------- observation_metadata(x1) observation_metadata(x2) observation_metadata(x3) observation_metadata(x4)[1:2, 1:3] class(observation_metadata(x4)) ## ----plot-examples------------------------------------------------------- sample_metadata(x1) sample_metadata(x2) sample_metadata(x3) sample_metadata(x4)[1:2, 1:3] class(sample_metadata(x4)) ## ----plot---------------------------------------------------------------- plot(biom_data(x4)) boxplot(as(biom_data(x4), "vector")) heatmap(as(biom_data(x4), "matrix")) ## ----write-biom-examples------------------------------------------------- outfile = tempfile() write_biom(x4, outfile) y = read_biom(outfile) identical(x4, y) ## ----compare-files-diff, eval=FALSE-------------------------------------- # # On Unix OSes # system(paste0("diff ", rich_sparse_file, outfile)) # # On windows # system(paste0("FC ", rich_sparse_file, outfile)) biomformat/inst/doc/biomformat.Rmd0000644000175400017540000001677313001263737020304 0ustar00biocbuildbiocbuild--- title: "The biomformat package vignette" author: "Paul J. McMurdie" output: BiocStyle::html_document: fig_height: 7 fig_width: 10 toc: yes toc_depth: 2 number_sections: true vignette: > %\VignetteIndexEntry{The biomformat package Vignette} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- `r library("knitr")` `r opts_chunk$set(cache=FALSE, fig.width=9, message=FALSE, warning=FALSE)` ## Paul J. McMurdie & Joseph N. Paulson If you find *biomformat* and/or its tutorials useful, please acknowledge and cite *biomformat* in your publications: Paul J. McMurdie and Joseph N Paulson (2015). *biomformat: An interface package for the BIOM file format.* R/Bioconductor package version 1.0.0. ## [phyloseq Home Page](https://github.com/joey711/biomformat/) phyloseq has many more utilities for interacting with this kind of data than are provided in this I/O-oriented package. ## Motivation for the biomformat package This is an [R Markdown document](http://www.rstudio.com/ide/docs/r_markdown). Markdown is a simple formatting syntax for authoring web pages. Further details on [R markdown here](http://www.rstudio.com/ide/docs/r_markdown). The BIOM file format (canonically pronounced "biome") is designed to be a general-use format for representing biological sample by observation contingency tables. BIOM is a recognized standard for [the Earth Microbiome Project](http://www.earthmicrobiome.org/) and is a [Genomics Standards Consortium](http://gensc.org/) candidate project. Please see [the biom-format home page](http://biom-format.org/) for more details. This demo is designed to provide an overview of the biom package to get you started using it quickly. The biom package itself is intended to be a utility package that will be depended-upon by other packages in the future. It provides I/O functionality, and functions to make it easier to with data from biom-format files. It does not (and probably should not) provide statistical analysis functions. However, it does provide tools to access data from BIOM format files in ways that are extremely common in R (such as `"data.frame"`, `"matrix"`, and `"Matrix"` classes). **Package versions** at the time (`r date()`) of this build: ```{r packages} library("biomformat"); packageVersion("biomformat") ``` # Read BIOM format Here is an example importing BIOM formats of different types into R using the `read_biom` function. The resulting data objects in R are given names beginning with `x`. ```{r read-biom-examples} min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") min_sparse_file = system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") rich_dense_file = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") rich_dense_char = system.file("extdata", "rich_dense_char.biom", package = "biomformat") rich_sparse_char = system.file("extdata", "rich_sparse_char.biom", package = "biomformat") x1 = read_biom(min_dense_file) x2 = read_biom(min_sparse_file) x3 = read_biom(rich_dense_file) x4 = read_biom(rich_sparse_file) x5 = read_biom(rich_dense_char) x6 = read_biom(rich_sparse_char) x1 ``` It would be hard to interpret and wasteful of RAM to stream all the data from a BIOM format file to the standard out when printed with `print` or `show` methods. Instead, a brief summary of the contents BIOM data is shown. # Access BIOM data To get access to the data in a familiar form appropriate for many standard R functions, we will need to use accessor functions, also included in the biom package. ### Core observation data The core "observation" data is stored in either sparse or dense matrices in the BIOM format file, and sparse matrix support is carried through on the R side via [the Matrix package](http://cran.r-project.org/web/packages/Matrix/index.html). The variables `x1` and `x2`, assigned above from BIOM files, have similar (but not identical) data. They are small enough to observe directly as tables in standard output in an R session: ```{r accessor-examples-table} biom_data(x1) biom_data(x2) ``` As you can see above, `x1` and `x2` are represented in R by slightly different matrix classes, assigned automatically based on the data. However, most operations in R will understand this automatically and you should not have to worry about the precise matrix class. However, if the R function you are attempting to use is having a problem with these fancier classes, you can easily coerce the data to the simple, standard `"matrix"` class using the `as` function: ```{r matrix-coercion} as(biom_data(x2), "matrix") ``` ### Observation Metadata Observation metadata is metadata associated with the individual units being counted/recorded in a sample, as opposed to measurements of properties of the samples themselves. For microbiome census data, for instance, observation metadata is often a taxonomic classification and anything else that might be known about a particular OTU/species. For other types of data, it might be metadata known about a particular genome, gene family, pathway, etc. In this case, the observations are counts of OTUs and the metadata is taxonomic classification, if present. The absence of observation metadata is also supported, as we see for the minimal cases of `x1` and `x2`, where we see `r NULL` instead. ```{r observ-meta} observation_metadata(x1) observation_metadata(x2) observation_metadata(x3) observation_metadata(x4)[1:2, 1:3] class(observation_metadata(x4)) ``` ### Sample Metadata Similarly, we can access metadata -- if available -- that describe properties of the samples. We access this sample metadata using the `sample_metadata` function. ```{r plot-examples} sample_metadata(x1) sample_metadata(x2) sample_metadata(x3) sample_metadata(x4)[1:2, 1:3] class(sample_metadata(x4)) ``` ### Plots The data really is accessible to other R functions. ```{r plot} plot(biom_data(x4)) boxplot(as(biom_data(x4), "vector")) heatmap(as(biom_data(x4), "matrix")) ``` # Write BIOM format The biom objects in R can be written to a file/connection using the `write_biom` function. If you modified the biom object, this may still work as well, but no guarantees about this as we are still working on internal checks. The following example writes `x4` to a temporary file, then reads it back using `read_biom` and stores it as variable `y`. The exact comparison of these two objects using the `identical` function shows that they are exactly the same in R. ```{r write-biom-examples} outfile = tempfile() write_biom(x4, outfile) y = read_biom(outfile) identical(x4, y) ``` Furthermore, it is possible to invoke standard operating system commands through the R `system` function -- in this case to invoke the `diff` command available on Unix-like systems or the `FC` command on Windows -- in order to compare the original and temporary files directly. Note that this is shown here for convenience, but not automatically run with the rest of the script because of the OS-dependence. During development, though, this same command is tested privately and no differences are reported between the files. ```{r compare-files-diff, eval=FALSE} # On Unix OSes system(paste0("diff ", rich_sparse_file, outfile)) # On windows system(paste0("FC ", rich_sparse_file, outfile)) ``` biomformat/inst/doc/biomformat.html0000644000175400017540000031602313001263737020515 0ustar00biocbuildbiocbuild The biomformat package vignette

Contents

0.1 Paul J. McMurdie & Joseph N. Paulson

If you find biomformat and/or its tutorials useful, please acknowledge and cite biomformat in your publications:

Paul J. McMurdie and Joseph N Paulson (2015). biomformat: An interface package for the BIOM file format. R/Bioconductor package version 1.0.0.

0.2 phyloseq Home Page

phyloseq has many more utilities for interacting with this kind of data than are provided in this I/O-oriented package.

0.3 Motivation for the biomformat package

This is an R Markdown document. Markdown is a simple formatting syntax for authoring web pages. Further details on R markdown here.

The BIOM file format (canonically pronounced “biome”) is designed to be a general-use format for representing biological sample by observation contingency tables. BIOM is a recognized standard for the Earth Microbiome Project and is a Genomics Standards Consortium candidate project. Please see the biom-format home page for more details.

This demo is designed to provide an overview of the biom package to get you started using it quickly. The biom package itself is intended to be a utility package that will be depended-upon by other packages in the future. It provides I/O functionality, and functions to make it easier to with data from biom-format files. It does not (and probably should not) provide statistical analysis functions. However, it does provide tools to access data from BIOM format files in ways that are extremely common in R (such as "data.frame", "matrix", and "Matrix" classes).

Package versions at the time (Mon Oct 17 20:07:58 2016) of this build:

library("biomformat"); packageVersion("biomformat")
## [1] '1.2.0'

1 Read BIOM format

Here is an example importing BIOM formats of different types into R using the read_biom function. The resulting data objects in R are given names beginning with x.

min_dense_file   = system.file("extdata", "min_dense_otu_table.biom", 
                               package = "biomformat")
min_sparse_file  = system.file("extdata", "min_sparse_otu_table.biom", 
                               package = "biomformat")
rich_dense_file  = system.file("extdata", "rich_dense_otu_table.biom", 
                               package = "biomformat")
rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", 
                               package = "biomformat")
min_dense_file   = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat")
rich_dense_char  = system.file("extdata", "rich_dense_char.biom", package = "biomformat")
rich_sparse_char  = system.file("extdata", "rich_sparse_char.biom", package = "biomformat")
x1 = read_biom(min_dense_file)
x2 = read_biom(min_sparse_file)
x3 = read_biom(rich_dense_file)
x4 = read_biom(rich_sparse_file)
x5 = read_biom(rich_dense_char)
x6 = read_biom(rich_sparse_char)
x1
## biom object. 
## type: OTU table 
## matrix_type: dense 
## 5 rows and 6 columns

It would be hard to interpret and wasteful of RAM to stream all the data from a BIOM format file to the standard out when printed with print or show methods. Instead, a brief summary of the contents BIOM data is shown.

2 Access BIOM data

To get access to the data in a familiar form appropriate for many standard R functions, we will need to use accessor functions, also included in the biom package.

2.0.1 Core observation data

The core “observation” data is stored in either sparse or dense matrices in the BIOM format file, and sparse matrix support is carried through on the R side via the Matrix package. The variables x1 and x2, assigned above from BIOM files, have similar (but not identical) data. They are small enough to observe directly as tables in standard output in an R session:

biom_data(x1)
## 5 x 6 Matrix of class "dgeMatrix"
##          Sample1 Sample2 Sample3 Sample4 Sample5 Sample6
## GG_OTU_1       0       0       1       0       0       0
## GG_OTU_2       5       1       0       2       3       1
## GG_OTU_3       0       0       1       4       2       0
## GG_OTU_4       2       1       1       0       0       1
## GG_OTU_5       0       1       1       0       0       0
biom_data(x2)
## 5 x 6 sparse Matrix of class "dgCMatrix"
##          Sample1 Sample2 Sample3 Sample4 Sample5 Sample6
## GG_OTU_1       .       .       1       .       .       .
## GG_OTU_2       5       1       .       2       3       1
## GG_OTU_3       .       .       1       4       .       2
## GG_OTU_4       2       1       1       .       .       1
## GG_OTU_5       .       1       1       .       .       .

As you can see above, x1 and x2 are represented in R by slightly different matrix classes, assigned automatically based on the data. However, most operations in R will understand this automatically and you should not have to worry about the precise matrix class. However, if the R function you are attempting to use is having a problem with these fancier classes, you can easily coerce the data to the simple, standard "matrix" class using the as function:

as(biom_data(x2), "matrix")
##          Sample1 Sample2 Sample3 Sample4 Sample5 Sample6
## GG_OTU_1       0       0       1       0       0       0
## GG_OTU_2       5       1       0       2       3       1
## GG_OTU_3       0       0       1       4       0       2
## GG_OTU_4       2       1       1       0       0       1
## GG_OTU_5       0       1       1       0       0       0

2.0.2 Observation Metadata

Observation metadata is metadata associated with the individual units being counted/recorded in a sample, as opposed to measurements of properties of the samples themselves. For microbiome census data, for instance, observation metadata is often a taxonomic classification and anything else that might be known about a particular OTU/species. For other types of data, it might be metadata known about a particular genome, gene family, pathway, etc. In this case, the observations are counts of OTUs and the metadata is taxonomic classification, if present. The absence of observation metadata is also supported, as we see for the minimal cases of x1 and x2, where we see instead.

observation_metadata(x1)
## NULL
observation_metadata(x2)
## NULL
observation_metadata(x3)
##            taxonomy1         taxonomy2              taxonomy3
## GG_OTU_1 k__Bacteria p__Proteobacteria c__Gammaproteobacteria
## GG_OTU_2 k__Bacteria  p__Cyanobacteria    c__Nostocophycideae
## GG_OTU_3  k__Archaea  p__Euryarchaeota     c__Methanomicrobia
## GG_OTU_4 k__Bacteria     p__Firmicutes          c__Clostridia
## GG_OTU_5 k__Bacteria p__Proteobacteria c__Gammaproteobacteria
##                     taxonomy4             taxonomy5         taxonomy6
## GG_OTU_1 o__Enterobacteriales f__Enterobacteriaceae    g__Escherichia
## GG_OTU_2        o__Nostocales        f__Nostocaceae g__Dolichospermum
## GG_OTU_3 o__Methanosarcinales f__Methanosarcinaceae g__Methanosarcina
## GG_OTU_4   o__Halanaerobiales   f__Halanaerobiaceae  g__Halanaerobium
## GG_OTU_5 o__Enterobacteriales f__Enterobacteriaceae    g__Escherichia
##                                taxonomy7
## GG_OTU_1                             s__
## GG_OTU_2                             s__
## GG_OTU_3                             s__
## GG_OTU_4 s__Halanaerobiumsaccharolyticum
## GG_OTU_5                             s__
observation_metadata(x4)[1:2, 1:3]
##            taxonomy1         taxonomy2              taxonomy3
## GG_OTU_1 k__Bacteria p__Proteobacteria c__Gammaproteobacteria
## GG_OTU_2 k__Bacteria  p__Cyanobacteria    c__Nostocophycideae
class(observation_metadata(x4))
## [1] "data.frame"

2.0.3 Sample Metadata

Similarly, we can access metadata – if available – that describe properties of the samples. We access this sample metadata using the sample_metadata function.

sample_metadata(x1)
## NULL
sample_metadata(x2)
## NULL
sample_metadata(x3)
##         BarcodeSequence  LinkerPrimerSequence BODY_SITE Description
## Sample1    CGCTTATCGAGA CATGCTGCCTCCCGTAGGAGT       gut   human gut
## Sample2    CATACCAGTAGC CATGCTGCCTCCCGTAGGAGT       gut   human gut
## Sample3    CTCTCTACCTGT CATGCTGCCTCCCGTAGGAGT       gut   human gut
## Sample4    CTCTCGGCCTGT CATGCTGCCTCCCGTAGGAGT      skin  human skin
## Sample5    CTCTCTACCAAT CATGCTGCCTCCCGTAGGAGT      skin  human skin
## Sample6    CTAACTACCAAT CATGCTGCCTCCCGTAGGAGT      skin  human skin
sample_metadata(x4)[1:2, 1:3]
##         BarcodeSequence  LinkerPrimerSequence BODY_SITE
## Sample1    CGCTTATCGAGA CATGCTGCCTCCCGTAGGAGT       gut
## Sample2    CATACCAGTAGC CATGCTGCCTCCCGTAGGAGT       gut
class(sample_metadata(x4))
## [1] "data.frame"

2.0.4 Plots

The data really is accessible to other R functions.

plot(biom_data(x4))

boxplot(as(biom_data(x4), "vector"))

heatmap(as(biom_data(x4), "matrix"))

3 Write BIOM format

The biom objects in R can be written to a file/connection using the write_biom function. If you modified the biom object, this may still work as well, but no guarantees about this as we are still working on internal checks. The following example writes x4 to a temporary file, then reads it back using read_biom and stores it as variable y. The exact comparison of these two objects using the identical function shows that they are exactly the same in R.

outfile = tempfile()
write_biom(x4, outfile)
y = read_biom(outfile)
identical(x4, y)
## [1] FALSE

Furthermore, it is possible to invoke standard operating system commands through the R system function – in this case to invoke the diff command available on Unix-like systems or the FC command on Windows – in order to compare the original and temporary files directly. Note that this is shown here for convenience, but not automatically run with the rest of the script because of the OS-dependence. During development, though, this same command is tested privately and no differences are reported between the files.

# On Unix OSes
system(paste0("diff ", rich_sparse_file, outfile))
# On windows
system(paste0("FC ", rich_sparse_file, outfile))
biomformat/inst/extdata/0000755000175400017540000000000013001231115016331 5ustar00biocbuildbiocbuildbiomformat/inst/extdata/min_dense_otu_table.biom0000644000175400017540000000217413001231115023204 0ustar00biocbuildbiocbuild { "id":null, "format": "Biological Observation Matrix 1.0.0-dev", "format_url": "http://biom-format.org", "type": "OTU table", "generated_by": "QIIME revision XYZ", "date": "2011-12-19T19:00:00", "rows":[ {"id":"GG_OTU_1", "metadata":null}, {"id":"GG_OTU_2", "metadata":null}, {"id":"GG_OTU_3", "metadata":null}, {"id":"GG_OTU_4", "metadata":null}, {"id":"GG_OTU_5", "metadata":null} ], "columns": [ {"id":"Sample1", "metadata":null}, {"id":"Sample2", "metadata":null}, {"id":"Sample3", "metadata":null}, {"id":"Sample4", "metadata":null}, {"id":"Sample5", "metadata":null}, {"id":"Sample6", "metadata":null} ], "matrix_type": "dense", "matrix_element_type": "int", "shape": [5,6], "data": [[0,0,1,0,0,0], [5,1,0,2,3,1], [0,0,1,4,2,0], [2,1,1,0,0,1], [0,1,1,0,0,0]] } biomformat/inst/extdata/min_sparse_otu_table.biom0000644000175400017540000000254113001231113023377 0ustar00biocbuildbiocbuild { "id":null, "format": "Biological Observation Matrix 1.0.0-dev", "format_url": "http://biom-format.org", "type": "OTU table", "generated_by": "QIIME revision XYZ", "date": "2011-12-19T19:00:00", "rows":[ {"id":"GG_OTU_1", "metadata":null}, {"id":"GG_OTU_2", "metadata":null}, {"id":"GG_OTU_3", "metadata":null}, {"id":"GG_OTU_4", "metadata":null}, {"id":"GG_OTU_5", "metadata":null} ], "columns": [ {"id":"Sample1", "metadata":null}, {"id":"Sample2", "metadata":null}, {"id":"Sample3", "metadata":null}, {"id":"Sample4", "metadata":null}, {"id":"Sample5", "metadata":null}, {"id":"Sample6", "metadata":null} ], "matrix_type": "sparse", "matrix_element_type": "int", "shape": [5, 6], "data":[[0,2,1], [1,0,5], [1,1,1], [1,3,2], [1,4,3], [1,5,1], [2,2,1], [2,3,4], [2,5,2], [3,0,2], [3,1,1], [3,2,1], [3,5,1], [4,1,1], [4,2,1] ] } biomformat/inst/extdata/min_sparse_otu_table_hdf5.biom0000644000175400017540000010201013001231114024276 0ustar00biocbuildbiocbuildHDF  `  xTREE`HEAPX observationsample8 @id ` @type ` H format-url` P format-version@ H generated-by` Hcreation-date` H shape@ 0 nnz@TREEhHEAPX8matrixidsmetadatagroup-metadata GCOL No Table ID otu tablehttp://biom-format.orgexample2014-07-29T16:15:43.318377GG_OTU_4GG_OTU_5GG_OTU_3 GG_OTU_2 GG_OTU_1 Sample4 Sample5 Sample6Sample3Sample2Sample18SNODxQQSTREE&HEAPX dataindicesindptr8SNOD(NNQBK L@N ?@4 4 deflateSPTREE'8(SNOD80x9x^c``AB P>8htu4} Tx^cbF fb fb&(ĆɃ@|%x^c````b6 b^ b-x^```H`. 1a|N4>B泡فg8Yx^c`0?ؓG @i(.} Tx^cd```bF(LH|al$ x^c````bV bn b~ `7x^cg``H` fGh|4>/  deflateH1SXTREE!_(  deflate:SXTREE( deflateCSPTREE"( L@NTREE&HEAPX`NPNQTREE&HEAPX QPQSTREEVHEAPX8Smatrixidsmetadatagroup-metadata `TVTREEaHEAPX Vdataindicesindptr8SNOD(HpHu8T`TV~~Ѐ ?@4 4 deflatePYSPTREE"(SNOD@Xbl  deflatecSXTREE (  deflatemSXTREE( deflateXvSPTREE$)~ЀTREEaHEAPXPpTREEaHEAPXPbiomformat/inst/extdata/rich_dense_char.biom0000644000175400017540000000644413001231113022307 0ustar00biocbuildbiocbuild{ "id":null, "format": "Biological Observation Matrix 1.0.0-dev", "format_url": "http://biom-format.org", "type": "OTU table", "generated_by": "QIIME revision XYZ", "date": "2011-12-19T19:00:00", "rows":[ {"id":"GG_OTU_1", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}}, {"id":"GG_OTU_2", "metadata":{"taxonomy":["k__Bacteria", "p__Cyanobacteria", "c__Nostocophycideae", "o__Nostocales", "f__Nostocaceae", "g__Dolichospermum", "s__"]}}, {"id":"GG_OTU_3", "metadata":{"taxonomy":["k__Archaea", "p__Euryarchaeota", "c__Methanomicrobia", "o__Methanosarcinales", "f__Methanosarcinaceae", "g__Methanosarcina", "s__"]}}, {"id":"GG_OTU_4", "metadata":{"taxonomy":["k__Bacteria", "p__Firmicutes", "c__Clostridia", "o__Halanaerobiales", "f__Halanaerobiaceae", "g__Halanaerobium", "s__Halanaerobiumsaccharolyticum"]}}, {"id":"GG_OTU_5", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}} ], "columns":[ {"id":"Sample1", "metadata":{ "BarcodeSequence":"CGCTTATCGAGA", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample2", "metadata":{ "BarcodeSequence":"CATACCAGTAGC", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample3", "metadata":{ "BarcodeSequence":"CTCTCTACCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample4", "metadata":{ "BarcodeSequence":"CTCTCGGCCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample5", "metadata":{ "BarcodeSequence":"CTCTCTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample6", "metadata":{ "BarcodeSequence":"CTAACTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}} ], "matrix_type": "dense", "matrix_element_type": "unicode", "shape": [5,6], "data": [["0","sky","1","0","clouds","0"], ["5","1","0","2","3","1"], ["0","lightning","1","4","2","0"], ["2","1","1","0","gray","1"], ["0","1","1","0","0","bottle"]] } biomformat/inst/extdata/rich_dense_otu_table.biom0000644000175400017540000000631613001231115023350 0ustar00biocbuildbiocbuild{ "id":null, "format": "Biological Observation Matrix 1.0.0-dev", "format_url": "http://biom-format.org", "type": "OTU table", "generated_by": "QIIME revision XYZ", "date": "2011-12-19T19:00:00", "rows":[ {"id":"GG_OTU_1", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}}, {"id":"GG_OTU_2", "metadata":{"taxonomy":["k__Bacteria", "p__Cyanobacteria", "c__Nostocophycideae", "o__Nostocales", "f__Nostocaceae", "g__Dolichospermum", "s__"]}}, {"id":"GG_OTU_3", "metadata":{"taxonomy":["k__Archaea", "p__Euryarchaeota", "c__Methanomicrobia", "o__Methanosarcinales", "f__Methanosarcinaceae", "g__Methanosarcina", "s__"]}}, {"id":"GG_OTU_4", "metadata":{"taxonomy":["k__Bacteria", "p__Firmicutes", "c__Clostridia", "o__Halanaerobiales", "f__Halanaerobiaceae", "g__Halanaerobium", "s__Halanaerobiumsaccharolyticum"]}}, {"id":"GG_OTU_5", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}} ], "columns":[ {"id":"Sample1", "metadata":{ "BarcodeSequence":"CGCTTATCGAGA", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample2", "metadata":{ "BarcodeSequence":"CATACCAGTAGC", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample3", "metadata":{ "BarcodeSequence":"CTCTCTACCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample4", "metadata":{ "BarcodeSequence":"CTCTCGGCCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample5", "metadata":{ "BarcodeSequence":"CTCTCTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample6", "metadata":{ "BarcodeSequence":"CTAACTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}} ], "matrix_type": "dense", "matrix_element_type": "int", "shape": [5,6], "data": [[0,0,1,0,0,0], [5,1,0,2,3,1], [0,0,1,4,2,0], [2,1,1,0,0,1], [0,1,1,0,0,0]] } biomformat/inst/extdata/rich_sparse_char.biom0000644000175400017540000000667513001231113022514 0ustar00biocbuildbiocbuild{ "id":null, "format": "Biological Observation Matrix 1.0.0-dev", "format_url": "http://biom-format.org", "type": "OTU table", "generated_by": "QIIME revision XYZ", "date": "2011-12-19T19:00:00", "rows":[ {"id":"GG_OTU_1", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}}, {"id":"GG_OTU_2", "metadata":{"taxonomy":["k__Bacteria", "p__Cyanobacteria", "c__Nostocophycideae", "o__Nostocales", "f__Nostocaceae", "g__Dolichospermum", "s__"]}}, {"id":"GG_OTU_3", "metadata":{"taxonomy":["k__Archaea", "p__Euryarchaeota", "c__Methanomicrobia", "o__Methanosarcinales", "f__Methanosarcinaceae", "g__Methanosarcina", "s__"]}}, {"id":"GG_OTU_4", "metadata":{"taxonomy":["k__Bacteria", "p__Firmicutes", "c__Clostridia", "o__Halanaerobiales", "f__Halanaerobiaceae", "g__Halanaerobium", "s__Halanaerobiumsaccharolyticum"]}}, {"id":"GG_OTU_5", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}} ], "columns":[ {"id":"Sample1", "metadata":{ "BarcodeSequence":"CGCTTATCGAGA", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample2", "metadata":{ "BarcodeSequence":"CATACCAGTAGC", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample3", "metadata":{ "BarcodeSequence":"CTCTCTACCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample4", "metadata":{ "BarcodeSequence":"CTCTCGGCCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample5", "metadata":{ "BarcodeSequence":"CTCTCTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample6", "metadata":{ "BarcodeSequence":"CTAACTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}} ], "matrix_type": "sparse", "matrix_element_type": "unicode", "shape": [5, 6], "data":[[0,2,"sky"], [1,0,"clouds"], [1,1,"lightning"], [1,3,"gray"], [1,4,"bottle"], [1,5,"1"], [2,2,"1"], [2,3,"4"], [2,5,"2"], [3,0,"2"], [3,1,"1"], [3,2,"1"], [3,5,"1"], [4,1,"1"], [4,2,"1"] ] } biomformat/inst/extdata/rich_sparse_otu_table.biom0000644000175400017540000000660413001231114023546 0ustar00biocbuildbiocbuild{ "id":null, "format": "Biological Observation Matrix 1.0.0-dev", "format_url": "http://biom-format.org", "type": "OTU table", "generated_by": "QIIME revision XYZ", "date": "2011-12-19T19:00:00", "rows":[ {"id":"GG_OTU_1", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}}, {"id":"GG_OTU_2", "metadata":{"taxonomy":["k__Bacteria", "p__Cyanobacteria", "c__Nostocophycideae", "o__Nostocales", "f__Nostocaceae", "g__Dolichospermum", "s__"]}}, {"id":"GG_OTU_3", "metadata":{"taxonomy":["k__Archaea", "p__Euryarchaeota", "c__Methanomicrobia", "o__Methanosarcinales", "f__Methanosarcinaceae", "g__Methanosarcina", "s__"]}}, {"id":"GG_OTU_4", "metadata":{"taxonomy":["k__Bacteria", "p__Firmicutes", "c__Clostridia", "o__Halanaerobiales", "f__Halanaerobiaceae", "g__Halanaerobium", "s__Halanaerobiumsaccharolyticum"]}}, {"id":"GG_OTU_5", "metadata":{"taxonomy":["k__Bacteria", "p__Proteobacteria", "c__Gammaproteobacteria", "o__Enterobacteriales", "f__Enterobacteriaceae", "g__Escherichia", "s__"]}} ], "columns":[ {"id":"Sample1", "metadata":{ "BarcodeSequence":"CGCTTATCGAGA", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample2", "metadata":{ "BarcodeSequence":"CATACCAGTAGC", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample3", "metadata":{ "BarcodeSequence":"CTCTCTACCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"gut", "Description":"human gut"}}, {"id":"Sample4", "metadata":{ "BarcodeSequence":"CTCTCGGCCTGT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample5", "metadata":{ "BarcodeSequence":"CTCTCTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}}, {"id":"Sample6", "metadata":{ "BarcodeSequence":"CTAACTACCAAT", "LinkerPrimerSequence":"CATGCTGCCTCCCGTAGGAGT", "BODY_SITE":"skin", "Description":"human skin"}} ], "matrix_type": "sparse", "matrix_element_type": "int", "shape": [5, 6], "data":[[0,2,1], [1,0,5], [1,1,1], [1,3,2], [1,4,3], [1,5,1], [2,2,1], [2,3,4], [2,5,2], [3,0,2], [3,1,1], [3,2,1], [3,5,1], [4,1,1], [4,2,1] ] } biomformat/inst/extdata/rich_sparse_otu_table_hdf5.biom0000644000175400017540000013355013001231114024455 0ustar00biocbuildbiocbuildHDF  h`  xTREE`HEAPX observationsample8 @id ` @type ` H format-url` P format-version@ H generated-by` Hcreation-date` H shape@ 0 nnz@TREEhHEAPX8matrixidsmetadatagroup-metadata GCOL No Table ID otu tablehttp://biom-format.orgexample2014-07-29T16:16:36.617320GG_OTU_4GG_OTU_5GG_OTU_3 GG_OTU_2 GG_OTU_1 f__Methanosarcinaceae g__Methanosarcina s__ k__Bacteria p__Firmicutes c__Clostridiao__Halanaerobialesf__Halanaerobiaceaeg__Halanaerobiums__Halanaerobiumsaccharolyticum k__Bacteriap__Proteobacteriac__Gammaproteobacteriao__Enterobacterialesf__Enterobacteriaceaeg__Escherichias__c__Nostocophycideae o__Nostocalesf__Nostocaceaeg__Dolichospermum s__! k__Archaea"p__Euryarchaeota#c__Methanomicrobia$o__Methanosarcinales%g__Escherichia&s__' k__Bacteria(p__Cyanobacteria)o__Enterobacteriales*f__Enterobacteriaceae+c__Gammaproteobacteria,p__Proteobacteria- k__Bacteria.Sample4/Sample50Sample61Sample32Sample23Sample14CATGCTGCCTCCCGTAGGAGT5CATGCTGCCTCCCGTAGGAGT6CATGCTGCCTCCCGTAGGAGT7CATGCTGCCTCCCGTAGGAGT8CATGCTGCCTCCCGTAGGAGT9CATGCTGCCTCCCGTAGGAGT: CTCTCGGCCTGT; CTCTCTACCAAT< CTAACTACCAAT= CTCTCTACCTGT> CATACCAGTAGC? CGCTTATCGAGA@ human skinA human skinB human skinC human gutD human gutE human gutFskinGskinHskinIgutJgutKguthSNOD^0^P`TREE&HEAPX dataindicesindptr8SNOD(H[p[]BK L@N ?@4 4 deflateDSPTREE'8(SNOD80x9x^c``AB P>8htu4} Tx^cbF fb fb&(ĆɃ@|%x^c````b6 b^ b-x^```H`. 1a|N4>B泡فg8Yx^]Y @Cպk]N&XxI)~/24L25LZR_Fk3o}ȼy22?" %\VignetteIndexEntry{The biomformat package Vignette} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- `r library("knitr")` `r opts_chunk$set(cache=FALSE, fig.width=9, message=FALSE, warning=FALSE)` ## Paul J. McMurdie & Joseph N. Paulson If you find *biomformat* and/or its tutorials useful, please acknowledge and cite *biomformat* in your publications: Paul J. McMurdie and Joseph N Paulson (2015). *biomformat: An interface package for the BIOM file format.* R/Bioconductor package version 1.0.0. ## [phyloseq Home Page](https://github.com/joey711/biomformat/) phyloseq has many more utilities for interacting with this kind of data than are provided in this I/O-oriented package. ## Motivation for the biomformat package This is an [R Markdown document](http://www.rstudio.com/ide/docs/r_markdown). Markdown is a simple formatting syntax for authoring web pages. Further details on [R markdown here](http://www.rstudio.com/ide/docs/r_markdown). The BIOM file format (canonically pronounced "biome") is designed to be a general-use format for representing biological sample by observation contingency tables. BIOM is a recognized standard for [the Earth Microbiome Project](http://www.earthmicrobiome.org/) and is a [Genomics Standards Consortium](http://gensc.org/) candidate project. Please see [the biom-format home page](http://biom-format.org/) for more details. This demo is designed to provide an overview of the biom package to get you started using it quickly. The biom package itself is intended to be a utility package that will be depended-upon by other packages in the future. It provides I/O functionality, and functions to make it easier to with data from biom-format files. It does not (and probably should not) provide statistical analysis functions. However, it does provide tools to access data from BIOM format files in ways that are extremely common in R (such as `"data.frame"`, `"matrix"`, and `"Matrix"` classes). **Package versions** at the time (`r date()`) of this build: ```{r packages} library("biomformat"); packageVersion("biomformat") ``` # Read BIOM format Here is an example importing BIOM formats of different types into R using the `read_biom` function. The resulting data objects in R are given names beginning with `x`. ```{r read-biom-examples} min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") min_sparse_file = system.file("extdata", "min_sparse_otu_table.biom", package = "biomformat") rich_dense_file = system.file("extdata", "rich_dense_otu_table.biom", package = "biomformat") rich_sparse_file = system.file("extdata", "rich_sparse_otu_table.biom", package = "biomformat") min_dense_file = system.file("extdata", "min_dense_otu_table.biom", package = "biomformat") rich_dense_char = system.file("extdata", "rich_dense_char.biom", package = "biomformat") rich_sparse_char = system.file("extdata", "rich_sparse_char.biom", package = "biomformat") x1 = read_biom(min_dense_file) x2 = read_biom(min_sparse_file) x3 = read_biom(rich_dense_file) x4 = read_biom(rich_sparse_file) x5 = read_biom(rich_dense_char) x6 = read_biom(rich_sparse_char) x1 ``` It would be hard to interpret and wasteful of RAM to stream all the data from a BIOM format file to the standard out when printed with `print` or `show` methods. Instead, a brief summary of the contents BIOM data is shown. # Access BIOM data To get access to the data in a familiar form appropriate for many standard R functions, we will need to use accessor functions, also included in the biom package. ### Core observation data The core "observation" data is stored in either sparse or dense matrices in the BIOM format file, and sparse matrix support is carried through on the R side via [the Matrix package](http://cran.r-project.org/web/packages/Matrix/index.html). The variables `x1` and `x2`, assigned above from BIOM files, have similar (but not identical) data. They are small enough to observe directly as tables in standard output in an R session: ```{r accessor-examples-table} biom_data(x1) biom_data(x2) ``` As you can see above, `x1` and `x2` are represented in R by slightly different matrix classes, assigned automatically based on the data. However, most operations in R will understand this automatically and you should not have to worry about the precise matrix class. However, if the R function you are attempting to use is having a problem with these fancier classes, you can easily coerce the data to the simple, standard `"matrix"` class using the `as` function: ```{r matrix-coercion} as(biom_data(x2), "matrix") ``` ### Observation Metadata Observation metadata is metadata associated with the individual units being counted/recorded in a sample, as opposed to measurements of properties of the samples themselves. For microbiome census data, for instance, observation metadata is often a taxonomic classification and anything else that might be known about a particular OTU/species. For other types of data, it might be metadata known about a particular genome, gene family, pathway, etc. In this case, the observations are counts of OTUs and the metadata is taxonomic classification, if present. The absence of observation metadata is also supported, as we see for the minimal cases of `x1` and `x2`, where we see `r NULL` instead. ```{r observ-meta} observation_metadata(x1) observation_metadata(x2) observation_metadata(x3) observation_metadata(x4)[1:2, 1:3] class(observation_metadata(x4)) ``` ### Sample Metadata Similarly, we can access metadata -- if available -- that describe properties of the samples. We access this sample metadata using the `sample_metadata` function. ```{r plot-examples} sample_metadata(x1) sample_metadata(x2) sample_metadata(x3) sample_metadata(x4)[1:2, 1:3] class(sample_metadata(x4)) ``` ### Plots The data really is accessible to other R functions. ```{r plot} plot(biom_data(x4)) boxplot(as(biom_data(x4), "vector")) heatmap(as(biom_data(x4), "matrix")) ``` # Write BIOM format The biom objects in R can be written to a file/connection using the `write_biom` function. If you modified the biom object, this may still work as well, but no guarantees about this as we are still working on internal checks. The following example writes `x4` to a temporary file, then reads it back using `read_biom` and stores it as variable `y`. The exact comparison of these two objects using the `identical` function shows that they are exactly the same in R. ```{r write-biom-examples} outfile = tempfile() write_biom(x4, outfile) y = read_biom(outfile) identical(x4, y) ``` Furthermore, it is possible to invoke standard operating system commands through the R `system` function -- in this case to invoke the `diff` command available on Unix-like systems or the `FC` command on Windows -- in order to compare the original and temporary files directly. Note that this is shown here for convenience, but not automatically run with the rest of the script because of the OS-dependence. During development, though, this same command is tested privately and no differences are reported between the files. ```{r compare-files-diff, eval=FALSE} # On Unix OSes system(paste0("diff ", rich_sparse_file, outfile)) # On windows system(paste0("FC ", rich_sparse_file, outfile)) ```