During the last 3 years, a number of approaches for the normalization of RNA sequencing data have emerged in the literature, differing both in the type of bias adjustment and in the statistical strategy adopted. However, as data continue to accumulate, there has been no clear consensus on the appropriate normalization method to be used or the impact of a chosen method on the downstream analysis. In this work, we focus on a comprehensive comparison of seven recently proposed normalization methods for the differential analysis of RNA-seq data, with an emphasis on the use of varied real and simulated datasets involving different species and experimental designs to represent data characteristics commonly observed in practice. Based on this comparison study, we propose practical recommendations on the appropriate normalization method to be used and its impact on the differential analysis of RNA-seq data.

BACKGROUND: Most microarray studies are made using labelling with one or two dyes which allows the hybridization of one or two samples on the same slide. In such experiments, the most frequently used dyes are Cy3 and Cy5. Recent improvements in the technology (dye-labelling, scanner and, image analysis) allow hybridization up to four samples simultaneously. The two additional dyes are Alexa488 and Alexa494. The triple-target or four-target technology is very promising, since it allows more flexibility in the design of experiments, an increase in the statistical power when comparing gene expressions induced by different conditions and a scaled down number of slides. However, there have been few methods proposed for statistical analysis of such data. Moreover the lowess correction of the global dye effect is available for only two-color experiments, and even if its application can be derived, it does not allow simultaneous correction of the raw data. RESULTS: We propose a two-step normalization procedure for triple-target experiments. First the dye bleeding is evaluated and corrected if necessary. Then the signal in each channel is normalized using a generalized lowess procedure to correct a global dye bias. The normalization procedure is validated using triple-self experiments and by comparing the results of triple-target and two-color experiments. Although the focus is on triple-target microarrays, the proposed method can be used to normalize p differently labelled targets co-hybridized on a same array, for any value of p greater than 2. CONCLUSION: The proposed normalization procedure is effective: the technical biases are reduced, the number of false positives is under control in the analysis of differentially expressed genes, and the triple-target experiments are more powerful than the corresponding two-color experiments. There is room for improving the microarray experiments by simultaneously hybridizing more than two samples.

BACKGROUND: In individually dye-balanced microarray designs, each biological sample is hybridized on two different slides, once with Cy3 and once with Cy5. While this strategy ensures an automatic correction of the gene-specific labelling bias, it also induces dependencies between log-ratio measurements that must be taken into account in the statistical analysis. RESULTS: We present two original statistical procedures for the statistical analysis of individually balanced designs. These procedures are compared with the usual ML and REML mixed model procedures proposed in most statistical toolboxes, on both simulated and real data. CONCLUSION: The UP procedure we propose as an alternative to usual mixed model procedures is more efficient and significantly faster to compute. This result provides some useful guidelines for the analysis of complex designs.

MOTIVATION: In cDNA microarray experiments all samples are labeled with either Cy3 or Cy5. Systematic and gene-specific dye bias effects have been observed in dual-color experiments. In contrast to systematic effects which can be corrected by a normalization method, the gene-specific dye bias is not completely suppressed and may alter the conclusions about the differentially expressed genes. METHODS: The gene-specific dye bias is taken into account using an analysis of variance model. We propose an index, named label bias index, to measure the gene-specific dye bias. It requires at least two self-self hybridization cDNA microarrays. RESULTS: After lowess normalization we have found that the gene-specific dye bias is the major source of experimental variability between replicates. The ratio (R/G) may exceed 2. As a consequence false positive genes may be found in direct comparison without dye-swap. The stability of this artifact and its consequences on gene variance and on direct or indirect comparisons are addressed. AVAILABILITY: http://www.inapg.inra.fr/ens_rech/mathinfo/recherche/mathematique

We would like to thank K. Dobbin, J. Shih and R. Simon for their comments about Martin-Magniette et al. (2005). Their remarks relate to the design of microarray experiments and notably about the use of dye-swaps. We, however, want to make it clear that our manuscript primarily focuses on the detection, quantification and correction of the gene-specific dye-bias introduced in dual-color microarray experiments. The most important point in Martin-Magniette et al. (2005) is the following: by dint of studying technical errors, we will be able to identify and then to remove most of them. For the first time, in Martin-Magniette et al. (2005), we were able to quantify the gene-specific dye bias by calculating the Labelling Bias Index, (LBI). The LBI measured on different array types shows that this artifact seems to be very low in some cases and could be thus neglected. However, it is high in other cases and it cannot thus be neglected for data issued from these array types. This measure is of crucial importance as it allows users to evaluate the impact of this bias on their data. Moreover, we think that each platform should at least know in what class it belongs for each array types. Although this artifact is not lowered nor understood, it is simply dangerous to underestimate it. The gene-specific dye bias is not an inevitability and can be well controlled, as we point out in our paper. Recently, in another paper, Dobbin et al. (2005) studied the gene-specific dye bias. Although they reached the same conclusions, some of their remarks are explained in Martin-Magniette et al. (2005): e.g. Dobbin et al. (2005) have found that ‘(the gene-specific) dye bias appears to have masked the true differential expression’. This is explained in Martin-Magniette et al. (2005): the variance associated to a gene is overestimated by the dye bias effect in model (2) of Martin-Magniette et al. Dye-swaps constitute a simple and effective design to remove gene-specific dye bias when it is high. Nevertheless, we agree with Dobbin et al. (2005), that a balanced block design may be better than dye-swaps in some situations. As the former designs allow the use of more biological samples, the estimation of the biological variability will be more precise. Even if balanced block designs are statistically more efficient, the following considerations should be taken into account before choosing the experimental design:The experimental design must be adapted not only to the research question but also to the amount of biological material available. Finally, class prediction or differential expression, e.g. the question of interest, do not necessarily imply the same experimentaldesign. Even with rigorous experimental procedures some sources of variability remain (quality and yield of target purification, labelling efficiency, …). Performing dye-swaps will allow to differentiate biological from technical variability. It is often difficult to balance the dye for every treatment in complex designs, when samples are hardly available. For instance, such situations are encountered in sex-balanced medicalstudies. Moreover, some redundant experimental procedures (quality control of mRNA, preparation of targets for indirect dye-labelling) used in dye-swap experiments, decrease the financial cost to <2-fold the cost of a single slide hybridization, thus rendering this design much more attractive.

Following publication of this work [1], the authors became aware that Local FDR has been originally defined by Efron & al. (2001) in a mixture model framework [2].

BACKGROUND: Thousands of genes in a genomewide data set are tested against some null hypothesis, for detecting differentially expressed genes in microarray experiments. The expected proportion of false positive genes in a set of genes, called the False Discovery Rate (FDR), has been proposed to measure the statistical significance of this set. Various procedures exist for controlling the FDR. However the threshold (generally 5%) is arbitrary and a specific measure associated with each gene would be worthwhile. RESULTS: Using process intensity estimation methods, we define and give estimates of the local FDR, which may be considered as the probability for a gene to be a false positive. After a global assessment rule controlling the false positive error, the local FDR is a valuable guideline for deciding wether a gene is differentially expressed. The interest of the method is illustrated on three well known data sets. A R routine for computing local FDR estimates from p-values is available at http://www.inapg.fr/ens_rech/mathinfo/recherche/mathematique/outil.html. CONCLUSIONS: The local FDR associated with each gene measures the probability that it is a false positive. It gives the opportunity to compute the FDR of any given group of clones (of the same gene) or genes pertaining to the same regulation network or the same chromosomic region.