Microarray experiments and factors which affect their reliability
- 3.1k Downloads
Oligonucleotide microarrays belong to the basic tools of molecular biology and allow for simultaneous assessment of the expression level of thousands of genes. Analysis of microarray data is however very complex, requiring sophisticated methods to control for various factors that are inherent to the procedures used. In this article we describe the individual steps of a microarray experiment, highlighting important elements and factors that may affect the processes involved and that influence the interpretation of the results. Additionally, we describe methods that can be used to estimate the influence of these factors, and to control the way in which they affect the expression estimates. A comprehensive understanding of the experimental protocol used in a microarray experiment aids the interpretation of the obtained results. By describing known factors which affect expression estimates this article provides guidelines for appropriate quality control and pre-processing of the data, additionally applicable to other transcriptome analysis methods that utilize similar sample handling protocols.
Reviewers: This article was reviewed by Dr. Janet Siefert, Dr. Leonid Hanin, and Dr. I King Jordan.
KeywordsMicroarrays Microarray pre-processing Quality control Transcriptome profiling Measurement bias
3′ in vitro transcription
3′ untranslated region
Background Intensity Probes
External RNA controls
Polymerase chain reaction
Oligonucleotide microarrays belong to the most common tools used to describe changes in gene expression levels caused by altering the physical or chemical conditions. Microarrays can be also used to track differential expression patterns among various tissues and thus evaluate variability among individuals [1, 2, 3], they are used in SNP (single-nucleotide polymorphism) genotyping [4, 5, 6, 7] and identification of transcription factor binding sites using the ChIP-chip (ChIP: chromatin immunoprecipitation) method [8, 9, 10, 11, 12]. Microarrays are also used to estimate genomic copy number using Comparative Genomic Hybridization (CGH) arrays [13, 14, 15, 16] and in resequencing [17, 18, 19, 20, 21, 22].
Microarray analysis offers a variety of methods allowing, among other, identification of genes which might be significant in a specific cellular response mechanism or a particular gene expression pattern that characterizes a particular disease. To obtain significant results, microarray data need to undergo statistical processing to differentiate between signal changes caused by direct experimental factors and arising from the indirect experimental factors such as specific methods used, as well as from inaccuracies of the measurements. This level of processing challenges led to studies of the compatibility of different microarray platforms [23, 24, 25, 26, 27, 28] which usually is achieved by standardizing protocols and data analysis pipelines [29, 30]. Selection of an appropriate statistical method for microarray processing is a significant subject of scientific discussion and although microarrays have been in use for more than fifteen years, many issues related to data analysis remain unresolved.
The most discussed issues concern the algorithms used for the data normalization [31, 32], whose goal is to eliminate differences between samples that originate from technical aspects of the microarray handling which may confound the biological differences in a given experimental setup. A similar goal underlies methods used for batch-effect removal, a step which is crucial when comparing datasets that originate from different times and laboratories . Other frequently-discussed issues concern the identification of sample differentiating genes [34, 35] and evaluation of noise level in the sample , as well as methods to evaluate contamination or damage on the microarray’s surface [37, 38]. The most commonly used microarrays, produced by Affymetrix, are known for additional issues related to their particular design which influence the final results. These include problems resulting from several measurements of expression level for a single gene [39, 40], incorrect assignments of probes to genes [41, 42], incorrect evaluation of the background level and non-specific probe hybridization signals , and the effects of distinct probe features on data processing algorithms .
The most significant disadvantages of microarrays include the high cost of a single experiment, the large number of probe designs based on sequences of low-specificity, as well as the lack of control over the pool of analyzed transcripts since most of the commonly used microarray platforms utilize only one set of probes designed by the manufacturer. Other weaknesses of microarrays are their relatively low accuracy, precision and specificity  as well as the high sensitivity of the experimental setup to variations in hybridization temperature , the purity and degradation rate of genetic material , and the amplification process  which, together with other factors, may impact the estimates of gene expression.
The most widespread microarray is the Affymetrix 3′IVT (3′ in vitro transcription), i.e. HG-U133A or HG-U133_Plus_2, which is assembled as 11 sets of perfect match (PM) probes consisting of 25 nt sequences, which in most cases were chosen out of 600 nt sequence fragments located near the 3′ end of a specific transcript. For every PM probe on the microarray, a MM (mismatch) probe exists in which all nucleotides but one are identical to those on the corresponding PM probe but the original 13th nucleotide is replaced by a non-complementary one. The rationale behind the MM probes is to gauge the level of nonspecific hybridization , although the usefulness of this concept has been doubted (see further on).
The most recent generation of Affymetrix microarrays, such as the HuGene 1.0ST, is constructed using probes similar to the standard PM probes but with affinity not to the noncoding part of the 3′ end but rather to the individual exons in a given transcript. In this design the MM probes are replaced by the Background Intensity Probes (BGP), which are designed to evaluate background intensity levels for probes of different sequence characteristics. BGP are a set of about 1000 probes, non-complementary to any human gene sequence, with a variable ratio of GC nucleotides in the sequence. This approach enables a better evaluation of non-specific hybridization across the microarray compared with MM probes, for which the signal often exceeds the PM signal due to probe-specific effects . Additionally, lowering the number of probes which evaluate non-specific hybridization allows inserting of a much higher number of PM probes. The probe set in the new generation of whole transcript microarrays is constructed with two levels, exon and gene level. The exon probe set includes 4 probes on average, which are tailored for individual exons, and then these are clustered, usually in groups of around 25, creating sets for individual genes. Using this approach it is possible to determine levels of individual differently-spliced transcripts.
Another popular system is the Agilent microarray platform which was built using the SurePrint technology that allows using considerably longer, 60 nt-long probes. While probes are longer than in the Affymetrix system, the number of probes per gene is considerably lower, 8 on average in the most expensive set of exon microarrays (2 × 400 k) or 2 in the least expensive platform (8 × 60 k). As the Agilent probes are longer than those in the Affymetrix microarrays, the system tends to be more specific which is an obvious advantage, but on the other hand the lower number of probes per gene makes Agilent microarrays more sensitive to single nucleotide variations. These latter should not affect the signal if they result from amplification errors , but they may influence the expression estimates resulting from characteristic features of the sample analyzed. In the case of the Affymetrix microarray system these sources of error will only have a minor impact, as they influence signal only in an individual probe for a transcript or a transcript-specific probe-set. Single nucleotide polymorphisms do not block the hybridization but lower its efficiency, which can be interpreted as a significant decrease of gene expression, a feature which is used to estimate the level of nonspecific hybridization using mismatch probes [54, 55] or to assess allelic frequencies using SNP microarrays . In the Affymetrix systems the signal from one badly designed probe, which may be based on inaccurate data from a sequence database, can be easily eliminated from further analysis  without significant decrease in the precision of gene expression estimate, while in the Agilent systems the same design glitch might cause significant difficulties in the evaluation of gene expression levels.
Microarrays provide expression data for thousands of genes, but platform differences contribute to low accuracy of microarrays and for this reason they are only used to identify potentially significant genes in the experimental conditions studied. Precise assessment of the expression level of these presumably significant genes requires additional studies using more accurate methods such as real-time PCR (polymerase chain reaction) which, in turn, are not suitable for large-scale analyses. However, some steps of the microarray protocol are shared by the validation methods, affecting data quality in a similar manner.
Biological background of microarray experiments
Step I: RNA isolation
Reference genes found on a typical Affymetrix 3’IVT microarray. Amplification and hybridization control RNAs are added in various proportions and quantities as indicated in the last column. The amplification control transcripts are added using various dilutions which results in an estimated copy numbers ranging from one copy per 6,667 to 100,000 transcripts in the studied RNA sample. The hybridization control consists of biotinylated and fragmented cRNAs added in various amounts that result in a final concentrations ranging from 1.5 to 100 pM
ACTB - β-actin gene responsible for the structure of the cell
GAPDH – enzyme which takes part in glycolysis
STAT1 – transcription factor
Gene coding for 18S rRNA subunit
Gene coding for 28S rRNA subunit
Gene coding for 18S rRNA subunit - version 2
Gene coding for 28S rRNA subunit - version 2
Amplification control (Poly-A spike)
AFFX-DapX / AFFX-r2-Bs-dap
Dap gene of B.Subtilis bacteria - proportions 1:6,667
AFFX-ThrX / AFFX-r2-Bs-thr
Thr gene of B.Subtilis bacteria - proportions 1:25,000
AFFX-PheX / AFFX-r2-Bs-phe
Phe gene of B.Subtilis bacteria - proportions 1:50:000
AFFX-LysX / AFFX-r2-Bs-lys
Lys gene of B.Subtilis bacteria - proportions 1:100,000
Hybridization control (Bacterial spike)
AFFX-BioB / AFFX-r2-Ec-bioB
BioB gene of E.Coli bacteria – quantity 1.5 pM
AFFX-BioC / AFFX-r2-Ec-bioC
BioC gene of E.Coli bacteria – quantity 5 pM
AFFX-BioDn / AFFX-r2-Ec-bioD
BioD gene of E.Coli bacteria – quantity 25 pM
AFFX-CreX / AFFX-r2-P1-cre
Cre gene of P1 bacteriophage – quantity 100 pM
Each of the control probe-sets exists in three variants, each targeting a different region of the selected transcript - its central section and the 3′- and 5′-ends. This allows assessing the degradation rate of individual transcripts by examining the 3′/5′ probe-set signal ratios, which can be compared to the threshold defined by the manufacturer and ratios obtained for other microarrays, in order to assess the homogeneity of degradation level across individual samples. In order to aid the assessment of post-experimental RNA degradation, more complex methods have been developed including RNA degradation plots  or mixed effect models based on individual probe and transcript characteristics .
Step II: cDNA synthesis
The second strand of the cDNA is then created by using the first strand as a template. Addition of ribonuclease causes RNA cleavage at nonspecific sites, leaving only short fragments attached to the cDNA (Fig. 2). These fragments are then used as primers for the polymerase which synthesizes the second strand of the cDNA, removing the remaining mRNA fragments found on its way. Measurement of cDNA concentration, which allows standardizing it across various samples, is not a part of the standard experimental procedure for eukaryotic cells, due to the presence of other nucleic acid species that affect the spectrophotometric measurement, whose removal requires additional cDNA purification. This step is strongly influenced by any previous RNA degradation, which leads to the creation of truncated mRNAs (from the 5′-end) . When oligo-dT primers are used during cDNA synthesis these truncated mRNAs are read from the 3′-end only to the position of truncation, and the remaining part is lost due to the lack of poly-A. In such a situation probes located further from the 3′-end usually show lower signal intensity, a phenomenon which is the basis of RNA degradation plots used to assess the mRNA quality . In order to reduce this effect, on 3′IVT microarrays probes from a single set are selected based on a very small region of 600 bp located close to the 3′-end of the mRNA. To further reduce this bias sophisticated methods have been developed that take into account the location of regions targeted by probes in order to correct the signal intensities [67, 68].
The 3′-end bias does not occur when random primers are used for the cDNA synthesis. Random primers do not require a poly-A tail since they can attach to any region of the mRNA and not only to its 3′-end, promoting synthesis in a 3′ ➔ 5′ direction, and a very strong 5′-end bias can be observed as shown in ref. .
Although many of the available cDNA synthesis kits include a combination of oligo-dT and random primers, kits based solely on oligo-dT are commonly used especially for the 3′-IVT platform where 3′-UTR sequences are of the highest importance since they are targeted by oligonucleotide probes.
Oligo-dT-based cRNA synthesis introduces an additional bias that may affect the results of a microarray experiment. First of all, because of the mRNA degradation problem, oligo-dT primers are a good choice only if the region of interest is located in the vicinity of the 3′-UTR, since large distances between the region targeted by probes and the poly-A can decrease the precision of expression level estimates . If the analysis requires the entire transcript as in the case of WT (whole-transcript) microarray platforms where individual exons are analyzed, random primers are required. Additionally, oligo-dT is assumed to bind only to the poly-A tail of the transcript, requiring a long continuous strand of A nucleotides, as shown in Fig. 2. However, partial primer complementarity (i.e. complementarity of only 8 adenine nucleotides in the primer’s sequence) is sufficient for the reaction initiation, and due to the random nature of the attachment it can also bind to the A-strands found commonly in the UTRs . Further, with increasing concentration of oligo-dT the chance of attaching multiple oligonucleotides to a single mRNA are increased. In such situation the synthesis may start from two distinct regions but the reaction located closer to the 3′-end might be blocked by the second reaction, again producing truncated cDNA products . This phenomenon can therefore affect the entire probe-set signal intensity of the targeted transcript if its sequence includes simple repeats built predominantly of A nucleotides.
Step III: Amplification and labeling
In this step the newly-synthesized cDNA is replicated (amplified) in a process of in vitro transcription. The goal of this step is to obtain a large quantity of cRNA containing biotinylated C and U nucleotides that will be required in the subsequent steps . For this purpose another fragment of oligo-dT is used, marked in red in Fig. 2, which serves as a promoter for the T7 bacteriophage polymerase.
The efficiency of this reaction and its consistency between samples has a decisive impact on the final experimental outcomes . There are many factors which influence the efficiency of this reaction including the structural properties of the cDNA itself which, depending on the GC content, can affect the efficiency of the polymerase  and form secondary structures . This step is completed with a cleanup and quantification of the cRNA which allows for control of the total reaction yield and purity of the sample. The product of the amplification reaction can be observed in lanes three and four of the electrophoresis gel (Fig. 3). rRNA is no longer visible, and due to the variability in length of the cRNAs there are no easily distinguishable bands visible on the gel.
Post-experimental control of cRNA level variations, utilizes the signals of probes targeting a reference RNA (poly-A spike) added prior to cDNA synthesis and signals of housekeeping genes which should be on a similar level across all samples. The poly-A spike contains transcripts of five B. subtilis genes (Dap, Lys, Phe, Thr, and Trp) which are added in various proportions to the isolated RNA. Since they all include a poly-A tail they undergo the same procedure as the RNA analyzed, independently of its condition. Lys gene RNA is added at the lowest concentration (1:100,000 of the total RNA) which is close to the sensitivity level of the microarray. Its detection in at least half of the microarrays of a given experiment is a good indicator of a properly conducted procedure. The remaining reference RNAs are added in increasing concentrations Lys < Phe < Thr < Dap with Dap being the highest and close to the probe signal intensity saturation level.
The amplification products no longer have the T7 promoter, although the spacer sequence between the promoter and the (T)24 primer (green in Fig. 3) is also amplified . Since this fragment is copied with each cRNA its quantity is very large, and since it can bind to probes having a similar sequence it might affect their signal intensity . It is believed that the process of amplification might be the source of inconsistent signals among samples, as it depends highly on the experiment conditions and the transcript structure [74, 76], becoming the main motivation for the development of microarray protocols that do not require RNA amplification .
Step IV and V: cRNA fragmentation and hybridization
cRNAs obtained in the previous step are cut into 50–100 nt fragments shown in lanes five and six of the electrophoresis gel (Fig. 3). After this, another set of external RNA controls (ERCs) that originates from P1 bacteriophage and E. coli bacteria (termed bacterial spikes) is added to the RNA pool. Similarly to the poly-A spike, bacterial RNA is added in various concentrations with the following relations satisfied: bioB < bioC < bioD < Cre (group 4 in Table 1). BioB, bioC and bioD originate from the E. coli genes used in the synthesis of biotin, while Cre is isolated from P1 bacteriophage where its gene product serves as a recombinase . This bacterial spike is already converted to cRNA and fragmented allowing to control the hybridization process, independently of the efficiency of labeling and amplification used in the previous steps to obtain cRNA . After this the mixture of various cRNAs is transferred on to the microarray chip, initiating the hybridization process.
Hybridization is the most time-consuming step of the entire microarray procedure. During approximately 16 h, in which microarrays are incubated in a hybridization oven set to 45 °C, the cRNA binds to the specific probes attached to the glass surface of the microarray chip. The dynamics of the hybridization process depends on many factors which, as in the amplification step, depend on both the reaction conditions and structural properties of the individual cRNA molecules which may significantly affect the experimental outcomes [79, 80]. Prolonged hybridization can cause sample drying and uneven distribution of the material on the surface of the chip. Additionally, evaporation of some of the water can change the salt concentration in the buffers and significantly affect the efficiency of the process .
Problems detected by different control probe-sets and their possible reasonsa
Poor quality of the mRNA analyzed
Problems during amplification/labeling
Problems during hybridization/washing
Inaccurate preparation of bacterial spike
Inaccurate preparation of bacterial poly-A spike
Bacterial spike controls are a good indicator of problems that may occur during the hybridization procedure, although they fail to detect uneven hybridization, since the probe-set intensity is obtained after summarizing signals of over 20 individual probes, spread over the entire surface of the microarray (3′IVT arrays) or located in a small region at the middle of the array (WT arrays). For this purpose the quality control of each sample should include the analysis of an image of the microarray surface, which is either a complete scan saved in a DAT file, or more commonly a recreated image based on the individual probe intensities stored in a CEL file [83, 84].
The main assumption made in design of a microarray is that probes targeting a single transcript are placed randomly on its surface. For this reason, variations in the signal intensity of specific regions suggest reasons other than the biological variation between the analyzed mRNAs. Such differences among regions, termed image artifacts, are mostly caused by bubbles of air or small levels of impurities, which were added into the microarray cartridge with the experimental solutions . Such artifacts appear very commonly, although they usually have a very small size and are handled efficiently by summarization methods, which are insensitive to a small number of outlying values. The main problem occurs when the artifact covers a significant percentage of the array surface or its intensity is extremely high and close to the saturation level of the probes. Such artifacts are mainly caused by uneven hybridization and affect not only the expression estimates from probes located in its region, but also the remaining probe signals. This latter effect is due to data processing, which utilizes expression levels of all or of a significant fraction of the probes on the microarray .
Microarray surface artifacts can be visualized by either creating an image, based on single probe expression intensities in a convenient (usually logarithmic) scale, or by analyzing differential images created by subtracting the signal of each probe on a single microarray from that on another reference array created by, for example, calculating the median intensity level of each probe across all microarrays in a single experiment . If a defective array is found, probes affected by an aberration may be separated and removed from the subsequent data analysis or even recreated using imputation techniques [38, 37, 85, 86]. Microarrays affected by a very large aberration should be removed from the study, as they no longer serve as a reliable source of information.
Step VI: Washing and staining
Washing follows the cRNA hybridization and is used to remove cRNA non-specifically bound to the microarray surface. Again, in this step small variations in the reaction conditions may affect the expression estimates . Depending on the conditions of the washing process (temperature, salt concentration, calcium and magnesium ion levels in the buffer) non-specifically bound cRNA is removed with varying efficiency, affecting the sensitivity and background level of the entire microarray. The binding strength of cRNAs depends not only on their complementarity level but also on the temperature of the hybridization and their sequence characteristics, mainly the GC content  and specific base positions inside the sequence . Separation between the binding strength of non-specifically bound GC-rich cRNA and GC-poor cRNA with perfect complementarity is not very sharp, affecting the final intensity level of cRNAs depending on their sequence characteristics , which can be only reduced using sequence-based normalization approaches during the data pre-processing step [89, 90].
The washing process is followed by staining of the hybridized cRNA using a streptavidin-phycoerythrin complex (Fig. 2). Streptavidin is a protein with high binding affinity to the biotinylated nucleotides used in the cRNA preparation, while phycoerythrin is a fluorescent dye used for quantitation of the hybridized cRNA. The quality of the fluorophore used significantly affects the fluorescence intensity of the microarray, decreasing its sensitivity if it is exposed for too long to daylight .
Step VII: Scanning
In this step the microarray cartridge is placed in the microarray scanner where the fluorescence of the phycoerythrin bound to the cRNA is excited using a laser. The level of fluorescence is measured by the scanner’s detector and is assumed to be proportional to the amount of cRNA bound to the corresponding probe. The length of this process depends on the size of the microarray and in most cases lasts around 10 min for a single array. During the scanning process all arrays are placed inside the scanner’s chamber so that the fluorescence intensity is not affected by differences in the length of exposure to daylight, which could increase the differences among microarrays in both the scale of the measurements and the sensitivity level. It is advised to scan each microarray only once, since each subsequent scan decreases the fluorescence intensity by 10–20 %, due to decay of the fluorophore . The fluorescence intensity of cyanine-based dyes also used in microarray experiments, such as Cy5, can be further affected by the ozone concentration in the laboratory, a factor which is both time- and location-dependent, and can become a major source of among-experiment inconsistencies [60, 93].
Step VIII: Data pre-processing
The last stage involves data pre-processing which starts by analyzing the microarray image stored in the DAT file, whose goal is to obtain single fluorescence intensity for each probe based on the 16 pixels of the original microarray image. This step is performed by the Affymetrix software and returns a CEL file as an output, in which each probe, at a specific position on the microarray, has a signal intensity assigned to it. These individual probe intensities are used in the subsequent preprocessing steps, during which each array is standardized by first estimating and then subtracting the background signal in order to reduce the effect of non-specific hybridization . Following step is to perform normalization procedure which reduces the differences in probe intensities that originate from differences in experimental conditions and cRNA concentration [31, 94]. The final step of pre-processing is the summarization, in which a single expression estimate is calculated for each probe-set based on the intensity of the individual probe signals . Summarization step is highly dependent on the quality of the probe and probeset definitions which are in many cases low due to inaccurate transcriptome data at the time of microarray design. This can result in probesets targeting transcripts of multiple genes due to low probe specificity, probes that do not map any of the known transcripts [41, 42] or multiple probesets that map the same gene [39, 40], requiring the development of methods used for the validation of existing probes and for probeset redefinition [41, 42, 96].
Selection of the pre-processing strategy can have a very large impact on the experimental outcomes  and often requires a few assumptions which are not always acceptable. The main assumption made by pre-processing methods is that the total level of mRNA in the cell does not vary significantly among samples, regardless of the experimental conditions and cell lines used. This assumption is required for the standardization approaches based on mean and median scaling or more complex approaches, such as quantile normalization , and its natural consequence is that the amount of differentially-expressed features with increased or decreased levels will be always similar. For example, in the case of global transcript level changes in cells with inhibited transcription, one might expect to detect predominantly transcript down-regulation, whereas after applying quantile normalization it is very probable that a significant number of up-regulated transcripts will be observed, due to intensity distribution transformations.
Another important assumption is forced by the massively parallel experimentation of the microarray technique which allows for assessing expression level of thousands of genes simultaneously. We have to assume that the reaction conditions for each individual gene were similar while knowing that due to various molecular properties of the analyzed RNA/DNA fragments it is impossible to properly optimize each of the individual reactions. Most of the data processing methods make this assumption although some standardization methods also exist that utilize probe and RNA/DNA sequence information in order to reduce the signal differences resulting from sub-optimal amplification and hybridization conditions that affect gene expression estimates to a varying degree [89, 90].
Despite successful studies of reproducibility  and specificity , microarrays have been often subject of criticism as a method which fails to identify relevant information that can be transferred directly into clinical applications . The main reason is that statistical significance often differs from biological relevance due to a very limited number of samples or to the influence of other factors, such as cellular heterogeneity or variability of the morphological features, which are difficult to separate from the studied features. This highlights the importance of experimental design which utilizes an adequate number of samples and biological replicates to answer questions defined in the project.
The capabilities of microarray studies are limited, since the measurement of transcript levels provides only a rough estimate of the intracellular conditions at a specific time point, and is affected by a plethora of experiment-specific factors. The process of discovery of new drugs, using expression or genotyping microarrays, is therefore uneven in pace and in some cases might be even misleading. However, microarrays can be successfully used to validate the effects of existing drugs by helping to identify their targets and off-target effects . Microarrays are becoming less popular due to the decreasing costs of the RNA-seq methods , although one has to remember that some of the steps used in the microarray procedure, with their drawbacks and limitations, are also utilized in other techniques including the RNA-seq approaches [101, 102, 103]. Despite the evolution of experimental procedures, the fundamental principles behind microarray experiments remain similar and their understanding is also an essential step towards appropriate interpretation of the data provided by more advanced but related methods.
Reviewer’s report 1: Dr. Janet Siefert
Jaksik et al. have written a review article on the use of microarrays and the cautions and complications of using the results of them to evaluate research data. The Kimmel lab has considerable experience, over several years, with microarray data so the expertise of his team to evaluate and write such an article is well placed. I find their review to be comprehensive and thorough. It will be of considerable use to anyone considering employing microarray experiments as well as those who need to troubleshoot previous use of microarrays and accompanying statistical evaluations. Although the published literature offers a number of articles reviewing microarray use, it is the expertise of this team, as statisticians actively working with numerous microarray data sets, that makes this article valuable to the researching community.
Reviewer’s report 2: Dr. Leonid Hanin
This is an interesting article that lists and analyzes in detail various sources of errors and inconsistencies associated with microarray technology. It represents an important step towards answering the following fundamental question: Is microarray technology a reliable tool for furthering our understanding of biological systems at the genomic level or is it bound to produce a lot of biological/technological artefacts and largely generate false knowledge? I believe the main deficiency of the article is that it is entirely qualitative in that no quantitative estimates of the impact of various factors identified in this work or of their relative importance were given. The basic question that a researcher utilizing microarrays would ask is whether impact of these factors is minor or major. The article does not provide any information or opinion in this regard. Given that the processes collectively forming microarray technology are either biochemical or physical in nature, estimating the effects of various factors on gene expression signals quantitatively seems to be in principle possible. For example, here are two relevant publications, just from the top of my head, about the physics of DNA/RNA hybridization, see also references therein:
1. E. Carlon, T. Heim (2006), Thermodynamics of RNA/DNA hybridization in high-density oligonucleotide microarrays, Physica A 362: 433–449.
2. A. Ferrantini, E. Carlon (2008), On the relationship between perfect matches and mismatches in Affymetrix Genechips, Gene 422: 1–6.
On a more technical level, I have the following questions and comments.
Authors’ response: Description of probe design flaws resulting from inaccurate transcriptome data were added to the description of the data pre-processing step.
“Summarization step is highly dependent on the quality of the probe and probeset definitions which are in many cases low due to inaccurate transcriptome data at the time of microarray design. This can result in probesets targeting transcripts of multiple genes due to low probe specificity, probes that do not map any of the known transcripts [41, 42] or multiple probesets that map the same gene [39, 40], requiring the development of methods used for the validation of existing probes and for probeset redefinition [41, 42, 96]. “
Authors’ response: Cellular heterogeneity is a major problem in many biological studies and can indeed significantly affect the results of a microarray study. However because this problem is unrelated to the technical aspects of the microarray protocol, we find it to be outside of the scope of this article.
Authors’ response: Based on our experience we suggest using at least 3 replicates for studies based on cell lines. From a statistical point of view a minimum of 3 samples are required for the estimate of standard deviation to be valid. Higher number of replicates might be highly beneficial for experiments dealing with poor quality material or studies aiming to detect small differences in gene expression. For experiments based on samples extracted from multiple patients, replicates are usually not necessary since the confidence level increases with the number of patients studied.
Authors’ response: The fluorescence intensity of probe is proportional to the RNA level of corresponding gene although, as shown by Held et al. in 2006 the relationship is not linear whether or not the data are pre-processed.
Authors’ response: Minor comments of the reviewer have been taken into account. Corresponding changes in the manuscript are highlighted in grey.
1. Introduction, line 2. “physical or chemical conditions”. Perhaps biological conditions too?
2. Introduction, line 4. It seems like “their” should be inserted between “evaluate” and “variability”.
3. Introduction, paragraph 2, sentence 2. Aren’t “specific methods used” and “inaccuracies of the measurements” themselves “experimental factors”?
Author’s response: Definitions of accuracy and precision can be found in ref. 45.
In the case of microarrays specificity refers to the ability of a probe to bind a unique target sequence. A specific probe provides signal proportional only to the amount of the target sequence, while non-specific probe signal will be a result of interaction with more than one target sequence. The specificity of a probe can be diminished by cross-hybridization, also called non-specific hybridization, a phenomenon in which sequences that are not strictly complementary according to the Watson–Crick rules bind to each other.
5. Introduction, last sentence. Delete “have”.
7. Microarray Structure, last paragraph. Platform differences contribute to low accuracy of microarrays, so “despite” seems to be out of place.
8. Step I: RNA isolation, paragraph 1. “…ribosomal RNA… is rarely studied”. I think rRNA is studied quite extensively in phylogeny and pharmacology.
9. Step I: cDNA synthesis, paragraph 1. Shouldn’t the figure referred to here be 4 rather than 3?
10. Conclusions, paragraph 1. What is the significance of “morphological features” in this context?
11. Caption to Fig. 1. What is the purpose of probe sets A and B? Also, does “corresponds to” mean “proportional to”, see technical comments 4?
12. Figure 2. I think it would be better if the steps of microarray experiment shown in the figure correspond to the steps described under Biological background of microarray experiments.
13. Table 2. What about the other two combinations of control probe-set outcomes involving an error?
14. The authors are encouraged to proofread their submission. There are a few places with missing or extra commas, instances where article “the” can (or perhaps should) be removed, etc.
Reviewer’s report 3: Dr. I King Jordan
This reviewer provided no comments for publication.
We thank Ron Hancock, Anna Lalik and Robert Herok for very helpful discussions. This work was supported by Polish National Science Center grant DEC-2012/04/A/ST7/00353 (RJ, JR and MK) and POIG.02.03.01-00-040/13 (MI).
- 4.Erickson S, MacLeod SL, Hobbs CA. Cheek swabs, SNP chips, and CNVs: assessing the quality of copy number variant calls generated with subject-collected mail-in buccal brush DNA samples on a high-density genotyping microarray. BMC Med Genet. 2012;13:51. doi: 10.1186/1471-2350-13-51.PubMedCentralCrossRefPubMedGoogle Scholar
- 16.Ahn J, Mann K, Walsh S, Shehab M, Hoang S, et al. Validation and implementation of array comparative genomic hybridisation as a first line test in place of postnatal karyotyping for genome imbalance. Mol Cytogenet. 2010;3(9). doi: 10.1186/1755-8166-3-9.
- 32.Li C, Hung Wong W. Model-based analysis of oligonucleotide arrays: model validation, design issues and standard error application. Genome Biol. 2001;2(8). RESEARCH0032. http://www.genomebiology.com/2001/2/8/research/0032.
- 35.Smyth GK. Linear models and empirical bayes methods for assessing differential expression in microarray experiments. Statistical applications in genetics and molecular biology. 2004;3:Article3. doi: 10.2202/1544-6115.1027.
- 40.Schneider S, Smith T, Hansen U. SCOREM: statistical consolidation of redundant expression measures. Nucleic Acids Res. 2011;40(6):e46. doi: 10.1093/nar/gkr1270.
- 51.Affymetrix. GeneChip Expression Analysis - Technical Manual. 2004:185.Google Scholar
- 58.Affymetrix. 3′ IVT Express Kit User Manual. 2012. http://www.affymetrix.com.
- 59.Grillo G, Turi A, Licciulli F, Mignone F, Liuni S, Banfi S, et al. UTRdb and UTRsite (RELEASE 2010): A collection of sequences and regulatory motifs of the untranslated regions of eukaryotic mRNAs. Nucleic Acids Res. 2010;38(Database issue):D75–80. doi: 10.1093/nar/gkp902.PubMedCentralCrossRefPubMedGoogle Scholar
- 61.Mignone F, Grillo G, Licciulli F, Iacono M, Liuni S, Kersey PJ, et al. UTRdb and UTRsite: a collection of sequences and regulatory motifs of the untranslated regions of eukaryotic mRNAs. Nucleic Acids Res. 2005;33(Database issue):D141–6. doi: 10.1093/nar/gki021.PubMedCentralCrossRefPubMedGoogle Scholar
- 81.Affymetrix. Gene Expression Assay and Data Analysis - Hybridization time. 2012. http://www.affymetrix.com/support/help/faqs/ge_assays/faq_15.jsp.
- 89.Hulsman M, Mentink A, van Someren EP, Dechering KJ, de Boer J, Reinders MJ. Delineation of amplification, hybridization and location effects in microarray data yields better-quality normalization. BMC bioinformatics. 2010;11:156. doi: 10.1186/1471-2105-11-156.PubMedCentralCrossRefPubMedGoogle Scholar
- 91.Munier M, Jubeau S, Wijaya A, Morancais M, Dumay J, Marchal L, et al. Physicochemical factors affecting the stability of two pigments: R-phycoerythrin of Grateloupia turuturu and B-phycoerythrin of Porphyridium cruentum. Food Chem. 2014;150:400–7. doi: 10.1016/j.foodchem.2013.10.113.CrossRefPubMedGoogle Scholar
- 92.Affymetrix. Gene Expression Assay and Data Analysis - Microarray scanning. 2012. http://www.affymetrix.com/estore/support/help/faqs/ge_assays/faq_8.jsp.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.