Proteomics pp 271-288 | Cite as

Designing Successful Proteomics Experiments

  • Daniel RudermanEmail author
Part of the Methods in Molecular Biology book series (MIMB, volume 1550)


Because proteomics experiments are so complex they can readily fail, and do so without clear cause. Using standard experimental design techniques and incorporating quality control can greatly increase the chances of success. This chapter introduces the relevant concepts and provides examples specific to proteomic workflows. Applying these notions to design successful proteomics experiments is straightforward. It can help identify failure causes and greatly increase the likelihood of inter-laboratory reproducibility.

Key words

Design of experiments Randomization Bias Variance 



I thank Dr. Parag Mallick for introducing me to the field of experimental design and Dr. Nicholas Graham for comments on the manuscript.


  1. 1.
    Montgomery DC (2013) Design and analysis of experiments, 8th edn. John Wiley, Hoboken, NJGoogle Scholar
  2. 2.
    Box GEP, Hunter JS, Hunter WG (2005) Statistics for experimenters : design, innovation, and discovery. Wiley series in probability and statistics, 2nd edn. Wiley-Interscience, Hoboken, NJGoogle Scholar
  3. 3.
    Quinn GP, Keough MJ (2002) Experimental design and data analysis for biologists. Cambridge University Press, Cambridge, UKCrossRefGoogle Scholar
  4. 4.
    NIST/SEMATECH e-Handbook of statistical methods.
  5. 5.
    Hu J, Coombes KR, Morris JS (2005) The importance of experimental design in proteomic mass spectrometry experiments: some cautionary tales. Brief Funct Genomic Proteomic 3(4):322–331CrossRefPubMedGoogle Scholar
  6. 6.
    Maes E, Cho WC, Baggerman G (2015) Translating clinical proteomics: the importance of study design. Expert Rev Proteomics 12(3):217–219CrossRefPubMedGoogle Scholar
  7. 7.
    Prinz F, Schlange T, Asadullah K (2011) Believe it or not: how much can we rely on published data on potential drug targets? Nature: pp 1–2. doi: papers3://publication/doi/ 10.1038/nrd3439-c1
  8. 8.
    Begley CG, Ellis LM (2012) Drug development: raise standards for preclinical cancer research. Nature 483 (7391).[Online]. doi: papers3://publication/uuid/0C526949-4171-407F-834A-A82A37275234Google Scholar
  9. 9.
    Begley CG, Ioannidis JPA (2015) Reproducibility in science: improving the standard for basic and preclinical research. Circ Res 116(1):116–126CrossRefPubMedGoogle Scholar
  10. 10.
    The National Institutes of Health (2015) Enhancing Reproducibility through Rigor and Transparency.
  11. 11.
    Oberg AL, Vitek O (2009) Statistical design of quantitative mass spectrometry-based proteomic experiments. J Proteome Res 8(5):2144–2156CrossRefPubMedGoogle Scholar
  12. 12.
    Riter LS, Vitek O, Gooding KM, Hodge BD, Julian RK (2005) Statistical design of experiments as a tool in mass spectrometry. J Mass Spectrom 40(5):565–579CrossRefPubMedGoogle Scholar
  13. 13.
    Karp NA, Lilley KS (2007) Design and analysis issues in quantitative proteomics studies. Proteomics 7(Suppl 1):42–50CrossRefPubMedGoogle Scholar
  14. 14.
    Leek JT, Scharpf RB, Bravo HC, Simcha D, Langmead B, Johnson WE, Geman D, Baggerly K, Irizarry RA (2010) Tackling the widespread and critical impact of batch effects in high-throughput data. Nat Rev Genet 11(10):733–739CrossRefPubMedGoogle Scholar
  15. 15.
    Cairns DA (2011) Statistical issues in quality control of proteomic analyses: good experimental design and planning. Proteomics 11(6):1037–1048CrossRefPubMedGoogle Scholar
  16. 16.
    Cairns DA (2014) Statistical issues in the design and planning of proteomic profiling experiments. Clin Proteomics 18:223–236Google Scholar
  17. 17.
    Leitner A, Aebersold R (2013) SnapShot: mass spectrometry for protein and proteome analyses. Cell 154(1):252–252e251. doi: 10.1016/j.cell.2013.06.025 CrossRefPubMedGoogle Scholar
  18. 18.
    Gallien S, Bourmaud A, Kim SY, Domon B (2014) Technical considerations for large-scale parallel reaction monitoring analysis. J Proteomics 100:147–159. doi: 10.1016/j.jprot.2013.10.029 CrossRefPubMedGoogle Scholar
  19. 19.
    Montgomery DC (2013) Introduction to statistical quality control, 7th edn. Wiley, Hoboken, NJGoogle Scholar
  20. 20.
    Bramwell D (2013) An introduction to statistical process control in research proteomics. J Proteomics 95(C):3–21, doi: papers3://publication/doi/ 10.1016/j.jprot.2013.06.010 CrossRefPubMedGoogle Scholar
  21. 21.
    Krokhin OV, Spicer V (2009) Peptide retention standards and hydrophobicity indexes in reversed-phase high-performance liquid chromatography of peptides. Anal Chem 81(22):9522–9530, doi: papers3://publication/doi/ 10.1021/ac9016693 CrossRefPubMedGoogle Scholar
  22. 22.
    Krokhin OV (2006) Sequence-specific retention calculator. algorithm for peptide retention prediction in ion-pair RP-HPLC: application to 300- and 100-Å pore size C18 sorbents. Anal Chem 78(22):7785–7795. doi: 10.1021/ac060777w CrossRefPubMedGoogle Scholar
  23. 23.
    Geiger T, Wisniewski JR, Cox J, Zanivan S, Kruger M, Ishihama Y, Mann M (2011) Use of stable isotope labeling by amino acids in cell culture as a spike-in standard in quantitative proteomics. Nat Protoc 6(2):147–157, doi: papers3://publication/doi/ 10.1038/nprot.2010.192 CrossRefPubMedGoogle Scholar
  24. 24.
    Levin Y, Hradetzky E, Bahn S (2011) Quantification of proteins using data-independent analysis (MSE) in simple andcomplex samples: a systematic evaluation. Proteomics 11(16):3273–3287CrossRefPubMedGoogle Scholar
  25. 25.
    Ivanov AR, Colangelo CM, Dufresne CP, Friedman DB, Lilley KS, Mechtler K, Phinney BS, Rose KL, Rudnick PA, Searle BC, Shaffer SA, Weintraub ST (2013) Interlaboratory studies and initiatives developing standards for proteomics. Proteomics 13(6):904–909, doi: papers3://publication/doi/ 10.1002/pmic.201200532 CrossRefPubMedGoogle Scholar
  26. 26.
    Rudnick PA, Clauser KR, Kilpatrick LE, Tchekhovskoi DV, Neta P, Blonder N, Billheimer DD, Blackman RK, Bunk DM, Cardasis HL, Ham A-JL, Jaffe JD, Kinsinger CR, Mesri M, Neubert TA, Schilling B, Tabb DL, Tegeler TJ, Vega-Montoto L, Variyath AM, Wang M, Wang P, Whiteaker JR, Zimmerman LJ, Carr SA, Fisher SJ, Gibson BW, Paulovich AG, Regnier FE, Rodriguez H, Spiegelman C, Tempst P, Liebler DC, Stein SE (2010) Performance metrics for liquid chromatography-tandem mass spectrometry systems in proteomics analyses. Mol Cell Proteomics 9(2):225–241CrossRefPubMedGoogle Scholar
  27. 27.
    Ma Z-Q, Polzin KO, Dasari S, Chambers MC, Schilling B, Gibson BW, Tran BQ, Vega-Montoto L, Liebler DC, Tabb DL (2012) QuaMeter: multivendor performance metrics for LC–MS/MS proteomics instrumentation. Anal Chem 84(14):5845–5850CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Wang X, Chambers MC, Vega-Montoto LJ, Bunk DM, Stein SE, Tabb DL (2014) QC metrics from CPTAC raw LC-MS/MS data interpreted through multivariate statistics. Anal Chem 86(5):2497–2509CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Bereman MS, Johnson R, Bollinger J, Boss Y, Shulman N, MacLean B, Hoofnagle AN, MacCoss MJ (2014) Implementation of statistical process control for proteomic experiments via LC MS/MS. J Am Soc Mass Spectrom 25(4):581–587. doi: 10.1007/s13361-013-0824-5 CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Burkhart JM, Vaudel M, Zahedi RP, Martens L, Sickmann A (2011) iTRAQ protein quantification: a quality-controlled workflow. Proteomics 11(6):1125–1134, doi: papers3://publication/doi/ 10.1002/pmic.201000711 CrossRefPubMedGoogle Scholar
  31. 31.
    Herbrich SM, Cole RN, West KP Jr, Schulze K, Yager JD, Groopman JD, Christian P, Wu L, O'Meally RN, May DH, McIntosh MW, Ruczinski I (2013) Statistical inference from multiple iTRAQ experiments without using common reference standards. J Proteome Res 12(2):594–604. doi: 10.1021/pr300624g CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Greco G, Boltner A, Letzel T (2014) Optimization of jet stream ESI parameters when coupling agilent 1260 infinity analytical SFC system with agilent 6230 TOF LC/MS., Agilent Technologies, Google Scholar
  33. 33.
    Geer LY, Markey SP, Kowalak JA, Wagner L, Xu M, Maynard DM, Yang X, Shi W, Bryant SH (2004) Open mass spectrometry search algorithm. J Proteome Res 3(5):958–964. doi: 10.1021/pr0499491 CrossRefPubMedGoogle Scholar
  34. 34.
    Bezerra MA, Santelli RE, Oliveira EP, Villar LS, Escaleira LA (2008) Response surface methodology (RSM) as a tool for optimization in analytical chemistry. Talanta 76(5):965–977CrossRefPubMedGoogle Scholar
  35. 35.
    Coscollà C, Navarro-Olivares S, Martí P, Yusà V (2014) Application of the experimental design of experiments (DoE) for the determination of organotin compounds in water samples using HS-SPME and GC–MS/MS. Talanta 119:544–552CrossRefPubMedGoogle Scholar
  36. 36.
    Eliasson M, Rännar S, Madsen R, Donten MA, Marsden-Edwards E, Moritz T, Shockcor JP, Johansson E, Trygg J (2012) Strategy for optimizing LC-MS data processing in metabolomics: a design of experiments approach. Anal Chem 84(15):6869–76CrossRefPubMedGoogle Scholar
  37. 37.
    Székely G, Henriques B, Gil M, Ramos A, Alvarez C (2012) Design of experiments as a tool for LC–MS/MS method development for the trace analysis of the potentially genotoxic 4-dimethylaminopyridine impurity in glucocorticoids. J Pharmaceut Biomed Anal 70:251–8CrossRefGoogle Scholar
  38. 38.
    Maes K, Van Liefferinge J, Viaene J, Van Schoors J, Van Wanseele Y, Béchade G, Chambers EE, Morren H, Michotte Y, Vander Heyden Y, Claereboudt J, Smolders I, Van Eeckhaut A (2014) Improved sensitivity of the nano ultra-high performance liquid chromatography-tandem mass spectrometric analysis of low-concentrated neuropeptides by reducing aspecific adsorption and optimizing the injection solvent. J Chromatogr A 1360:217–228CrossRefPubMedGoogle Scholar
  39. 39.
    Passeport E, Guenne A, Culhaoglu T, Moreau S, Bouyé J-M, Tournebize J (2010) Design of experiments and detailed uncertainty analysis to develop and validate a solid-phase microextraction/gas chromatography–mass spectrometry method for the simultaneous analysis of 16 pesticides in water. J Chromatograph A 1217(33):5317–5327CrossRefGoogle Scholar
  40. 40.
    Raji MA, Schug KA (2009) Chemometric study of the influence of instrumental parameters on ESI-MS analyte response using full factorial design. Int J Mass Spectrom 279:100–106CrossRefGoogle Scholar
  41. 41.
    Zhou Y, Song J-Z, Choi FF-K, Wu H-F, Qiao C-F, Ding L-S, Gesang S-L, Xu H-X (2009) An experimental design approach using response surface techniques to obtain optimal liquid chromatography and mass spectrometry conditions to determine the alkaloids in Meconopsi species. J Chromatogr A 1216(42):7013–7023CrossRefPubMedGoogle Scholar
  42. 42.
    Switzar L, Giera M, Lingeman H, Irth H, Niessen WMA (2011) Protein digestion optimization for characterization of drug–protein adducts using response surface modeling. J Chromatogr A 1218(13):1715–1723, doi: papers3://publication/doi/ 10.1016/j.chroma.2010.12.043 CrossRefPubMedGoogle Scholar
  43. 43.
    Mischak H, Allmaier G, Apweiler R, Attwood T, Baumann M, Benigni A, Bennett SE, Bischoff R, Bongcam-Rudloff E, Capasso G, Coon JJ, D'Haese P, Dominiczak AF, Dakna M, Dihazi H, Ehrich JH, Fernandez-Llama P, Fliser D, Frokiaer J, Garin J, Girolami M, Hancock WS, Haubitz M, Hochstrasser D, Holman RR, Ioannidis JPA, Jankowski J, Julian BA, Klein JB, Kolch W, Luider T, Massy Z, Mattes WB, Molina F, Monsarrat B, Novak J, Peter K, Rossing P, Sánchez-Carbayo M, Schanstra JP, Semmes OJ, Spasovski G, Theodorescu D, Thongboonkerd V, Vanholder R, Veenstra TD, Weissinger E, Yamamoto T, Vlahou A (2010) Recommendations for biomarker identification and qualification in clinical proteomics. Science Transl Med 2 (46): 46ps42Google Scholar
  44. 44.
    Anderle M, Roy S, Lin H, Becker C, Joho K (2004) Quantifying reproducibility for differential proteomics: noise analysis for protein liquid chromatography-mass spectrometry of human serum. Bioinformatics 20(18):3575–3582CrossRefPubMedGoogle Scholar
  45. 45.
    Liu P, Hwang JTG (2007) Quick calculation for sample size while controlling false discovery rate with application to microarray analysis. Bioinformatics 23(6):739–746CrossRefPubMedGoogle Scholar
  46. 46.
    Fisher RA, Others (1926) The arrangement of field experiments. doi: citeulike-article-id:10709753Google Scholar
  47. 47.
    Westbrook JA, Noirel J, Brown JE, Wright PC, Evans CA (2015) Quantitation with chemical tagging reagents in biomarker studies. Proteomics Clin Appl 9(3-4):295–300. doi: 10.1002/prca.201400120 CrossRefPubMedGoogle Scholar
  48. 48.
    Oberg AL, Mahoney DW, Eckel-Passow JE, Malone CJ, Wolfinger RD, Hill EG, Cooper LT, Onuma OK, Spiro C, Therneau TM, Bergen Iii HR (2008) Statistical analysis of relative labeled mass spectrometry data from complex samples using ANOVA. J Proteome Res 7(1):225–233, doi: papers3://publication/doi/ 10.1021/pr700734f CrossRefPubMedPubMedCentralGoogle Scholar
  49. 49.
    Niepel M, Spencer SL, Sorger PK (2009) Non-genetic cell-to-cell variability and the consequences for pharmacology. Curr Opin Chem Biol 13(5–6):556–561, doi: CrossRefPubMedPubMedCentralGoogle Scholar
  50. 50.
    Liu H, Sadygov RG, Yates JR 3rd (2004) A model for random sampling and estimation of relative protein abundance in shotgun proteomics. Anal Chem 76(14):4193–4201. doi: 10.1021/ac0498563 CrossRefPubMedGoogle Scholar
  51. 51.
    Mercier C, Truntzer C, Pecqueur D, Gimeno J-P, Belz G, Roy P (2009) Mixed-model of ANOVA for measurement reproducibility in proteomics. J Proteomics 72(6):974–981, doi: CrossRefPubMedGoogle Scholar
  52. 52.
    Day SJ, Altman DG (2000) Blinding in clinical trials and other studies. BMJ 321(7259):504CrossRefPubMedPubMedCentralGoogle Scholar
  53. 53.
    Begley CG (2013) Six red flags for suspect work. Nature 497(7450):433–4CrossRefPubMedGoogle Scholar
  54. 54.
    MacCoun R, Perlmutter S (2015) Blind analysis: hide results to seek the truth. Nature 526(7572):187–189CrossRefPubMedGoogle Scholar
  55. 55.
    Gatto L, Christoforou A (2014) Using R and Bioconductor for proteomics data analysis. Biochim Biophys Acta 1844(1 Pt A):42–51CrossRefPubMedGoogle Scholar
  56. 56.
    Colangelo CM, Chung L, Bruce C, Cheung K-H (2013) Review of software tools for design and analysis of large scale MRM proteomic datasets. Methods 61(3):287–298CrossRefPubMedPubMedCentralGoogle Scholar
  57. 57.
    Schwacke JH, Hill EG, Krug EL, Comte-Walters S, Schey KL (2009) iQuantitator: a tool for protein expression inference using iTRAQ. BMC Bioinformatics 10:342. doi: 10.1186/1471-2105-10-342 CrossRefPubMedPubMedCentralGoogle Scholar
  58. 58.
    Gonzalez-Galarza FF, Lawless C, Hubbard SJ, Fan J, Bessant C, Hermjakob H, Jones AR (2012) A critical appraisal of techniques, software packages, and standards for quantitative proteomic analysis. OMICS 16(9):431–442, doi: papers3://publication/doi/ 10.1089/omi.2012.0022 CrossRefPubMedPubMedCentralGoogle Scholar
  59. 59.
    Kleinbaum DG, Kleinbaum DG (1998) Applied regression analysis and other multivariable methods, 3rd edn. Duxbury, Pacific GroveGoogle Scholar
  60. 60.
    Choi M, Chang CY, Clough T, Broudy D, Killeen T, MacLean B, Vitek O (2014) MSstats: an R package for statistical analysis of quantitative mass spectrometry-based proteomic experiments. Bioinformatics 30(17):2524–2526CrossRefPubMedGoogle Scholar
  61. 61.
    Kerr NL (1998) HARKing: hypothesizing after the results are known. Pers Soc Psychol Rev 2(3):196–217CrossRefPubMedGoogle Scholar
  62. 62.
    Kuhn M. Desirability: desirability function optimization and ranking.
  63. 63.
    Derringer G, Suich R (1980) Simultaneous optimization of several response variables. J Qual Technol 12:214–219Google Scholar

Copyright information

© Springer Science+Business Media LLC 2017

Authors and Affiliations

  1. 1.Lawrence J. Ellison Institute for Transformative Medicine of USCKeck School of Medicine of USCLos AngelesUSA

Personalised recommendations