Journal of Computer-Aided Molecular Design

, Volume 33, Issue 1, pp 19–34 | Cite as

Convolutional neural network scoring and minimization in the D3R 2017 community challenge

  • Jocelyn Sunseri
  • Jonathan E. King
  • Paul G. Francoeur
  • David Ryan KoesEmail author


We assess the ability of our convolutional neural network (CNN)-based scoring functions to perform several common tasks in the domain of drug discovery. These include correctly identifying ligand poses near and far from the true binding mode when given a set of reference receptors and classifying ligands as active or inactive using structural information. We use the CNN to re-score or refine poses generated using a conventional scoring function, Autodock Vina, and compare the performance of each of these methods to using the conventional scoring function alone. Furthermore, we assess several ways of choosing appropriate reference receptors in the context of the D3R 2017 community benchmarking challenge. We find that our CNN scoring function outperforms Vina on most tasks without requiring manual inspection by a knowledgeable operator, but that the pose prediction target chosen for the challenge, Cathepsin S, was particularly challenging for de novo docking. However, the CNN provided best-in-class performance on several virtual screening tasks, underscoring the relevance of deep learning to the field of drug discovery.


Protein–ligand scoring Machine learning Neural networks Virtual screening D3R Drug design data 



J.S. is supported by a fellowship from The Molecular Sciences Software Institute under NSF Grant ACI-1547580. This work is supported by R01GM108340 from the National Institute of General Medical Sciences and by a GPU donation from the NVIDIA corporation.


  1. 1.
    Wang J-C, Lin J-H (2013) Scoring functions for prediction of protein-ligand interactions. Curr Pharm Des 19(12):2174–2182Google Scholar
  2. 2.
    Colwell LJ (2018) Statistical and machine learning approaches to predicting protein-ligand interactions. Curr Opin Struct Biol 49:123–128Google Scholar
  3. 3.
    Braga RC, Alves VM, Silva AC, Nascimento MN, Silva FC, Liao LM, Andrade CH (2014) Virtual screening strategies in medicinal chemistry: the state of the art and current challenges. Curr Top Med Chem 14(16):1899–1912Google Scholar
  4. 4.
    Pérez-Sianes J, Pérez-Sánchez H, Díaz F (2016) Virtual screening: a challenge for deep learning. In: Mohamad MS, Rocha M, Fdez-Riverola F, De Paz JF, De Paz JF (eds) 10th International Conference on practical applications of computational biology and bioinformatics. Springer, Basel, pp 13–22Google Scholar
  5. 5.
    Sliwoski G, Kothiwale S, Meiler J, Lowe EW (2014) Computational methods in drug discovery. Pharmacol Rev 66(1):334–395Google Scholar
  6. 6.
    Jansen JM, Amaro RE, Cornell W, Tseng YJ, Patrick Walters W (2012) Computational chemistry and drug discovery: a call to action. Future Med Chem 4(15):1893–1896Google Scholar
  7. 7.
    Boutros PC, Margolin AA, Stuart JM, Califano A, Stolovitzky G (2014) Toward better benchmarking: challenge-based methods assessment in cancer genomics. Genome Biol 15(9):462Google Scholar
  8. 8.
    Gathiaka S, Liu S, Chiu M, Yang H, Stuckey JA, Kang YN, Delproposto J, Kubish G, Dunbar JB, Carlson HA et al (2016) D3r grand challenge 2015: evaluation of protein-ligand pose and affinity predictions. J Comput-Aided Mol Des 30(9):651–668Google Scholar
  9. 9.
    Gaieb Z, Liu S, Gathiaka S, Chiu M, Yang H, Shao C, Feher VA, Walters WP, Kuhn B, Rudolph MG et al (2018) D3r grand challenge 2: blind prediction of protein-ligand poses, affinity rankings, and relative binding free energies. J Comput-aided Mol Des 32(1):1–20Google Scholar
  10. 10.
    Jiménez Luna J, Skalic M, Martinez-Rosell G (2018) K deep: Protein-ligand absolute binding affinity prediction via 3d-convolutional neural networks. J Chem Inf Model 58(2):287–296Google Scholar
  11. 11.
    Mobley DL, Graves AP, Chodera JD, McReynolds AC, Shoichet BK, Dill KA (2007) Predicting absolute ligand binding free energies to a simple model site. J Mol Biol 371(4):1118–1134Google Scholar
  12. 12.
    Aldeghi M, Heifetz A, Bodkin MJ, Knapp S, Biggin PC (2016) Accurate calculation of the absolute free energy of binding for drug molecules. Chem Sci 7(1):207–218Google Scholar
  13. 13.
    Stjernschantz E, Oostenbrink C (2010) Improved ligand-protein binding affinity predictions using multiple binding modes. Biophys J 98(11):2682–2691Google Scholar
  14. 14.
    Kim R, Skolnick J (2008) Assessment of programs for ligand binding affinity prediction. J Comput Chem 29(8):1316–1331Google Scholar
  15. 15.
    Ashtawy HM, Mahapatra NR (2012) A comparative assessment of ranking accuracies of conventional and machine-learning-based scoring functions for protein-ligand binding affinity prediction. IEEE/ACM Trans Comput Biol Bioinform 9(5):1301–1313Google Scholar
  16. 16.
    Carlson HA (2016) Lessons learned over four benchmark exercises from the community structure—activity resource. J Chem Inf Model 56:951–954Google Scholar
  17. 17.
    Smith RD, Damm-Ganamet KL, Dunbar JB Jr, Ahmed A, Chinnaswamy K, Delproposto JE, Kubish GM, Tinberg CE, Khare SD, Dou J et al (2015) Csar benchmark exercise 2013: evaluation of results from a combined computational protein design, docking, and scoring/ranking challenge. J Chem Inf Model 56(6):1022–1031Google Scholar
  18. 18.
    Carlson HA, Smith RD, Damm-Ganamet KL, Stuckey JA, Ahmed A, Convery MA, Somers DO, Kranz M, Elkins PA, Cui G et al (2016) Csar 2014: a benchmark exercise using unpublished data from pharma. J Chem Inf Model 56(6):1063–1077Google Scholar
  19. 19.
    Harder E, Damm W, Maple J, Wu C, Reboul M, Xiang JY, Wang L, Lupyan D, Dahlgren MK, Knight JL, Kaus JW, Cerutti DS, Krilov G, Jorgensen WL, Abel R, Friesner RA (2016) OPLS3: a force field providing broad coverage of drug-like small molecules and proteins. J Chem Theory Comput 12(1):281–296. Google Scholar
  20. 20.
    Yin S, Biedermannova L, Vondrasek J, Dokholyan NV (2008) MedusaScore: an accurate force field-based scoring function for virtual drug screening. J Chem Inf Model 48(8):1656–1662. Google Scholar
  21. 21.
    Case DA, Cheatham TE, Darden T, Gohlke H, Luo R, Merz KM, Onufriev A, Simmerling C, Wang B, Woods RJ (2005) The Amber biomolecular simulation programs. J. Comput. Chem. 26(16):1668–1688. ISSN 1096-987X.
  22. 22.
    Cheng T, Li X, Li Y, Liu Z, Wang R (2009) Comparative assessment of scoring functions on a diverse test set. J Chem Inf Model 49(4):1079–1093. Google Scholar
  23. 23.
    Ewing TJ, Makino S, Skillman AG, Kuntz ID (2001) DOCK 4.0: search strategies for automated molecular docking of flexible molecule databases. J Comput-Aided Mol Des 15(5):411–28Google Scholar
  24. 24.
    Brooks BR, Bruccoleri RE, Olafson BD (1983) CHARMM: a program for macromolecular energy, minimization, and dynamics calculations. J Comput Chem 4(2):187–217 ISSN 1096-987XGoogle Scholar
  25. 25.
    Lindahl E, Hess B, Van Der Spoel D (2001) GROMACS 3.0: a package for molecular simulation and trajectory analysis. J Mol Model 7(8):306–317 ISSN 1610-2940Google Scholar
  26. 26.
    Jorgensen WL, Maxwell DS, Tirado-Rives J (1996) Development and testing of the OPLS all-atom force field on conformational energetics and properties of organic liquids. J Am Chem Soc 118(45):11225–11236 ISSN 0002-7863Google Scholar
  27. 27.
    Jones G, Willett P, Glen RC, Leach AR, Taylor R (1997) Development and validation of a genetic algorithm for flexible docking. J Mol Biol 267(3):727–748. Google Scholar
  28. 28.
    Koes DR, Baumgartner MP, Camacho CJ (2013) Learned lessons in empirical scoring with smina from the CSAR 2011 benchmarking exercise. J Chem Inf Model. Google Scholar
  29. 29.
    Eldridge MD, Murray CW, Auton TR, Paolini GV, Mee RP (1997) Empirical scoring functions: I. The development of a fast empirical scoring function to estimate the binding affinity of ligands in receptor complexes. J Comput-Aided Mol Des 11(5):425–445Google Scholar
  30. 30.
    Böhm HJ (1994) The development of a simple empirical scoring function to estimate the binding constant for a protein-ligand complex of known three-dimensional structure. J Comput-Aided Mol Des 8(3):243–256 ISSN 0920-654XGoogle Scholar
  31. 31.
    Wang R, Lai L, Wang S (2002) Further development and validation of empirical scoring functions for structure-based binding affinity prediction. J Comput-Aided Mol Des 16(1):11–26 ISSN 0920-654XGoogle Scholar
  32. 32.
    Korb O, Stützle T, Exner TE (2009) Empirical scoring functions for advanced protein-ligand docking with PLANTS. J Chem Inf Model 49(1):84–96. ISSN 1549-9596
  33. 33.
    Friesner RA, Banks JL, Murphy RB, Halgren TA, Klicic JJ, Mainz DT, Repasky MP, Knoll EH, Shelley M, Perry JK, Shaw DE, Francis P, Shenkin PS (2004) Glide: a new approach for rapid, accurate docking and scoring. 1. Method and assessment of docking accuracy. J Med Chem 47(7):1739–1749. Google Scholar
  34. 34.
    Trott O, Olson AJ (2009) AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J Comput Chem. ISSN 1096-987X
  35. 35.
    Huang SY, Zou X (2010) Mean-force scoring functions for protein-ligand binding. Annu Rep Comp Chem 6:280–296 ISSN 1574-1400Google Scholar
  36. 36.
    Muegge I, Martin YC (1999) A general and fast scoring function for protein-ligand interactions: a simplified potential approach. J Med Chem 42(5):791–804. Google Scholar
  37. 37.
    Gohlke H, Hendlich M, Klebe G (2000) Knowledge-based scoring function to predict protein-ligand interactions. J Mol Biol 295(2):337–356Google Scholar
  38. 38.
    Zhou H, Skolnick J (2011) GOAP: a generalized orientation-dependent, all-atom statistical potential for protein structure prediction. Biophys J 101(8):2043–2052. Google Scholar
  39. 39.
    Mooij WT, Verdonk ML (2005) General and targeted statistical potentials for protein-ligand interactions. Proteins 61(2):272–287. Google Scholar
  40. 40.
    Ballester PJ, Mitchell JBO (2010) A machine learning approach to predicting protein-ligand binding affinity with applications to molecular docking. Bioinformatics 26(9):1169. ISSN 1367-4803
  41. 41.
    Huang SY, Zou X (2006) An iterative knowledge-based scoring function to predict protein-ligand interactions: II. Validation of the scoring function. J Comput Chem 27(15):1876–1882. ISSN 1096-987X
  42. 42.
    Rojas R (2013) Neural networks: a systematic introduction. Springer Science and Business Media, BerlinGoogle Scholar
  43. 43.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444Google Scholar
  44. 44.
    Durrant JD, McCammon JA (2010) Nnscore: a neural-network-based scoring function for the characterization of protein-ligand complexes. J Chem Inf Model 50(10):1865–1871. Google Scholar
  45. 45.
    Durrant JD, McCammon JA (2011) Nnscore 2.0: a neural-network receptor-ligand scoring function. J Chem Inf Model 51(11):2897–2903. Google Scholar
  46. 46.
    Chupakhin V, Marcou G, Baskin I, Varnek A, Rognan D (2013) Predicting ligand binding modes from neural networks trained on protein-ligand interaction fingerprints. J Chem Inf Model 53(4):763–772. Google Scholar
  47. 47.
    Ashtawy HM, Mahapatra NR (2015) Machine-learning scoring functions for identifying native poses of ligands docked to known and novel proteins. BMC Bioinform 16(6):1–17. ISSN 1471-2105
  48. 48.
    Jorissen RN, Gilson MK (2005) Virtual screening of molecular databases using a support vector machine. J Chem Inf Model 45(3):549–561. Google Scholar
  49. 49.
    Zilian David, Sotriffer Christoph A (2013) Sfcscore rf: a random forest-based scoring function for improved affinity prediction of protein-ligand complexes. Journal of chemical information and modeling 53(8):1923–1933. Google Scholar
  50. 50.
    Gomes J, Ramsundar B, Feinberg EN, Pande VS (2017) Atomic convolutional networks for predicting protein-ligand binding affinity. arXiv preprint arXiv:1703.10603
  51. 51.
    Wallach I, Dzamba M, Heifets A (2015) Atomnet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery. arXiv preprint arXiv:1510.02855
  52. 52.
    Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Solla SA, Leen TK, Müller KR (eds) Advances in neural information processing systems. MIT Press, London, pp 2224–2232Google Scholar
  53. 53.
    Schütt KT, Kindermans PJ, Sauceda HE, Chmiela S, Tkatchenko A, Müller K-R (2017) Moleculenet: a continuous-filter convolutional neural network for modeling quantum interactions. arXiv preprint arXiv:1706.08566
  54. 54.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Solla SA, Leen TK, Müller KR (eds) Advances in neural information processing systems. MIT Press, London, pp 1097–1105Google Scholar
  55. 55.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9Google Scholar
  56. 56.
    He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR, abs/1512.03385.arXiv:1512.03385Google Scholar
  57. 57.
    Ragoza M, Hochuli J, Idrobo E, Sunseri J, Koes DR (2017a) Protein-ligand scoring with convolutional neural networks. J Chem Inf Model 57(4):942–957Google Scholar
  58. 58.
    Ragoza M, Turner L, Koes DR (2017) Ligand pose optimization with atomic grid-based convolutional neural networks. arXiv preprint arXiv:1710.07400
  59. 59.
    Hochuli J, Helbling A, Skaist T, Ragoza M, Koes DR (2018) Visualizing convolutional neural network protein-ligand scoring. arXiv preprint arXiv:1803.02398
  60. 60.
    Trott O, Olson AJ (2010) AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. J Comput Chem 31(2):455–461. Google Scholar
  61. 61.
    Liu Z, Minyi S, Han L, Liu J, Yang Q, Li Y, Wang R (2017) Forging the basis for developing proteinligand interaction scoring functions. Acc Chem Res 50(2):302–309. Google Scholar
  62. 62.
    Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093
  63. 63.
    rdkit. RDKit: Open-Source Cheminformatics. Accessed 6 Nov 2017
  64. 64.
    Kufareva I, Ilatovskiy AV, Abagyan R (2011) Pocketome: an encyclopedia of small-molecule binding sites in 4d. Nucleic Acids Res 40(D1):D535–D540Google Scholar
  65. 65.
    DeLano WL, Schrödinger, LLC. The PyMOL molecular graphics system, version 1.8. (2015)Google Scholar
  66. 66.
    O’Boyle NM, Banck M, James CA, Morley C, Vandermeersch T, Hutchison GR (2011) Open babel: an open chemical toolbox. J Cheminf 3(1):33Google Scholar
  67. 67.
    Shewchuk LM, Hassell AM, Ellis B, Holmes WD, Davis R, Horne EL, Kadwell SH, McKee DD, Moore JT (2000) Structure of the tie2 rtk domain: self-inhibition by the nucleotide binding loop, activation loop, and c-terminal tail. Structure 8(11):1105–1113Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computational & Systems Biology, School of MedicineUniversity of PittsburghPittsburghUSA

Personalised recommendations