Reporting on the Performance and Usability of Planning Support Systems—Towards a Common Understanding

  • Haozhi PanEmail author
  • Brian Deal


To assess the implementation of Planning Support Systems (PSSs) and scenario planning strategies, a common understanding of model goals, performance, and usability are among several issues that need to be addressed. Without a common understanding, resulting policies that are drawn from model outcomes could be compromised. Reporting methods of PSS results should be objective, reasonable, understandable, and useful. We describe a process including several methods of reporting PSS results that allows for minor variation and understandably communicating the results to PSS users. This approach includes finding meaningful resolution and probability matching of PSS model results. We show that our approach is a theoretically reasonable and objective method for reporting PSS results in planning practice. We test our approach using the LEAM PSS in an application in northern Illinois. In this case, our model results inform planners using an easily comprehensible spatial resolution over which the simulation provides useful information for future land-use scenarios. We also apply our reporting methods to compare a preferred scenario to a business-as-usual scenario. The result prompts local planners and stakeholders to rethink their plan implementation strategy for preferred scenario implementation. We conclude that next steps for PSS model development will require both technical and practical focuses in the model reporting and implementation arena.


Planning support systems Model performance Usefulness Scenario planning Spatial analytics Land-use 



The work used the ROGER supercomputer, which is supported by NSF under grant number: 1429699. We thank Shaowen Wang’s assistance with super-computing technical assistance, which was made possible through the CyberGIS Center Help Desk.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Supplementary material

12061_2019_9296_MOESM1_ESM.docx (20 kb)
ESM 1 (DOCX 20 kb)


  1. Al-Ahmadi, K., See, L., Heppenstall, A., & Hogg, J. (2009). Calibration of a fuzzy cellular automata model of urban dynamics in Saudi Arabia. Ecological Complexity, 6, 80–101.CrossRefGoogle Scholar
  2. Al-kheder, S., Wang, S., & Shan, J. (2008). Fuzzy inference guided cellular automata urban-growth modelling using multi- temporal satellite images. International Journal of Geographical Information Science, 22(11–12), 1271–1293.CrossRefGoogle Scholar
  3. Almeida, C. M., Monteiro, A. M. V., Camara, G., Soares-Filho, B. S., Cerqueira, G. C., Pennachin, C. L., & Batty, M. (2005). GIS and remote sensing as tools for the simulation of urban land-use change. International Journal of Geographical Information Science, 26(4), 759–774.Google Scholar
  4. Andrews, C. J. (2002). Humble analysis: the practice of joint fact-finding. Praeger: Greenwood Publishing Group.Google Scholar
  5. Ballestores, F., Jr., & Qiu, Z. (2012). An integrated parcel-based land use change model using cellular automata and decision tree. Proceedings of the International Academy of Ecology and Environmental Sciences, 2(2), 53.Google Scholar
  6. Biermann, S. (2011). Planning support systems in a multi-dualistic spatial planning context. Journal of Urban Technology, 18(4), 5–37.CrossRefGoogle Scholar
  7. Caruso, G., Rounsevell, M., & Cojocaru, G. (2005). Exploring a spatio-dynamic neighbourhood-based model of residential behaviour in the Brussels periurban area. International Journal of Geographical Information Science, 19(2), 103–123.CrossRefGoogle Scholar
  8. Chakraborty, A., Kaza, N., Knaap, G., & Deal, B. (2011). Robust plans and contingent plans. Journal of the American Planning Association, 77(3), 251–266.CrossRefGoogle Scholar
  9. Champlin, C., te Brömmelstroet, M., & Pelzer, P. (2018). Tables, tablets and flexibility: evaluating planning support system performance under different conditions of use. Applied Spatial Analysis and Policy, 1–25.
  10. Chaudhuri, G., & Clarke, K. C. (2014). Temporal accuracy in urban growth forecasting: a study using the SLEUTH model. Transactions in GIS, 18(2), 302–320.CrossRefGoogle Scholar
  11. Clarke, K. C., & Gaydos, L. J. (1998). Loose-coupling a cellular automaton model and GIS: long-term urban growth prediction for San Francisco and Washington/Baltimore. International Journal of Geographical Information Science, 12(7), 699–714.CrossRefGoogle Scholar
  12. Costanza, R. (1989). Model goodness of fit: a multiple resolution procedure. Ecological Modelling, 47, 199–215.CrossRefGoogle Scholar
  13. Deal, B., Pan, H., Pallathucheril, V., & Fulton, G. (2017a). Urban resilience and planning support systems: the need for sentience. Journal of Urban Technology, 24(1), 29–45.Google Scholar
  14. Deal, B., Pan, H., Timm, S., & Pallathucheril, V. (2017b). The role of multidirectional temporal analysis in scenario planning exercises and planning support systems. Computers, Environment and Urban Systems, 64, 91–102.CrossRefGoogle Scholar
  15. Demetriou, D., Stillwell, J., & See, L. (2012). An integrated planning and decision support system (IPDSS) for land consolidation: theoretical framework and application of the land-redistribution modules. Environment and Planning. B, Planning & Design, 39(4), 609–628.CrossRefGoogle Scholar
  16. Fry, J., et al. (2011). Completion of the 2006 National Land Cover Database for the conterminous United States. PE&RS, 77(9), 858–864.Google Scholar
  17. Geertman, S. (2002). Participatory planning and GIS: a PSS to bridge the gap. Environment and Planning. B, Planning & Design, 21, 21–35.CrossRefGoogle Scholar
  18. Geertman, S. (2006). Potentials for planning support: a planning-conceptual approach. Environment and Planning. B, Planning & Design, 33(6), 863–880.CrossRefGoogle Scholar
  19. Goodchild, M. F. (1986). Spatial Autocorrelation: Concepts and Techniques in Modern Geography. Norwich: Geo Books.Google Scholar
  20. Klosterman, R. E. (1997). Planning support systems: a new perspective on computer-aided planning. Journal of Planning Education and Research, 17(1), 45–54.CrossRefGoogle Scholar
  21. Klosterman, R. E. (1999). The what if? Collaborative planning support system. Environment and Planning. B, Planning & Design, 26, 393–408.CrossRefGoogle Scholar
  22. Klosterman, R. E., & Pettit, C. J. (2005). An update on planning support systems. Environment and Planning B, 32(4), 477–484.CrossRefGoogle Scholar
  23. Lau, K. H., & Kam, B. H. (2005). A cellular automata model for urban land-use simulation. Environment and Planning. B, Planning & Design, 32, 247–263.CrossRefGoogle Scholar
  24. Li, X., & Yeh, A. G. (2004). Data mining of cellular automata’s transition rules. International Journal of Geographical Information Science, 18(8), 723–744.CrossRefGoogle Scholar
  25. Li, X., Yang, Q., & Liu, X. (2008). Discovering and evaluating urban signatures for simulating compact development using cellular automata. Landscape and Urban Planning, 86, 177–186.CrossRefGoogle Scholar
  26. Li, X., Gong, P., Yu, L., & Hu, T. (2017). A segment derived patch-based logistic cellular automata for urban growth modeling with heuristic rules. Computers, Environment and Urban Systems, 65, 140–149.CrossRefGoogle Scholar
  27. Ligmann-Zielinska, A., & Sun, L. (2010). Applying time-dependent variance-based global sensitivity analysis to represent the dynamics of an agent-based model of land use change. International Journal of Geographical Information Science, 24(12), 1829–1850.CrossRefGoogle Scholar
  28. Liu, X., Li, X., Liu, L., & Ai, B. (2008). A bottom-up approach to discover transition rules of cellular automata using ant intelligence. International Journal of Geographical Information Science, 22(11–12), 1247–1269.CrossRefGoogle Scholar
  29. Marsden, R. (2015). A web based information system for planning support in Barnsley. Applied Spatial Analysis and Policy, 8(2), 131–153.CrossRefGoogle Scholar
  30. McHenry County Regional Planning Commission. (2010). McHenry County 2030 comprehensive plan. Retrieved from Accessed 29 May 2018.
  31. Nedović-Budić, Z., Kan, R. G., Johnston, D. M., Sparks, R. E., & White, D. C. (2006). CommunityViz-based prototype model for assessing development impacts in a naturalized floodplain—EmiquonViz. Journal of Urban Planning and Development, 132(4), 201–210.CrossRefGoogle Scholar
  32. Pan, H., Deal, B., Chen, Y., & Hewings, G. (2018a). A reassessment of urban structure and land-use patterns: distance to CBD or network-based?—Evidence from Chicago. Regional Science and Urban Economics, 70, 215–228.CrossRefGoogle Scholar
  33. Pan, H., Deal, B., Destouni, G., Zhang, Y., & Kalantari, Z. (2018b). Sociohydrology modeling for complex urban environments in support of integrated land and water resource management practices. Land Degradation & Development, 29(10), 3639–3652.CrossRefGoogle Scholar
  34. Pan, H., Zhang, L., Cong, C., Deal, B., & Wang, Y. (2019). A dynamic and spatially explicit modeling approach to identify the ecosystem service implications of complex urban systems interactions. Ecological Indicators, 102, 426–436.Google Scholar
  35. Parker, D. C., Manson, S. M., Janssen, M. A., Hoffmann, M. J., & Deadman, P. (2003). Multi-agent systems for the simulation of land-use and land-cover change: a review. Annals of the Association of American Geographers, 93(2), 314–337.CrossRefGoogle Scholar
  36. Pelzer, P. (2017). Usefulness of planning support systems: a conceptual framework and an empirical illustration. Transportation Research Part A: Policy and Practice, 104, 84–95.Google Scholar
  37. Pelzer, P., Geertman, S., van der Heijden, R., & Rouwette, E. (2014). The added value of planning support systems: a practitioner’s perspective. Computers, Environment and Urban Systems, 48, 16–27.Google Scholar
  38. Pelzer, P., Geertman, S., & van der Heijden, R. (2016). A comparison of the perceived added value of PSS applications in group settings. Computers, Environment and Urban Systems, 56, 25–35.CrossRefGoogle Scholar
  39. Pettit, C. J. (2005). Use of a collaborative GIS-based planning-support system to assist in formulating a sustainable-development scenario for Hervey Bay, Australia. Environment and Planning. B, Planning & Design, 32(4), 523–545.CrossRefGoogle Scholar
  40. Pettit, C. J., Klosterman, R. E., Delaney, P., Whitehead, A. L., Kujala, H., Bromage, A., & Nino-Ruiz, M. (2015). The online what if? Planning support system: A land suitability application in Western Australia. Applied Spatial Analysis and Policy, 8(2), 93–112.Google Scholar
  41. Pontius, R. G., Jr., Peethambaram, S., & Castella, J.-C. (2011). Comparison of three maps at multiple resolutions: a case study of land change simulation in Cho Don district, Vietnam. Annals of the Association of American Geographers, 101(1), 45–62.CrossRefGoogle Scholar
  42. Pontius, R. G., Boersma, W., Castella, J.-C., Clarke, K., de Nijs, T., Dietzel, C., et al. (2008). Comparing the input, output, and validation maps for several models of land change. The Annals of Regional Science, 42(1), 11–37.CrossRefGoogle Scholar
  43. Russo, P., Lanzilotti, R., Costabile, M. F., & Pettit, C. J. (2018). Towards satisfying practitioners in using planning support systems. Computers, Environment and Urban Systems, 67, 9–20.CrossRefGoogle Scholar
  44. Saarloos, D. J., Arentze, T. A., Borgers, A. W., & Timmermans, H. J. (2008). A multi-agent paradigm as structuring principle for planning support systems. Computers, Environment and Urban Systems, 32(1), 29–40.CrossRefGoogle Scholar
  45. Silva, C., Bertolini, L., te Brömmelstroet, M., Milakis, D., & Papa, E. (2017). Accessibility instruments in planning practice: Bridging the implementation gap. Transport Policy, 53, 135–145.Google Scholar
  46. te Brömmelstroet, M. (2009). The relevance of research in planning support systems: a response to Janssen et al. Environment and Planning. B, Planning & Design, 36(1), 4–7.CrossRefGoogle Scholar
  47. te Brömmelstroet, M. (2012). Transparency, flexibility, simplicity: from buzzwords to strategies for real PSS improvement. Computers, Environment and Urban Systems, 36(1), 96–104.CrossRefGoogle Scholar
  48. te Brömmelstroet, M. (2013). Performance of planning support systems: what is it, and how do we report on it? Computers, Environment and Urban Systems, 41, 299–308.CrossRefGoogle Scholar
  49. te Brömmelstroet, M. (2017a). PSS are more user-friendly, but are they also increasingly useful? Transportation Research Part A: Policy and Practice, 104, 96–107.Google Scholar
  50. te Brömmelstroet, M. (2017b). Towards a pragmatic research agenda for the PSS domain. Transportation Research Part A: Policy and Practice, 104, 77–83.Google Scholar
  51. te Brömmelstroet, M., & Bertolini, L. (2008). Developing land use and transport PSS: meaningful information through a dialogue between modelers and planners. Transport Policy, 15(4), 251–259.CrossRefGoogle Scholar
  52. te Brömmelstroet, M. T., & Schrijnen, P. M. (2010). From planning support systems to mediated planning support: a structured dialogue to overcome the implementation gap. Environment and Planning B: Planning and Design, 37(1), 3–20.Google Scholar
  53. Torrens, P. M. (2003). Cellular automata and multi-agent systems as planning support tools. In Planning support systems in practice (pp. 205–222). Berlin: Springer.CrossRefGoogle Scholar
  54. Turner, M. G., Costanza, R., & Sklar, F. H. (1989). Methods to evaluate the performance of spatial simulation models. Ecological Modeling, 48, 1–18.CrossRefGoogle Scholar
  55. Veldkamp, A., & Verburg, P. (2004). Modelling land use change and environmental impact. Journal of Environmental Management, 72(1), 1–3.CrossRefGoogle Scholar
  56. Vonk, G., & Ligtenberg, A. (2010). Socio-technical PSS development to improve functionality and usability—sketch planning using a Maptable. Landscape and Urban Planning, 94(3–4), 166–174.CrossRefGoogle Scholar
  57. Waddell, P. (2002). UrbanSim modeling urban development for land use, transportation, and environmental planning. Journal of the American Planning Association, 68(3), 297–314.CrossRefGoogle Scholar
  58. Waddell, P. (2011). Integrated land use and transportation planning and modelling: addressing challenges in research and practice. Transport Reviews, 31(2), 209–229.CrossRefGoogle Scholar
  59. White, R., & Engelen, G. (1994). Cellular dynamics and GIS: modelling spatial complexity. Geographical System, (1), 237–253.Google Scholar
  60. Wu, F. (2002). Calibration of stochastic cellular automata: the application to rural-urban land conversions. International Journal of Geographical Information Science, 16(8), 795–851.CrossRefGoogle Scholar
  61. Yang, Q., Li, X., & Shi, X. (2008). Cellular automata for simulating land use changes based on support vector machines. Computers & Geosciences, 34(2008), 592–602.CrossRefGoogle Scholar
  62. Yang, T., Pan, H., Hewings, G., & Jin, Y. (2019). Understanding urban sub-centers with heterogeneity in agglomeration economies—where do emerging commercial establishments locate? Cities, 86, 25–36.CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Regional Economics Applications LaboratoryUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Department of Landscape ArchitectureUniversity of Illinois at Urbana-ChampaignChampaignUSA

Personalised recommendations