Skip to main content
SpringerLink
Log in

Reliability and One-Year Stability of the PIN3 Neighborhood Environmental Audit in Urban and Rural Neighborhoods

  • Published:
Journal of Urban Health Aims and scope Submit manuscript

Abstract

Reliable and stable environmental audit instruments are needed to successfully identify the physical and social attributes that may influence physical activity. This study described the reliability and stability of the PIN3 environmental audit instrument in both urban and rural neighborhoods. Four randomly sampled road segments in and around a one-quarter mile buffer of participants’ residences from the Pregnancy, Infection, and Nutrition (PIN3) study were rated twice, approximately 2 weeks apart. One year later, 253 of the year 1 sampled roads were re-audited. The instrument included 43 measures that resulted in 73 item scores for calculation of percent overall agreement, kappa statistics, and log-linear models. For same-day reliability, 81% of items had moderate to outstanding kappa statistics (kappas ≥ 0.4). Two-week reliability was slightly lower, with 77% of items having moderate to outstanding agreement using kappa statistics. One-year stability had 68% of items showing moderate to outstanding agreement using kappa statistics. The reliability of the audit measures was largely consistent when comparing urban to rural locations, with only 8% of items exhibiting significant differences (α < 0.05) by urbanicity. The PIN3 instrument is a reliable and stable audit tool for studies assessing neighborhood attributes in urban and rural environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Sallis JF, Owen N, Fisher EB. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: theory, research, and practice. 4th ed. San Francisco, CA: Jossey-Bass; 2008. p. 465–86.

  2. Sallis JF, Cervero RB, Ascher W, Henderson KA, Kraft MK, Kerr J. An ecological approach to creating active living communities. Annu Rev Public Health. 2006;27:297–322.

    Article  PubMed  Google Scholar 

  3. Sallis JF. Measuring physical activity environments: a brief history. Am J Prev Med. 2009;36(4 Suppl):S86–92.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Brownson RC, Hoehner CM, Day K, Forsyth A, Sallis JF. Measuring the built environment for physical activity: state of the science. Am J Prev Med. 2009;36(4 Suppl):S99–123.e112.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Bethlehem JR, Mackenbach JD, Ben-Rebah M, Compernolle S, Glonti K, Bárdos H, et al. The SPOTLIGHT virtual audit tool: a valid and reliable tool to assess obesogenic characteristics of the built environment. Int J Health Geogr. 2014;13:52.

  6. Clarke P, Ailshire J, Melendez R, Bader M, Morenoff J. Using Google Earth to conduct a neighborhood audit: reliability of a virtual audit instrument. Health Place. 2010;16(6):1224–9.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Bader MDM, Mooney SJ, Lee YJ, Sheehan D, Neckerman KM, Rundle AG, et al. Development and deployment of the Computer Assisted Neighborhood Visual Assessment System (CANVAS) to measure health-related neighborhood conditions. Health Place. 2015;31:163–72.

  8. Mooney SJ, Bader MDM, Lovasi GS, Neckerman KM, Teitler JO, Rundle AG. Validity of an ecometric neighborhood physical disorder measure constructed by virtual street audit. Am J Epidemiol. 2014;180(6):626–35.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Rundle AG, Bader MD, Richards CA, Neckerman KM, Teitler JO. Using Google Street View to audit neighborhood environments. Am J Prev Med. 2010;40(1):94–100.

    Article  Google Scholar 

  10. Brownson RC, Hoehner CM, Brennan LK, Cook RA, Elliott MB, McMullen KM. Reliability of 2 instruments for auditing the environment for physical activity. J Phys Act Health. 2004;1(3):191–208.

    Article  Google Scholar 

  11. Nickelson J, Wang AR, Mitchell QP, Hendricks K, Paschal A. Inventory of the physical environment domains and subdomains measured by neighborhood audit tools: a systematic literature review. J Environ Psychol. 2013;36:179–89.

    Article  Google Scholar 

  12. Millstein RA, Cain KL, Sallis JF, Conway TL, Geremia C, Frank LD, et al. Development, scoring, and reliability of the Microscale Audit of Pedestrian Streetscapes (MAPS). BMC Public Health. 2013;13:403.

  13. Malecki KC, Engelman CD, Peppard PE, Nieto FJ, Grabow ML, Bernardinello M, et al. The Wisconsin Assessment of the Social and Built Environment (WASABE): a multi-dimensional objective audit instrument for examining neighborhood effects on health. BMC Public Health. 2014;14

  14. Su M, Du YK, Liu QM, et al. Objective assessment of urban built environment related to physical activity—development, reliability and validity of the China Urban Built Environment Scan Tool (CUBEST). BMC Public Health. 2014;14:109.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Scanlin K, Haardoefer R, Kegler MC, Glanz K. Development of a pedestrian audit tool to assess rural neighborhood walkability. J Phys Act Health. 2014;11(6):1085–96.

    Article  PubMed  Google Scholar 

  16. Leung CW, Gregorich SE, Laraia BA, Kushi LH, Yen IH. Measuring the neighborhood environment: associations with young girls’ energy intake and expenditure in a cross-sectional study. Int J Behav Nutr Phys Act. 2010;7:52.

  17. Parsons JA, Singh G, Scott AN, Nisenbaum R, Balasubramaniam P, Jabbar A, et al. Standardized observation of neighbourhood disorder: does it work in Canada? Int J Health Geogr. 2010;9:6.

  18. Schaefer-McDaniel N. Neighborhood stressors, perceived neighborhood quality, and child mental health in New York City. Health Place. 2009;15(1):148–55.

    Article  PubMed  Google Scholar 

  19. Michael YL, Keast EM, Chaudhury H, Day K, Mahmood A, Sarte AF. Revising the senior walking environmental assessment tool. Prev Med. 2009;48(3):247–9.

    Article  PubMed  Google Scholar 

  20. Franzini L, Elliott MN, Cuccaro P, Schuster M, Gilliland MJ, Grunbaum JA, et al. Influences of physical and social neighborhood environments on children's physical activity and obesity. Am J Public Health. 2009;99(2):271–8.

  21. Furr-Holden CD, Smart MJ, Pokorni JL, et al. The NIfETy method for environmental assessment of neighborhood-level indicators of violence, alcohol, and other drug exposure. Prev Sci. 2008;9(4):245–55.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  22. Zhu X, Lee C. Walkability and safety around elementary schools economic and ethnic disparities. Am J Prev Med. 2008;34(4):282–90.

    Article  PubMed  Google Scholar 

  23. King D. Neighborhood and individual factors in activity in older adults: results from the neighborhood and senior health study. J Aging Phys Act. 2008;16(2):144–70.

    Article  PubMed  Google Scholar 

  24. Kamphuis CB, Giskes K, Kavanagh AM, et al. Area variation in recreational cycling in Melbourne: a compositional or contextual effect? J Epidemiol Community Health. 2008;62(10):890–8.

    Article  PubMed  CAS  Google Scholar 

  25. Foltete J-C, Piombini A. Urban layout, landscape features and pedestrian usage. Landsc Urban Plan. 2007;81:225–34.

    Article  Google Scholar 

  26. Zenk SN, Schulz AJ, Mentz G, House JS, Gravlee CC, Miranda PY, et al. Inter-rater and test-retest reliability: methods and results for the neighborhood observational checklist. Health Place. 2007;13(2):452–65.

  27. Suminski RR, Heinrich KM, Poston WS, Hyder M, Pyle S. Characteristics of urban sidewalks/streets and objectively measured physical activity. J Urban Health. 2008;85(2):178–90.

    Article  PubMed  Google Scholar 

  28. Clifton KJ, Livi Smith AD, Rodriguez D. The development and testing of an audit for the pedestrian environment. Landsc Urban Plan. 2007;80(1–2):95–110.

    Article  Google Scholar 

  29. Hoehner CM, Ivy A, Ramirez LK, Handy S, Brownson RC. Active neighborhood checklist: a user-friendly and reliable tool for assessing activity friendliness. Am J Health Promot. 2007;21(6):534–7.

    Article  PubMed  Google Scholar 

  30. Wright PA, Kloos B. Housing environment and mental health outcomes: A levels of analysis perspective. J Environ Psychol. 2007;27(1):79–89.

    Article  PubMed  PubMed Central  Google Scholar 

  31. McMillan TE. The relative influence of urban form on a child’s travel mode to school. Transp Res A Policy Pract. 2007;41(1):69–79.

    Article  Google Scholar 

  32. Boarnet MG, Day K, Alfonzo M, Forsyth A, Oakes M. The Irvine-Minnesota inventory to measure built environments: reliability tests. Am J Prev Med. 2006;30(2):153–9.

    Article  PubMed  Google Scholar 

  33. Andresen EM, Malmstrom TK, Miller DK, Wolinsky FD. Reliability and validity of observer ratings of neighborhoods. J Aging Health. 2006;18(1):28–36.

    Article  PubMed  Google Scholar 

  34. Laraia BA, Messer L, Kaufman JS, Dole N, Caughy M, O’Campo P, et al. Direct observation of neighborhood attributes in an urban area of the US south: characterizing the social context of pregnancy. Int J Health Geogr. 2006;5:11.

  35. Craig CL, Brownson RC, Cragg SE, Dunn AL. Exploring the effect of the environment on physical activity: a study examining walking to work. Am J Prev Med. 2002;23(2 Suppl):36–43.

    Article  PubMed  Google Scholar 

  36. Pikora TJ, Bull FC, Jamrozik K, Knuiman M, Giles-Corti B, Donovan RJ. Developing a reliable audit instrument to measure the physical environment for physical activity. Am J Prev Med. 2002;23(3):187–94.

    Article  PubMed  Google Scholar 

  37. Caughy MO, O'Campo PJ, Patterson J. A brief observational measure for urban neighborhoods. Health Place. 2001;7(3):225–36.

    Article  PubMed  CAS  Google Scholar 

  38. Loukaitou-Sideris A, Liggett R, Iseki H, Thurlow W. Measuring the effects of built environment on bus stop crime. Environ Plan B: Plan Design. 2001;28(2):255–80.

    Article  Google Scholar 

  39. Evenson KR, Sotres-Alvarez D, Herring AH, Messer L, Laraia BA, Rodríguez DA. Assessing urban and rural neighborhood characteristics using audit and GIS data: derivation and reliability of constructs. Int J Behav Nutr Phys Act. 2009;6:44.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Knapp EA, Nau C, Brandau S, DeWalle J, Hirsch AG, Bailey-Davis L, et al. Community Audit of Social, Civil, and Activity Domains in Diverse Environments (CASCADDE). Am J Prev Med. 2017;52(4):530–40.

  41. Brown AF, Morris DM, Kahn KL, et al. The healthy community neighborhood initiative: rationale and design. Ethn Dis. 2016;26(1):123–32.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Frizzelle BG, Evenson KR, Rodriguez DA, Laraia BA. The importance of accurate road data for spatial applications in public health: customizing a road network. Int J Health Geogr. 2009;8:24.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Census 2000 urban and rural classifications. https://www.census.gov/geo/reference/ua/urban-rural-2000.html. Accessed Sept 26, 2017.

  44. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.

    Article  PubMed  CAS  Google Scholar 

  45. Tanner MA, Young MA. Modeling agreement among raters. J Am Stat Assoc. 1985;80(389):175–80.

    Article  Google Scholar 

  46. Tanner MA, Young MA. Modeling ordinal scale disagreement. Psychol Bull. 1985;98(2):408–15.

    Article  PubMed  CAS  Google Scholar 

  47. Agresti A. An introduction to categorical data analysis. New York, NY: Wiley; 1996.

Download references

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Funding

Funding for this study was provided by the National Institutes of Health (NIH) /National Cancer Institute (#CA109804-01). AKP was supported by a National Research Service Award T32 post-doctoral research fellowship, funded by the NIH, National Heart, Lung, and Blood Institute (#T32HL007055).

Author information

Authors and Affiliations

  1. Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina – Chapel Hill, 123 W Franklin St, Building C, Suite 410, Chapel Hill, NC, 27516, USA

    Anna K. Porter & Fang Wen

  2. Department of Statistical Science, Duke University, Durham, NC, USA

    Amy H. Herring

  3. Department of City and Regional Planning, Institute for Transportation Studies, University of California, Berkeley, Berkeley, CA, USA

    Daniel A. Rodríguez

  4. Oregon Health & Science University - Portland State University School of Public Health, Portland, OR, USA

    Lynne C. Messer

  5. Division of Community Health Sciences, School of Public Health, University of California, Berkeley, Berkeley, CA, USA

    Barbara A. Laraia

  6. Department of Epidemiology, Gillings School of Global Public Health, Center for Health Promotion and Disease Prevention, University of North Carolina – Chapel Hill, Chapel Hill, NC, USA

    Kelly R. Evenson

Authors
  1. Anna K. Porter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fang Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amy H. Herring
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel A. Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lynne C. Messer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Barbara A. Laraia
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kelly R. Evenson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna K. Porter.

Ethics declarations

This study was reviewed and approved by the Institutional Review Board at UNC.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Porter, A.K., Wen, F., Herring, A.H. et al. Reliability and One-Year Stability of the PIN3 Neighborhood Environmental Audit in Urban and Rural Neighborhoods. J Urban Health 95, 431–439 (2018). https://doi.org/10.1007/s11524-018-0243-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11524-018-0243-9

Keywords

Search

Navigation