Measuring and modelling perceptions of the built environment for epidemiological research using crowd-sourcing and image-based deep learning models

2022  Journal Article

Measuring and modelling perceptions of the built environment for epidemiological research using crowd-sourcing and image-based deep learning models

Pub TLDR

The research aims to measure and predict perceptions of the built environment, such as nature quality, beauty, relaxation, and safety, using crowd-sourced methods and deep learning models. By analyzing over 36,000 image comparisons, the study developed models that explained significant portions of these perceptions and found weak to moderate correlations with traditional built environment measures. Future applications will explore the associations between these perception measures and mental health in the Washington State Twin Registry.

DOI: 10.1038/s41370-022-00489-8    PubMed ID: 36369372
 

College of Health researcher(s)

Abstract

Background

Perceptions of the built environment, such as nature quality, beauty, relaxation, and safety, may be key factors linking the built environment to human health. However, few studies have examined these types of perceptions due to the difficulty in quantifying them objectively in large populations.

Objective

To measure and predict perceptions of the built environment from street-view images using crowd-sourced methods and deep learning models for application in epidemiologic studies.

Methods

We used the Amazon Mechanical-Turk crowdsourcing platform where participants compared two street-view images and quantified perceptions of nature quality, beauty, relaxation, and safety. We optimized street-view image sampling methods to improve the quality and resulting perception data specific to participants enrolled in the Washington State Twin Registry (WSTR) health study. We used a transfer learning approach to train deep learning models by leveraging existing image perception data from the PlacePulse 2.0 dataset, which includes 1.1 million image comparisons, and refining based on new WSTR perception data. Resulting models were applied to WSTR addresses to estimate exposures and evaluate associations with traditional built environment measures.

Results

We collected over 36,000 image comparisons and calculated perception measures for each image. Our final deep learning models explained 77.6% of nature quality, 68.1% of beauty, 72.0% of relaxation, and 64.7% of safety in pairwise image comparisons. Applying transfer learning with the new perception labels specific to the WSTR yielded an average improvement of 3.8% for model performance. Perception measures were weakly to moderately correlated with traditional built environment exposures for WSTR participant addresses; for example, nature quality and NDVI (r = 0.55), neighborhood area deprivation (r = −0.16), and walkability (r = −0.20), respectively.

Significance

We were able to measure and model perceptions of the built environment optimized for a specific health study. Future applications will examine associations between these exposure measures and mental health in the WSTR.

Impact statement

Built environments influence health through complex pathways. Perceptions of nature quality, beauty, relaxation and safety may be particularly import for understanding these linkages, but few studies to-date have examined these perceptions objectively for large populations. For quantitative research, an exposure measure must be reproducible, accurate, and precise––here we work to develop such measures for perceptions of the urban environment. We created crowd-sourced and image-based deep learning methods that were able to measure and model these perceptions. Future applications will apply these models to examine associations with mental health in the Washington State Twin Registry.

Larkin, A., Krishna, A., Chen, L., Amram, O., Avery, A.R., Duncan, G.E., Hystad, P. (2022) Measuring and modelling perceptions of the built environment for epidemiological research using crowd-sourcing and image-based deep learning modelsJournal of Exposure Science & Environmental Epidemiology32
 
Publication FAQ

FAQ: Perceptions of the Built Environment

What are "perceptions" of the built environment, and why are they important to study?

Perceptions of the built environment refer to how individuals subjectively experience and interpret their surroundings. These perceptions, encompassing feelings of safety, the perceived quality of nature, or the overall attractiveness of a space, can significantly influence behaviors like physical activity and mental health states like well-being. Understanding these perceptions is crucial because they provide insights into the complex ways that the built environment affects human health beyond mere physical characteristics.

How were perceptions of the built environment measured in this study?

This study utilized a crowdsourcing platform, Amazon Mechanical Turk (AMT), to gather data on perceptions. Participants were presented with pairs of Google Street View (GSV) images and asked to compare them based on specific perceptions, including nature quality, beauty, relaxation, and safety. They indicated their preference and the strength of their feeling using a slider. This method, combined with algorithms to adjust for individual biases, allowed for the collection of large-scale, quantifiable data on perceptions.

How were deep learning models used to predict perceptions from street view images?

The study employed deep learning models, specifically convolutional neural networks (CNNs), to predict perceptions from GSV images. These models were first pre-trained using a massive dataset of image comparisons from the PlacePulse 2.0 study. Then, they were fine-tuned using the perception data gathered from AMT, specifically tailored to the Washington State Twin Registry (WSTR) study area. This transfer learning approach enabled the models to accurately capture the complex visual features associated with each perception.

What were the key findings regarding the accuracy of the deep learning models in predicting perceptions?

The deep learning models achieved high accuracy in predicting perceptions, ranging from 77.6% for nature quality to 64.4% for safety. These models consistently outperformed previous models like those used in the PlacePulse study. Notably, the models were most effective in predicting perceptions when comparing dissimilar images and in areas with a higher degree of urban development.

How did the study address the potential biases associated with crowdsourced data collection?

The study acknowledged several potential biases in crowdsourced data and implemented strategies to mitigate them. These included dynamic image sampling to ensure comparisons between similar environments, adjustments for individual voting tendencies, and quality control measures to exclude unreliable responses. These steps enhanced the reliability and generalizability of the collected perception data.

How do perceptions of the built environment relate to traditional built environment measures?

The study compared the predicted perceptions with traditional built environment measures such as NDVI (greenness), walkability scores, and area deprivation indices. The correlations were generally low to moderate, suggesting that perceptions capture unique aspects of the built environment not fully captured by these objective measures.

What are the limitations of using street view images and crowdsourced data to study perceptions?

The study highlighted several limitations:

  • Limited Scope: Street view images only capture streetscapes and may not represent perceptions related to sounds, smells, or non-street areas.
  • Sampling Bias: Reaching specific populations or geographic areas through crowdsourcing can be challenging.
  • Image Variability: The quality, season, weather, and time of day of street view images can introduce variations in perceptions.
  • Generalizability: Crowdsourced data may not fully represent diverse populations.

What are the implications of this research for future studies in environmental epidemiology?

This study offers valuable tools and insights for future research:

  • It provides a framework for measuring and modeling perceptions, enabling researchers to incorporate these crucial factors into epidemiological studies.
  • It highlights the importance of considering perceptions alongside traditional measures for a more comprehensive understanding of how the built environment impacts health.
  • It advances the use of deep learning and street view imagery to assess complex environmental exposures, opening new avenues for research on the exposome – the totality of environmental exposures throughout life.