A framework for enriching survey datasets using passive data and machine learning, with an application to transferring attitudinal variables across transport surveys

Source abstract

Declining response rates make it increasingly critical for survey designers across disciplines to utilize mechanisms that facilitate timesaving and reduce the burden on the part of respondents. In practice, this often means that questionnaires are shortened, yielding increased response rates but reduced information/ variables available for modeling purposes. Here, this challenge is addressed by using machine learning and regression algorithms, alongside passive data augmentation, to develop and apply a predictive transfer learning-based framework for enriching surveys with information from other datasets. We demonstrate the framework by applying it to supplement and enrich the U.S. National Household Travel Survey (NHTS) with psychometric data (e.g., attitudes), which have been shown in the literature to have the ability to explain and predict behavior, but which are often not captured on household travel surveys. It is, to our knowledge, among the first times such an approach has been applied: (1) in a framework-based systematic approach in transportation; and (2) to transferring attitudinal variables across survey datasets. The application shown here explains up to 25% of the variance in observed attitudes, yielding correlations of up to 0.5 between observed and predicted attitudinal variables. Future applications of the framework presented in this paper have the potential to improve travel demand forecasting and behavioral predictions; and, even more broadly, may be used to enrich other large-scale behavior-based surveys with external variables, thereby providing more diverse and robust data streams for use in an array of modeling efforts.