A multi-view deep learning framework for urban tree health assessment via fusion of aerial multispectral and street-level RGB imagery

Source abstract

Urban trees provide critical ecosystem services, but city-scale crown-dieback monitoring remains challenging. We evaluated a multi-view deep-learning proof of concept for discriminating low ( < 25 % ) from high ( ≥ 50 % ) crown dieback in urban Pinus pinea using UAV multispectral and Google Street View RGB imagery. The cohort comprised 111 expert-labelled trees (68 high-dieback and 43 low-dieback), each represented by a four-band, 7 × 7 m UAV patch and six Street View images. Using ResNet-18 and dependency-aware five-fold cross-validation that kept 67 components intact, we compared Multispectral UAV, Street View, probability-averaging fusion, feature-level fusion, and a separate-product UAV RGB-only baseline. Multispectral UAV and Street View achieved pooled out-of-fold AUCs of 0.906 and 0.834, respectively. Average Fusion achieved the highest pooled AUC (0.930), whereas Feature Fusion achieved the highest accuracy (0.847) and balanced accuracy (0.832). Relative to Multispectral UAV, the Average Fusion AUC difference was 0.024 (95% component-bootstrap CI: −0.022 to 0.069), and adjusted paired comparisons did not establish fusion superiority. Multispectral UAV exceeded UAV RGB-only by 0.116 in AUC (95% component-bootstrap CI: 0.033–0.212; Holm-adjusted p = 0.014 ), whereas the 9.0 percentage-point accuracy difference was not statistically resolved (95% CI: −2.7 to 21.0). Controlled synthetic occlusion and tree-level disagreement analyses characterised non-identical error patterns. External, multi-season, prospective studies spanning the full dieback continuum are required before broader inference or operational use.