CROCUS Program Science

University of Notre Dame

University of Notre Dame’s research will deal with the development of Microscale Models for Urban Environmental Prediction and Observational Support  (within the topic Street Scale: Urban Environmental SMART Sustainable Solutions (UES3), collaborating with Dr. Rao Kotamarthi at ANL)

The Urban Environmental SMART Sustainable Solutions (UES3) numerical modeling component development of CROCUS proposal will cover micro scale physics and modeling, and will account for interactions between social, ecological and environmental drivers impacting urban systems in a changing climate. UES3 will resolve urban form (geometry: shape, size and configuration), urban fabric (physical characteristics: landuse and material composition), and urban functionality (energy use) and will serve as the “digital twin”for individual buildings to the scale of an urban ward in Chicago (~ of few square kms). The UES3 is being developed using OpenFOAM computational fluid dynamics code at University of Notre Dame in collaboration with the Argonne National Laboratory (ANL).  OpenFOAM modeling platform is a widely used open source code for performing small scale CFD simulations, and well suited for 1-10 m resolution simulation of urban factitious elements. It can be used for developing urban parameterizations that can be scaled to a city as well as assessing neighborhood level needs and SMART solutions.  OpenFOAM is used with Reynolds-Averaged Navier-Stokes (RANS) simulation of wind field, in association with real urban morphologies mapped by airborne lidars and meshed with native OpenFOAM meshing utility to create unstructured mesh. Flow simulations have been performed for applications to simulations for the university campus in Singapore by Wang et al., (2021).

UES3 is currently capable of performing both RANS and Large Eddy Simulations (LES) over complex geometry with moist thermodynamics and radiative transfer modeling using ray-tracing methods for the heterogeneous urban topography. Vegetation modules are in development, and are capable of handling both large patches of tree canopies and solitary trees to provide more refined representation of urban streets, with parameterized tree generation. In the next stage, to further equip the model towards becoming the digital twin for urban neighborhoods, modules for buildings with various complexity will be developed, with potential coupling with building energy model EnergyPlus for targeted assessment. In addition, and through mapping with OpenStreetMap data, roads and traffic will be subsequently included which will be imperative for evaluation of pollutant dispersion and noise level. We have been developing an urban noise model over the past decade, which can be parsed with the UES3 (Ovenden et al. 2009; Voropayev et al. 2017).  All modules will be developed with anticipation of increasing level of details for urban environment for reusable and sustainable model development. We plan to run this model using external boundary conditions produced by Urban System Scale Model) USSYM of ANL using various strategies the CROCUS investigators have investigated for coupling microscale-to-mesoscale models in recent years (Haupt et al., 2019; Mirocha et al., 2018; Haupt et al., 2017).

While OpenFOAM offers many advantages for modular integration of components such as radiative transfer to the UES3, it is not yet GPU ready. Consequentially, a code co-developed by Argonne and that is being further developed by a number of Exascale Computing Project (ECP) projects, the  nekRS, an open-source, highly-scalable, and GPU/exascale-ready code, will be used with UES3 (Fischer et al., 2022). Recently nekRS has been tested against codes that were part of the ExaWind benchmark (an ECP project) for atmospheric boundary layer simulations (Kolev et al., 2020, 2021, 2022) and its results have been compared against other widely used CFD codes in wind and atmospheric boundary layer for DOE applications such as Nalu-Wind and AMR-Wind (Min et al., submitted). We will use high-order nekRS to perform detailed turbulence resolving simulations LES under different Local Climate Zones (LCZs) in the Chicago area. nekRS has similar numerical capabilities as OpenFOAM (Rezaeiravesh et al. 2022) with the additional benefit to be highly scalable (Fischer et al. 2021).  The LES simulations for the Chicago area can be used to improve the performance  of the RANS models used in OpenFOAM as well as for the development of reduced order models to simulate thermal effects (Kaneko et al., 2020) and effects from the obstacles (such as buildings or trees) geometry and orientation (Fytanidis et al., 2021) that can be used for the upscaling of the street-scale to the ~ 100 m grid used by USSYM (Haupt et al., 2019; Mirocha et al., 2018; Haupt et al., 2017) . Additionally, we are steadily making improvements in the nekRS code. As part of the project, we are planning to include some of the capabilities of UES3 (e.g. tree shading modules and land surface temperature submodels) in nekRS. The advantage of doing so is to eventually transition to a version of nekRS that would include the physics already available on USE3 but which will also be capable to perform simulations exascale supercomputers and is GPU ready. The modeling framework that CROCUS envisages is pictorially depicted below, and University of Notre Dame work mainly revolves around OpenFOAM modeling and its nexus with improving NekRS as well as providing support for developing low order models for upscaling to USSYM.

Overall, CROCUS will push boundaries of enhanced spatial resolution using adaptive meshes and Argonne Leadership Computing Facility (ALCF), integrate CMS and invest in machine learning (ML) and data assimilation to capture non-linearity without loss of model complexity in urban processes, and result in advancing fundamental urban knowledge and developing solutions (Sharma et al. 2021). Given the availability of the microscale model simulation results and the U-IFL field efforts, it is proposed that ML will be utilized for multiple aspects of the microscale model and we plan to participate in this activity to the extent possible. Combined with field measurement data, ML methods such as deep learning can be used to learn turbulence closure models, achieving better/faster-than-PDE turbulence models (Duraisamy et al., 2019, Xiao et al., 2019). Moreover, data assimilation techniques can be used to adapt the learned closures based on real-time data acquisition (Maulik et al., 2022). Such data-driven models are typically cheaper to evaluate than their high-fidelity counterparts – this will allow for orders-of-magnitude faster predictions of flow-fields, particularly for an urban canopy model. Embedded sub-models such as trees and buildings can also be included as data-driven model to allow greater expressiveness of the forward-model.

Observational Datasets

We expect to use observational datasets for improving, evaluating and developing necessary new physics parameterizations. For the USE3 in particular, we will be using community/citizen datasets extensively for model evaluation. For example, changes in land surface temperature (LST) over short distances, assessing the changes in LST from mitigation actions. Measurements of wind flow and turbulence collected using Doppler lidars, sonic anemometers around buildings and urban canyons as a part of the BGC sites and the mobile BGC laboratory measurements will be used to evaluate and improve the flow representation in the model with USE3 RANS and nekRS LES simulations, which will be used to improve the model representations and inform the observational strategy in an iterative Model-Experiment (MODEX) framework. We will use profile data collected during the campaign to initialize and develop tendency terms for forcing the microscale simulations and connecting the microscales with mesoscale.  Resources permitting, University of Notre Dame group will provide expertise and instruments for data collection in the City of Chicago, and ANL is expected to provide overarching framework for the placement of equipment, subject to the approval of land owners.

Simulation Types

OpenFOAM and nekRS models will be used for performing 24-hr time scale simulations for a single neighborhood (for example Westelawn in Chicago and the sustainable square mile being developed by the BIG community) to refine the solution for case studies developed using USSYM. The microscale models developed/refined by Notre Dame group will also resolve building scale features of energy and momentum at this spatial scale and the results will be used to improve flow drag parameterizations as well as heat and pollution diffusion rates in the USSYM. The BGC mobile observations will be used to constrain the surface fluxes. The nekRS simulation will be performed for similar spatial scales but for a few hours to represent different atmospheric stability conditions for improving the RANS model performance in the USE3 OpenFOAM.

Applications

The model for instance will be able to resolve the effect of tree shading on LST at a street level. Effect of trees/vegetation and other mitigation actions on particle removal rates would not be possible with models of lower spatial resolutions.