FloodNet - Computer Vision for Urban Street Flood Detection
- Charlie Mydlarz, Research Associate Professor, NYU CUSP, FloodNet
- Chen Feng, Assistant Professor of Civil and Urban Engineering & Mechanical Aerospace Engineering, NYU CUSP
In NYC, sea level rise has led to a dramatic increase in flood risk, particularly in low-lying and coastal neighborhoods. Urban flood water can impede pedestrian and vehicle mobility, and also can contain a diverse array of contaminants, including fuels, raw sewage, and industrial/household chemicals. For this capstone project, the team will train, test and deploy computer vision (CV) and deep learning (DL) models for the detection of street flood events. Existing labelled datasets will be used for training. In addition, an unlabelled NYC street image dataset will be provided for labelling and training of a NYC centric model.
Category: Urban Environment
Project Description & Overview
In NYC, sea level rise has led to a dramatic increase in flood risk, particularly in low-lying and coastal neighborhoods. Urban flood water can impede pedestrian and vehicle mobility, and also contains a diverse array of contaminants, including fuels, raw sewage, and industrial/household chemicals.
The FloodNet project is interested in evaluating whether a longitudinally deployed fleet of CV flood sensors can monitor urban flooding events in real-time. This data can improve resiliency by: (1) allowing residents to identify navigable transportation routes and make informed decisions to avoid exposure to flood water contaminants, and (2) informing city agencies in targeting flood control improvements through data-driven decision making.
The capstone team will train, test and deploy CV/DL models for the detection of street flood events. Existing labeled datasets will be used for training. In addition, an unlabelled NYC street image dataset will be provided. The labeling strategy of this dataset will be determined by the team. Unsupervised or weakly supervised DL approaches could also be explored.
The team will work through 3 stages:
- Literature review on privacy and ethical concerns – 20%
The team will complete a review on CV ethics/privacy concerns in urban sensing.
- Labeling of NYC traffic cam data – 30%
The team will collate, clean and label the existing NYC open data using custom scripts that draw from flood related datasets to utilize the large amounts of unlabeled data.
- Training of NYC centric flood model with realtime implementation – 50%
A NYC centric flood detection model will be trained and tuned to provide accurate flood detections under a range of environmental conditions. In addition a realtime implementation of this model will be developed to provide actionable flood notifications to key stakeholders.
Stage 2 will involve the generation of a NYC centric dataset using existing unlabelled images collected from NYC streets in both flood and non flood conditions including the use of weather data and floods reporting from other sources.
- Machine learning
- Dimensionality reduction
- Supervised learning
- Semi-/Weak-supervised learning
- Computer vision experience
- Good code and data management skills
- Python SciPy stack and PyTorch DL library
Learning Outcomes & Deliverables
The team will be using a broad range of urban analytics approaches that will result in proven abilities in: computer vision, remote sensing, data science, and machine learning.
The expected deliverables for each project stage are:
- A literature review on the ethical and privacy concerns surrounding urban sensing and CV solutions.
- A labeled flood dataset of NYC traffic camera data for open sourcing on the data platform Zenodo.
- The NYC centric model with performance levels exceeding a given threshold under varying real world conditions, including a realtime implementation of the model for flood detection alerting.
All deliverables will be based around Jupyter notebooks and committed to a well documented public GitHub repository.