Project Team Members:

Shwetha TR – 01FB15EEC239
Sri Amoghavarsha BS – 01FB15EEC247
Sruthi R – 01FB15EEC249

Project Abstract:
The aim of the project is to develop an understanding of foggy scenes for outdoor applications like autonomous driving and mobile mapping. Semantic Foggy Scene Understanding (SFSU) is a relatively lesser explored field when compared to image de-hazing and semantic scene understanding with weather clear images and videos. Due to the difficulty of collecting and annotating foggy images, we depart from this traditional paradigm and generate synthetic fog into real, weather-clear images. 
The pipeline of the project is as follows:

fog simulation on real outdoor scenes
training with pairs of synthetic foggy images and semantic annotations
obtain the training set- pairs of foggy images and weather- clear counterparts
scene understanding of real foggy scenes

Fog simulation is achieved by modelling the effect of fog as a function that maps the radiance of the clear scene to the observed radiance at the camera sensor, which is related to the distance of various objects from the camera, and in turn related to depth. The resulting pair of the clear image and its corresponding depth map forms the basis of our synthesis.
The dataset is used to train two semantic segmentation models, and an object detector for foggy scenes. The models are trained in two fashions: 1) by supervised learning using the obtained high- quality foggy images and 2) by an attempt at semi-supervised learning which combines the results obtained from 1) with an unsupervised transfer from weather- clear images to their foggy counterparts using an additional set of images. The model is then cross-validated using a set of images of driving scenes in presence of fog. 
Expectations:
The project will show how the synthetic data can boost the performance of unsupervised learning methods like Convolutional Neural Networks, for semantic segmentation and object detection and extend this to challenging real foggy scenes. 
The dataset will be used to conduct the following experiments as well:

Comparison of Fog Simulation Approaches – Nearest neighbour interpolation and a truncated version without guided filtering
Benefit of Fine-tuning on Synthetic Fog – Four de-hazing options: 1. no de-hazing at all 2. Multi -scale convolutional neural networks 3. Dark channel prior 4. non-local image de-hazing
Increasing Returns at Larger Distance
Training on Varying levels of synthetic fog- which correspond to different values of attenuation coefficient Beta

These experiments shall prove that image de-hazing faces challenges on real outdoor foggy data but is marginally more helpful for synthetic data. In addition to existing applications like autonomous driving, the algorithms developed for object detection and segmentation, and experiments on increasing the large distance gain can also be incorporated into airplane systems to curtail losses and delays due to bad weather. 
The project will help in understanding fundamental Machine Learning algorithms and utilise them in novel applications.
References:

Publication: https://arxiv.org/pdf/1708.07819.pdf
Dataset: 

http://people.ee.ethz.ch/~csakarid/SFSU_synthetic/