Real-time Defogging for Better Visibility in Fogged Scenes is a most challenging problem statement, and it has numerous very beneficial applications. One can find many of the research works related to single image dehazing, but they are useless unless they can be implemented in a real-time scenario. From classical to deep learning-based, there are many approaches for single image dehazing, but all of these are taking too much execution time and failing for real-time implementation. As well, these require a high amount of computational resources and fail for a portable real-time solution.
Human vision is limited to only a specific band of light wavelength . There are other bands available that contain much more information, even in the presence of fog. Like using a thermal camera, you can have better visibility and an idea of the surrounding scene. But thermal cameras are costly. We will collect data from the non-human visible range using comparable cheap cameras, enhance the image from that portion of the non-human visible range, and fuse those data using deep learning-based enhancement.
The main features of these system will be:
1. real-time implementation
2. be able to use as portable system
Beyond the visible spectrum (400–700 nm), different advanced semiconductor based camera sensors show notable sensitivity and can record wavelengths as high as 1100 nm, which in the electromagnetic spectrum is known as the near-infrared. Even though image sensor material is capturing this radiation, it is eliminated by placing a filter in front of the image sensor considering it as noise. Many tasks related to computer vision can be improved when NIR information is retained rather than removed. The NIR portion of the spectrum has inherent qualities that enhance sharpness and reduce ambient haze. It is a better approach as a replacement of only software based dehazing algorithms associated with only RGB Color Images. We here are capturing both Visible and NIR range of electromagnetic spectrum and fuse those information to get a better dehazed output.
Right now, due to lack of actual fogged scene, we have no provision to collect actual data. We found some general RGB-NIR Scene Dataset and separated some fog-type Color-IR paired image data and did unsupervised enhancement on it towards defogging.
On the IR image we applied latest state of the art deeplearning based edge detection algorithm RCF (Richer Convolutional Features).
Among the CNN-based approaches for edge detection, the Richer Convolutional Features (RCF) method is often considered one of the best. Here's why:
- Richer Feature Extraction: RCF leverages features from all convolutional layers, not just the deeper ones. This allows the model to capture both fine and coarse details, leading to more accurate edge detection.
- Multi-Scale Representation: By integrating features from multiple scales, RCF can handle various object sizes and complexities in an image, improving edge localization and detection.
- Deep Supervision: Like HED, RCF uses deep supervision, meaning it provides edge predictions at multiple stages of the network. This helps to refine edges progressively and ensures that even early layers contribute meaningfully to the final output.
- State-of-the-Art Performance: RCF has been shown to achieve state-of-the-art results on several benchmark datasets for edge detection, demonstrating its effectiveness and robustness.
While RCF is powerful, the choice of the best approach may also depend on specific requirements, such as computational resources, the complexity of the images, and the level of detail needed. For certain applications, other methods like CEDN or HED may also be suitable.
https://mmcheng.net/rcfedge/
https://github.com/yun-liu/RCF-PyTorch
Next, we have fused the RCF output with the actual visible data.
import cv2
import numpy as np
color_image = cv2.imread('whiteroofs-copy-0.jpg')
edges_colored = cv2.imread('whiteroofs_ss.png')
# Set the edge pixels to red (or any color you prefer)
edges_colored[np.where((edges_colored == [255, 255, 255]).all(axis=2))] = [0, 0, 255]
# Overlay the edge image on the original image
overlay_image = cv2.addWeighted(color_image, 0.8, edges_colored, 0.2, 0)
# Save or display the result
cv2.imwrite('whiteroofs.jpg', overlay_image)
cv2.waitKey(0)
cv2.destroyAllWindows()The AMD UM790 Pro Mini PC AMD Ryzen 9 7940HS AI Hardware & AI Software Stack is beneficial in building a real-time, implementable portable solution.
A display will be attached, which will display the AMD Mini PC processed live dual camera video.
Comments