DroneCaps: Recognition Of Human Actions In UAV Videos Using Capsule Networks With Binary Volume Comparisons

Understanding human actions from videos captured by UAVs is a challenging task in computer vision due to the unfamiliar viewpoints of individuals and changes in their size due to the camera’s location and motion.

This work proposes DroneCaps, a capsule network architecture for multi-label HAR (Human Action Recognition) in videos captured by UAVs. DroneCaps uses features computed by 3D convolution neural networks plus a new set of features computed by a novel Binary Volume Comparison layer.

All these features, in conjunction with the learning power of CapsNets, allow understanding and abstracting the different viewpoints and poses of the depicted individuals very efficiently, thus improving multi-label HAR.

The evaluation of the DroneCaps architecture’s performance for multi-label classification shows that it outperforms state-of-the-art methods on the Okutama-Action dataset.

Read more at: https://ieeexplore.ieee.org/document/9190864

Desertification Glassland Classification and Three-Dimensional Convolution Neural Network Model for Identifying Desert Grassland Landforms with UAV Hyperspectral Remote Sensing Images

Based on deep learning, a Desertification Grassland Classification (DGC) and three-dimensional Convolution Neural Network (3D-CNN) model is established.

The F-norm paradigm is used to reduce the data; the data volume was effectively reduced while ensuring the integrity of the spatial information. Through structure and parameter optimization, the accuracy of the model is further improved by 9.8%, with an overall recognition accuracy of the optimized model greater than 96.16%.

Accordingly, high-precision classification of desert grassland features is achieved, informing continued grassland remote sensing research.

Read more at: https://link.springer.com/article/10.1007/s10812-020-01001-6

Vision-Based Obstacle Avoidance for UAVs via Imitation Learning with Sequential Neural Networks

This paper explores the feasibility of a framework for vision-based obstacle avoidance techniques that can be applied to UAVs (Unmanned Aerial Vehicles) where such decision-making policies are trained upon supervision of actual human flight data.

The neural networks are trained based on aggregated flight data from human experts, learning the implicit policy for visual obstacle avoidance by extracting the necessary features within the image. The images and flight data are collected from a simulated environment provided by Gazebo, and Robot Operating System is used to provide the communication nodes for the framework.

The framework is tested and validated in various environments with respect to four types of neural network including fully connected neural networks, two- and three-dimensional CNNs (Convolutional Neural Networks), and Recurrent Neural Networks (RNNs). Among the networks, sequential neural networks (i.e., 3D-CNNs and RNNs) provide the better performance due to its ability to explicitly consider the dynamic nature of the obstacle avoidance problem.

Read more at: https://link.springer.com/article/10.1007/s42405-020-00254-x

Tree Species Classification of UAV Hyperspectral and RGB Imagery with Deep Learning Convolutional Neural Networks

Interest in UAV solutions in forestry applications is growing.

Using UAVs, datasets can be captured flexibly and at high spatial and temporal resolutions when needed.

In forestry applications, fundamental tasks include the detection of individual trees, tree species classification, biomass estimation, etc. Deep Neural Networks (DNN) have shown superior results when comparing with conventional machine learning methods such as MLP (Multi-Layer Perceptron) in cases of huge input data.

The objective of this research is to investigate 3D Convolutional Neural Networks (3D-CNN) to classify three major tree species in a boreal forest: pine, spruce, and birch. The proposed 3D-CNN models were employed to classify tree species in a test site in Finland. The classifiers were trained with a dataset of 3039 manually labelled trees. Then the accuracies were assessed by employing independent datasets of 803 records.

To find the most efficient set of feature combination, it was compared the performances of 3D-CNN models trained with HS (HyperSpectral) channels, Red-Green-Blue (RGB) channels, and Canopy Height Model (CHM), separately and combined. It is demonstrated that the proposed 3D-CNN model with RGB and HS layers produces the highest classification accuracy. The producer accuracy of the best 3D-CNN classifier on the test dataset were 99.6%, 94.8%, and 97.4% for pines, spruces, and birches, respectively.

The best 3D-CNN classifier produced ~5% better classification accuracy than the MLP with all layers. The results suggest that the proposed method provides excellent classification results with acceptable performance metrics for HS datasets. The results show that pine class was detectable in most layers. Spruce was most detectable in RGB data, while birch was most detectable in the HS layers. Furthermore, the RGB datasets provide acceptable results for many low-accuracy applications.

Read more at: https://www.mdpi.com/2072-4292/12/7/1070

Classification of Grassland Desertification in China Based on Vis-NIR UAV Hyperspectral Remote Sensing

In this study, a vis-NIR (visual Near Infra Red) hyperspectral remote sensing system for UAVs (Unmanned Aerial Vehicles) was used to analyze the type and presence of vegetation and soil of typical desertified grassland in Inner Mongolia using a DBN (Deep Belief Network), 2D CNN (2D Convolutional Neural Network) and 3D CNN (3D Convolutional Neural Network).

The results show that these typical deep learning models can effectively classify hyperspectral data on desertified grassland features. The highest classification accuracy was achieved by 3D CNN, with an overall accuracy of 86.36%. This study enriches the spatial scale of remote sensing research on grassland desertification, and provides a basis for further high-precision statistics and inversion of remote sensing of grassland desertification.

Read more: https://www.spectroscopyonline.com/view/classification-grassland-desertification-china-based-vis-nir-uav-hyperspectral-remote-sensing

Deep Learning Classification of 2D Orthomosaic Images and 3D Point Clouds for Post-Event Structural Damage Assessment

Aerial imaging from UAVs (Unmanned Aerial Vehicles) permits highly detailed site characterization, in particular in the aftermath of extreme events with minimal ground support, to document current conditions of the region of interest.

However, aerial imaging results in a massive amount of data in the form of two-dimensional (2D) orthomosaic images and three-dimensional (3D) point clouds. Both types of datasets require effective and efficient data processing workflows to identify various damage states of structures.

This study aims to introduce two deep learning models based on both 2D and 3D convolutional neural networks to process the orthomosaic images and point clouds, for post windstorm classification. In detail, 2D CNN (2D Convolutional Neural Networks) are developed based on transfer learning from two well-known networks: AlexNet and VGGNet.

In contrast, a 3DFCN (3D Fully Convolutional Network) with skip connections was developed and trained based on the available point cloud data. Within this study, the datasets were created based on data from the aftermath of Hurricanes Harvey (Texas) and Maria (Puerto Rico). The developed 2DCNN and 3DFCN models were compared quantitatively based on the performance measures, and it was observed that the 3DFCN was more robust in detecting the various classes. 

This demonstrates the value and importance of 3D Datasets, particularly the depth information, to distinguish between instances that represent different damage states in structures.

Read more: https://www.mdpi.com/2504-446X/4/2/24/htm

Accurate 3D Facade Reconstruction using UAVs

Automatic reconstruction of a 3D model from images using multi-view Structure-from-Motion methods has been one of the most fruitful outcomes of computer vision.

These advances combined with the growing popularity of Micro Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools ubiquitous for large number of Architecture, Engineering and Construction applications among audiences, mostly unskilled in computer vision.

However, to obtain high-resolution and accurate reconstructions from a large-scale object using SfM, there are many critical constraints on the quality of image data, which often become sources of inaccuracy as the current 3D reconstruction pipelines do not facilitate the users to determine the fidelity of input data during the image acquisition.

In this paper, it is presented and advocate a closed-loop interactive approach that performs incremental reconstruction in real-time and gives users an online feedback about the quality parameters like Ground Sampling Distance (GSD), image redundancy, etc on a surface mesh. It is also proposed a novel multi-scale camera network design to prevent scene drift caused by incremental map building, and release the first multi-scale image sequence dataset as a benchmark.

Further, it is evaluated the system on real outdoor scenes, and show that the interactive pipeline combined with a multi-scale camera network approach provides compelling accuracy in multi-view reconstruction tasks when compared against the state-of-the-art methods.

More info:

https://www.researchgate.net/publication/272791949_Building_with_Drones_Accurate_3D_Facade_Reconstruction_using_MAVs