Deep Learning Classification of 2D Orthomosaic Images and 3D Point Clouds for Post-Event Structural Damage Assessment

 

Efficient and rapid data collection techniques are necessary to obtain transitory information in the aftermath of natural hazards, which is not only useful for post-event management and planning, but also for post-event structural damage assessment.

Aerial imaging from UAVs permits highly detailed site characterization, in particular in the aftermath of extreme events with minimal ground support, to document current conditions of the region of interest. However, aerial imaging results in a massive amount of data in the form of two-dimensional (2D) orthomosaic images and three-dimensional (3D) point clouds.

Both types of datasets require effective and efficient data processing workflows to identify various damage states of structures. This manuscript aims to introduce two deep learning models based on both 2D and 3D convolutional neural networks to process the orthomosaic images and point clouds, for post windstorm classification.

In detail, 2D CNN (2D Convolutional Neural Networks) are developed based on transfer learning from two well-known networks AlexNet and VGGNet. In contrast, a 3DFCN (3D Fully Convolutional Network) with skip connections was developed and trained based on the available point cloud data. Within this study, the datasets were created based on data from the aftermath of Hurricanes Harvey (Texas) and Maria (Puerto Rico). The developed 2DCNN and 3DFCN models were compared quantitatively based on the performance measures, and it was observed that the 3DFCN was more robust in detecting the various classes.

This demonstrates the value and importance of 3D Datasets, particularly the depth information, to distinguish between instances that represent different damage states in structures.

Read more: https://www.mdpi.com/2504-446X/4/2/24

Classification of Grassland Desertification in China Based on Vis-NIR UAV Hyperspectral Remote Sensing

In this study, a vis-NIR (visual Near Infra Red) hyperspectral remote sensing system for UAVs (Unmanned Aerial Vehicles) was used to analyze the type and presence of vegetation and soil of typical desertified grassland in Inner Mongolia using a DBN (Deep Belief Network), 2D CNN (2D Convolutional Neural Network) and 3D CNN (3D Convolutional Neural Network).

The results show that these typical deep learning models can effectively classify hyperspectral data on desertified grassland features. The highest classification accuracy was achieved by 3D CNN, with an overall accuracy of 86.36%. This study enriches the spatial scale of remote sensing research on grassland desertification, and provides a basis for further high-precision statistics and inversion of remote sensing of grassland desertification.

Read more: https://www.spectroscopyonline.com/view/classification-grassland-desertification-china-based-vis-nir-uav-hyperspectral-remote-sensing

Deep Learning Classification of 2D Orthomosaic Images and 3D Point Clouds for Post-Event Structural Damage Assessment

Aerial imaging from UAVs (Unmanned Aerial Vehicles) permits highly detailed site characterization, in particular in the aftermath of extreme events with minimal ground support, to document current conditions of the region of interest.

However, aerial imaging results in a massive amount of data in the form of two-dimensional (2D) orthomosaic images and three-dimensional (3D) point clouds. Both types of datasets require effective and efficient data processing workflows to identify various damage states of structures.

This study aims to introduce two deep learning models based on both 2D and 3D convolutional neural networks to process the orthomosaic images and point clouds, for post windstorm classification. In detail, 2D CNN (2D Convolutional Neural Networks) are developed based on transfer learning from two well-known networks: AlexNet and VGGNet.

In contrast, a 3DFCN (3D Fully Convolutional Network) with skip connections was developed and trained based on the available point cloud data. Within this study, the datasets were created based on data from the aftermath of Hurricanes Harvey (Texas) and Maria (Puerto Rico). The developed 2DCNN and 3DFCN models were compared quantitatively based on the performance measures, and it was observed that the 3DFCN was more robust in detecting the various classes. 

This demonstrates the value and importance of 3D Datasets, particularly the depth information, to distinguish between instances that represent different damage states in structures.

Read more: https://www.mdpi.com/2504-446X/4/2/24/htm

The methodology of documenting cultural heritage sites using photogrammetry, UAV, and 3D printing techniques: the case study of Asinou Church in Cyprus

As the affordability, reliability and ease-of-use of Unmanned Aerial Vehicles (UAV) advances, the use of aerial surveying for cultural heritage purposes becomes a popular choice, yielding an unprecedented volume of high-resolution, geo-tagged image-sets of historical sites from above.

As well, recent developments in photogrammetry technology provide a simple and cost-effective method of generating relatively accurate 3D models from 2D images. These techniques provide a set of new tools for archaeologists and cultural heritage experts to capture, store, process, share, visualise and annotate 3D models in the field.

This paper focuses on the methodology used to document the cultural heritage site of Asinou Church in Cyprus using various state of the art techniques, such as UAV, photogrammetry and 3D printing. Hundreds of images of the Asinou Church were taken by a UAV with an attached high resolution, low cost camera. These photographic images were then used to create a digital 3D model and a 3D printer was used to create a physical model of the church.

Such a methodology provides archaeologists and cultural heritage experts a simple and cost-effective method of generating relatively accurate 3D models from 2D images of cultural heritage sites.

(Read more...)

Structure Analysis and Optimization of Transitioning UAV

With the aim to develop more efficient aircraft configurations, the Blended-Wing-Body (BWB) Unmanned Aerial Vehicles have grown attention in recent years. Compared to conventional aircraft configurations, the BWB structure has several advantages in aerodynamics and fuel efficiency.

Topology Optimization (TO) is also a relatively new structure optimization approach which has applied successfully in automotive industry for a considerable time. In this paper, topology optimization method will be applied on a special BWB structure UAV called BITU on both 2D and 3D models in ABAQUS.

The optimization goal is to minimize compliance energy under specified loading and boundary conditions which will be computed in modeling and simulation section. Finally, optimized result compared to initial design will demonstrate TO is a rational and efficient design tool for structure optimization, especially in Aircraft industry.

How to capture in 3D without scanning… or lenses

According to a recent publication in Optica, researchers working under Laura Waller at UC Berkeley have developed an “easy-to-build camera that produces 3D images from a single 2D image without any lenses.” The “DiffuserCam” can capture high-definition 3D information in large volume without the use of a scanning mechanism or difficult-to-manufacture micro-circuitry.

But, you ask, How does it work? The magic lies in Waller’s use of “computational imaging,” which is “an approach that examines how hardware and software can be used together to design imaging systems.” In other words, the hardware and software are thought of as two smaller, complementary parts of a holistic imaging system. Despite the DIY approach, the DiffuserCam is no joke, as the camera can reconstruct 100 million voxels (points in 3D space) from an 1.3-megapixel image. As a comparison, the article mentions that the new iPhone X captures 12-megapixel photos. It means that you could use that image sensor to build a a DiffuserCam that captures more than a billion points.

The camera hardware is  “essentially a bumpy piece of plastic” that is “placed on top of an image sensor,” a summary article explains. This simplicity makes it versatile and easy to construct. Researchers in Waller’s lab say they can build it using any image sensor, and practically any bumpy piece of plastic, including privacy glass stickers, Scotch tape, and conference pass holders.

The lensless DiffuserCam consists of a diffuser placed in front of a sensor. The system turns a 3D scene into a 2D image on the sensor. After a one-time calibration, an algorithm is used to reconstruct 3D images computationally. The result is a 3D image reconstructed from a single 2D measurement.

This approach enables the researchers to unload the complexity off of the hardware and into the processing algorithms. This proves the old adage that “software eats hardware,” and brings with it one of the classic benefits of code: You only have to make the complex software once, because after that it can be easily copied and distributed.

Waller says her team’s work proves out the usefulness of computational imaging, showing that this holistic approach enables breakthroughs in performance and price that were not possible before. Still, she warns, “this is a very powerful direction for imaging, but requires designers with optical and physics expertise as well as computational knowledge.”

Photogrammetry Software

Some of the most interesting uses of UAVs deployment in science and conservation are terrain mapping and vegetation analysis.

Data capture for mapping or vegetation analysis can be done by multirotor or fixed-wing methods, with the former a good choice for smaller areas that can be covered in less than 20 minutes and the latter better for bigger landscapes that may require up to an hour of flight time (or more).

The photogrammetry software to perform this work is rapidly evolving but at this time the best options carry a rather stunning range of pricing. Tools include both desktop processing and server-farm processing approaches. You have to explore the options to see what matches your end use. Here is a list of processing software for your exploration:

3dr Site Scan

https://3dr.com/enterprise/features/

Agisoft Photoscan Professional Edition

http://www.agisoft.com/

AgPixel

http://www.agpixel.com/

BladeEdge

http://edgedata.net/bladeedge/ (Wind Turbines)

Contextcapture

https://www.bentley.com/en/products/brands/contextcapture

Correlator 3D

http://www.simactive.com/en/correlator3d/photogrammetry-software/features

Datumate

http://www.datumate.com/

Drone Data Management System

https://event38.com/drone-data-management-system/

Drone Mapper

https://dronemapper.com/

Drone2Map for ArcGIS

http://www.esri.com/products/drone2map

DroneDeploy

https://www.dronedeploy.com/

Geomatica

http://www.pcigeomatics.com/software/geomatica/professional

Global Mapper for UAV

http://blog.bluemarblegeo.com/2017/03/22/global-mapper-for-uav-operations/

Icaros One Button

https://www.icaros.us/onebuttonpro/

Imagestation

http://www.hexagongeospatial.com/products/power-portfolio/%20imagestation

Imagine UAV (ERDAS Imagine) https://www.geosystems.de/en/products/imagine-uav/benefits/

Maps Made Easy

https://www.mapsmadeeasy.com/

Open Drone Map

http://opendronemap.org/

Photomodeler

http://www.photomodeler.com/index.html (3D models)

Pix4Dmapper Pro

https://pix4d.com/

Precision Mapper

http://www.precisionhawk.com/precisionmapper

Propeller Aero

https://www.propelleraero.com/

Reality Capture

https://www.capturingreality.com (3D models)

SlantView by SlantRange

http://www.slantrange.com/slantview/

Summit Evolution

http://www.datem.com/

Trimble Business Center

https://geospatial.trimble.com/products-and-solutions/trimble-business-center

Unlimited Aerial

http://holistic-imaging.com/unlimited-aerial/

IIDT 3D maps with UAV

Reconstruction of a 3D object from 2D pictures, to measure the areas, depth of a specific surface. This can be used for digital animation and for Intelligence purposes.

GeoDragon: 3D ISR

A sensor system designed to create 3-D reconstructions in near-real time and output high-resolution digital elevation models, LIDAR-like datasets, and wide area maps has been flown on an Arcturus T-20 Tier II UAV.

The system, called GeoDragon, is enclosed in a wing-mounted pod and is capable of high resolution 2-D and 3-D image capture. Urban Robotics in Portland, OR, which designed the sensor, says among its "unique aspects" are low weight (equating to longer endurance and loiter time,) small operational footprint on the ground (1 or 2 full size pick-up trucks,) a quick mount/dismount pod, quiet operation (the T-20 utilizes a modified 4-stroke engine,) and the ability to fly simultaneously with other payloads on the T-20, such as EO/IR.

GeoDragon imagery is post-processed using automated 3-D algorithms to rapidly generate large mapping and modeling datasets. Urban Robotics develops software and hardware solutions for 3-D ISR, remote sensing and geospatial applications, including collection, post-processing, and data management. The aircraft was built by Arcturus UAV in Rohnert Park, CA. Urban Robotics says the GeoDragon adds significant 3-D imaging and mapping capabilities to the T-20 UAV. The system is scheduled to be released in mid-2014.