Geraint West, Adrian Bodenmann, Darryl Newborough, and Blair Thornton
October 21, 2020A project to address the twin challenge of covering large areas of seabed at high resolution, while also processing the gathered data within expedition time-scales, has been delivering promising results. BioCam project participants Geraint West, Adrian Bodenmann, Darryl Newborough, and Blair Thornton explain
It’s often said that we know more about the surface of the moon than our own ocean floor – despite half the moon being permanently hidden from our gaze. The challenges involved in imaging these two environments are uniquely different. Imaging the seafloor, in a detailed and repeatable way to allow for meaningful monitoring, poses a dual challenge: being able to cover large, often remote and deep areas quickly, at high resolution.
Existing systems tend to offer either millimetric resolution over small areas or low resolution coverage over wide areas. High resolution imaging also tends to require imaging systems to be within 1-2 m of objects of interest, inviting accidents. The end result for either method is usually extensive, lengthy post-processing on the vast amounts of gathered data.
A new, 3D deep-sea imaging system, developed by the University of Southampton, with support from Sonardyne, under the Natural Environment Research Council’s OCEANIDS Marine Sensor Capital “BioCam” program, has gone a long way to resolving these challenges.
During its first 24-hour deployment, a BioCam fitted to an Autosub6000 autonomous underwater vehicle (AUV), from the UK’s National Oceanography Centre (NOC), mapped more than 50 times the area of a football pitch at photographic resolution – a rate 40 times that of conventional high resolution imaging systems. By the end of this dive, it had already created the largest continuous visual map of the seafloor ever obtained in UK waters, and went on to map more during the rest of the expedition. The project has also developed a data processing pipeline which provides expedition time-scale information that helps scientists know exactly where they should go back to get better data, during the same expedition – instead of such information emerging from the data months or even years later, when it’s too late to do anything about it.
A main driver for this capability has been to help gather the information needed to efficiently monitor seabed biology and ecosystems, such as the cold-water coral found at the Darwin Mounds, a deep water Marine Protected Area (MPA) in the North Atlantic’s Northern Rockall Trough, 160 km north west of the Scottish mainland. The area features ~100 m diameter and 5 m high coral covered mounds in 710-1,029 m water depth. It was first discovered by NOC scientists in 1998 and has been protected from bottom contact fishing since 2003.
Cold-water coral reefs are biological hotspots, providing rich habitats for diverse marine life. They are slow growing (mm or cm/year), fragile and extremely sensitive to environmental changes and human activities such as trawling. Any damage can be irreversible, making monitoring critical. Since healthy coral can only be distinguished from dead frameworks by their color, imaging – at a subcentimeter-level and in color – is critical, which means underwater cameras are a must. But because these reefs are sparsely distributed over city-sized areas, monitoring can be time-consuming and costly.
3D reconstruction of whale carcass discovered by Biocam at the Darwin Mounds. Image: Sonardyne
BioCam hopes to address this type of challenge. The project, launched in 2017, builds on the earlier SeaXerocks system and leverages advances in lighting and higher resolution detector technology to enable 3D imaging capable of mapping the seafloor at millimetric-scale, from up to 10 m altitudes. This enables increased area coverage and safer altitude operation from significantly faster-moving AUVs, potentially enabling data gathering at significantly faster rates than existing technologies. The BioCam design has been underpinned by electro-mechanical design, testing and manufacturing support from Sonardyne, with a key focus on ensuring the system was smaller, lighter and less power hungry so that it could be used on long-endurance AUVs.
At its core is a pair of high-dynamic range scientific CMOS (complementary metal–oxide–semiconductor) cameras, each with 2,560 x 2,160 pixel resolution – double that of SeaXerocks. These are mounted in a 4000 m-rated titanium housing, optimized for AUV mounting, that also includes low-power electronics to control the lasers, strobe illumination, communication, data acquisition and processing.
Like SeaXerocks, BioCam uses laser lines, but has two lasers, which are pulsed in synchronization with the camera. These were installed on the NOC’s Autosub6000 to provide an ~8 m swath able to cover several hectares per hour at ~3 mm resolution, enabling large, high resolution 3D texture maps to be created by combining strobe illuminated images and laser line measurements.
Building explorable 3D reconstructions
If anything, this results in even more data being generated. So, a further focus of the project has been automating the creation of large explorable 3D reconstructions and data processing – within a comparable timeframe to the data acquisition.
The process to mesh the strobed and laser images with vehicle navigation data has been largely automated, using the relative position between the camera poses and the bathymetry, to generate depth maps corresponding to each strobe illuminated image frame (having already calibrated the stereo camera after assembly, before an expedition). The images are then color corrected, using the depth maps and images, and then reprojected on to the bathymetry mesh, creating explorable 3D visual reconstructions.
To rapidly summarize what has been observed, the data processing pipeline uses self-calibrating machine learning algorithms that automatically group similar looking patches of the seafloor, and generates maps showing the patterns they form. To make this easy to understand, the images that are most representative of each group are tagged to the maps, and a user can use any image as a query, to generate maps of seafloor regions that look similar to the query image. Furthermore, the global community of experts are able to participate in this process through online labelling platforms and use of standard formats.
BioCam was put to the test onboard the NOC’s Autosub6000 during a cruise over the Darwin Mounds. Tens of thousands of stereo pair images were taken with an average coverage rate of ~16,000 m2/h (not including the overlaps needed to build a continuous mosaiced map of the seafloor) – an order of magnitude higher than conventional millimeter resolution imaging surveys. During one of the dives a whale carcass was found in 956 m water depth. The 3D reconstruction enabled it to be measured, at 7.51-m long and 1.03 m wide at its skull.
Unsupervised clustering of images from one of the dives automatically organized seabed features by color: pink and purple representing coral; blue, light blue, orange, green and beige representing rippled sand; and lime green and red representing xenophyophores (a large, single cell organism). This data could then be queried. For example, a selected image of coral could be used to identify similar images. The results of such a query showed that the corals were most densely distributed in rings around the base of the mounds; a crown rather than a cap. A separate query of an image showing xenophyophores – a recognized Vulnerable Marine Ecosystem indicator species – showed strong correlation in the tails of the mounds.
One of the goals for the BioCam project was to enable the data it gathers to be easily disseminated to the broader scientific community during an ongoing expedition. A web-based open-access annotation platform Squidle+ was used, with the data still available under campaign dy108‐109_nerc_oceanids_class.
Outlook
The Darwin Mounds survey showed the ability to survey a large area quickly, but also to quantify the distribution of species over that area – in an equivalent timescale. Potential uses for this capability are already emerging, such as surveying seafloor ecology around decommissioned sites in the North Sea or surveying areas for natural gas or CO2 leaks. What’s more, this could be done without the need for a crewed research vessel on hand, with shore based launch and recovery. The next steps for BioCam include the possibility of real-time, in situ image processing and remote awareness of this processed information.
BioCam system installation on NOC Autosub6000. Image: Sonardyne
No comments:
Post a Comment