Recommended Citation
Postprint version. Published in 2020 IEEE 14th International Conference on Semantic Computing (ICSC), February 3-5, 2020, San Diego, CA, February 1, 2020, pages 17-24.
The definitive version is available at https://doi.org/10.1109/ICSC.2020.00010.
Abstract
As the world explores opportunities to develop offshore renewable energy capacity, there will be a growing need for pre-construction biological surveys and post-construction monitoring in the challenging marine environment. Underwater video is a powerful tool to facilitate such surveys, but the interpretation of the imagery is costly and time-consuming. Emerging technologies have improved automated analysis of underwater video, but these technologies are not yet accurate or accessible enough for widespread adoption in the scientific community or industries that might benefit from these tools. To address these challenges, we developed a website that allows us to: (1) Quickly play and annotate underwater videos, (2) Create a short tracking video for each annotation that shows how an annotated concept moves in time, (3) Verify the accuracy of existing annotations and tracking videos, (4) Create a neural network model from existing annotations, and (5) Automatically annotate unwatched videos using a model that was previously created. The website was seeded with 50 hours of high-resolution underwater videos that were generously provided by the Monterey Bay Aquarium Research Institute (MBARI). The biology students that were part of the project created more than 30,000 annotations that range over more than 20 concepts. About 3,000 of these annotations were then verified for accuracy by our marine biology experts. Using both validated and unvalidated annotations and automatically generated annotations from trackings, our software was able to count the number of Rathbunaster californicus (starfish) and Strongylocentrotus fragilis (sea urchin) with count accuracy of 97% and 99%, respectively, and F1 score accuracy of 0.90 and 0.81, respectively.
Disciplines
Computer Sciences
Copyright
2020 IEEE.
Number of Pages
8
Publisher statement
Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
URL: https://digitalcommons.calpoly.edu/csse_fac/268