The Department of Energy’s (DOE’s) Water Power Technologies Office (WPTO) is working to advance the testing and development of reliable, cost-effective marine energy technologies. The Triton Initiative is a WPTO project at Pacific Northwest National Laboratory (PNNL) dedicated to supporting this mission. One of the goals of Triton is to explore and test practical and efficient tools for environmental monitoring that help reduce barriers to marine energy permitting and development.
Regulators who permit and license marine energy deployments need information that address concerns about the potential environmental impacts of marine energy devices. This includes the behavior and life functions of the marine animals that live in dynamic ocean habitats suitable for harvesting marine energy. Questions about how to measure and monitor behavioral responses of marine wildlife when they encounter a marine energy device inspired Triton’s Marine Wildlife Detection and Tracking Project. This project is led by Alicia Amerson, a project manager and marine biologist at PNNL, who is passionate about biodiversity and human-wildlife coexistence research, has a decade of experience in marine mammal research, including work using aerial technologies to conduct her research. Amerson first considered using unoccupied aerial vehicles (UAV) equipped with cameras for the detection and monitoring of marine wildlife at marine energy test sites. UAVs are proven to be valuable tools for marine mammal research. However, little has been done to advance capabilities for tracking and detecting the presence, absence, and displacement of marine mammals around marine energy devices. Displacement is the possible loss of habitat due to disturbance, such as human activities. Displacement may occur in the form of redistribution of aquatic species or complete avoidance of an area, and marine energy researchers and stakeholders want to know if marine mammals are displaced from their habitats due to the installation of an array of marine energy devices.
The primary constraint of UAVs for this type of research is flight time due to their short battery life. Typically, UAVs can only be in the air for about 20 to 30 minutes before needing to return for a fresh battery. Additionally, when conducting marine wildlife studies from the shore or a boat, short flight durations and environmental conditions can make it difficult locate marine mammals that may only come to the surface periodically to take a breath. To effectively observe and monitor animal behavior with UAVs, specialized cameras and sensors with longer flight times are required. These limitations prompted Amerson to reach out to the Triton principal investigator and Earth scientist, Joseph Haxel, who posed the question “what if we use a tethered balloon instead?”
Existing technology, new application
The Atmospheric Radiation Measurement (ARM) is a DOE scientific user facility dedicated to atmospheric climate research efforts. PNNL and Sandia National Laboratories (Sandia) are of the nine DOE national laboratories contributing to the management and operation of this program. Through ARM, projects conduct atmospheric climate research using helium-based meteorological tethered balloon systems (TBS) across the country. Each balloon can collect atmospheric data up to 1500 meters altitude and can operate in extreme conditions, like the Arctic. TBS can fly for long durations (days to weeks), day or night, and in a variety of weather conditions such as high winds and extreme temperatures. These attributes make them ideal for applications in high-energy and open ocean environments.
Excited about the possible use of a TBS for Triton’s efforts, Amerson connected with Sandia researcher Darielle (Dari) Dexheimer, an expert on using TBS and a licensed pilot through the Federal Aviation Administration. Dexheimer also manages the TBS facility at Sandia for the ARM program. She was eager to expand TBS capabilities with open ocean research and when Amerson proposed this idea, it was an immediate connection for the two scientists and their respective institutions. Dexheimer has previously deployed the TBS from a barge in Louisiana which sparked an interest in applications related to marine wildlife research. Adapting this technology for marine wildlife detection allows researchers to overcome the UAVs limitation associated with battery, payload, and weather-related flight constraints. Both researchers were thrilled about partnering to trial the TBS to monitor marine mammal behavior around marine energy installations. These efforts are particularly compelling because it allows researchers to integrate commercial-off-the-shelf sensors and cameras with novel aerial technologies to address important data gaps around animal behavior for marine energy regulators.
Before these environmental stressor related questions could be explored, the team needed to determine if the TBS technology was truly fit for the job. The team thoroughly examined how to develop an ideal platform for offshore deployments, what environmental conditions the balloon can fly in—such as areas of high wind and wave action—and what permits would be needed to fly a balloon in the open ocean and around protected marine mammal species.
Another critical component of this project included determining which cameras and sensors were well-suited to collect the data needed to observe animals in water, how many sensors could practically be deployed on a TBS, and what type of preliminary data were required to validate the technology package for observing marine wildlife. To account for some of the range of marine biodiversity found in marine energy test site areas in the United States, the team identified payload options that could monitor different types and sizes of marine animals, ranging from small seabirds to large whales up to 70 feet in length. They decided to test two thermal cameras, one multispectral camera, and one optical camera.
Amerson worked with PNNL researchers Andre Coleman and Ilan Gonzalez-Hirshfeld to identify a multispectral camera capable of creating an image with different wavelengths of light, including those outside the spectrum visible to humans. Multispectral cameras collect data on the reflection of light energy off objects in the environment. In particular, the team chose a multi-spectral camera utilizing the coastal blue range of the light spectrum, which enables wildlife detection and tracking above and below the water’s surface up to several meters in depth. Thermal cameras, another sensor type, detect temperature contrasts; with capabilities to locate the body of a marine mammal swimming on the surface or their breathing based on minor changes in temperatures. Optical cameras (ones that capture visible red, green, and blue wavelengths [RBG for short]) produce familiar images like those from a cell phone or point-and-shoot camera. An RBG camera is essential to help determine species identity of marine animals and is also essential to ground truth the data outputs from the multispectral and thermal cameras. To achieve high-resolution images, cameras attached to UAVs or balloons need to be stable and are typically mounted on a gimbal that compensates for motion. For this project the Sandia research team engineered an arm that has a gimbal on each end to hold the different cameras attached to the tether of the balloon.
Putting the tethered balloon to the test
Validation is a key step in exploring any new application of an environmental monitoring technology. As part of this project, the PNNL and Sandia teams traveled to La Porte, Texas, where they completed a six-day technology validation effort with support of boat operators, Jimmy Flynn and Travis Griggs from University of Houston-Clear Lake. The field effort included Amerson and two other Triton researchers, Haxel and Garrett Staines from PNNL, and Dexheimer and the Sandia team, including David Novick, Brent Peterson, and Casey Longbottom. After a year and a half of planning, the teams came together in person for the first time. “Meeting everyone was exciting and rewarding after facing several challenges to make the timing and details for this project come together,” says Amerson.
To maximize the system validation efforts in Texas, as well as to establish consistent evaluation methodologies, the Triton team designed and fabricated multiple marine mammal models to act as proxies–also known as surrogates—to detect and identify during TBS testing. The surrogates included a three-foot, three-dimensional sea lion made of silicone, a nine foot whale named Ethel, and a 20-foot whale lovingly dubbed Large Marge. The whale surrogates were made using polyvinyl chloride (PVC) pipe, black tarp, and pool noodles for floatation. All three were tested for buoyancy at the PNNL-Sequim campus prior to the field effort in Texas.
The tethered balloon was 22 feet in diameter and took almost 15 bottles of helium to fill. Dexheimer and her experienced team operated the TBS with skill, ascending the balloon and sensor package to altitudes up to 250 meters. A traditional recreational vehicle trailer was converted into the operations station with computers, bench space, and tools to fix anything, monitor the system, track live outputs, and to try to keep the mosquitos away! The TBS was housed on a flatbed trailer which could be moved close to the shoreline to maximize the over water view while the TBS performed operations at different altitudes.
The first test conducted in La Porte aimed to validate the TBS and payload and assess the sensor package’s ability to detect the marine surrogates on land. Ethel, the small whale, and the sea lion were placed on the grass near the TBS operations area. The TBS ascended to a low altitude where the sensor settings were adjusted to obtain images and orient the cameras, all while operators calibrated them using hand-held remote devices. These calibrations and practice sessions were necessary to make sure that accurate detection of the surrogates were underway once they were deployed in the water and more difficult to view. Additionally, these efforts allowed the team to identify and troubleshoot any challenges or technical issues. “These initial tests can be the most challenging, and it was exciting for the team to work through them and communicate a path forward to safely conduct the work and obtain data needed to validate the system and sensors,” shares Amerson. Once initial photos were captured and data from sensors were obtained on land by the Sandia team, the Triton team took to the sea and began to tow the surrogates from the University of Houston research vessel.
Fieldwork often presents unexpected challenges that need to be met with resourcefulness and creativity. Initially, the small whale surrogate struggled to stay above water, and it took the first in-water trial for the team to identify and address this buoyancy issue. In response, they added more floatation in the form of pool noodles to both the PVC surrogates, allowing them to be fluidly towed through the bay. Once the marine mammal surrogates were towing successfully, the TBS was flown at various altitudes, at different times of day, and in different weather conditions to validate the performance of the multispectral, RGB, and thermal cameras to detect the moving surrogates. The team completed a total of seven flights in six days, capturing thousands of images to analyze.
At the end of the field campaign, Ethel and Large Marge were celebrated for their contributions to science and retired to their permanent home on the wall of the University of Houston-Clear Lake atmospheric research laboratory warehouse, but that doesn’t mean the project is over. Data from these deployments will help train artificial intelligence and machine learning programs to detect and track marine wildlife in ocean habitats where marine energy devices may be deployed—all with the aim to build knowledge around the potential movement and behavior of animals in relation to these devices.
Steps toward a broader vision
In the next phase of the project, the project team will deploy a TBS to detect wild marine mammals in the open ocean. The team is exploring opportunities to fly in areas where large baleen whales, like blue, fin, humpback, and gray whales use the Oregon coast and open ocean habitats to forage and migrate. These data will provide key opportunities for developing TBS capabilities for detection and monitoring marine animals in their natural habitats.
PacWave, a marine energy test site, is located approximately 7 miles off the shore of Newport, Oregon sharing habitat with large whales. Deploying the TBS in these waters will provide the use-case necessary to evaluate how practical and effective these techniques are for monitoring in high-energy environments. If successful, using these technologies around the boundaries of a marine energy installation would allow researchers to evaluate a tool designed to directly observe, and hopefully better understand animal movement and behavior in the proximity of wave energy converters or other marine energy devices. In turn, regulators and permitting decision-makers could use this type of information to help mitigate possible concerns regarding animal interactions or future marine energy installation-based impacts.
If the TBS system proves to be a plausible monitoring method in high-energy environments, other innovative applications could be explored. “These systems may be designed to operate autonomously which is the long-term vision Dari and I have for this project,” says Amerson. Environmental monitoring at sea is typically expensive and possibly unsafe due to the unpredictable nature of offshore environments where there are high energetic resources. “If we can help reduce the human component of intensive environmental monitoring, it would make data collection safer, and if we provide a technology capable of long-term deployment from an autonomous platform, we could collect data during the day or night around boundaries of marine energy sites. This is unprecedented.” Amerson adds, “in an application like this, the tethered balloon system and camera technologies could inform animal movement and behavior, monitor other environmental variables or human infrastructure, or collect data for other atmospheric or research programs in climate change research.” She concludes, “the sky is really the limit, and we are excited to see where the research goes in the future and all the possibilities there are for the TBS application at sea.”
Check out this video highlighting the project:
Written by Cailene Gunn.