Simon Fraser University (SFU), the Pacific Salmon Foundation, and Wild Salmon Centre have teamed up with First Nations fisheries to explore the use of artificial intelligence (AI) to detect and count wild salmon.
Salmon are a vital part of the livelihoods, food security and cultures of communities along the Pacific Northwest coast. However, their populations are under threat and their returns are unpredictable. Maintaining the sustainability of salmon fisheries requires effective monitoring of stocks, often done by volunteers and in-river cameras—which demands time and resources.
SFU computing science professor Jiangchuan Liu and biological sciences/resource and environmental management professor Jonathan Moore are working with the Pacific Salmon Foundation and partners to develop AI technology to track wild salmon returns. The collaboration includes salmon watershed scientist William Atlas of the Wild Salmon Centre in Portland, the Gitanyow Fisheries Authority and the Skeena Fisheries Commission. SFU graduate students have also contributed to the research—in particular, PhD candidate Sami Ma contributed to the model and algorithm design as well as deploying the system in the wild.
The research team worked with the two British Columbia (B.C.) First Nations fishery programs to develop and test AI deep learning models for automated video enumeration of salmon. The AI was trained to detect and track salmon passing through First Nation-run weirs at the Bear River and the Kitwanga River in Northwestern B.C.
The researchers gathered and annotated more than 500,000 frames of video data encompassing 12 species, while training the AI to recognize the different species and genders of fast moving and migrating fish. The AI, appropriately nicknamed “salmonvision,” was then tested at the Coquitlam and Bear Rivers and found to be more than 90 percent accurate for detecting coho and 80 percent at detecting sockeye salmon, when compared with data gathered by fisheries staff.
Thanks to support from the BC Salmon Restoration and Innovation Fund and the Department of Fisheries and Oceans Canada plans are in the works to collaborate with more First Nation communities in B.C. and Yukon on using AI as an additional tool to support sustainable salmon fisheries.
The article Wild salmon enumeration and monitoring using deep learning empowered detection and tracking, outlines the researchers’ methods and findings and was published in Frontiers of Marine Science.
We spoke to Jiangchuan Liu and Jonathan Moore about their research.
What are some of the benefits of using artificial intelligence (AI) to count salmon?
Liu: Automating salmon counting with AI can allow a greater amount of footage to be reviewed for Indigenous-led fisheries, enhancing the quality of the data being gathered. For the First Nations, it helps provide governance and sovereignty over their salmon fisheries and rivers. The system will identify and count each salmon species in real-time, freeing up technician time and resources to concentrate on other efforts in making informed in-season fishing decisions for responsible fishing in this changing climate.
Moore: In this world of climate change and unpredictable salmon returns, in order to manage fisheries sustainably, we need to know how many fish are coming back successfully to spawn. Thus, as “salmonvision” is developed and applied, it provides a tool to count fish more efficiently. This information allows managers to either open or close fisheries for sustainability.
Tell us about some of the challenges your team encountered when teaching AI to detect fish.
Liu: Gathering salmon data from the wild itself is a big challenge. Most of the working sites in northern B.C. have no infrastructure support—no roads, no power, no internet. We have to rely on solar panels or diesel generators for power supply, which are not reliable nor sufficient for computing tasks. We are exploring Starlink for transmitting the data, which is better than earlier generation geosynchronous satellite communication services, but is still power-hungry and heavily affected by terrain and weather, particularly in the Great Bear Rainforest with 30-meter-tall trees. Underwater images can be easily distorted by water quality and movement, thereby affecting the detection performance.
In addition, certain salmon species can have very similar morphologies especially in different environments, making it challenging for the detection model to distinguish the species when processing video in new rivers and environments. Training models directly from footage of the target river could remedy the issue which will be an experiment in comparing river-specific detectors against a general-use salmon detector.
Do you have an update on this work? Has the AI been employed at other sites, and what are you planning for the future?
Liu: The dataset is currently being expanded with data from the Koeye and KwaKwa rivers in B.C. with more samples of species other than Coho which should improve the overall performance of the model. Our current plan is to move model architectures to YOLO v8, which is a computer visioning tool that stands for “You Only Look Once.” New tools have been implemented that incorporate multi-object tracking which is required for counting, speeding up the entire pipeline to make all aspects fully real-time.
The salmon monitoring system will be further refined and deployed at the same sites in this upcoming season after resolving issues with the lack of alternative power sources other than solar power, and in the future the system will be deployed at even more Indigenous-stewarded river sites as the system is complete and proves beneficial.
We are also exploring other sensing tools, in particular, underwater sonar radar, to track salmon. Sonar radar utilizes sound wave reflections to visualize objects and structures. Unlike video data, sonar data encounters issues such as acoustic shadow, reverberations and background speckle noises, making tracking salmon a challenge. We are working toward a multi-frame detection solution to capture spatial and temporal differences between adjacent sonar frames. The solution also features a streamlined detection pipeline and real-time performance.
AI has the potential to enhance and aid in so many tasks—from stocking grocery store shelves to farming, to facial recognition and more. What do you think are some of the news ways AI will be employed into the future?
Liu: Given the rise of large language models (LLMs) through usages such as ChatGPT, there is a lot of potential for integration into many different fields in an interactive manner. These models can be employed towards essentially any industry and become more general to provide contextual assistance on more than text input and response. However, due to LLMs being mainly text-based, more advancements in computer vision and other computation inputs would be required to achieve such goals. Future prospects could include assistance in data processing and analytics and in products that require reasoning from vision inputs.
How important is collaboration across regions and across disciplines to ensure the health of salmon populations?
Moore: Salmon migrations connect diverse ecosystems and face diverse challenges. These fish also connect diverse communities of people, from the Indigenous rightsholders that have stewarded these areas for millennia to agency managers, to diverse stakeholders to scientists. Collaboration brings different perspectives together, and this joining of perspectives and insights offers potential steps forward to understanding, protecting, and managing wild salmon.
For more, read the CBC News story: First Nations harness power of AI to monitor wild salmon stocks in B.C.
SFU's Scholarly Impact of the Week series does not reflect the opinions or viewpoints of the university, but those of the scholars. The timing of articles in the series is chosen weeks or months in advance, based on a published set of criteria. Any correspondence with university or world events at the time of publication is purely coincidental.
For more information, please see SFU's Code of Faculty Ethics and Responsibilities and the statement on academic freedom.