When Trackers Date Fish

A Benchmark and Framework for Underwater Multiple Fish Tracking

Weiran Li, Yeqiang Liu, Qiannan Guo, Yijie Wei, Hwa Liang Leo, Zhenbo Li*
China Agricultural University National University of Singapore

Abstract

Multiple object tracking (MOT) technology has made significant progress in terrestrial applications, but underwater tracking scenarios remain underexplored despite their importance to marine ecology and aquaculture. We present Multiple Fish Tracking Dataset 2025 (MFT25), the first comprehensive dataset specifically designed for underwater multiple fish tracking, featuring 15 diverse video sequences with 408,578 meticulously annotated bounding boxes across 48,066 frames. Our dataset captures various underwater environments, fish species, and challenging conditions including occlusions, similar appearances, and erratic motion patterns. Additionally, we introduce Scale-aware and Unscented Tracker (SU-T), a specialized tracking framework featuring an Unscented Kalman Filter (UKF) optimized for non-linear fish swimming patterns and a novel Fish-Intersection-over-Union (FishIoU) matching that accounts for the unique morphological characteristics of aquatic species.

15
Sequences
408K
Annotated Boxes
34.1
HOTA Score
44.6
IDF1 Score
SU-T Framework Overview
Overview of the SU-T framework, optimizing unscented Kalman Filter (UKF) and Fish-IoU for non-linear swimming patterns.

MFT25 Dataset

  • First comprehensive benchmark for underwater MOT
  • Includes both actual farm & laboratory collections
  • Diverse range of fish species and turbidity conditions
  • 48,066 frames meticulously annotated

SU-T Framework

  • Unscented Kalman Filter (UKF) for non-linear motion
  • Fish-IoU optimized for aquatic body shapes
  • State-of-the-art performance on MFT25
  • Robust against occlusion and rapid swimming

Performance Comparison

Method Class Year HOTA ↑ IDF1 ↑
FairMOTJDE202122.22626.867
CMFTNetJDE202222.43227.659
TransTrackTF202130.42635.215
TransCenterTF202327.89630.278
TrackFormerTF202230.36135.285
TFMFTTF202425.44033.950
SORTSDE201629.06334.119
ByteTrackSDE202231.75840.355
BoT-SORTSDE202226.84836.847
OC-SORTSDE202325.01734.620
Deep-OC-SORTSDE202324.84834.176
HybridSORTSDE202432.25838.421
HybridSORT†SDE202432.70541.727
SU-T (Ours) SDE 2025 33.351 41.717
SU-T† (Ours) SDE 2025 34.067 44.643

Downloads

MFT25 Dataset

Includes 15 video sequences, annotation files, etc.

BaiduYun (Pwd: wrbg)

Pretrained Models

Includes SU-T base model, SU-T with ReID module, and checkpoints.

BaiduYun (Pwd: 9uqc)

Contact