Parametric Segmental Switching Linear Dynamic Systems (SLDSs)

Parametric Segmental Switching Linear Dynamic Systems (PS-SLDS)

Paper

Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems
Sang Min Oh, James M. Rehg, Tucker Balch, Frank Dellaert
International Journal of Computer Vision (IJCV) Special Issue on Learning for Vision, May 2008. Vol.77(1-3). Pages 103-124.

Honey Bee Dance Data

The experimental data are zipped and downloadable.
In case you would prefer to create your own videos of your experimental results on top of the honey bee dance videos,
you can do so by downloading the original videos below, and use TeamView software.
In case you publish the results on the honey bee dance dataset, please cite our IJCV paper if possible.

Instructions (Possibly out of date. Please read the relevant sections only).
Data

Experimental Results

Dancer bees are automatically tracked by a appearance based tracker from video sequences. The tracked bee is shown in green rectangle in the left figure below.
The right figure shows a stylized bee dance through which bees talk to the other bees about the orientation and distance to the food sources.

The automatically obtained bee trajectories are labeled based on the learned models. The experimental results described in detail along with the video clips show the capabilities of tested models : PS-SLDS and original SLDS.
The experimental results show the superior recognition capabilities of the proposed PS-SLDS model over the original SLDS model. The label inference results on all data sequences are shown below. The three color strips in each figure represent SDLS Viterbi, SLDS Variational Approximation, PS-SLDS Viterbi and the ground-truth (GT) labels from the top to the bottom (See figures in Video Clips section below). The x-axis represents time flow from left to right and the color is the label at that corresponding video frame. Key : waggle (green), right-turn (red), left-turn (blue).
The superior recognition abilities of PS-SLDS can be observed from the presented results. The PS-SLDS results are closer to ground truth or comparable to SLDS results in all sequences. Especially, the sequences 1, 2 and 3 are challenging. The tracking results based on the vision-based tracker was more noisy. In addition, the patterns of switching in dance modes and the durations in each dance regime are more irregular than the other sequences.
It can be observed that most of the over-segmentations that appear in the SLDS labeling results disappear in the PS-SLDS labeling results. PS-SLDS estimates still introduce some errors, especially in the sequence 1 and 3. However, assuming that even an expert human can introduce labeling noise, the labeling capabilities of PS-SLDS are superior.

Video Clips : Presented Result and Original Videos

The video clips below shows the automatic labeling results of the bee trajectories based on the presented PS-SLDS work. There are additional

PS-SLDS : results show the bee trajectories superimposed with colored labels on the original tracking data. Labels are obtained as the result of inference on the hidden motion patterns.
SLDS : results show the inferred labels using approximate Viterbi/variational approximation method.
Original Videos : are the original videos used to create the movie clips of PS-SLDS and SLDS results.
- For those who are interested in visualizing your own behavior analysis results on the actual honey bee videos, you can use TeamView Software developed at Georgia Tech. However, note that the orientation visualization convention changed in TeamView Software, hence you would need to turn off the oriented triangle when you create your own movies. Maye you would need to convert the video format/codec to fit TeamView in case the evolving TeamView software does not support the original videos anymore.(as of Jun 29, 2008, the uploaded formats work on TeamView without any problem).
- For those who are interested in tracking, note that the uploaded videos on the page are not the ones used for tracking.
The raw format tracking data are 6~7 Giga byte each, and currently not available on this page. The original tracking videos are available only for the sequences 4~6.
If you're interested in using them, please send e-mail to the first author of the IJCV paper, and have your anonymous FTP server open. We may upload the videos into your ftp servers as time permits.

Data 1 : PS-SLDS SLDS Original Video

Data 2 : PS-SLDS SLDS Original Video

Data 3 : PS-SLDS SLDS Original Video

Data 4 : PS-SLDS SLDS Original Video

Data 5 : PS-SLDS SLDS Original Video

Data 6 : PS-SLDS SLDS Original Video