Parametric Segmental Switching Linear Dynamic Systems (PS-SLDS)


Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems
Sang Min Oh, James M. Rehg, Tucker Balch, Frank Dellaert
International Journal of Computer Vision (IJCV) Special Issue on Learning for Vision, May 2008. Vol.77(1-3). Pages 103-124.

Honey Bee Dance Data

The experimental data are zipped and downloadable.
In case you would prefer to create your own videos of your experimental results on top of the honey bee dance videos,
you can do so by downloading the original videos below, and use TeamView software.
In case you publish the results on the honey bee dance dataset, please cite our IJCV paper if possible.

Experimental Results

Dancer bees are automatically tracked by a appearance based tracker from video sequences. The tracked bee is shown in green rectangle in the left figure below.
The right figure shows a stylized bee dance through which bees talk to the other bees about the orientation and distance to the food sources.

The automatically obtained bee trajectories are labeled based on the learned models. The experimental results described in detail along with the video clips show the capabilities of tested models : PS-SLDS and original SLDS.
The experimental results show the superior recognition capabilities of the proposed PS-SLDS model over the original SLDS model. The label inference results on all data sequences are shown below. The three color strips in each figure represent SDLS Viterbi, SLDS Variational Approximation, PS-SLDS Viterbi and the ground-truth (GT) labels from the top to the bottom (See figures in Video Clips section below). The x-axis represents time flow from left to right and the color is the label at that corresponding video frame. Key : waggle (green), right-turn (red), left-turn (blue).
The superior recognition abilities of PS-SLDS can be observed from the presented results. The PS-SLDS results are closer to ground truth or comparable to SLDS results in all sequences. Especially, the sequences 1, 2 and 3 are challenging. The tracking results based on the vision-based tracker was more noisy. In addition, the patterns of switching in dance modes and the durations in each dance regime are more irregular than the other sequences.
It can be observed that most of the over-segmentations that appear in the SLDS labeling results disappear in the PS-SLDS labeling results. PS-SLDS estimates still introduce some errors, especially in the sequence 1 and 3. However, assuming that even an expert human can introduce labeling noise, the labeling capabilities of PS-SLDS are superior.

Video Clips : Presented Result and Original Videos

The video clips below shows the automatic labeling results of the bee trajectories based on the presented PS-SLDS work. There are additional

  Data 1   : PS-SLDS SLDS Original Video

  Data 2   : PS-SLDS SLDS Original Video

  Data 3   : PS-SLDS SLDS Original Video

  Data 4   : PS-SLDS SLDS Original Video

  Data 5   : PS-SLDS SLDS Original Video

  Data 6   : PS-SLDS SLDS Original Video