Video methods: Setup and standards

In the summer of 2021, OpenBehavior organized a working group on methods for video analysis. A Slack channel was started to discuss the methods and a series of virtual meetings were held that involved conversations among users and developers. Some in the group decided to write a commentary on the methods, with goals of explaining how to set up video methods in a lab and highlighting issues that developers and advanced users should consider, such the need to openly share datasets and code, how to compare algorithms and their parameters, and documentation and community-wide standards. The results of those efforts led to a manuscript available at https://arxiv.org/abs/2204.02842. We hope that it helps new users get started with video recordings and analysis and also encourages more widespread use and continued development of the tools.

The paper provides a brief overview on video methods, with a focus on tools for pose estimation. The first part of the paper covers what we felt is needed to create a basic setup for video recordings in animal experiments. Advice is given on hardware and software (cameras, data formats, positioning of cameras, materials used in behavioral arenas, and considerations for lighting). Next, the paper describes a typical analysis pipeline for using pose estimation tools and analyzing results from them with supervised and unsupervised machine learning algorithms and other approaches. 

The second part of the paper covers what the group considered to be best practices for data sharing and tool evaluation. We emphasized the need for a common data format, more widespread sharing of raw and processed videos, and the need for direct comparisons among existing and in-development methods and more attention given to how the parameters of a given method influence the accuracy and speed of pose estimation models. This part of the paper closes with a discussion of the need for benchmarks and guidelines for code and reproducibility in animal pose estimation.

We hope that our review of the current state of open-source tools for behavioral video analysis will be helpful to the community.