Introduction

Publications

uBAM: Unsupervised Behavior Analysis and Magnification using Deep Learning arxiv
Biagio Brattoli, Uta Buechler, Michael Dorkenwald, Philipp Reiser, Linard Filli, Fritjof Helmchen, Anna-Sophia Wahl, Bjoern Ommer

Abstract

Motor behavior analysis composes a non-invasive diagnostic approach that is essential in biomedical research and clinical diagnostics for analyzing brain function and adjusting treatment. Current computer-based detailed behavior analysis is limited, since it requires physical or virtual markers to estimate trajectories. Besides the tedious annotation effort required for marking keypointsor training a detector, users need to characterize the behavior deviation beforehand by providing characteristic keypoints. We introduce uBAM, a novel fully automatic approach for detailed behavior analysis and magnification of deviations based on state-of-the-art deep learning. We propose an unsupervised learning of posture and behavior representations. These enable behavior comparison across subjects directly in videos without requiring keypoints or annotations during training or testing. A generative model with a novel disentanglement of appearance and behavior magnifies subtle behavior differences even across subjects. Evaluations on rodents and human patients with different neurological diseases demonstrate the wide applicability of our approach.