From: Adrian Freed <adrian@cnmat.berkeley.edu>
Subject: clustering on guass maps for dance etc.
Date: 1 July, 2013 9:55:10 AM PDT
To: John MacCallum <john@cnmat.berkeley.edu>
Cc: Sha Xin Wei <shaxinwei@gmail.com>, Andrew Schmeder <andy@enchroma.com>, Rama Gottfried <rama.gottfried@gmail.com>
I have an intuition fueled by a half understanding developed over a drink with Xin Wei that could pan
out into something…
With that disclaimer, the idea is to compute correlations betweens orientations of body parts of dancers (a dancer and between
dancers). The first step is to project the orientation vectors onto a sphere (i.e. a gauss map). Then we do cluster analyses on this.
I am not sure how this is done adapting classical k-means things - I guess you can just replace the usual coordinate discretization with a graph. I wonder if one should go straight for a spherical harmonic representation so as to be able to characterize shapes? Of particular interest would be to compute the autocorrelation to identify periodicities in orientation change. I am not good enough
to be sure but I suspect the Wiener–Khinchin theorem holds so these periodicities can be estimated efficiently with FFT's. While I am wildly speculating here it would be nice to compute the linking number of pairs of paths traced by the body closed in some
ingenious way based on the aformentioned periodicities. This would capture braid structures that occur if you imagine
hands and feet with ribbons tied to them or braid structures in patterned dance (e.g. Ceili Dance).
I will stop these speculations here as it is time to ignore me and plunge into the literature. I have attached something, John,
that addresses your interest in using kurtosis of bodliy motion to drive migrators. This paper came up when I searched for "gauss map cluster" so we should be able to use this as a seed to harvest related papers. What is interesting about this paper is they don't construct a surface description so we can use it perhaps on the depth map data from Kinect.
I am trying to drag Andy into this partly to help us avoid some silly false leads but also because I suspect there is something interesting here for aerial dance which gives more orientation freedom than landed dance.
p.s. There is something deeply silly about what I am suggesting (and philosophically disturbing from Xin Wei's point of view) if we use Kinect skeletonization data. With enough crunch you can do better
than kinect by deriving the kinematics from the imager data using local invariance structure (as hinted in the attached paper). For testing on our slow laptops and getting our feet wet the Kinect is a handy starting point though.
p.p.s. My puzzling over the right arithmetic to use for those quality measures that the kinect software gives us (0, 0.5, 1.0)
lead me to indicator functions which are usually 0 or 1 so as to bridge set membership or other predicates to number. They are however generalized to be continuous in fuzzy set theory which presumably has a lot more to say about how we should operate with them.
I am particularly interested now in how to leverage the dynamics of such functions.