DexterCap: Affordable and Automated Capture of Complex Hand-Object Interactions

How to read this

Category

Academic paper presenting a capture system and an accompanying dataset

Contributions

• Builds a low cost capture system for hand and object interaction, aimed at making large scale acquisition of manipulation data affordable
• Uses dense, character coded marker patches so tracking survives the severe self occlusion that closely spaced fingers cause
• Provides an automated reconstruction pipeline that needs minimal manual cleanup, which is the usual bottleneck in finger capture
• Releases DexterHand, a dataset of fine grained hand and object interactions spanning simple primitives up to articulated objects such as a Rubik's Cube

Context

Hand capture has long been the awkward gap in performance capture: the fingers occlude each other constantly and the motions that matter are small. DexterCap attacks that with marker design rather than with a learned prior, which puts it closer to the classical marker based capture tradition than to the video based hand pose estimation line.

Correctness

The claims rest on the capture system working in practice and on the released dataset, which is the sort of contribution that is verified by other people using it rather than by a benchmark number. Marker based capture also carries its own caveat: the patches are on the hand, so what is captured is a marked hand, not a bare one, and contact with objects is mediated by that.

Clarity

Practical and system oriented, closer to a build description than to a mathematical paper, which makes it approachable if you have ever run a capture session.

How to read it

First pass, look at the hardware photographs and the marker patch design, since that is the actual idea. Second pass, read the reconstruction pipeline for how much automation is genuinely there, because minimal manual effort is a claim worth quantifying. Third pass is really for anyone planning to capture hands themselves, or wanting the DexterHand data, in which case go straight to the dataset description.