MIT media lab: recompose MIT media lab: recompose
feb 17, 2011

MIT media lab: recompose

‘recompose’ input/output actuated surface, by the MIT media lab

recompose‘, the work of matthew blackshaw, anthony devincenzi, dávid lakatos, daniel leithinger, and hiroshi ishii at america’s MIT media lab, integrates user input and visual output into a single device. responding to both physical touch and also gestural interaction, the system offers new ways of interacting in realtime with three-dimensional representations.

video demonstration of ‘recompose’ video courtesy of fastcodesign

‘recompose’ consists of 120 physical tiles, mounted on small rods that rise or sink in response to user input as on a typewriter. in addition to responding to direct presses, however, the keys of ‘recompose’ also react to gestural input, such that moving one’s fingers over keys or making the gesture of pulling up or pushing down will cause the same effect.

the device currently recognizes five kinds of user behaviour: ‘selection’ involves the projection of light onto the device’s surface; the ‘actuation’ gesture will raise the selected keys; and ‘translation’, ‘rotation’, and ‘scale’ interactions all modify the selected input accordingly.

the green light cast from a mounted projector indicates the selected area, while the user’s hand gesture instructs the device to raise these keys

not merely a user interface, ‘recompose’ can also be used for visualization, such as the three-dimensional display of graphs, or the representation of the relative ‘pressure’ of user gestural input.

because it is at once input and output device, ‘recompose’ offers completely new modes of visualization functionality, such as the ability to transform via gestures three-dimensional changes in the surface. although for the current model, this can produce only the scaling, rotation, and other manipulations of simple shapes, one can easily imagine the ways in which a more finely detailed surface might provide three-dimensional interactive visualizations of modeling projects, CAD designs, or other visual and infographic data.

‘recompose’ responds to both tactile and gestural input

the device is based on team member daniel leithinger’s ‘relief‘ table, a similar input/output device for tactile input. to this basic model, ‘recompose’ adds a depth camera and projector above the table. with input from the camera, computer vision detects user interaction and determines the position, orientation, and depth of the hands and fingers and relays the desired changes to the individual keys.

concept diagram of gestural interactions, clockwise from top left: selection (2 images), actuation, translation, scaling (both images), and rotation (both images).

the view of input as modeled through computer vision

via fastcodesign

  • Sad. The MIT Media lab presenting second-year-in-university level projects.

    Pavla says:
  • @Pavia

    I think it’s brilliant. I don’t see how it could be used yet, but maybe you have done something more impressive? Please share.

    Roger says:

have something to add? share your thoughts in our comments section below.

comments policy
designboom's comment policy guidelines
generally speaking, if we publish something, it's because we're genuinely interested in the subject. we hope you'll share this interest and if you know even more about it, please share! our goal in the discussion threads is to have good conversation and we prefer constructive opinions. we and our readers have fun with entertaining ones. designboom welcomes alerts about typos, incorrect names, and the like.
the correction is at the discretion of the post editor and may not happen immediately.

what if you disagree with what we or another commenter has to say?
let's hear it! but please understand that offensive, inappropriate, or just plain annoying comments may be deleted or shortened.

- please do not make racist, sexist, anti-semitic, homophobic or otherwise offensive comments.
- please don't personally insult the writers or your fellow commenters.
- please avoid using offensive words, replacing a few letters with asterisks is not a valid workaround.
- please don't include your website or e-mail address in your comments for the purpose of self-promotion.
- please respect jury verdicts and do not discuss offensively on the competition results
(there is only one fist prize, and designboom usually asks renown professionals to help us to promote talent.
in addition to the awarded designs, we do feel that almost all deserve our attention, that is why we publish
the best 100-200 entries too.)

a link is allowed in comments as long as they add value in the form of information, images, humor, etc. (links to the front page of your personal blog or website are not okay). unwelcome links (to commercial products or services of others, offensive material etc. ) will be redacted. and, ... yes, spam gets banned. no, we do not post fake comments.


a diverse digital database that acts as a valuable guide in gaining insight and information about a product directly from the manufacturer, and serves as a rich reference point in developing a project or scheme.

technology news