breakfast making robots at TUM
breakfast making robots at TUM breakfast making robots at TUM
jun 13, 2011

breakfast making robots at TUM

created by researchers at technical university munich, the robot TUM-rosie prepares a traditional bavarian breakfast

led by dr. michael beets and dr. bernd radig, a team of researches in the IAS (intelligent autonomous systems) group at germany’s technical university munich demonstrate advances in the fields of robotics with two robots that go shopping for food and prepare and serve a traditional bavarian breakfast. the event was organized by the german research collaborative CoTeSys (‘cognition for technical systems’).

the two robots are the US-designed TUM-james and german-engineered TUM-rosie. in the two-part demonstration (a follow-up to last year’s event, during which the pair prepared pancakes), TUM-james first simulates shopping, examining a shelf of products, making a selection, and then carrying and organizing the groceries at ‘home’. then TUM-rosie collects and boils sausages, skims them out, and transfers them to a serving bowl, while TUM-james uses an electric bread slicer to cut a french baguette.

15x-sped video of the robots preparing breakfast and grocery-shopping (full length, annotated videos are embedded below)

much of the robots’ activity is learning-based. their vision is based on output from kinect sensors, while surface-based matching handles the differentiation of objects, such as which colour bowl is the destination for a particular task, accomplished through the ‘learning’ of the 3D models of these objects in order to pinpoint them even if their location is altered. likewise, in the shopping task, TUM-james recognizes objects on the grocery shelf via 3D perception algorithms from the open-source ‘point cloud library’ (PCL) combined with the ‘objects of daily use finder’.

both robots evaluate their grasp of objects using torque and position sensors. any failures in task execution are handled using learned knowledge about the process; for example, if TUM-rosie fails to get the sausage out of the pot, it immediately reattempts the action.

after ‘shopping’, TUM-james determines where in the kitchen the object should be placed using the IAS ‘knowrob‘ (‘knowledge processing for robots’) system, which provides a way for the robot to calculate where the most semantically similar object already in the kitchen is located. this method reduces the amount of information that must be encoded about each object; instead, the robot is able to use information about things it has already seen or what is already available in the environment, and make ‘logical’ decisions when handling new materials.

TUM-james uses an electric slicer to slice bread for breakfast

TUM-james goes grocery-shopping, using object recognition databases to select what is needed

the inset view indicates the robot’s vision, as it adjusts its arm to grasp the shopping basket and fold down the handle

diagram depicts the functioning of the ‘objects of daily use finder’ object recognition software

full-length, annotated video of the robots preparing breakfast

full-length, annotated video of TUM-james ‘shopping’ for groceries

via IEEE spectrum

    have something to add? share your thoughts in our comments section below.
    all comments are reviewed for the purposes of moderation before publishing.

    comments policy
    LOG IN
    designboom's comment policy guidelines
    generally speaking, if we publish something, it's because we're genuinely interested in the subject. we hope you'll share this interest and if you know even more about it, please share! our goal in the discussion threads is to have good conversation and we prefer constructive opinions. we and our readers have fun with entertaining ones. designboom welcomes alerts about typos, incorrect names, and the like.
    the correction is at the discretion of the post editor and may not happen immediately.

    what if you disagree with what we or another commenter has to say?
    let's hear it! but please understand that offensive, inappropriate, or just plain annoying comments may be deleted or shortened.

    - please do not make racist, sexist, anti-semitic, homophobic or otherwise offensive comments.
    - please don't personally insult the writers or your fellow commenters.
    - please avoid using offensive words, replacing a few letters with asterisks is not a valid workaround.
    - please don't include your website or e-mail address in your comments for the purpose of self-promotion.
    - please respect jury verdicts and do not discuss offensively on the competition results
    (there is only one fist prize, and designboom usually asks renown professionals to help us to promote talent.
    in addition to the awarded designs, we do feel that almost all deserve our attention, that is why we publish
    the best 100-200 entries too.)

    a link is allowed in comments as long as they add value in the form of information, images, humor, etc. (links to the front page of your personal blog or website are not okay). unwelcome links (to commercial products or services of others, offensive material etc. ) will be redacted. and, ... yes, spam gets banned. no, we do not post fake comments.


    a diverse digital database that acts as a valuable guide in gaining insight and information about a product directly from the manufacturer, and serves as a rich reference point in developing a project or scheme.

    technology news

    keep up with our daily and weekly stories
    492,671 subscribers
    - see sample
    - see sample