onformative trains AI to sculpt 3D models from a cube of voxels

what happens when ai becomes the creator?

In the age of technology, inspired by the rising surge of AI-driven design and its controversial impact on human creators, onformative’s ‘AI Sculpting’ questions: what happens when AI becomes the creator? How can human creativity and machine-made output feed each other?

With AI being increasingly adapted in the fields of design, our role as the human creator is changing rapidly. Machines, much faster at learning and reproducing visual output than humans ever could, are driven by efficiency, meanwhile human creators are mostly motivated by curiosity. Exploring this juxtaposition and curious to seek an equilibrium between two co-creators — man and machine — the Berlin-based design studio began an AI research project to program, observe, and reinterpret the evolution of a machine learning how to sculpt a 3D model. ‘The core aspect of this co-creation process is that we — to an extent — let go of control,’ notes the team.

Their experiment, presented as a curated series of digital artworks in a culminating 3D-generated exhibition environment, charts the journey of a simple cube being transformed into an increasingly recognizable shape with every iteration, eventually taking on the shape of a human form.

all images courtesy onformative

onformative trains Ai to transform a voxel cube into a statue

Throughout history, man has held a keen interest in the manipulation of objects and materials of the moment for aesthetic purposes. Transferring this curiosity to the age of technology and artificial intelligence, design studio onformative engineered a machine-learning process and AI model as an emblematic traditional sculpting tool. ‘To best experience AI as the creator and to learn from it became our main pursuit,’ notes the team. The AI developed different strategies seeking constant improvement on its way to materializing a given form. By feeding it with different tools, rules, and rewards through reinforcement learning, the team was able to steer the process — but not the outcome — revealing unpredictable final forms.

Set to achieve the given goal of sculpting a 3D model, the AI was trained through reinforcement learning based on rewards and punishments. The agent, defined as a certain machine-learning model, was programmed to seek maximum reward in a trial and error process. In the voxel-based environment — which started out as one big cube — infinite data and a clear reward structure were provided for the agent to move through. From this initial structure, the agent needed to remove mass to get closer to a predefined target state.

onformative’s ‘AI Sculpting’ questions: How can human creativity and machine-made output feed each other?

With each step, the agent is able to decide where to go, whether to remove a mass of voxels around itself, and how. To enable its learning, it was conditioned in a specific way: when extraneous mass was removed it was rewarded and when mass that ought to be part of the final sculpture was removed, it was punished. Through trial and error the agent created its own strategies to achieve the desired shape.

One very striking outcome of ‘AI Sculpting’ was the enormous parallelism in the training process, wherein the AI was able to simultaneously perform many training sets, resulting in a great variety of sculptural output. Observing the machine’s evolution of the learning curve, onformative noted the strategies, behaviors, and visual outputs, continually experimenting with different parameters and predefined rules.

the project programs, observes, and reinterprets the evolution of a machine learning how to sculpt a 3D model

Visualizing AI through the eyes of the machine

Further exploring the concept of co-creation, onformative decided to take on the view of the agent to see the process from the creator’s perspective. In an attempt to interpret the agent’s decision-making, the design team experimented with visualizing AI data such as confidence, or penalty and reward for individual steps within the 3D environment. By highlighting the agent’s path through the block, for example, they gained another perspective on the creation process itself.

the AI developed different strategies seeking constant improvement on its way to materializing a human form

Finally, the team experimented with their creative expression by implementing different tools for the agent to choose from. Through the reinforcement learning set-up, this process allowed more complexity and resulted in a variety and depth of strategies as well as visual outcomes. By regarding the traces of different tools on the surface of the emerging shapes, the sculpting process becomes visible. Various tools have their own unique fingerprint — from rough to fine and quick to slow — each decision the agent made by choosing a different tool or way of orientation had a different visual outcome and impact.

the AI was trained through reinforcement learning based on rewards and punishments

The Curation: uniting AI and Human Creativity

‘By studying the visual results along the process, our role as a designer became one of a curator,’ notes the team. A number of outcomes were selected and defined as sculptures to present as digital artworks. In this stage, human creativity comes into play again, as onformative had to consider which machine-made manipulations of the cube were deemed presentable, and how they should be exhibited.

‘In a scenario like this, we collaborate with AI. The outcome is not a reproduction of our creativity, it is unforeseen. We receive unprecedented results, which re-inspire our way of creating, highlighting the importance of exploring the concept of co-creating with AI.’

the voxel-based environment started as a cube, and the agent removed mass to get closer to the predefined target