From Text To Robotic Assembly: 3D Generative AI And Discrete Robotic Assembly For Making Physical Objects
Interview with Alexander Htet Kyaw – MIT
Alexander Htet Kyaw is a researcher at MIT working on generative AI systems that move beyond text-to-3D visualization toward physical, manufacturable assembly.
His work converts text generated meshes into discrete component systems that account for structural performance, robotic fabrication constraints, and assembly feasibility.
Ahead of his presentation at CDFAM DC, we spoke with him about closing the gap between what a generated object looks like and what it takes to actually build it.
Can you give us a brief overview of your current research at MIT and what you’ll be presenting at CDFAM?
My current research at MIT focuses on how generative AI can move beyond producing digital 3D assets and begin producing physical, manufacturable objects.
While 3D printing has made it easier to fabricate complex geometries directly from digital models, my work focuses more on modular, discrete assembly systems, where objects are constructed from reusable components.
In this context, many current text-to-3D systems are very good at generating meshes that visually resemble a prompt, but those meshes usually do not account for how an object can be broken down into parts, assembled, or fabricated within real-world constraints.
The gap between a generated 3D mesh of what something ‘looks’ like, and an object that performs a given requirement AND can be manufactured is significant. How does your research bridge that gap, and at what stage does the system begin reasoning about the physical properties like stiffness or material distribution?
The system bridges the gap between “what something looks like” and “how it can be made” by converting generated meshes into a discrete assembly representation.
Once the object is discretized, the system can evaluate constraints such as connectivity, overhangs, vertical stacking, component count, reachability, and assembly sequence.
In addition, there’s a parallel investigation on structural robotic assembly, with an emphasis on load-bearing performance. The longer-term goal is to connect generative design, functional decomposition, assembly logic, and structural behavior more tightly.
What role do vision language models play in the pipeline, and how does the system understand a text prompt’s spatial structural requirements?
A vision language model (VLM) is a type of AI system that can understand both images and text together. For example, the VLM can take in a text prompt alongside multi-view rendered images of the generated object and reason about its functional requirements.
On the robotic assembly side, what constraints from the fabrication process feed back into how assemblies are generated or decomposed, and how is that data flow managed?
The robotic assembly process creates many constraints that directly affect the design. The robot has limits in reach, grasping and collision. The object also has to remain feasible during assembly, not just after it is completed.
These constraints feed back into the generation and decomposition process through feasibility checks and assembly planning.
The system moves from prompt to generated mesh, to discrete component representation, to feasibility evaluation, to assembly sequence, and finally to robot execution.
Discrete assembly implies modularity and reusability. How does the system decide on component boundaries, to reuse or create a novel component, and how much of that is currently rule-based versus learned?
Right now, the system works with a known set of library components that the system already understands how to use based on predetermined rules. Because the components are reusable, the system prioritizes mapping geometry into existing modular units rather than inventing a new component every time.
Finally, what key takeaway do you want the audience to have from your presentation, and what connections are you hoping to make at the event?
The main takeaway I want the audience to have is that text-to-physical generation is not simply text-to-3D followed by fabrication.
Generative AI can become part of a physically grounded fabrication loop, rather than only a tool for producing visual concepts.
At CDFAM, I’m hoping to connect with people working across computational design, manufacturing, robotics, structural systems, and AI.
I’m especially interested in conversations around how we can make generative systems more useful for real manufacturing and how discrete, modular systems might support more sustainable ways of making.
