0:01 My name is Ruben Verhack. I co-ounded Datameister about two years ago. I have a strong background in computer science. I double PhD in in 3D representations. Over the last few years we’ve been looking into how actually bringing this more into you know applications. I’m going to present here a a case that we did on AI accelerated automotive design. And the idea is I start off with a question what do these artworks have in what do these artworks have in common?

0:36 So first one is Grenica by Picasso. Second one is the 18th sonnet by Shakespeare and then we have the Michelangelo the D by Malanchulio. The the idea is that Picasso had actually could have drawn this in any other in any style that he wanted. He could have done it in for the realistic way, but he chose to limit himself to like a cubist language and and limiting himself in in the types of shapes and forms.

1:04 Same for Shakespeare. He could have conveyed his information in any free form text. But the beauty is in the fact that he he fitted it in this kind of like solid strict disciplined manner. And the same for Michelangelo. He did this whole thing in marble and the fact that like he constrained himself to do to keep himself fixed to this medium is actually what makes it into a masterpiece brings me to the point that a masterpiece is a combination of talent and constraints.

1:36 Okay, but what about the following? This is the typical AI slop that that everybody has been seeing in their feeds for the last few years. And why don’t we perceive this as masterpieces? Because technologically that it’s great that we can do this. But because it’s so easy and there so much freedom and there’s no constraints, you you typically fall down into cliches because despite there’s been like a huge enormous investment in general purpose visual AI, it doesn’t really immediately lead to masterpieces.

2:16 This is because like an AI when given too much freedom, it it falls back to cliches. It it it has this kind of knowledge of like a mean knowledge of what something should look like. And when when AI is uncon is unconstrained, it tends to do the obvious. However, in reality, constraints shape all design. Especially in industrial design you have hard points and hard points are fixed constraints in design that these can be physical locations or the dimensions of the objects or rules that cannot be moved or changed without major impact.

2:57 You don’t want to be creative on these things. And automotive, for example, this is when you start designing something, a car, you typically already know what wheelbase you’re going to use or or other package data. And then there’s also a ton of regulatory constraints that you need to adhere by. Same for architecture. You need to take into account some load bearing load bearing structures or elevator shaft.

3:26 Any any characteristics of the plot of land you’re working on and also in consumer electronics you’re going to buy some offtheshelf components you want to design around those things that that are already known so for example where the batteries are or any cooling zones and so the goal is to explore the creative in between space shaped by these constraints and this brings us to a project that we did with ground studio Grand Studio is a is a car design company from Torino.

3:54 They’re known for for designing Maserati like this Delada supercar and Alfa Romeo. They saw the the the rise of AI image generation generation and they saw their clients even like bringing ideas of like hey I think it around with my journey. This is something I would like to have. But the problem is that these AI generations they they start from surface whereas actually design starts from the inside.

4:31 For example in a car the position of the the the engine the where where the people need to be and the steering column they all matter. Just by changing the type of engine has a complete influence on the exterior look of the car. If it’s if the engine is in front or in the back or if you have a four-wheel drive, it will define largely how your how your car will look like.

4:59 So ground studio they have this developed this hardware product it’s a kind of a creative platform where that completely aligns the digital and physical world. So, it’s a it’s a seating buck that’s completely configurable. So, the seats can be changed in in position. The steering wheel can be completely changed. And this allows you as a designer to actually experience the design that you have in mind that you’ve up till that point only done virtually.

5:34 And you can add some other props to it. So that it serves as well as like a great communication platform between the stakeholders, the the designers, the engineers, the clients and you can functionally test the car. So you can step into it. You can feel how far is everything like do I like this form of inter interacting? How how the the feeling of like getting into the car.

6:00 This is something you cannot do with a clay model. So this is what they already had. So this this digify product it’s a fantastic platform for creative design plus it helps to define you know these hard points these things that are then fixed plus it actually already largely defines the general shape of the vehicle. I don’t know if you can see it, but like just by repositioning these seats and how how high the platform is, you can already kind of tell if it’s going to be an SUV or a compact car or a delivery truck.

6:33 So, the need is to AI accelerate the ideation process. So, now that you have these hard points and you want to kind of design the exterior, that would be really great. And then you have all these AI image generators. But the problem is that these AI generators, they they create cars that are not manufacturable. They ignore all these real world constraints. Plus, they they lack practicality. Typically, it’s harder for to iterate on your design.

7:07 You often hit this kind of 80% there thing, but like actually closing that last 20% is always the problem. And the results are typically not edit friendly or not in the form that that actually the client wants it. So our first take on this was basically use the AI generate generators and and adapt them to the fixed geometries. So on top left you have then what we call a style input like an AI generated image where they were happy about the general style and look and feel of the car and then you had like a basic geometry of the car that adheres to this digify data like a general rough outline of the car.

7:47 We would then develop like a sort of a blender interface where you can put like a camera viewpoint on this base geometry and to then transfer this style information but onto actually the geometry that is allowed that that you know 100% sure like this is something we can make. This already allowed us to to iterate on on the designs. You could like take regions and regenerate them but keeping the style intact but like allowing for like creative exploration.

8:23 So here you have three different fronts. So each time with different kind of lights different feelings and then this you can steer with other control signals. You can also for example control the atmosphere because for the clients it’s also important like if if the client is in like in more desert country or in Scandinavian country or in more urban environment you want to quickly visualize these ideas that you have because this is what it’s about.

8:57 It’s still in this ideation phase. Now the problem here is that for example in this case where you start off with like more of a brake vehicle and you want to map it more onto an SUV type of geometry you will try start to see that this method starts to break two problems. The first is it’s a fiction before physics. So you first dream and then you try to retrofit it onto reality and this will always kind of break if what you’ve imagined is too far away from of actually where you want to land.

9:32 And the second problem is that there’s this tension between geometry and texture and that which we completely ignored in this first version because the base geometry is it was in this case completely fixed but actually what you want is that these style ideas of for example the the light catcher these these indents actually should trickle to the geometry but that actually doesn’t happen at the at this moment.

10:05 So we we thought of a better way as basically by constraining a 3D generator and I’m going to use like a toy example but a literal toy example where we thought like oh let’s make a a banana copter and so we draw and drew a banana some some blades we defined it with with GBT just to get a good starting image and we then build further on trellis a to open source model for 3D generation.

10:39 These typically always work in the same way. You have a 2D image input, typically multiple images. You encode this into a latent space or a a model like an abstract representation of what the 3D geometry is. And then you can decode that into a 3D output which is typically in a mesh or custom view or some some views so that you can minimize the reconstruction error. And you want that the 3D output looks as much as your 2D inputs.

11:08 And the way that trellis operates is so you start off with your input data and then you have your generative model and it has put a three-dimensional structure in this latent space and that is the the the big clue in trellis whereas in most genative models this latent space is kind of one big abstract fingerprint of your geometry but the the the stuff is not localized in three dimensions.

11:40 Now it now it is and this one decodes into goshion splatting decoder which which gives you the the views and texture or you can decode it into a mesh which give you the mesh. So this was the base method that we worked on. And to recap the the benefits of this structured latent space is the fact that like you have your latent space as a voxalized method which is basically a compressed version of your 3D space and every voxil is actually a one local geometry and texture descriptor.

12:14 So this is not the resolution that you can decode in. This is the res the resolution where your data is modeled on. The cool thing about this is that it’s completely differentiable. So this means that we can plug in all the advances of even 2D generative models now into the mix which helps us to actually completely constrain this whole generation process. So for example, we want to use an offtheshelf component.

12:50 We we want to buy the blades and and the engine and but we want to design around this component. Second constraint is we know how big we want our helicopter to to be. So we define the boundaries. We kind of inject these constraints right into this latent space which which we then call the constraint structured lat space for regions where we allow the optimization to to take place or not take place or with some weights or with separate control signals.

13:28 This still then decodes into u 3D representations such as a gaution splat or or as a mesh. And as you can see here in the in the mesh reconstruction, this is the exact component coming back. This is not like a regenerated component or something. This is the exact constraint that we want to put in. Plus, we can add more and stuff to the mix. So like if if we draw on one of the output images that we want to have actually to write DM on there we can force it to update the texture so that it actually gets updated onto this in this structured latency representation but we can also do the the opposite if we we can add geometry to it.

14:15 If we for example draw a landing gear as extra, we can have it actually just kind of add this as extra geometry. So this gives us a a method which is like controllable like iterable design method. So you have fixed components that you can use. The dimensions were set. You could you can update the texture. You can update the geometry. And this is just like the start.

14:57 This is just the toy example. Okay. Bananas cars. Let’s go back. So where we had the the style input image and the base geometry, we can now actually update the geometry and we can choose where to actually update the geometry where I I didn’t point it out in the first time, but you have the the spoiler on the on the back of the stylist image, but you don’t have it on the base geometry.

15:31 It was not showing in in first method but now you actually see that like the the spoiler was added to the base geometry. Don’t know if the projection is that good, but trust me, you can actually see these creases, these indents actually now trickling back to the geometry. And then you can plug it back into the thing and you can start taking viewpoints from around this geometry based on your style inputs.

16:08 So this solves kind of the two problems that we had the fiction before physics. We we uncoupled this the sequential nature. Because we now we can we can tackle the whole thing as one optimization problem. It can take into account a lot of your wishes and then it will try to find an optimal solution that that aderes to your constraints. And also we we actually can now have the tech the geometry be updated by these image inputs.

So as the takeway here is that basically the crux here was to actually use the structured latent space as a as it’s powerful representation for control generation which you can add. You can play around with this. So that we arrived at an iterable editable constraint image driven geometry supported exploration methods and what enables is that the client is then able to do a lot more faster iterations fewer design to engineering loops and have higher novelty or a higher creativity within the within the constraints that that was set out.

17:27 So like I said like we founded data meer two years ago. We’re now about a team of 14 people. We’re bootstrapped and we focus on on developing these AI capabilities typically for visual modalities and spec specifically 3D the anal the analysis and the generation of this and we we do this so that other companies can build a business around our AI capabilities. So like feel free to talk to me during lunch.

17:57 I’m here for two days. To learn more about the CDFAM computational design symposium series, to see the archives of previous presentations, and to learn about future events, visit CDFAM.com.