CDFAM NYC 2023 · New York · 14–15 June 2023

Design Computation Human

Listen as a podcast

Abstract

Opening keynote of the CDFAM 2023 symposium in NYC, featuring Onur Yüce Gün, a distinguished figure in the field of computational design. In this insightful presentation, Onur delves into the intricate relationship between design computation and the human element, drawing on his extensive experience and knowledge.

For all presentations form CDFAM 23 in NYC visit the archive.

Throughout the keynote, Onur shares his journey, beginning with his arrival in the United States two decades ago to pursue a master’s degree at MIT in computational design through to tleading the computational design team at New Balance.

He reflects on his career, which has been dedicated to elucidating the concept of computational design and its practical applications. Onur’s unique approach often challenges conventional methods, as he strives to bring a human-centric perspective to computational design.

A significant part of the talk focuses on the evolution of computational aesthetics and the role of computers in rendering artistic concepts.

Onur discusses his early experiments with image processing and machine learning, detailing his innovative use of algorithms to analyze and recreate images. He emphasizes the importance of understanding the interplay between meaning and information in design, referencing influential works and theories in the field.

Transcript

From YouTube’s automatic captions, lightly cleaned; expect some errors. Each timestamp opens the video at that moment.

Read the full transcript · 4,624 words

0:01 We’re going to spend some time setting this up, hopefully around 20 seconds, and then we will start. So I’m super happy to be here, the reason being the conference is the title starts with computational design, so yes, exactly. So the fun thing, my colleague this, it’s kind like giving me a lot of energy is has been in it for a long time too. So I came to I stepped on the US soil 20 years ago to do my degree in, U, master’s degree at MIT in computational design, and for two decades my affiliations and jobs have been about explaining what computational design is. And on the side, Al, I also tried to do, you know, some projects, as as we were discussing, discussing with Danan, for this conference, you know, I was I didn’t that I was going to go first, but I submited the title Design Computation Human, and then, you know, I Tred to kind of like bring computation back to being human, and, you know, reveal what computers cannot do. As a computational designer, so I can use my computational strengths to do the best I can, so that’s my approach, it’s slightly different, sometimes it causes troubles for me, but, you know, I go with that.

So my question when I came to MIT was, okay, how can a computer render this, right? So that was my question, computational aesthetics, I like the image, but can I teach the computer to render that? So today I just, you know, type it, this is a, you know, AI generated image, so you just give the prompt, it creates it for you, but if you go back 19 years, that was kind of like rough, and I didn’t know much about, I live AR by the way, I a programmer. What I realized was, okay, you know, mean understand in products, in, you know, vision, but also there’s the information that I have to kind of like deal with all the time, right? So if I have to quantify anything, I have to go into it, look at it, understand it, and then, you know, subdivide it into like little components, and then so you can combine them back and make something out of it, actually help the machine make something out of that, unlike me, because I don’t, I don’t have to kind of like subdivide the world into pieces while I’m looking at it, I understand there’s a room here, I see you, and so on and so forth. So there’s a really great article about it, this subject of meaning and information, by David B, if you’re interested, just check out, David Bom is a, you know, famous physicist who passed away quite a while ago.

2:34 So what I did was, at the time, I’m like okay, let me take this picture, so I, I heard about this thing which is called gray levels, right, so you can do some bitmap processing, and I was like okay, I can, I can try that, let let let me learn a little bit of like programming for that. So I thought myself programming in Max, 3D Max, Max script, and I started analyzing the images in 3D Max, right, so I was plotting the let’s say I was segregating the darks and brighter pixels, I was trying to also look into composition of the images, and then I was trying to do some like age detection, which I learned from, you know, a friend at electronics department, I was like okay, to to detect objects in the scene. So he was like okay, there’s this algorithm which, you know, helps you differentiate between the determine the shorelines, you know, as you’re kind of like from the satellit pictures, so I was like okay, let me try that. And then I made another algorithm, I made it up, you know, I was like okay, if I go from left to right, how much the grays are changing, like brightness is changing from left to right, and I plotted them. I combined the three, you know, algorithms, I set some scores, I sorted them, I let, I let the computer render images, they were like 32 pixels to 16 pixels, overnight, okay, and then next day I went and, you know, found the selected images and rendered them in a large format, and then I prepared this UI, you know, to let’s say set your rules for rendering, and so on and so forth.

4:02 And then I realized, wow, that was a teases on computational aesthetics that I did, I had no idea, and I also didn’t know that it was about machine learning, because it was image processing, feature detection, segmentation, fitness scoring, and so on and so forth, right? So so then I started seeing these people, like AI experts, you know, image prompt engineers, kind of coming forward and front, and, you know, talking about, yeah, journey is amazing, as I was okay, well, maybe, maybe I have to think about it a little more. So I have this 10 years between my Master’s thesis and the PHD thesis, so in between I got really intrigued about like what computers cannot do, as I was explaining, and I went back to my M, and I was like okay, I’m going to do a super blind system, I don’t want the computer to be smart, I just wanted to help me replicate myself.

4:55 And I I start drawing this on diagram, so if you look, look at that on diagram, which is kind of like open, you can it’s not like a flow chart, because I had problems with like inputs and outputs, I, in my mind, everything was flying in the air, so I was trying to represent that. So what’s happening here is that there’s a transparent surface, you make watercolor drawings, and then you can actually capture whatever you draw, and then you can project back to your drawing underneath, and then you can trace your own, you know, trace, so like tracing your own trace. So and I can talk about it in mathematical terms, so if you look at the schemas at the bottom, you see X goes to T of X, so something is being trans you know, translated, and on the surface zero goes to X is like nothing goes to something. And you can get what is interesting in this kind of computation is that it’s open-ended, because you end up like creating some weird stuff, let’s say the red circles, which weren’t inherent in the system, it was they weren’t there, right? So the human perception is interesting in the way that you can just kind of like make things up, right, but un likee a data set, we will discuss about that later.

6:02 But at the same time I was super interested in like keeping the the material qualities, the the things we touch, the things we interact with, right, so painting is really important in that, in that aspect, like as a designer, as an architect, as a kind of like, you know, hobbyist painter, I was really interested in not being able to touch the stuff that I was designing all the time. But also, like, you know, I think my point was okay, these things don’t have to be that far apart, if you know about shape grammars you can make these kind of visual computing, X go to X, X X goes to X plus part of X, and so on. So the difference here is that I’m not multiplying or adding up numbers, like 1, two, three, right, which are super abstract, I have shapes, and shapes are my units, and a shape doesn’t have a definition, it’s just a weird stroke, and it doesn’t have to be defin, so it can change over time, so it’s a pretty interesting way to compute, actually.

7:02 So if you want to kind of learn about it, you can read about this book, which is What Computers Still Cannot Do, which was published in, I think, like 80s or 90s, so this is already like three decades old, and there’s the first version of this book which is quite interesting. So I finished the PHD, I got, you know, I’m like okay, you know, I’m still having this ambulant feeling against computers, by the way, I’m really using them, right, so I don’t have anything against computer, I love it, I’m a computational designer, I my life with it, I sleep with it, I wake up with it, and so on and so forth. But, you know, I started doing these oil paintings, and, you know, taking the pictures of them, that’s exactly when the style again started emerging, right, so people were kind of like collecting these data sets for me, it was a retrospective thing, because I hit the data set already, it was my interest that kind of like captur this picture, and just, just put them together.

7:50 And then I trained a, you know, custom model on this, and I got, got these images which were, okay, I’m like interesting, but what are they, they’re not exactly my paintings, and and it was interesting to see the the reflection of the, you know, the patterns from the paintings, but at the same time the patterns from the photography, like you, you can see depth of field, you know, light blurs, and, you know, also those kind of like visual qualities in these images. So then I did this, there’s this sound, if I don’t know if you can play it, but not a big deal, okay, so that’s fine. Anyway, so, you know, there, there’s music, it’s it’s not a big deal, what’s happening is that’s the latent space of paintings that are trained by, you know, like the ML, machine, machine learning, the train model, and then I took them and I exploited them in space, and then I added these like tectonic levels, you know, because I’m an architect and lost painter, and then, you know, I’m designing shoes at the same time, so this was kind of like confused to me, but I like playing, and I was like okay, what am I doing really.

9:01 People ask me when I show these, they like what is the purpose of this, like why do you do that, right? So I was like okay, I was just cloud gazing, right, like looking at the clouds and trying to see shapes, and this kind of like connects with, you know, so Solnet says okay, you know, why am I doing this, so you need some sort of something to be efficient, purposeful and utilitarian in the modern world, and if it doesn’t work, if you cannot measure it, there’s no meaning into it, right? So like that’s how can we kind of like get away from that, because I benefited actually from this kind of like cloud gazing stuff a lot. So anyway, that was the critique, I think, of the of the book, and I kind of like brought it back here, and then of course this kind of like play, do we do that at work? Of course we do, and again, more than 3 years ago, we looked into, you know, some San models, and we were like sty Gan models, and we were like okay, what if we trained something on speed that would inspired designers to to use these things for, you know, making sketches, and, you know, and creating concepts.

10:06 And I wanted to move forward, like push it forward a little more than that, and, you know, recruited some of my friends and said hey, are you, are you on board to create something that we can we can showcase, we can kind of like demonstrate in the company, that’s that’s my take, usually I, you know, when we come up with ideas, I have to do the proof of concept and just push, push for those things. So, you know, this is I haven’t seen this in person, because I was away still, I haven’t seen it, I, I’m back to US, but this was 3D printed, you know, somebody I, I really don’t remember who did it, actually, you know, put this sock like an upper in it, and then sent the picture, photograph, to me. I’m like oh okay, like this is really doable, can you wear it? Yes. Can you run it? No, because it’s not the right material to do that, right, so it’s just visual, but it’s okay to make these kind of like, you know, again, like cloud, cloud gazing kind of stuff.

10:57 So these are some crops from the concepts that we’ve built on the office, so just to show that okay, we can’t push the digital stuff like crazy, right, and especially if you’re not manufacturing or printing them, right, so there’s there, there are no limits. So on the mathematical side, on the formula side, we, we show that well, is that useful, actually? 20, 20, 28 MIT, which was bringing this, you know, like super crazy concepts, and the knowledge that we developed at New Balance by 3D printing. I’m going to give a break here, and actually I this event is great, this is the second note about it, one was computational design, the other one is I already met some people that I have met digitally, but I’ve never seen them physically, right, so I saw those people, and then I also had friends who were, you know, that I haven’t seen for like 15, 20 years, they are here too, and then this class actually became, you know, a convers, online conversation, with some people, and I saw some of them here too.

12:01 So with that, again, like thank you for, you know, like letting me talk about this, this material today. So if you go back and like okay, we have the crazy digital stuff, we have the manufacturing, so how do we kind of like combine these things, and, you know, design is about kind of coming up with the problems that we can really restrict and find answers for. So motorcycle riding is an interesting thing, you know, it has nothing to do with like bicycle riding, or driving, if you ask me, and then your body position counts a lot, because similar to any sports you do, right, so your, your kind of ergonomics are really important, because you can do something wrong and things can go really wrong. Or all the the the thing, the material itself, is designed for a specific p purpose, so if you want to go fast, it looks different, the The Way You Haul the handlebars, where you put, what the where the pegs are, how you sit, you know, and you see some people kind of like going like this on the road, right.

So just kind of like borrowing that idea, as we were getting ready for this conference, you know, I asked in the group, can we do something similar for running, to explain it to people, because, you know, I learned when I started working for New BS, I knew nothing about shoes except wearing them, I can do my laces too, but then I said okay, what if we what if we explain the, you know, the economics in terms of like cushioning, you know, which changes tremendously depending on the activity, the drop that you have in the shoes, and put all those together to show that okay, it’s really kind of like the motorcycle design, you know, where things go is changing depending on the the purpose. But at the same time, just to represent that it’s so hard to design shoes, like how can you, because shoes are made by using lasts, which is the internal mold, right, you build around, so you’re seeing the section of the, you know, the the last, but you see the foot, the foot has not I mean, that correlations, connections, between the foot, last, and the shoe, but I mean computationally it’s, I would say, almost impossible to define, right?

14:13 So there’s a guy who knows less, and he, he takes it, and he’s like, he, he does this, he’s like oh, I have to shave it a little bit here, that’s it, that’s his knowledge, and I don’t know how that information will be kind of transferred to the next generation of L makers. So that’s designing for, right, and then we design with, so this is an interesting subject, because data, you know, it, it has been hot, it’s super hot now, because of 3D printing, AI, and so on. So data is an interesting thing, so here is data, for instance, by an interpretation of data by Timol, so he Tims, he’s kind of like attaching these pens to threes, and the wind is blowing, so the three is plotting something, making art, right, but at the same time it’s the data that’s representing the sness of the branch, and then, you know, the the you know the the magnitude of the wind, and and so on and so forth, right.

15:05 So depending on that, actually, data is something that we make, right, so I can’t say hey, data is exactly the thing that is found under your feet as you’re walking or running, so I can just say that, right, so you can, you can say okay, data is this, this data is that, and then where it comes from is actually it comes from wherever you look at for finding it, okay. So as an architect, we looked on the surface of buildings for making solar simulations, or to make the, you know, structural analysis of the buildings, to to look for that, so data is exactly at the location that you’re looking at. So here is an insert, pressure insert, that you put in your shoe, and once you start running, let’s say, we can measure the, you know, the the event, the pressure event, that happens as you’re running, from the kind of like landing and take off.

15:55 And then what you do is, okay, there are numbers, let’s split them, you know, let’s record these numbers, let’s parse them, let’s write a code, and Stace, let, you know, structure Stace, now let’s make sure that our structure looks good, let’s make sure it’s connected, it doesn’t break apart, let’s make sure it’s kind of carrying, it’s doing what it’s supposed to do, let’s explore some different options, let’s look at the, you know, softer ones, harder ones, and then let’s print them, so we can have a better understanding, because if you stay in the digital, well, it’s not tangible, if you’re wearing something you have to feel it, if you are feeling something, you know, it has to be sturdy, and it has to have a life, lifetime, lifespan, and it has to perform.

16:44 So we ended up doing maybe 1,400 parts so far, but for a specific study, for these shoes, triple cell technology, we called it, we created Two Shoes, one had this 4 foot part, the other one had a heel part, it was a really interesting process of kind of like creating, learning, designing, and the goal was okay, you know, let’s design these two shoes, which are by using the same material, which are behaving significantly different than each other, and let’s put these parts in different locations in the shoe, one is a for foot part, the other one is a heel part, and then let’s, let’s show that okay, we can customize things if needed, and by using the same material, so that’s, that’s a learning from design for EDI of manufacturing, right?

17:30 So I’m taking you from computational aesthetics to drawing, to paintings, to building making, which I didn’t show, and then to Dam, so what’s happening in Dam is there’s a big shift, in, you know, parts are changing, processes, methods and tests are all changing, and this is just acknowledge that okay, the material is the same, but you can do a lot by using the same material and same process, and make the thing perform really differently, and this also being in footware business, so what I learned a lot is actually footwear design empowered me to use the claim. So I always said okay, simulation is not reality, it’s not going to work, I always give the example of Bank of America building in Manhattan, by, you know, like Time Square, which was which which was a, you know, lead platinum building, there’s an article about it, you can, you should read it, simulations were a huge percentage off.

18:18 What is important here is that what you see on the 997 shoe, the colorful one, you see anap reveal, so there are nested components in there, let’s say five, six parts, you know, nested together, which are, you know, together doing something for the shoe to perform, whereas in the black one you have one single part, which is by lettuce design doing all those things together, no nesting, nothing, you just print it and it works. This is also bringing the the notion of how we understand the world, right, so you can represent and describe a sho in very different ways, in my mind it’s always this like blue, yellow, you know, green thing, because it’s like very mushy, that’s, that’s, that’s how I feel with my, you know, feet in it, so like all the other representations are not that, that accurate, but also the parts are, you know, by nature, they are just scientifically put there, because again it’s like data, because we want to understand and talk about it.

19:11 Can anybody tell where your finger ends and your palm starts? Well, there’s a line there, but it’s not I mean, if you scale in, there’s no line actually, everything is connected, right, same thing for trees, we cannot talk about trees, so we have to kind of like subdivide it into pieces, same for engines, right, so here different piston representations for you, they are like totally different, depending on what you’re trying to say about engines and pistons, because it can also become an artwork, right, so this is a there are lots of pistons there, and they’re just being used for the sake of making an artwork, okay.

19:44 So this is a scientific method, pretty much, you our assignment is for for science, for building, is to go to the, you know, landscape, go to the beach, and then we are asked to take a handful of grain, and we need to sort them into part like categories, right, so as we start categorizing parts with our scientific knife, right, we split everything into, let’s say, buckets, depending on the color, size, translucency, transparency, and so on, but then you pick a grein, and then it doesn’t fit into any of these clusters, what do you need to do? You need to go back and change your classification, or you need to kind of like cheat a little bit and just kind of, you know, ignore the that that grain. So this is from the book that you have just seen, it’s from The Zen and Art of Motorcycle Maintenance, by Robert Persk, talking about the scientific method, okay.

20:35 We’re coming to the end, okay, what do we learn by working with computers? One mutd is really important, so if I ask you, if the point is inside or outside, in the condition A or B, you’ll say like that, you know, which one is inside, which one is outside, counting the number of intersections, so you would see that if the number of intersections is odd, the point is inside, if it is even, it’s outside. Beautiful, we took something perceptual and solve it computationally, but the problem is we lost the curves, right, so like we, we lost the vision, that’s very important, when we say, for instance, computer vision, it’s or artificial intelligence, it has the vision, but the vision is super different than ours, same thing for artificial intelligence, which is like totally different than our understanding and learning processes.

21:20 So for design, you also have to come up with this binary things, work for a while, until you find the crocodile, which is is, you know, like super important, to say beyond the duck and rabbit, you have to search for this to add real value. And speaking about values is really important, because, you know, this is from Albert Dur, from 1514, so he said what is beautiful, I do not know, in the picture you see tools of computation and measurement, you know, like pretty, like the hourglass and the compass, and the, you know, like the hammer, so today we like okay, what can I do with Cat, GPM, M Journey, today I don’t know, before I go to bed, so you, you are kind of like constantly being bombarded with these tools, but, you know, where is the meaning in using those, so you have to be critical to find that.

22:05 I took this class from Patrick Winston at MIT, and I happened to, as a designer, read all the papers from like 1952 till our day, on seminal papers on AI, which helped me a lot to understand how AI works, which is also helping me, you know, comment on AI and so on, but beyond it, I think collaboration is really important, we really at the end here. So I have the New Balance design team with me at this conference, because we said okay, we’re going to go and learn a lot in the conference, so these are actually combining all this knowledge of like sculpting, but at the same time, you know, bringing data, bringing, let’s say, performance, to to shoe making, this is all done by the computational design team at work.

22:47 But then also what we do is to to kind of like connect everything, we are like okay, let’s, let’s bring the new tools, you know, we had the data, we have machine learning, let’s, let’s be inspirational, we are designers, let’s keep everything, and then, you know, do something with it, and again, like put the proof of concept on the table, so this is pretty much that. And I said okay, it was a month ago, this is the end, thank you very much. A month ago, you know, I said okay, why are we trying to constantly make shoes by knitting, because, you know, the patterns are so interesting, and they are shoes are too small, so I said make a sweatshirt, sweater, and I said if you make it, I’m going to wear it, so now I have to ful, fulfill my promise, right here, so this is the first data driven, machine learning made okay, all right sweatshirt, that was done actually one day before, completed one day before, we draw here as a team.

23:46 So I would like to thank the competitional design team at New, they will be around in this event, but we are missing our N expert, who is in another team, to combine everything, but again, yeah, we just kind of like create stuff and put it on the table to see if there’s any value or not. Thank you.

More from CDFAM NYC 2023

Mud y Robots: Keynote Presentation

Mud y Robots: Keynote Presentation

Ronald Rael · UC Berkeley / Emerging Objects

Roadmap to Replicators

Roadmap to Replicators

Neil Gershenfeld · MIT Center for Bits and Atoms

Scaling & Standardizing Patient Specific Design

Scaling & Standardizing Patient Specific Design

Adam Wentworth · Mayo Clinic

Computational Design Ops

Computational Design Ops

Ronnie Parsons · Mode Lab

Overcoming the Design Dilemma

Overcoming the Design Dilemma

Ryan Carter · Johns Hopkins APL

Generative Design & Digital Manufacturing at NASA Goddard

Generative Design & Digital Manufacturing at NASA Goddard

Ryan McClelland · NASA Goddard Space Flight Center

Digital Materials – Geometry and Data Combined

Digital Materials – Geometry and Data Combined

Marek Moffett · General Lattice

Ten Years Teaching Additive

Ten Years Teaching Additive

John Hart · MIT

10 Years of Applying DFAM

10 Years of Applying DFAM

Tim Simpson · Penn State University

MfAM Walks so DfAM Can Run!

MfAM Walks so DfAM Can Run!

Chelsea Cummings · The Barnes Global Advisors

Informatics and AI for Materials Development

Informatics and AI for Materials Development

Wes John Alder · Citrine Informatics

Predicting Fatigue Life in AM Metal Parts Using DigitalClone

Predicting Fatigue Life in AM Metal Parts Using DigitalClone

Behrooz Jalalahmadi · Sentient Science

Designing Material Properties with Advanced Toolpath Generation

Designing Material Properties with Advanced Toolpath Generation

Alex Roschli · Oak Ridge National Laboratory

Design Automation Using Machine Learning & Computational Design

Design Automation Using Machine Learning & Computational Design

Rhushik Matroja · Cognitive Design Systems

Learning to Generate Shapes

Learning to Generate Shapes

Karl Willis · Autodesk Research

Design for Artificial Intelligence

Design for Artificial Intelligence

Chris McComb · Carnegie Mellon University

Register for Updates and Discounts on CDFAM events.