Abstract
This recording is part of the sessions from the CDFAM Computational Design (+DfAM) Symposium 2023, featuring Chris McComb, Director, Human+AI Initiative
Introduction: Chris McComb introduces his work at Carnegie Mellon University, focusing on engineering design and analytics. He leads the human plus AI design initiative, emphasizing hybrid solutions that combine human and AI capabilities.
Design for Artificial Intelligence (DfAI): McComb proposes the idea of DfAI, a set of principles aimed at developing better AI at the intersection of computation, design, and manufacturing. He structures his talk around three themes: composition, collaboration, and caution.
AI Hierarchy of Needs: From data collection to AI achievement, McComb emphasizes the importance of data infrastructure, cleaning, analytics, and machine learning.
DfAI Principles: Six principles are introduced to support better data development, including digitization, valuation, metadata, machine readability, linkage integrity, and evolution.
Emerging Roles: McComb identifies three archetypes in DfAI: engineering design, AI developers, and data curators.
AI as a Tool: Teams benefit from AI-powered “easy buttons” that help in decision-making.
AI as a Coach: AI can provide targeted guidance recommendations, performing on par with human coaches.
Transcript
From YouTube’s automatic captions, lightly cleaned; expect some errors. Each timestamp opens the video at that moment.
Read the full transcript · 3,053 words
0:06 All right, can we get a big round of applause for the road crew, they have been awesome. So I’m Chris McComb, I’m on the academic side of things, and I’m really excited to share with you today some of our work on design for artificial intelligence, the big idea being how do we structure teams, organizations, work, in such a way that we can bring AI seamlessly into what we do. So I want to share a little bit about myself, first of all, the first time I really remember interacting or thinking about design was during a spelling bee, last century, during a spelling bee, third grade or something, and the judge said “Chris, your word is design,” and I stood up and barely confidently spelled it d-e-z-i-g-n, and I’ve learned from my mistakes, like all good engineers and algorithms do.
1:06 And if we fast forward a little bit, I now have the pleasure of working at Carnegie Mellon University as a professor who focuses on engineering design and analytics, and I also have the pleasure of leading our human plus AI design initiative, building off of the last talk very nicely. What we believe in the initiative is that really the best solutions that we’re going to find are hybrid solutions, solutions that bring together both human and AI to really do things that are magical and creative and interesting. In the initiative we bring together perspectives from across the university, including engineering, computer science, our Tepper School of Business as well, so we really come at this problem of how to use AI in engineering from a multi-disciplinary, in a transdisciplinary perspective, and one of the things that gets us most excited is working with need holders, working with all of you who bring really interesting, fascinating problems to bear.
2:10 And I think today all of us are here because we can find ourselves somewhere in this Venn diagram, maybe we’re outside of it, but probably somewhere inside of it. I come at it more from the computation and design side, but thanks to folks like Tim Simpson, I’m finding my way up towards additive manufacturing. You know, when I think about CDF am, right, manufacturing is hard enough, right, when we add the A on to that, additive manufacturing, the promise gets bigger but it also gets a little harder, right, when we add the D onto the front, DFAM, right, more promise but also some more challenges, add the C, same story, more challenges but also more promise. And when I started working in this area I saw, you know, something that resonates with what the last two speakers have said, which is we have a lot of challenges around data.
3:05 So together we propose this idea of design for, for artificial intelligence, a set of principles that’s oriented towards helping us develop better AI at this intersection of computation, design and manufacturing. So I’d like to structure my, my talk today around three ideas, three themes: composition, collaboration, and finally ending with a little word of caution. So, composition, I, I love this, this pyramid, the AI hierarchy of needs, because it takes us all the way from collecting the data up to achieving AI, right. And down at the base we need to generate, sense, log data, right, or work with Alexa to have our data produced, right, then we have, we need data infrastructure to store, to, to hold our data, warehousing, we need some cleaning and some preparation, right, ETL, then data, analytics, machine learning, and AI, and you could argue that maybe I’ve compressed the top end of this a little bit, that’s because I needed room to fit all of these in.
4:14 So these are our DFAI principles, and they’re really clustered near the base of the pyramid, and that’s intentional, because that’s where we think the greatest need is right now. In order to support better AI we need to support better data development. So these are six, what we think of as pretty general principles that we encourage people to adopt as they’re beginning their AI journey, in order to really seed the ground for subsequent AI development. The first one is simple, I think maybe for, for all of us in the room it seems a little trivial, but I was recently working with a construction company that wanted to start using more AI, and it’s a different world, right, a lot of construction is still driven by paper drawings, by PDFs, stuff that does not digitize very well necessarily, but as soon as we start digitizing data that opens up a whole slew of possibilities.
5:14 The next step, though, is attaching some value to that data, right, we need to assess both the current value of data sets and add that as essentially metadata to the data set itself, but also the potential future value, right, many of us are planning out several years in our organizations, right, and data needs to be part of that valuation. In a lot of cases we can think about data, AI, ML as another kind of capex. Next, and I loved hearing this in the last couple of talks, metadata, right, maybe in comparison to, to some colleagues who are more compute, pure computer science, we live and die by the metadata, right, this is why we hate STLs, right, because STLs hold so little metadata, right, we need surface finish, we need material, we need all of that information in order to make effective design AI.
6:12 The next idea is machine readability, right, once we have our data, once we have it warehoused, we need to make it operable for AI agents, for algorithms, things like that. And then the next point is one that’s especially important in manufacturing, linkage integrity, as we keep growing this digital thread, right, as the digital thread gets longer we need to be able to track assets across that digital thread, integrate and link that data. And then finally, evolution, I was looking for a good image of the evolution of 3D printing, and everything stopped at about 2018. And I was not the person to put together the new figure, but evolution is a really critical part, speaking with a few folks earlier, the last six months of AI and machine learning have been wild, right, so it’s essential to keep up with that and keep updating these generalizable principles.
7:13 So the, the big takeaway from this first scene, composition, is that AI is only going to be as strong as the data and the infrastructure that it’s built upon, and by paying more attention to the data up front we’re seeding the ground for subsequent AI development efforts, facile data, easy data, makes for easier AI development. So let’s say we, you know, embrace these principles, we start to develop AI solutions internally, through collaborations, what’s this collaboration now start to look like in this environment, how do our teams work, what do our teams look like, and what my, my lab is starting to see is that some core, what we call defy doers, have started to emerge, and we see sort of three archetypes, or three roles, emerging.
8:04 One is, you know, a little familiar, a sort of engineering design role, but then we also need AI developers, right, that might seem like a, oh yeah, of course we need people to develop the AI, but I think that the folks that we need in this community are really unique in that respect, right, because we need folks who have the AI knowledge but also the engineering and the manufacturing knowledge, right. So when I think about this AI developer persona, I picture somebody who, you know, had a Thompson undergrad degree and maybe went to Penn State for their AM master’s degree, or vice versa, right, has a mechanical engineering undergrad degree, came to CMU for an AI degree of some sort, right, really interdisciplinary in that respect.
8:51 And then we need people or organizations who are going to curate our design data, right, people who own the data are absolutely essential, and a lot of times, you know, like, like Francis said, these people are working in the background, right, and not many of us end up touching and curating data on a daily basis, but this is absolutely essential, and maybe even more important than who these people are is how they’re connected and supported within organization. But what I want to think about with the, the next few slides is, you know, what starts to happen when some of these people or some of these personas, these roles, are taken on by AI agents.
9:33 So my lab has been looking and thinking about this for a while, and we’ve run experiments that show that teams can benefit from AI as a tool, AI as a coach, and maybe potentially AI as a partner. These are all, you know, early stage experiments, and we conducted them in the context of a last mile delivery task. So we brought in teams of five or six students, we gave them the task of designing a drone delivery business, everything from configuring their drones to figuring out how to route them around and make deliveries, to actually figuring out how to turn a profit and make some money.
10:17 We worked with a fantastic team of developers at Penn State ARL to put together essentially a stripped down CAD environment for doing and supporting this work, so here’s our drone design environment, we have copters, we also have airfoils and wings, and we have an evaluation engine built in that gives us cost, speed, distance, and lifting capacity. Now the AI agents in this are essentially an easy button, in the first iteration, so you hit the easy button, it generates some potential solutions based off of your current one, that are improvements in one direction or another, and presents this option set to you. We have similar interfaces for the operations members of the team who are in charge of routing drones around our simulated city, and then we have a business role, this is really similar to operations but with some additional data analytics based into it, so they can start to compare and trade off between these different operations plans and routes, within the context of having the drone.
11:26 So the first experiment that we ran looked at AI as a tool, right, so essentially participants, everybody had access to an AI powered easy button that would help them out when they clicked on it, and we compared against teams that did not have that button, and the green icons here show the, the stuff that’s sort of a head nodding truth, yeah, they’d do better, they’d create better solutions, make more profit, yes, we also saw that they explored a little more broadly in the solution space. But what was interesting to us is that there was also a different focus for the human members of the team, so when we went back and looked at the chat logs, when we looked at their activity logs, we saw that they were chatting more, acting less, and that they were talking about higher value, right, so they were talking about, you know, sense making, about strategy, and a little bit less tactical, so integrating AI tools leads to a fundamental human shift in cognition.
12:24 Second, within the same drone design context, we started to look at AI as a coach, so here when I say coach this is a, an AI agent or a human member of the team that was in charge of monitoring the team process, nothing to do specifically with the content of what the team was doing, but process, who was talking to who, how often they were talking, how often they were logging designs, you know, rate of work, stuff like that. And in this experiment we constrained both the AI and the human to have a consistent set of templated messages, so we wanted to, as far as possible, try to blind the participants to what was the AI and what was the human. We saw that the AI, you know, performed about on par with the human coaches, the human moderators, but one really important difference is that the AI was able to make more targeted guidance recommendations.
13:19 So what does that mean, are human coaches would generally make broad announcements to the team and say, hey, I noticed you’re slowing down, can you work a little faster, or, hey, I noticed nobody’s talking anymore, can we talk some more, whereas the AI agent was able to absorb and use more detailed information in order to provide targeted interventions to individuals. So this is what we see as potentially some great symbiosis between a human manager and an AI coach, right, taking some of the strategy and the tactics. One thing that did surprise us a little bit was that, almost across the board, regardless of which condition, 25% of the time participants thought they were being coached by a human, 75 percent of the time they thought it was an AI, so I think this speaks to some things, yeah, some, some mistrust, potentially, of management, maybe not, but maybe a healthy mistrust of AI systems.
14:25 And then finally we looked at AI as a partner, so in this case we had some human teams, we had some hybrid teams where we ripped out half of the people and replaced them with AI agents, in this case we had a, a basic natural language wrapper around the agents, this work was done a couple of years ago, so nothing too sophisticated, but this experiment was all surprises. So when we set it up we wanted to essentially establish failure cases, so we wanted to understand how these teams failed when they had AI agents. So in the first half of the experiment it was a relatively simple drone design task, in the second half we started putting in size limitations, we put in no-fly zones in the town, and these were things that were fundamentally out of representation for our AI agents, so our AI agents couldn’t even see the obstacles.
15:19 What ended up happening though was that in both halves of the experiment the hybrid teams were on par with the human teams. We expected the second half our hybrid teams would tank, what happened instead was the team started to communicate to adapt, so we saw in the second half that communication, especially communication between human and AI elements of the team, skyrocketed, and when we started to look into what they were saying and the directions they were giving, there is really sort of folks in the team taking on the role of an AI handler, right, so people who are translating back into representation in order to guide the AI and enable them to remain as effective members of the team.
16:06 So to sum up this second C on collaboration, I just like to say that there are many different ways, shapes and forms that we can collaborate with AI agents, AI tools, AI algorithms, and they all potentially have different benefits. So moving on to the third C, a little bit of caution. So in this experiment we didn’t want to tank the performance of hybrid teams, we wanted our hybrid teams to do really well. So we, you know, one of our favorite applications in the lab is structural design, but apparently, yeah, I know, AI doesn’t like what I’m saying, I should have loaded the algorithm off of my laptop.
16:49 So we have a, essentially, a trust design AI that can provide recommendations and provide a little guidance to people as they work, you’ll see that pop up on the bottom here in a second, so essentially we show heat maps of where they should add new joints, members, etc, but also where they should remove material, or where they have an opportunity to remove some mass. So this is really a case again of AI as a tool, right, so in this experiment we had some participants who had access to this AI powered recommendation, some folks who didn’t, and what we ended up seeing is that across the board using this AI agent tanked performance, and this was fascinating, because when we let this thing crank on its own it outperforms humans, but when we allow it to make recommendations and we rely on people to interpret and use those recommendations, it doesn’t help.
17:46 What we saw in some of the exit interviews was that the folks using AI had a false belief that they’re performing better, this might have been, you know, tied to how we introduced the experiment or something like that, but this was, this was interesting, and then the, the one bright note was that at least initially it helped the worst performers, but after a while they too started to, to not do as well. So this is just a counter example to illustrate the point that as we start to use AI in our own work we should really embrace a trust but verify mindset, and I think especially as we enter a world of more performant AI agents we need to be cognizant of some of these risks.
18:33 You know, last night I was going to end on that note, but especially after this conference and meeting a bunch of folks here, I wanted to add something that’s a little more upbeat. I think that this fourth C is community, right, so I started off the talk by showing this Venn diagram, and you know, asking you all to find yourselves in this diagram, now you think that we’re all in different parts of it, right, maybe subtly different, maybe different corners of this space, but we all find ourselves here together because we care and we believe in CDF am, right. It’s going to take a lot of work, it’s going to take all of us, but I cannot think of a community that I’d rather work with. Thank you very much.
More from Chris McComb

AI and the Battle for the Soul of Design
CDFAM NYC 2025
Listen, search and cite
Listen to this talk as a podcast
More from CDFAM NYC 2023

Generative Design & Digital Manufacturing at NASA Goddard
Ryan McClelland · NASA Goddard Space Flight Center

Generate-to-Validate: Optimal Designs and the Need to Deliver Under Uncertainty
André Wilmes · Rafinex

Defining Performance not Just Geometry with Software Process + Material Science
Hardik Kabaria · Carbon

Reinventing the Wheel – Representing & Manufacturing Geometry with a Program
Mark Burhop · Siemens Technology

Interspecies Forms: Robotic Feedback Systems Mycelia Growth & Computational Form
Natalie Alima · Adidas / Biolab Studio

Designing Material Properties with Advanced Toolpath Generation
Alex Roschli · Oak Ridge National Laboratory

Design Automation Using Machine Learning & Computational Design
Rhushik Matroja · Cognitive Design Systems






















