0:04 Perfect. All right. Hi everybody. My name is Sergey Pigach and I’m a software engineer at CORE studio at Thornton Tomasetti and I’m very excited to talk to you today about applications of machine learning and artificial intelligence in the structural engineering practice. So to tell you a little bit about ourselves for dami is a global engineering firm. We have about 1,800 people offices all around the world and structural engineering is a big part of what we do.

0:34 But that’s not the only thing we do. So our practices range from sustainability, facades, protective design, applied science, you name it. And CORE studio is the R&D arm of Thorn Tamasi. So we come from very different backgrounds. Were a bunch of recovering architects and engineers and software developers all trying to build tools for the EC community and within CORE studio. So CORE studio is about 40 people and within CORE studio we have a dedicated core AI team which specifically focuses on applications of machine learning and AI for solving engineering problems.

1:11 So you can see our faces up there spearheaded by Robatani who is our CEO. The CTO of Toronto Tamasetti and also the head of CORE studio. All right. So why are we interested in using AI as a structural engineering firm? So if you’re familiar with how a regular AC project goes as you move through different project phases, your ability to iterate drops off pretty sharply and the cost of change starts to climb exponentially.

1:44 So something that would be an easy adjustment in say schematic design or design development becomes very very expensive if possible at all by the time you get to construction documents or god forbid that the thing is being built already right. So you really want to do all of your iteration ahead of time. So what can we improve in this design process? Well, first of all, we just have to say that iteration in early design is very important, right?

2:12 You want to be able to play around, try things out, and kind of figure out what really works early. Architects could benefit from engineering feedback at the outset of the project so that they can test their assumptions and so that they don’t end up having to make an expensive change later down the line. Engineers on the other hand could really benefit from not having to use CD level tools for schematic design because it’s a complete overkill.

2:41 You’re using tools which are incredibly precise but that take a long time set to set up. They’re clunky. They take a long time time to run and you’re doing this for just like if you need to tell your client, oh the columns are going to be approximately this size and oh also we’re going to change it like 10 times over the course of the design, right? So it’s kind of a waste of time and rapid feedback in early stages is far more important than 100% accuracy.

3:12 You want to be accurate enough, you want to be comfortably in the ballpark, but you don’t have to be 100% on the money. It’s much more important to be able to move quickly. So, how can AI help? Well, AI and ML inference are very fast. So, a lightweight, easy to use, ML app can give you a 90 to 99% accurate result in seconds as opposed to minutes or hours it would take an engineer to run a con conventional industry standard software.

3:42 Architects because of this also get access to significantly more highquality engineering feedback at the outset of the project compared to what they could previously afford frankly. Iteration becomes easy and fast and early feedback with regards to approximate number sizes, tonnages, embodied carbon cost and other highle metrics really help you zero in on optimal solutions early on. So to take a look at how we’ve been approaching this problem we have to teleport ourselves all the way back to 2008 when we built asterisk.

So asterisk was a web app and a Rhino plugin that allowed you to reference your massing reference your core. It would use our geometry service to slice your building into floor plates, generate all of the structure, and then use the series of ML models to predict the sizing for all of the structural members. And of course, you could like switch between different structural systems. You could pick concrete or steel, different facade types, different programs, and it would also give you a readout of kind of highle metrics, right?

4:57 Your tonnage, your cost, your carbon. So it was very nice to kind of really quickly prototype and test different ideas and of course you know you could click on any elements see all of its properties and then visualize by any attribute you want right do you want to visualize by section depth or embody carbon page whatever you want. So yeah, it was just a nice way to kind of quickly prototype things.

5:26 And then the idea was that because each iteration takes you less than a minute, you could then generate a bunch of them and come to this view where there’s this nice parallel coordinates graph, where you can just kind of really dial in, the design requirements of your project and kind of filter things down to what works for what it is you’re trying to do. And then you could also compare options side by side.

5:49 Now, when we built this thing, we were like, “Oh my god, this is going to be better than sliced bread. It’s going to be an instant hit.” And people were talking about asterisk when it came out in 2018. It just no one was using it, which was very frustrating for us. And we spent kind of a long time reflecting on why that is. And yeah, some mistakes were made I would say.

6:14 And most importantly, I think what caused our adoption problems with asterisk is that first of all, it was a little bit too rigid. So it had a very my way or the highway kind of way of designing a building. It gave you only one approach and it was hard to integrate into different workflows. We also didn’t really know who our user was. Like engineers found it lacking because they didn’t have enough control and then architects just found it a little too clunky, not streamlined enough for them.

6:49 But most importantly, we never managed to gain our users trust because the way asterisk worked, it was a little bit of a black box, right? It just kind of conjured up this building and it was like, “Hey, I made this. Do you like it?” And it’s like, “No, I don’t. I don’t know where this came from.” but you know we also did some things well so super fast iteration that’s a check building under a minute we did hit that benchmark provided highle feedback which was still valuable at the outside of the project and also generated quick good-looking visuals well good-looking at least for the time.

7:24 All right so the first thing that we really had to work on is our malops. So, I will show you. I love this video. This is also back from 2018 from our old office. What you’re seeing here is all of our workstations working overnight at the office running safe to, generate a concrete slab data set. Now, safe did not have an API back then. So, we actually used something called auto hotkey, which records your button click.

7:53 Yeah. Which records your button clicks and keystrokes. And it was incredibly fragile. But you know we managed to export the data set but we quickly realized that this is no way to live. Also just organizing our stuff was very difficult because you know we’re software developers. It’s like okay you need version tracking. What do you do GitHub? And then very quickly our GitHub repository just became this black hole that everything disappeared to and you just have like your Python scripts and notebooks and your pickle files from your models and CSV files from your data sets and no one knows what’s the latest version.

8:28 No one knows like what is currently in production and also some assets were just too large to be committed to GitHub. So this was also a complete mess. So the first thing that we really had to work on was establishing our own MLOps pipeline. So that’s why we invested significant time and resources into building Cortex. And Cortex is our as I said in-house MLOps platform which allows our data scientists to organize their work in projects or as we call them experiments.

9:03 And within each experiment, you’re able to as you’re working on your machine learning models as you’re training them locally or in the cloud, doesn’t matter. You’re able to log them to the tracking server and it logs all of your model assets alongside all of the metadata like who logged it, when, what version of what data set they used, what were the hyperparameters, what were the evaluation metrics.

9:25 So all of this information is getting getting stored in this tracking database. And then Cortex is also plugged into our deployment pipeline. So in a couple of button clicks we can just take the model that you logged and send it to Amazon SageMaker which is a product from AWS that allows you to easily deploy machine learning models as web APIs. So Cortex has a Python SDK. We host it in Pippi.

9:56 I mean you can pip install it. You won’t be able to use it because it’s our internal tool. But it’s up there and you know because it’s a Python SDK our data scientists can just use Cortex wherever they work. So if you’re in Jupyter notebook, Jupyter Lab, I don’t know if you’re in SageMaker Studio, you can just as you’re working as you’re building your models, you’re able to evaluate them and log them to the tracking server.

10:23 So essentially what we built with Cortex is we established this really streamlined pipeline from training your model again locally in the cloud doesn’t really matter collecting all of that information logging it through a tracking server and then with a few clicks being able to turn your model into a web API that’s authenticated and that’s ready to support a downstream application that you would be building on top of it.

10:51 Now a few technologies kind of converged at the same time. So around around the same time I just really wanted to build a Grasshopper plug-in and I decided to just on my own time to build swiftlet which is essentially an HTTP client for Grasshopper. So Swiftlet allows you to make web requests to whatever API you want. If you guys know what curl is, Swift is essentially curl for Grasshopper.

11:18 And so what swiftlet allowed us to do is you know now that we have cortex and we have this way of creating this pipeline for deploying machine learning models. Now we can call into those models from Grasshopper which is great. Now you might be wondering why do we want to be in Grasshopper? Well, we’re trying to solve structural engineering problems, right? And part of it is AI inference and part of it is geometry processing.

11:43 And Grasshopper and Rhino are just very powerful geometry engines. And also just the speed of prototyping with the visual programming interface is a lot faster. So that’s where we want to be for development. That’s not where we want to be for distribution, however, right? You’re not going to be emailing your Grasshopper file around the office like with your API credentials embedded in them. So this is where the last piece of the puzzle kind of falls into place with Shape Diver.

12:11 So these folks are based in Austria. They’re amazing. We’ve been working with them for a long time and their business is centered around basically web configurator platform running Grasshopper in the back end. So you can upload your Grasshopper file to their platform. They will parse it and extract a very clean UI that you can see in the browser with all of your sliders, all of your buttons that you want to expose and then they give you a very nice 3D viewer to go along with it.

12:38 So for us, this was kind of a perfect way to share our apps securely with the rest of the company. So what are the apps? Well, here are the apps. So first of all you’ll notice that we’re not trying to design a whole building right so we kind of took a step back scaled it down and what you’re looking at here is a steel bay designer running in shape diver calling into cortex for AI inference and for an engineer this is a much more comfortable place to be because they have a lot of controls they have a lot of buttons to press and knobs to turn also all of the assumptions are explicitly stated up front there’s a whole dashboard that kind of shows you what’s going on with your structure.

13:23 You can even click a button and download a PDF report which will tell you like all of the assumptions and all of the metrics and kind of list them out for you for further documentation. Here’s another example AI share wall copilot again one building system very granular control and you’ll also notice so on the right hand side so all of these green checks these are actually codebased validation checks that we do on top of inference.

13:54 So we use inference to get a speed advantage. It’s really really fast. But then we use conventional code checks to make sure that the inference results are actually legit. And this also helps really improve the trust that our engineers have in our tools. So I’ll just go quickly through some other examples. Here’s an AI glass designer for our facades team. Timber column stack designer. Here’s the steel connection co-pilot.

14:25 That kind of lets you design an entire floor for your building energy modeling tool. So our sustainability team put this together using our infrastructure which was a fun collaboration. And an important thing to note that there are some cases where we don’t actually have to use AI, right? So here’s an example of the gusset plate designer. The core AI team, they created a custom solver that was competitive in terms of speed with other AI apps.

14:56 So, we’re like, well, there’s no point in using the solver to generate synthetic data, to train a model, to deploy it to Cortex, to use for inference, just deploy the solver, right? And so that’s what we did. And this is just to say that like we don’t use AI just to say that like we’re using AI because it’s a fashionable thing to say. We use it where it’s appropriate and where it gives us an advantage.

15:19 In this case, conventional solver did just fine. Same here. Trust copilot very popular app. Core AI built a custom FA tool turned out to be super fast. We’re just using that. I’m pretty sure it’s still running on Cortex, but there’s no machine learning in this specific case. Same same FEA in the chevron brace frame app. So we built a whole collection of these different apps. This is just a small fraction of them.

15:48 And our engineers are using them on projects. They’re continuously testing them. We get a lot of feedback. If something doesn’t work, if they have questions, we constantly keep the dialogue going and we improve them continuously. One thing that I’ll say is that as per our company policy, we have an AI policy in place. Nothing that’s generated either through our ML apps or through generative design tools can ever ever ever be communicated directly to the client or put on a drawing set without many eyeballs reviewing these designs using conventional industry standard software.

16:28 So there’s always a QC phase before something becomes part of the official record so to speak. But what these apps allow you to do is in the meantime as you’re playing around and as you’re testing things to move super super fast and get very high quality feedback. So of course having all of these individual building systems in place we had the temptation to put the Humpty Dumpty back together.

And so what you see here is you know creatively named Asteris 2.0 which does full building design again using a lot of those systems that you saw earlier and you know this is this is tailored towards engineers. You have a lot of very granular controls. You see on the right hand side there are metrics and readouts about your tonnage, your materials, your whatever you want. And also the way we’re supporting different structural systems has changed.

17:24 So the original asterisk you could either do a steel building or a concrete building. In our case you can mix and match and also we support timber. So for example right now we’ll see that you know this is a steel structure but like okay what happens if we do timber floors and the rest is steel right? What happens to your cost and your tonnage and your carbon or what if I want a timber structure with concrete shear walls, right?

17:53 What would that look like? So, you can very quickly test those things out and get some highle feedback on your project. I’ll mention gener generative AI really quickly. So, you know, we spent a couple years now building out all of those individual AI apps. And of course, where MCP came out, we’re like, oh, we wonder since we have all of these APIs, can we just give them to an agent and see how the agent does?

18:18 And the agent does pretty well, I must say. So here you see Claude using the trust copilot that I show I showed you earlier to do sensitivity analysis between tonnage and deflection using MCP worked pretty great. Here’s an example of Glean. So Glean is a product that we bought as like our internal AI powered enterprise search but Glean allows you to build agents. So this agent is using three different design tools to design a concrete bay and it’s smart enough to do multiple to tool calls and say it’s like okay I need to design the floor slab first the columns and the footings and then it presents the findings in these nice tables.

So with generative AI right now all of this is unlike our AI apps which are used by our engineers. This is all pure R&D because agents are still unreliable. They can hallucinate or do something drastic. They’re not very good with long horizon tasks. They kind of lose the plot the longer they work on something. And there are a lot of cyber security concerns as well. So in conclusion, where does this leave us?

19:34 We do believe that AI tools can provide a sign significant productivity boost in early stages of design for architects and engineers allowing really fast iteration. The blackbox nature of AI can be mitigated by just being very transparent about your assumptions and by doing conventional code checks. Risks of generative AI such as hallucinations and incontext scheming can currently be managed at least at this level of technology. Let’s be clear.

20:00 Can be managed by standard QC practices with human review and review by industry standard tools and yeah it’s an exciting time to be alive and we think that things are going to be pretty weird for a few years and we’re excited to see where it goes. So thanks so much. To see the full recording of this and previous presentations, as well as information about future CDF events, visit CDFAM.com.