# Integrate a machine learning model into Dynamo to be used for Generative Design purposes

**URL:** <https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519>\
**Category:** Uncategorized\
**Created:** [September 23, 2021, 9:10pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519 "2021-09-23T21:10:19Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [September 23, 2021, 9:10pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/1 "2021-09-23T21:10:19Z")

</div>

Hello,  
I am trying to integrate a machine learning model within Dynamo for analysis purposes. The idea would be to use this ML model and 10 numeric variables to predict 2 outputs which are also numeric values. I thought that code blocks in Dynamo would help me do the work, but they didn’t. I am trying to figure out how “python script” nodes in Dynamo can help me execute the attached codes (part 1 and part 2). I’ve screenshotted the output window for “code part 2” to show that the inputs are represented by one array that consists of 10 numeric values, and that the outputs are represented by one array of 2 values.  
I also tried attaching “model\_autokeras.zip” folder which is the ML model saved as h5 in this zipped folder, but the format isn’t supported here. I’ll try attaching it in a comment.  
The purpose of this integration will be to use the ML model as the analysis part of the GD, and later the goals of the GD will be to minimize both outputs of the ML model (while varying the input values).

Thank you in advance!

 ![code part 1](https://us1.discourse-cdn.com/flex022/uploads/dynamobim/original/3X/c/4/c41f6c7a6c4a609ad914a53d9d51c33d8798f536.png)  
 ![code part 2 with the output window screenshotted](https://us1.discourse-cdn.com/flex022/uploads/dynamobim/original/3X/5/d/5d31c2734ff1b89bdf7fd05cdfa97d609b3e509d.png)

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [September 23, 2021, 10:05pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/2 "2021-09-23T22:05:25Z")

</div>

> **[model\_autokeras.zip](https://www.dropbox.com/s/jyw4op253y93kfy/model_autokeras.zip?dl=0)**
>
> Shared with Dropbox

---

<div class="post-metadata">

**Author:** ![GavinNicholls](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/gavinnicholls/32/190442_2.png) [@GavinNicholls](https://forum.dynamobim.com/u/GavinNicholls)\
**Post date:** [September 24, 2021, 12:12am UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/3 "2021-09-24T00:12:38Z")

</div>

You’ll need to load some external libraries by the looks of it. Don’t think pip will work.

This thread has some discussion on external library referencing:

> [@How to load external Python modules?](https://forum.dynamobim.com/t/how-to-load-external-python-modules/5678):
>
> After adding proper directory by appending sys.path variable still cannot load external modules. Not DLL’s, but regular Python scripts. Even built-in modules like “sqlite3” and “xml” are not visible. Very annoying…

Dynamo 2022 offers CPython but older builds are using IronPython. I believe numpy is in 2022 but not earlier builds. keras and tf aren’t in any by default as far as I’m aware.

---

<div class="post-metadata">

**Author:** ![solamour](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/solamour/32/153516_2.png) [@solamour](https://forum.dynamobim.com/u/solamour)\
**Post date:** [September 24, 2021, 2:55pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/4 "2021-09-24T14:55:09Z")

</div>

Hello @ritaselias - have you tried this wiki page entry? [Customizing Dynamo's Python 3 installation · DynamoDS/Dynamo Wiki · GitHub](https://github.com/DynamoDS/Dynamo/wiki/Customizing-Dynamo's-Python-3-installation)

We have plans to include some of the most common and popular Python 3 packages out-of-the-box, but haven’t had scope to do so yet.

---

<div class="post-metadata">

**Author:** ![c.poupin](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/c.poupin/32/89152_2.png) [@c.poupin](https://forum.dynamobim.com/u/c.poupin)\
**Post date:** [September 25, 2021, 2:24pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/5 "2021-09-25T14:24:11Z")

</div>

Hello @ritaselias  
I wrote an article recently, it might be able to help you (use Translate button)

> **[\[Dynamo+=Python\] Deep Learning avec TensorFlow](https://voltadynabim.blogspot.com/2021/07/dynamopython-deep-learning-avec.html)**
>
> Blog sur le BIM, du Python et d'la Dynamo appliqués aux électrons

> [@solamour](#):
>
> We have plans to include some of the most common and popular Python 3 packages out-of-the-box, but haven’t had scope to do so yet.

it’s a good new ❤  
in addition to the most well-known packages, the implementation of some packages could be very useful  
This would fill some feature gaps in PythonNet, although I am waiting for to test PythonNet 3 which brings new features and fixes.

> [@\[New Feature Preview\] Python 3 Support Issue Thread](https://forum.dynamobim.com/t/new-feature-preview-python-3-support-issue-thread/51649/63):
>
> You are welcome @c.poupin slight_smile Just wanted to add an alternative for COM interop. This can be achieved also using Python --\> COM integration, without going through .NET. There is a module called pywin32 that serves this purpose. Here is an example equivalent to the one I showed before but using this pywin32 instead of relying on a custom .NET library.

> [@\[New Feature Preview\] Python 3 Support Issue Thread](https://forum.dynamobim.com/t/new-feature-preview-python-3-support-issue-thread/51649/65):
>
> Hi @ustncz. I tried and got the same error you did. I think it’s because we are running Python from .NET, so the threading mode is already initialized. I was actually able to use the library if I commented out the offending line, where CoInitializeEx is called. Not sure if that causes other issues down the road, so please beware.

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [September 25, 2021, 3:04pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/6 "2021-09-25T15:04:18Z")

</div>

@GavinNicholls , @solamour , and @c.poupin thank you all for your help! I’m sure I’ll get back to you with more questions once I start solving my problem! Thanks again!

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [October 1, 2021, 4:04pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/7 "2021-10-01T16:04:26Z")

</div>

I followed all your suggestions but I’m getting the attached error. I tried searching for a solution online but no luck!  
Below is my code:

```auto

# Load the Python Standard and DesignScript Libraries
import sys
import os
import clr
import System

dirAppLoc = System.Environment.GetFolderPath(System.Environment.SpecialFolder.LocalApplicationData) 
sys.path.append(dirAppLoc + r'\python-3.8.3-embed-amd64\Lib\site-packages')

# force to reload module
if "tensorflow" in sys.modules:
    del sys.modules["tensorflow"]
    

import autokeras as ak
import numpy as np
import tensorflow as tf
from tensorflow import keras

#Load Best Model
from tensorflow.keras.models import load_model

loaded_model = load_model("C:/Users/rita2/Downloads/model_autokeras", custom_objects=ak.CUSTOM_OBJECTS)

# The inputs to this node will be stored as a list in the IN variables.
X_test = np.array([IN[0], IN[1], IN[2], IN[3], IN[4], IN[5], IN[6], IN[7], IN[8], IN[9]])

OUT =[]
#Predictions
predicted_y = loaded_model.predict([X_test])

# Assign your output to the OUT variable.
OUT = predicted_y

```

Any idea? Thanks again for your help! ![Picture1](https://us1.discourse-cdn.com/flex022/uploads/dynamobim/original/3X/a/c/aca2468bfb472a6a5cea156c126254074e5ae8b0.png)

---

<div class="post-metadata">

**Author:** ![c.poupin](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/c.poupin/32/89152_2.png) [@c.poupin](https://forum.dynamobim.com/u/c.poupin)\
**Post date:** [October 2, 2021, 9:44am UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/8 "2021-10-02T09:44:30Z")

</div>

Not sure, but the error seems to come from your Keras model ML (maybe an incompatibility with tensorflow version )

simple test with an Export (from Sublime Text, Python 3.8) and Import (from Dynamo CPython 3.8) of a Keras Model

![testExportImportModeltf](https://us1.discourse-cdn.com/flex022/uploads/dynamobim/original/3X/3/2/32a67025d3f42b705de716654b7b6abc3d2e97f7.gif)

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [October 5, 2021, 1:36pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/9 "2021-10-05T13:36:12Z")

</div>

Thank you very much for your help! Your solutions are very thorough and helpful, I appreciate it! I’m still struggling with another error. I will try solving it before I give up and post it here!

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [October 8, 2021, 4:45pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/10 "2021-10-08T16:45:25Z")

</div>

I made sure that I have same tensorflow version 2.4.0 for both my code and Dynamo. But, I’m getting a new error:

Warning: RuntimeError : Unable to restore a layer of class Custom\>MultiCategoryEncoding. Layers of class Custom\>MultiCategoryEncoding require that the class be provided to the model loading code, either by registering the class using @keras.utils.register\_keras\_serializable on the class def and including that file in your program, or by passing the class in a keras.utils.CustomObjectScope that wraps this load call. [’ File “”, line 26, in \n’, ’ File “C:\Users\rita2\AppData\Local\python-3.8.3-embed-amd64\Lib\site-packages\tensorflow\python\keras\saving\save.py”, line 212, in load\_model\n return saved\_model\_load.load(filepath, compile, options)\n’, ’ File “C:\Users\rita2\AppData\Local\python-3.8.3-embed-amd64\Lib\site-packages\tensorflow\python\keras\saving\saved\_model\load.py”, line 138, in load\n keras\_loader.load\_layers()\n’, ’ File “C:\Users\rita2\AppData\Local\python-3.8.3-embed-amd64\Lib\site-packages\tensorflow\python\keras\saving\saved\_model\load.py”, line 374, in load\_layers\n self.loaded\_nodes[node\_metadata.node\_id] = self.\_load\_layer(\n’, ’ File "C:\Users\rita2\AppData\Local\python-3.8.3-embed-amd64\Lib\site-packages\tensorflow\python\keras\

![new error](https://us1.discourse-cdn.com/flex022/uploads/dynamobim/original/3X/f/0/f076c0bd7fa46a18afdb44ff8a8c025c8930bd3a.png)

Thanks again for all your help!

---

<div class="post-metadata">

**Author:** ![c.poupin](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/c.poupin/32/89152_2.png) [@c.poupin](https://forum.dynamobim.com/u/c.poupin)\
**Post date:** [October 11, 2021, 10:22am UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/11 "2021-10-11T10:22:21Z")

</div>

look at this maybe it can help you

> <https://github.com/tensorflow/tensorflow/issues/45443>
>
> \*\*System information\*\*
> 
> \- OS Platform: Windows10
> \- TensorFlow version: 2.3.1
> …
> \- Python version: 3.8.4
> \- CPU
> 
> I used AutoKeras generate a model and saved as 'tf' format.
> model.save('ak\_best\_model', save\_format='tf')
> Then i use:
> keras.models.load\_model('ak\_best\_model') in the same script
> It works well...
> 
> But i open a new script and use Keras load the same model, error occured:
> 
> RuntimeError: Unable to restore a layer of class Custom\>MultiCategoryEncoding. Layers of class Custom\>MultiCategoryEncoding require that the class be provided to the model loading code, either by registering the class using @keras.utils.register\_keras\_serializable on the class def and including that file in your program, or by passing the class in a keras.utils.CustomObjectScope that wraps this load call.
> 
> here is my script:
> 
> df = pd.read\_excel('total.xlsx', dtype=np.float32)
> 
> y = df\[\['Static\_Fric', 'Rolling\_Fric'\]\].values
> x = df.drop(labels = \['Static\_Fric', 'Rolling\_Fric'\], axis =1)
> 
> x\_train, x\_test, y\_train, y\_test = train\_test\_split(
> x, y,
> test\_size=0.2,
> shuffle=True
> )
> keras\_model = load\_model(r'D:\\Program Files\\Desktop\\ML\\AutoKeras\\ak\_best\_model')

> <https://github.com/tensorflow/tensorflow/issues/44199#issuecomment-730297817>
>
> \*\*System information\*\*
> \- TensorFlow version: 2.3.1
> 
> 
> \*\*error message\*\*
> 
> 
> …
> \*\*RuntimeError: Unable to restore a layer of class TextVectorization. Layers of class TextVectorization require that the class be provided to the model loading code, either by registering the class using @keras.utils.register\_keras\_serializable on the class def and including that file in your program, or by passing the class in a keras.utils.CustomObjectScope that wraps this load call.\*\*
> 
> 
> 
> \*\*Standalone code to reproduce the issue\*\* 
> i built a model like below:
> \`\`\`
> model = tf.keras.Sequential(\[
> tf.keras.Input(shape=(1,), dtype=tf.string),
> vectorize\_layer(a TextVectorization layer that has adapt),
> layers.Embedding(max\_features + 1, embedding\_dim),
> layers.Dropout(0.2),
> layers.GlobalAveragePooling1D(),
> layers.Dropout(0.2),
> layers.Dense(1),
> layers.Activation('sigmoid')
> \])
> \`\`\`
> 
> and i saved model like this:
> \`model.save('./model/basic-text-class-export', save\_format='tf')\`
> 
> but when i loaded model like below, i got a error as shown before
> \`model = tf.keras.models.load\_model('./model/basic-text-class-export')\`
> 
> how can i do next?
> best wishes for you

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [October 21, 2021, 4:19pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/12 "2021-10-21T16:19:41Z")

</div>

It didn’t but I really appreciate your help!

---

<div class="post-metadata">

**Author:** ![Cezmi](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/cezmi/32/96065_2.png) [@Cezmi](https://forum.dynamobim.com/u/Cezmi)\
**Post date:** [December 5, 2021, 4:04am UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/13 "2021-12-05T04:04:08Z")

</div>

Hi @c.poupin,

I just read your article " [Dynamo+=Python] Deep Learning avec TensorFlow" ([[Dynamo+=Python] Deep Learning avec TensorFlow ~ VoltaDynaBim](https://voltadynabim.blogspot.com/2021/07/dynamopython-deep-learning-avec.html)). and I couldn’t install the “termcolor” which you have explained in your article pretty good. However, I couldn’t download that zip file to extract.

 ![image](https://us1.discourse-cdn.com/flex022/uploads/dynamobim/original/3X/7/1/710de8fc768531418f2ef36a35c6dfacbce1a64a.jpeg)

the only file available to download is “termcolor-1.1.0.tar.gz” file. Is it possible to send me the file or point me to a website so I can download that file?

Thank you,  
Cezmi

---

<div class="post-metadata">

**Author:** ![c.poupin](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/c.poupin/32/89152_2.png) [@c.poupin](https://forum.dynamobim.com/u/c.poupin)\
**Post date:** [December 5, 2021, 1:35pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/14 "2021-12-05T13:35:40Z")

</div>

Hi @Cezmi  
You can open or extract directly the tar.gz file with 7zip software

---

<div class="post-metadata">

**Author:** ![Cezmi](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/cezmi/32/96065_2.png) [@Cezmi](https://forum.dynamobim.com/u/Cezmi)\
**Post date:** [December 5, 2021, 5:53pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/15 "2021-12-05T17:53:14Z")

</div>

Hi again @c.poupin,

All good now! I was able to install the packages I need. Thank you so much and very appreciated!!

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [December 30, 2022, 8:55pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/16 "2022-12-30T20:55:50Z")

</div>

@c.poupin Thanks again for all your help. I paused my work for a while, this is why I didn’t get back to you earlier. But all your comments were super helpful and thoroughly answered my question. I am facing a new related issue now. My Python script node is working within Dynamo completely fine and serves as the objective function within my graph. It basically finds/calculates the outputs of my generative design study by calling/loading an Artificial Neural Network from an external file. The graph in Dynamo runs with no issue. However, when I create my GD study, the outputs turn out to have no value (\*). I saw a comment made by @jacob.small on a different post:

> [@Generative Design Error : Graphics & Output No Value](https://forum.dynamobim.com/t/generative-design-error-graphics-output-no-value/56745/2):
>
> Do you have any python nodes calling an external application which GD doesn’t have access to (ie: Revit API requests, or RSA), or code which is attempting to leverage multiple cores (something GD does automatically).

Could this be the reason my GD study does not find values for my outputs? If yes, how can I solve it?

Thanks everyone for your help! Much appreciated!

---

<div class="post-metadata">

**Author:** ![RMohareb](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/rmohareb/32/120534_2.png) [@RMohareb](https://forum.dynamobim.com/u/RMohareb)\
**Post date:** [December 30, 2022, 9:54pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/17 "2022-12-30T21:54:22Z")

</div>

> [@Can i use AI surrogate model inside dyanmo to run genrative design?](https://forum.dynamobim.com/t/can-i-use-ai-surrogate-model-inside-dyanmo-to-run-genrative-design/78445/12):
>
> @c.poupin global optimization global processing. [https://www.google.com/url?sa=t&source=web&rct=j&url=https://damassets.autodesk.net/content/dam/autodesk/research/publications-assets/pdf/learning-to-simulate-and-design-for-structural-engineering.pdf&ved=2ahUKEwibl9iemPT4AhWnXvEDHQIhBDsQFnoECAQQAQ&usg=AOvVaw3jWFKQytlIKermIqN-\_4vf](https://www.google.com/url?sa=t&source=web&rct=j&url=https://damassets.autodesk.net/content/dam/autodesk/research/publications-assets/pdf/learning-to-simulate-and-design-for-structural-engineering.pdf&ved=2ahUKEwibl9iemPT4AhWnXvEDHQIhBDsQFnoECAQQAQ&usg=AOvVaw3jWFKQytlIKermIqN-_4vf) Like this paper. They use AI moder for global optimization problems

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [January 3, 2023, 6:15pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/18 "2023-01-03T18:15:32Z")

</div>

My Dynamo graph had 10 outputs so I could export it and created a Generative Design (GD) study. The GD Study was working completely fine until I integrated/added the Python script node that finds two additional outputs using a previously trained artificial neural network. Any idea anyone? Thanks again

---

<div class="post-metadata">

**Author:** ![RMohareb](https://sea2.discourse-cdn.com/flex022/user_avatar/forum.dynamobim.com/rmohareb/32/120534_2.png) [@RMohareb](https://forum.dynamobim.com/u/RMohareb)\
**Post date:** [January 3, 2023, 6:35pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/19 "2023-01-03T18:35:23Z")

</div>

Hi @ritaselias  
I face the same problem. GD tools in Dynamo cannot work if you use pretrained Ml model. I add old post has this problem.

---

<div class="post-metadata">

**Author:** ![ritaselias](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@ritaselias](https://forum.dynamobim.com/u/ritaselias)\
**Post date:** [January 3, 2023, 6:49pm UTC](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519/20 "2023-01-03T18:49:06Z")

</div>

Yes but I noticed from your problem description that your GD study didn’t run at all. Mine is running (and completes the run), but outputs no value for most of the outputs. I don’t know but I’m still hoping there is a solution for that as it is running.  
Did you find a workaround for this problem?

[Next page](https://forum.dynamobim.com/t/integrate-a-machine-learning-model-into-dynamo-to-be-used-for-generative-design-purposes/68519.md?page=2)