Parallel operations in Dynamo / Designscript

Is it possible? If yes, how to do it?

I wrote Designscript code to create multiple road routes, there are several performance issues:

  1. I use loop to create them one by one, but if parallel operations can be done, I can create them in parallel.

  2. for each route, it’s consists of many curves, I use loop to create them one by one. I can also create them in parallel if Parallel operations can be done.

1 Like

you can try to use IronPython with Parallel LINQ

a small test

import clr
import time
from Autodesk.DesignScript.Geometry import *

from System.Threading.Tasks import *
import System

def drawlines(lstpt):
	outlines = []
	for idx, pt in enumerate(lstpt):
		if idx > 0:
			line = Line.ByStartPointEndPoint(lstpt[idx - 1], pt)
	return outlines		
lstpoints = IN[0]
start = time.time()
if IN[1]:
	threadResult = lstpoints.AsParallel().Select(lambda sublst: drawlines(sublst)).ToList()
	out = threadResult
	out = []
	for sublst in lstpoints:
# Calculate Run-time
time = ("%s s" % (time.time()-start))

OUT = time, out

I tried the code in Dy 2.10.0 sandbox, but t shows some warnings

Hi @Xiaofei_Ying

Because you forgot to add Boolean Input:

1 Like

My PC has i7-4790K,
the time with Parallel operations is 0.23 s,
the time without parallel operations is 0.36 s,

the gain (time consuming) is about (0.36-0.24) / 0.36 = 33.3%, while in your test the gain is about (0.5-0.24) / 0.5 = 50%

1 Like

My Config

Better to time this with TuneUp as it will incorporate the other aspects of the Dynamo environment (ie: preview and data marshaling) if I’m not mistaken.

Also a note to anyone who visits later: This will likely not work with Generative Design (requires 6+ cores for running each of the Dynamo Core engines concurrently) or Revit interactions (limited to one thread as a DB). Worth testing anyway.


Hi @c.poupin!

Parallel LINQ is great, Dynamo refreshes several times faster on large dataset!

Trying to test PLINQ with Geometry.IntersectAll:

Almost working, no crashes. But i have problems with list recreate.
Could you help with my code, please?
Thanks in advance!
Linq2.dyn (78.1 KB)

1 Like

Hello @Vladimir
it’s necessary to convert the result to List (to avoid overwriting the previous return variable)

import clr
import time
from Autodesk.DesignScript.Geometry import *

from System.Threading.Tasks import *
import System

def intrs(elA, elB):
	return elA.IntersectAll(elB) 

lstpoints = IN[0]
o = IN[1]
start = time.time()
res =[]
tst = 0
#oo = o
#ls = lstpoints
for ls,oo in zip(lstpoints,o):
	threadResult = ls.AsParallel().Select(lambda l: (intrs(l,oo))).ToList()

time = ("%s s" % (time.time()-start))

OUT = time, res
1 Like

Hello @c.poupin !

All works, but i feel, that for ls,oo in zip(lstpoints,o): in my code is bad for balancing quenue.
The idea is to send independant chunks (like [a[0], b[0]],[a[1], b[1]] ) to different threads.
The “a” chunks contains Rooms solids. The “b” chunks contains some walls solids.

Can i ask you, how to push two arguments into PLINQ?

Found .Zip here:

var a = new int[] { 1, 2 };
var b = new int[] { 3, 4 };

foreach (var n in a.Zip(b, (x, y) => x + y))
    // enumerates 4 (1 + 3), 6 (2 + 4)

And the question is how to apply it in PLINQ in Python. (Without Python loops)
Attached sample dyn.

Thank you!
Linq3.dyn (41.1 KB)

UPD: find some similar discussion: .net - How is the PLINQ AsParallel function passing data to a function when called within the same scope - Stack Overflow


you can unpack the arguments in the function

a simple example with AsParallel() and Zip()
for simple methods, Zip() is just as efficient

1 Like

Hi @c.poupin!
The last problem is some nulls data after PLINQ.
I can easily fix this with recursive loops by processing my list until nulls are gone.
But think the problem is Safe Threading:

As i have to use .AsOrdered() to keep my list, i think that ConcurrentQueue will be better than ConcurrentBag:

Trying to enqueue IN[0] with ConcurrentQueue.Enqueue(T) method, but lost in syntax.
Can you help, please?

Concurrent_.dyn (10.9 KB)

UPD: found out, that we can’t use ConcurrentQueue with PLINQ multithreading. The question is how we can make thread safe PLINQ to protect data and avoid data corruption. Thank you!

I don’t know all the PLINQ methods, but out of my curiosity, I compared some methods

here the results

for AsParallel() with ConcurrentQueue I used this code

import sys
import clr
import System
from Autodesk.DesignScript.Geometry import *
import Autodesk.DesignScript.Geometry as DS

from System.Collections.Concurrent import ConcurrentQueue
from System.Collections.Generic import List

def test(*args):
	lstPoints = [p.Centroid() for p in args]
	points = Point.PruneDuplicates(lstPoints)
	return points

ziplst = zip(*IN)
out = []

lstRtn = ConcurrentQueue[System.Array[DS.Point]]()
		.ForAll(lambda e : lstRtn.Enqueue(test(*e)))
OUT = lstRtn

Test AsParallel with Concurrent.dyn (27.8 KB)

1 Like

Hello @c.poupin,
Noticed errors if try to switch to CPython3 in your previous post:
AttributeError : ‘zip’ object has no attribute ‘AsParallel’.
Do you think its possible to use PLINQ in CPython?

CPython use a PythonNet bridge, it does not support PLINQ extension, and CPython engine is executed from an imported (non-main) module.

Try to use multithreading with CPython3 engine