Hello, My latest Dynamo project is a ‘clash importer’ that creates a sphere with various attributes of the ‘clash’. There are some 3rd party applications that do the same thing- i.e import the XML report from Navisworks and place a sphere marker at the coordinates. Using the tools in Navisworks, it is difficult to get a sense of the big picture of where the clashes are and their status. Refer below, colour coded by status in Navisworks I’m using Pentaho to get the XML into a better format- this tool is well worth a look if you do any database work or need to transform data between different formats & structures. If anyone is interested, I am happy to share the necessary Pentaho/Dynamo
As a plug for my blog- this post covers the Myth of clash detection In this project, there are a crazy amount of clashes- in the tens of thousands. However, Dynamo/Revit seems to hit a hard wall of how many records it can deal with.
20,000: no problem, takes about 5 mins to run in Dynamo
30,000: no-go, even if I let it run all weekend, Revit/Dynamo 'hangs', maxing out a processor
I would have expected 30,000 to take ~50% longer than 20,000. but just seems to be over some sort of threshold. There could be be some bad data, but looking through it it seems to be pretty consistent. The questions are:
Is there any way to see the progress/status of Dynamo as it is running, particularly to identify any processor or memory intensive steps ?
Is there any way to increase the threshold i.e configuration changes, assuming plenty of physical memory etc
(this is other than adding watches in Dynamo, or seeing various errors once it has run) It seems that as soon as you hit 'run', it is a bit of a mystery what Dynamo is doing. I am thinking of various activity/metrics/monitoring tools thanks Andrew 
I’m guessing that Revit is the biggest bottleneck in your process. Consider using a cube or even a tetrahedron instead of a sphere. Curved faces are bad. The less faces you have to render, the smaller the performance penalty.
And finally yes, the forum gobbles up your formatting every time you press edit.
P.S.
Not sure if this will help with performace but it should reduce your file size; try using transformed/translated typed direct shapes instead of actual families:
I know this is an old post, but I wanted to follow up on the question regarding a progress bar. Does anyone know if there is a way to add a progress bar to function as part of a Dynamo script run?
I think the issue would be that Dynamo would need to first work out the total amount of processing, then the progress at a point of time as a proportion of that
Progress Bars are inherently difficult to implement - because they all pretty much fake it.
You can have a graph with 100 nodes, and most of those will execute in microseconds, but one Python node could take 20 minutes - therefore your progress bar, if implemented on a per-node basis would go super fast for 99% of it then hang for the residual 19.9 minutes. To get an accurate progress bar, you would have to pre-calculate to then post-calculate… which kind of defeats the purpose
As a side note, we are developing a way to showcase how long each part of a graph takes to run using the TuneUp Extension - available to check progress on here: https://github.com/DynamoDS/TuneUp
This will showcase per-node runtimes (execution time) and also expose a Force Calculate All button as Dynamo uses Delta Compute - Where it will only re-run sections of the graph that need it (i.e have new data passing through). This allows us to be much more efficient in operations - especially with large datasets.
A good example is taking a bunch of geometry and booleaning them together, then translating them. If you translate again and again after it will be fast, but if you had to re-calculate the boolean operations over and over it would be slow!
We’ve had internal discussions around how to implement a feasible, but more importantly useful approach to showing that Dynamo is still running - happy to hear any ideas on this front too TuneUp is a nod towards starting to showcase this.
Have no idea how this works in your code behind, but something like a property of the node you could write to in python or OOTB nodes that you give a count total based on the inputs and then just i++ for each element of the loop. This could be an selected thing like the player’s Is Input property. I’d selected and given a count and the individual increment a PG goes across the bottonnor too of the node based on items processed not tine. That way it wouldn’t be “fake” but also wouldn’t indicate time remaining either, just a way to tell it’s still working. I donthis often in C# Forms to make sure things are still running.
Greating! I read that due to the incomplete implementation of the multiprocessing module in IPython it is impossible to use many processors. I can run IronPython without the need for a GIL but Python (CPython) requires a GIL and performance is improved through threading. Can statusbar be implemented in main thread? In the code sections call join and start and show statusbar? Anybody try?
Dynamo Sandbox is multi-threaded (Which is awesome!) and contains a UI thread and an execution thread (Also awesome), but some of our host environments are single-threaded (Such as Revit) that means both the UI and Execution are on the same thread. This pretty much stops us being able to implement UI features that still continue to function during the execution of the graph (i.e a progress bar).
As an aside, we’re looking into enabling CPython inside of Dynamo now (Firstly in tandem with IronPython) to expose a whole bunch of things
We’re just kicking off the work now - so not for a release or two sadly. Will release another Dynamo Public Roadmap in the not-to-distant future that will talk a little more about our plans
@solamour , how can we use it in collision(intersection) detection? How Does.Intersect can be multi-threaded with Dynamo? We can use some packages for fast detection collisions in Revit, but solid-solid collisions in Dynamo are just brute-force.
Thanks!
Hello @Vladimir, the nodes themselves are single threaded (We decouple the UI thread from the execution thread and enable delta-compute on the execution thread).
Spring Nodes by @Dimitar_Venkov has an example in this Mesh.ToPolysurface node that showcases one way to use Threading in Python inside Dynamo.
Please bear in mind that you will have to dispose of any geometry on those threads before they disappear as threading sits outside the scope of our recent work allowing for auto-garbage disposal in Python.
I 've implemented threading in my script and it works differently on different computers, somewhere longer, somewhere faster . Therefore, I am interested in the implementation of the multiprocessing module because now for large models the execution time of my script (script is used to find the shortest paths in electrical circuits) takes more than 12 minutes for 325 electrical circuits.
If anyone is interested, I am happy to share the necessary Pentaho/Dynamo
TuneUp is a nod towards starting to showcase this.
When it is possible to see?
. Therefore, I am interested in the implementation of the multiprocessing module because now for large models the execution time of my script (script is used to find the shortest paths in electrical circuits) takes more than 12 minutes for 325 electrical circuits.