Inbuilt limits for Python/custom node list size?

Packages
,
1

More a curiosity for the developers I thought I’d share/query…

I was running a few custom nodes today that use Python to retrieve model error warning associated elements. The nodes themselves are straightforward, one gets all error warnings and passes them in db format using the GetWarnings document method.

The second node takes the warning and retrieves the element(s) in the warning by iterating across the warning message to return sublists for each.

At small scales the node works fine, but for 1000+ warnings it fails to run and references an internal error. Placing the Python script on the canvas causes the same issue. If I chop the list into sets of 200 or so, then run the custom node @L2 it works fine.

Is there an inbuilt limit on iteration or a potential limit on memory I’m running into in this example? Images below of the nodes in case I’ve forgotten something in this case. This is the first time I’ve seen this happen. I’m not sure if maybe the issue is I am passing db elements in their raw format, although I couldn’t seem to find a Dynamo native conversion for them (dstype didn’t work if I recall).

errors
errors1163×807 138 KB

python
python663×648 46.1 KB

2

Perhaps…

if len(warnings) > 900:
   OUT = "Your model is in too bad a shape for anyone to automate a review. Go sit in the corner and think about what you did for awhile, and then go clean up your mess. Call me if you get stuck."
else:
   faillingIds, failingElements = [],[]
   ids = [ w.GetFailingElements() for w in warnings ]
   elements = [doc.GetElement(id) for id in ids)
OUT = elements

Just a thought…

7 Likes
3

Haha the BIM manager in me calls this a solution.

In all sense of genuine curiosity though, is there a limit we’re hitting here? I’m curious if its an iteration limit in Python (don’t think so, I’ve worked with huge sets before in Python itself), or an inbuilt safeguard in Dynamo’s python wrapper itself maybe.

I’ve been dealing with some larger scale company deployments recently, so if I can anticipate these sorts of things I’ll know where to build in assistance methods for Python like the chop/levels combination.

4

If there is, this is the first I have come across it. Depending on the error type that method could be returning 1000^2 elements… and for some reason my computer tends to say ‘this is dumb’ and quits on me before I can build up that many warnings… took that as a sign to ‘call it a night myself.

I am wondering if it’s related to the code or the warning type or something else. What is the most common warning type of the 1000+?

1 Like
5

no, I don’t think theres any kind of limit in DS or IronPython for number of objects you can iterate over. There will be memory limitations of course and possible bugs - I would be curious what the actual exception is, but for that I think you’ll need to debug dynamo.

3 Likes
6

In this case we had almost every warning type under the sun present (at least 40 types). I tried feeding in reduced sets with each filtered out and had the same result. If I figure out why it happens I’ll report back, otherwise it’s just one of those mysteries for now I guess…

1 Like