Parallel execution of compounds

[jipo]

Hi,

first of all sorry about my bad english...

I found one interesting problem and I can't decide if this is a bug or feature I know that ICE is able to execute Per Point compounds in parallel and that makes ICE fast, but I found one inconsistency in simultanous execution of Per Object and Per Point compounds.
I have this situation:

1) I have Per Object compound that uses Repeat node, in each iteration it chooses point by ID and executes my compound jp_strand_compute_spiral_points that computes some my per point data. Please see img1.jpg ( i hope that it's correctly attached,this is my first post here)

2) Inside this compound, Point ID is compared with input port ID and some functionality on that point is executed. Simple comparison in image img3.jpg

3) Point with given ID computes some per point position data by compound jp_strand_spiral on img2.jpg

The problem is, that repeated per object compound is executed simultanously with per point compounds (so the repeat continues on next iteration although jp_strand_spiral is not finished yet), so the code on img2.jpg is not executed on correct points. For example i am logging the value of the ID input port and it changes its value IN THE MIDDLE of the compound execution, on the different stages of the inner compound execution it has a different value, because outside per object repeat node started next iteration and new ID is chosen.

The result is,that jp_strand_spiral is executed on some bad points with ID's that normally are not choosed by ID filter on img3.jpg.

There is a workaround,i tried to copy values of the input ports to the each point (simple switch to point context by FirstValid node) and it works, but it is an interesting problem. If some per point code access per object data,it is not guaranteed that have the same value during compound execution, because other per object code may change it "under your hands"


And the question is, is this correct "by ICE design" or it is a bug? From the programmer point of view, per object and per point code cannot run in parallel, only multiple per point codes can do it,because per point contexts are indenpendent....

Any opinions?


thanks,

JIPO

[Chris_TC]

The problem is, that repeated per object compound is executed simultanously with per point compounds (so the repeat continues on next iteration although jp_strand_spiral is not finished yet)

I'm 99% sure that this is not the case. ICE evaluates left to right and top to bottom.

it changes its value IN THE MIDDLE of the compound execution, on the different stages of the inner compound execution it has a different value

You need to go through your tree bit by bit and figure out what's going on. I'm quite certain that the problem lies somewhere within your tree. Do you overwrite the value anywhere within your compound? Do you always read the correct iterator value (i.e. before you set it again)? By the way, the Repeat with Counter is probably more suited because it iterates automatically after every evaluation.

[Chris_TC]

Also, how do you choose the Point ID you feed into your compound? The subtree that connects to this port of the compound could be the problem as well.

And finally, I think that using an If node with execute ports but leaving one port empty can sometimes give issues. But I'm not entirely sure. I tend to avoid it though.

[jipo]

The problem is, that repeated per object compound is executed simultanously with per point compounds (so the repeat continues on next iteration although jp_strand_spiral is not finished yet)

I'm 99% sure that this is not the case. ICE evaluates left to right and top to bottom.

it changes its value IN THE MIDDLE of the compound execution, on the different stages of the inner compound execution it has a different value

You need to go through your tree bit by bit and figure out what's going on. I'm quite certain that the problem lies somewhere within your tree. Do you overwrite the value anywhere within your compound? Do you always read the correct iterator value (i.e. before you set it again)? By the way, the Repeat with Counter is probably more suited because it iterates automatically after every evaluation.

well,I have 2 relatively strong indications that my theory is correct:
1) everything works fine after a simple change. I simply copy ID (and other input port values) from the input port by Set Data node to the point context (local copies) at the beginning of the compound node
2) my problem occurs only when jp_strand_compute_spiral_points runs slow (this compound computes StrandPositions,when it computes only few segments,it finishes befere next repeat iteration,when it computes >5000 strand segments,problem happens. The reason of the slow execution is described in the "set in array implies copy array?" topic on this forum.Set In Array is very slow on the big array with the constatnt size, but that is a different story with different solution:-)

yes,i know that ICE evaluates left->right,top->bottom, but i think that ONLY nodes in the same context


maybe, but i debugged mu compound 2 days,i made a lot of logs and everything looks like sync problem

[xsisupport]

I don't have a specific answer for you, but here's some general points about ICE that I've gathered from comments made by devs in various posts/emails:

- Loops should be avoided whenever possible. They don't parallelise very well.
- Repeat nodes force a re-evaluation of everything upstream
- Multithreading is based on chunks of data, not ports
- There may be pre-processing and caching of input ports

[Chris_TC]

I've never used this node and am not too sure what it does. But maybe a Delay Set Data could help you?