Python WISH LIST Hijack Thread

[Letterbox]

Moderator edit: the following thread has been extracted
from the 2014 WISH LIST to prevent diluting the message of that thread any further - HB

Cynic? Beta or No? All of the comments = few ICE requests !

Python & ICE, 2 me is a step backwards, using only one core again because of Python, i dont think so. Sorry iamvfx, had to say this.

A better future IS multithreaded everything, from Nurbs to bevel. Question = can you even buy single core cpu anymore?

[iamVFX]

Python & ICE, 2 me is a step backwards, using only one core again because of Python, i dont think so. Sorry iamvfx, had to say this.

No problem, Chris, Python sucks inside of Softimage, it's true. Look at what Python can do outside of it coupled with a pair of skilled hands:

[Hirazi Blue]

file exchange with cubase and reason

I like where you're going with this: next up, creating VST's directly in ICE?

But seriously...
@Letterbox - AFAIK Python isn't the problem here. "Vanilla" Python can for instance do multithreading. The biggest bottleneck between Python and Softimage seems to lie, as far as I can tell from numerous threads on the topic, in the way Python is "hooked into" Softimage via PyWin32/the COM scripting API.

On the SI-Wiki an alternative to using PyWin32 was/is given, but SWIG itself seems complicated enough to not make this feasible for the largely clueless "faint of heart" (like myself).

Native Python support (without PyWin32) is possible using SWIG on the C++ API headers to generate the code for Python to talk to XSI. So, rather than using the COM Scripting API, your Python code would communicate with XSI via the C++ API.

(Quoted from here)

So as this is a wishlist thread: Native Python support (without PyWin32)

edit: removed the qualification "effortlessly" from my remark about multithreading as the intended hyperbole might be lost on some

[luceric]

the 2.x python intepreter cannot multithread. that's not a limitation of XSI, it's also true outside of XSI. It's the python interpreter itself that can only run one thread at a time.

[iamVFX]

the 2.x python intepreter cannot multithread.

What's this, then?

[csaez]

No problem, Chris, Python sucks inside of Softimage, it's true. Look at what Python can do outside of it coupled with a pair of skilled hands:

Doing something similar to Nanode is definitively possible yes, but not by integrating it. Nanode evaluation graph system is really (really) not as good as Fabric Core one . Nanode graph evaluation is done in Python, so you can't evaluate some branches in parallel for example. But building a node graph editor a la Houdini would be great for sure!

Guillaume

From "Visually develop?" discussion on Creation Platform mailing list.

[luceric]

the 2.x python intepreter cannot multithread.

What's this, then?

Did you read the link you sent?

"Due to the Global Interpreter Lock, in CPython only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation)"

Python supports the concept threads, I know that and everyone and his brother knows that.
but the python interpreter is not re-entrant and will only run one at a time. The threads do not run in parallel and do not use multicore. So there is zero performance benefit in Softimage or outside of it.

[Letterbox]

Languages like C, C++, Java, and C#, parallel programs by executing multiple threads. The global interpreter lock in the CPython and PyPy runtimes preclude this option.

+1 for Java

[Hirazi Blue]

@luceric - But at the risk of getting my avatar chewed off as well, what about the multiprocessing module?

[Letterbox]

@luceric - But at the risk of getting my avatar chewed off as well, what about the multiprocessing module?

He doesn't chew he bites, so to save you having a chunk missing, look up "Global Interpreter Lock" + python, and you'll see that it was never designed to do it. Even though many are asking for the GIL to be removed, its not an easy task. To Paraphrase someone else... Don't get your hope up to high, expect nothing, then you wont be disappointed ;) lol

Save you the trouble of finding it... http://www.artima.com/weblogs/viewpost. ... ead=214235

[luceric]

the point of my reply was to say that the fact that python uses only one core is a python problem, not a softimage/pywin problem. converting it to embedded python+SWIG cannot fix that.

with the multiprocessing module, you could launch from your script multiple python interpreter process to use multiple core, but that's something you would have to do all the work for to get working. Because they are separate processes, if you launched python sub-processes, those processes would not have access to any softimage objects or data from the parent process, you'd have to build your system to copy the data to them and read the result back, through files or pipe or something. Again this is not related to pywin or softimage. multiprocessing has a lot of overhead and is better for things where you are dealing with large data on disk and don't need to talk back and forth with a host.

[iamVFX]

Nanode graph evaluation is done in Python, so you can't evaluate some branches in parallel for example.

This awkward moment of realization that multithreaded ICE performance is equal to one threaded python application...

[Letterbox]

multiprocessing has a lot of overhead and is better for things where you are dealing with large data on disk and don't need to talk back and forth with a host.

That's too generic a statement, and there are many examples that it doesn't hold for. It may confuse people whom have never done that kind of programming. That said, i think it's covered what python can and cant do.

[Hirazi Blue]

@luceric - Thanks for your thorough explanation.
My earlier post might have been badly worded, but I mentioned the SWIG bindings in a slightly different context. I was under the impression "the way Python is 'hooked into' Softimage via PyWin32/the COM scripting API" to be the true "biggest bottleneck" between Python and Softimage. I didn't mean to imply the SWIG bindings were a solution to Python multithreading/multiprocessing woes.
;)