Handling of massive amounts of objects
-
- Posts: 440
- Joined: 06 Jun 2009, 14:01
- Contact:
Handling of massive amounts of objects
I did a little test recently. I exported a large number of objects from a max scene to obj to see where else I can load it. In max about 60% where instances, which where resolved to unique objects in obj. 2.3 gb filesize, 15 000 000 polygons, about 15000 objects.
Maya didn't recognized the obj file at all. (Yes I loaded the obj plugin, which is off by default)
Max just canceled the import without any comment at about 30%.
Softimage took 20 min to gather scene file statistics, the started to load until 32 gig ram where full and began swapping happily. I canceled the import after an hour.
Cinema 4d, the same as SI.
Blender loaded the scene in about half an hour and became unusable slow.
Modo to my surprise loaded the scene completely in about 20 min and it was completely editable. I rotated all 150000 objects 90 degree with one single parent in 30 sec. I move an object out of the hierarchie in 10 sec. I was able to orbit the scene, select single objects and focus on them. It took 30 min to select all objects.
I do not know what the reasons are for all AD products to fail this test, but it surprised me quite a bit. I at least expected maya to recognize a obj file. I expected max to be able to reimport what it exported. And most of all I expected SI to be memory efficient.
Maya didn't recognized the obj file at all. (Yes I loaded the obj plugin, which is off by default)
Max just canceled the import without any comment at about 30%.
Softimage took 20 min to gather scene file statistics, the started to load until 32 gig ram where full and began swapping happily. I canceled the import after an hour.
Cinema 4d, the same as SI.
Blender loaded the scene in about half an hour and became unusable slow.
Modo to my surprise loaded the scene completely in about 20 min and it was completely editable. I rotated all 150000 objects 90 degree with one single parent in 30 sec. I move an object out of the hierarchie in 10 sec. I was able to orbit the scene, select single objects and focus on them. It took 30 min to select all objects.
I do not know what the reasons are for all AD products to fail this test, but it surprised me quite a bit. I at least expected maya to recognize a obj file. I expected max to be able to reimport what it exported. And most of all I expected SI to be memory efficient.
I'm now part of an endangered species...
Re: Handling of massive amounts of objects
Softimage has never been good with thousands of objects.
Re: Handling of massive amounts of objects
ha yes, the daily Blender promo! But seriously.. I wonder if you would have had more chance in 3dsmax or Maya if the file was under 2 bit in size (about 2.1 gig) It's very typical on Windows where both "long" and "integers" and C library functions file pointer are 32-bit, to fail with text files around that.
ha ha ha ha ha ha. funny man.Lord Futzi Voldemort wrote: And most of all I expected SI to be memory efficient.
go modo!Lord Futzi Voldemort wrote:Modo to my surprise loaded the scene completely in about 20 min and it was completely editable.[...]. It took 30 min to select all objects.
Re: Handling of massive amounts of objects
OPs question remains unanswered ;) or is it...
-FABRIC ENGINE? delayed data load?
-maya xgen???
-FABRIC ENGINE? delayed data load?
-maya xgen???
- Hirazi Blue
- Administrator
- Posts: 5107
- Joined: 04 Jun 2009, 12:15
Re: Handling of massive amounts of objects
@luceric - Really, "the daily Blender promo"?
Stay safe, sane & healthy!
-
- Posts: 440
- Joined: 06 Jun 2009, 14:01
- Contact:
Re: Handling of massive amounts of objects
No comment on maya's failure?luceric wrote:ha yes, the daily Blender promo! But seriously.. I wonder if you would have had more chance in 3dsmax or Maya if the file was under 2 bit in size (about 2.1 gig) It's very typical on Windows where both "long" and "integers" and C library functions file pointer are 32-bit, to fail with text files around that.
ha ha ha ha ha ha. funny man.Lord Futzi Voldemort wrote: And most of all I expected SI to be memory efficient.
go modo!Lord Futzi Voldemort wrote:Modo to my surprise loaded the scene completely in about 20 min and it was completely editable.[...]. It took 30 min to select all objects.
I'm now part of an endangered species...
Re: Handling of massive amounts of objects
I did, you'd have to check if the file is recognized when it's a little under 2 gig of size. I'm not sure what "not recongnized" means, I can only guess there was an error loading
Re: Handling of massive amounts of objects
Yes, latest modo can be quite fast at rotating camera, but modeling tools are still slower than XSI/Maya/Max etc.
P.S. About Maya, 2013 version fails at importing files with filenames starting with number. For me at least.
P.S. About Maya, 2013 version fails at importing files with filenames starting with number. For me at least.
Re: Handling of massive amounts of objects
softiamge 2014 vewport is less reactive and robust (on my system)
-
- Posts: 440
- Joined: 06 Jun 2009, 14:01
- Contact:
Re: Handling of massive amounts of objects
Not recognized is what maya said. All other apps recognized the file, so there is nothing wrong with it. And you may not know it, but this happens in maya sometimes with considerably smaller files too.luceric wrote:I did, you'd have to check if the file is recognized when it's a little under 2 gig of size. I'm not sure what "not recongnized" means, I can only guess there was an error loading
But to make it clear. I only posted this because I found this result surprising. If there can be a conclusion drawn at all, then maybe that it looks like as if tools with newer code seem to be more prepared for the future than others.
I couldn't test with houdini, so the image isn't complete. I have some other very large data sets somewhere, maybe I'll test again soon.
Oh, and yes I intend to go modo. Not now but in a in one or two years.
I'm now part of an endangered species...
Re: Handling of massive amounts of objects
I used to have more luck importing big obj files using the fbx importer (menu "file->import fbx/switch to obj file format"). A bit obscured
Also with such files it's sometimes a good idea to have no 3D viewport open before importing if you plan to do poly redcution or something.
Also with such files it's sometimes a good idea to have no 3D viewport open before importing if you plan to do poly redcution or something.
softimage resources section updated Jan 5th 2024
-
- Posts: 440
- Joined: 06 Jun 2009, 14:01
- Contact:
Re: Handling of massive amounts of objects
OK, somehow maya managed to read the file on another machine after all and now I get this:
// Error: line 0: OBJ file line 14635518: index out of range for face creation. //
// Error: line 0: Error reading file. //
Again all other apps read the file (max to a certain point). I think I could eventually load it in SI if I upgraded to 64 gig of RAM.
// Error: line 0: OBJ file line 14635518: index out of range for face creation. //
// Error: line 0: Error reading file. //
Again all other apps read the file (max to a certain point). I think I could eventually load it in SI if I upgraded to 64 gig of RAM.
I'm now part of an endangered species...
Re: Handling of massive amounts of objects
I have mixed feelings. I think it's a test of the .obj importer. If Maya can't handle huge .obj files, it just means that no studios is using it that way, because otherwise it would be fixed. Autodesk is in daily meetings with dreamworks, weta, dysney, ilm, lightstorm. Scalability is always part of the discussion; there are people here dedicated to working on that. I don't use Modo (obviously) but it was said that Modo is not scalable, and the user base was hoping The Foundry will do something about it and fix it - there was a lot of discussion about that around The Foundry acquisition, and talked about in the fxguide interview by John Knoll. (Maybe they mean another kind of scalability - reference models perhaps?) It's not necessarily a question of newcode/old code. You can load directly alembic to GPU in Maya, that's all new code, as is all the new asset management and Viewport 2.0 work, and the future opensubdiv work and bifrost work. Also, the softimage code is newer than maya and (much) newer than max, but struggles with larger number of objects, possibly because we focused on structures to support operator stacks, property inheritances (which is used visibility, materials, etc), and other architectural goodies, which are not important when you want to load a 15,000 static props.Lord Futzi Voldemort wrote:Not recognized is what maya said. All other apps recognized the file, so there is nothing wrong with it. And you may not know it, but this happens in maya sometimes with considerably smaller files too.luceric wrote:I did, you'd have to check if the file is recognized when it's a little under 2 gig of size. I'm not sure what "not recongnized" means, I can only guess there was an error loading
But to make it clear. I only posted this because I found this result surprising. If there can be a conclusion drawn at all, then maybe that it looks like as if tools with newer code seem to be more prepared for the future than others.
I couldn't wrap my head around Modo modeling workflows when I tried it, for some reasons, but many people can, for sure.Lord Futzi Voldemort wrote:I couldn't test with houdini, so the image isn't complete. I have some other very large data sets somewhere, maybe I'll test again soon.
Oh, and yes I intend to go modo. Not now but in a in one or two years.
-
- Posts: 440
- Joined: 06 Jun 2009, 14:01
- Contact:
Re: Handling of massive amounts of objects
I was just able to load the data in maya as fbx (which is good when your client has the data in fbx format) and I have to admit, maya was quite responsive and the viewport lightning fast until it froze completely.
I'm now part of an endangered species...
Re: Handling of massive amounts of objects
The performance problems with 10,000+ objects have bitten me a handful of times. But I really wouldn't want to miss out on the things you mentioned above. These are things that help me big time in my day-to-day workflow, so I can live with having to work around the many-objects-issue every now and then.luceric wrote:Also, the softimage code is newer than maya and (much) newer than max, but struggles with larger number of objects, possibly because we focused on structures to support operator stacks, property inheritances (which is used visibility, materials, etc), and other architectural goodies, which are not important when you want to load a 15,000 static props.
-
- Posts: 440
- Joined: 06 Jun 2009, 14:01
- Contact:
Re: Handling of massive amounts of objects
I managed to load the data in softimage via fbx as well, which seems much more memory friendly than the hopelessly outdated obj importer.
Performance is surprisingly not much worse than maya, except from the viewport. Viewport 2.0 in maya is great but also unreliable in my experience.
I played half an hour with the scene with no issues other than slow screen updates when selecting which were still in a tolerable range given the huge amount of data. After I saved the scene it reloaded pretty fast.
So after all I think it looks not that bad after all for SI. Max however becomes rather unstable and laggy even when only selecting single objects or switching editors, it freezes a lot and I have to restart it after using a heavy scene before I load something else to prevent it from locking completely.
Maya has it's own issues. I wasn't really surprised that it couldn't read the obj file, after all it sometimes refuses to read simple jpgs or tifs often enough. It could be a coincidence that it locked up with this data, but it did, and such coincidences are not seldom. For Si however the bottleneck clearly is the poor viewport. I wasn't able to get any lighting like headlight for the shaded view until I reloaded the scene. Also the importers need heavy work. (I wonder when that will happen
I was surprised blender could load the data at all although it became unusable slow. C4D can handle large amounts of data. I had scenes with a similar amount of objects which it handled fairly good. I think the obj importer was also the problem here (didn't test fbx).
The really big surprise was modo, because given the complaints of modo users regarding performance I didn't expect it to perform in a tolerable range at all. And it was one of the two apps that had no problems reading the obj file. It is definitely worth to keep an eye on it, although I don't think the foundry intends to compete against max maya or si with it. We will rather see it as a part of the foundry's full production framework consisting of nuke, catana, mari and modo, all interlinked directly or via hiero. (something that can bring AD in serious trouble IMHO)
Performance is surprisingly not much worse than maya, except from the viewport. Viewport 2.0 in maya is great but also unreliable in my experience.
I played half an hour with the scene with no issues other than slow screen updates when selecting which were still in a tolerable range given the huge amount of data. After I saved the scene it reloaded pretty fast.
So after all I think it looks not that bad after all for SI. Max however becomes rather unstable and laggy even when only selecting single objects or switching editors, it freezes a lot and I have to restart it after using a heavy scene before I load something else to prevent it from locking completely.
Maya has it's own issues. I wasn't really surprised that it couldn't read the obj file, after all it sometimes refuses to read simple jpgs or tifs often enough. It could be a coincidence that it locked up with this data, but it did, and such coincidences are not seldom. For Si however the bottleneck clearly is the poor viewport. I wasn't able to get any lighting like headlight for the shaded view until I reloaded the scene. Also the importers need heavy work. (I wonder when that will happen
I was surprised blender could load the data at all although it became unusable slow. C4D can handle large amounts of data. I had scenes with a similar amount of objects which it handled fairly good. I think the obj importer was also the problem here (didn't test fbx).
The really big surprise was modo, because given the complaints of modo users regarding performance I didn't expect it to perform in a tolerable range at all. And it was one of the two apps that had no problems reading the obj file. It is definitely worth to keep an eye on it, although I don't think the foundry intends to compete against max maya or si with it. We will rather see it as a part of the foundry's full production framework consisting of nuke, catana, mari and modo, all interlinked directly or via hiero. (something that can bring AD in serious trouble IMHO)
I'm now part of an endangered species...
Who is online
Users browsing this forum: No registered users and 29 guests