Procedural Music Pinecone with Softimage ICE and Unreal

Comments on work in progress
Post Reply
User avatar
Draise
Posts: 891
Joined: 09 Oct 2012, 20:48
Skype: ondraise
Location: Colombia

Procedural Music Pinecone with Softimage ICE and Unreal

Post by Draise » 12 Apr 2017, 06:33

Image
Frickin finally!

This little animation - that I will soon publish with a song compliments to Gabriel Lucien - will be my first of hopefully a series of techniques that will open doors.

I did it totally on my own accord to study a number of topics, probably a couple of weeks of manuels, tutorials and blood, sweat and tear mainly focused on two topics. One is procedurally created and animated geometry in ICE, Softimage; and the other is GPU rendering with Alembic inside Unreal Engine. None of these things I knew at all before starting the project.

So far... Unreal Engine is still in development, but holy cow can you do potentially a lot in there.

But by far ICE is way simpler to use than Blueprint......

This has been a long experiment. I spent a long time trying to render out this thing. Once I followed a tutorial on how to make procedural stairs, I perfected it and made a pinecone. After that I learnt how to import sound from sound files into Blender, cook it to geometry, and isolate frequencies in Audacity. Once I got that done, learnt how to export animation from Blender through FBX, then got that into SI to drive my pinecone in a somewhat procedural and creative way. Then I learnt how to use Alembic files in Unreal Engine, which is free, to then try render it out. I learnt that I can't cache large files, or have all the material clusters dissappear, it will crash UE. Later I learnt a lot about Blueprint, shading, and how to render realtime - also.. how to render a sequence with the textures. UE has a feature missing where media player texures will not render with the frame count, but with a timer - and if your card is rendering NOT at real time, they textures render really quickly meanwhile everything else doesn't. I also figured out that Motion Blur does not apply to Alembic Cache files. I also learnt that Virtual Dub with Avisynth with MV Tools, what I use for post motion blur, also finds it tricky for fast moving chunks. Also, Helix YV12 codec kinda sucks. Anyone know of any other yv12 codecs out there? I spent all day today, well afternoon, trying to find a way around the texture issue with UE, finally opting out of the new Media Player system for a Flipbook system.. Which was alright. In the process I rendered it in the dark, and thought.. HEY! That looks great. May as well splice it in.

I finally edited and did a little post in Blender and viola!

I'll be publishing the video soon.

User avatar
Draise
Posts: 891
Joined: 09 Oct 2012, 20:48
Skype: ondraise
Location: Colombia

Re: Procedural Music Pinecone with Softimage ICE and Unreal

Post by Draise » 13 Apr 2017, 04:15

Lucien Maguel - Mami te extraño - Promo Video Study



Here is a bit on my workflow to do this:
1. Use High and Low pass and frequency analysis in Audacity to isolate frequencies.
2. Import the sound files into Blender and bake them to a translation axis on a cube to export to FBX
3. Export the animation with FBX into Softimage
4. Build the ICE system to clone the shards onto curves. Think about what to do, what to hook up to what.. that is the tricky part haha.
5. Build the color beat system in ICE. Build the pen shape in Blender and drive a push deformer in Softimage.
6. Hook up the input from the cubes to set values, some animated, on what I wanted to drive in ICE
7. Export Alembic files. I did them in chunks as I had one bad chunk - realising that Unreal Engines needs all material ID's present, and will crash if one dissappears.
8. Render the color beat system from Softimage to create sprites for Unreal Engine.
9. Import the sprites to Unreal and hook them up with Blueprint.
10. Setup the shaders, the lighting, the Matinee and Sequencer relationship, the camera and due to the Alembic chunks: animate the visibility
11. Render - which is ridiculously fast in Unreal.
12. Post effects with Virtual Dub, Avisynth and MVtools for motion blur (Unreal won't put motion blur on Alembic Cache files) and Blender Composites - though you can get a really decent image from Unreal right away.
13. Edit in Blender

Tadaa!

All this... is the broken down version as I did a lot of experrimentation and trial and error learning the relationships between softwares, limits of Unreal Engine and playing video and rendering with the Sequencer, and the way Softimage handles data with ICE - which ultimately is really fun and easy. I had to also learn the basics of Unreal Engine practically from scratch, including building translucent shaders, Blueprint and hooking up parameters to Blueprint, and playing stuff with Blueprint for the rendering.

Notes to keep in mine.. I can render out to 4K without problem with UE, and then use Virtual Dub to give me that added AA sharpness for 1080p HD quality that my Video Card was lacking with. Work more on the artistic concept more. Have shards with the exported alembic file that contain the materials so that the particles can dissappear without crashing Unreal Engine. Still not sure how.... Learn more Unreal Engine and ICE. It is powerful.

User avatar
FXDude
Posts: 1129
Joined: 19 Jun 2012, 21:59

Re: Procedural Music Pinecone with Softimage ICE and Unreal

Post by FXDude » 14 Apr 2017, 00:17

Ice Viz! :)

Have you tried the (or one of the?) waveform to fcuve addon(s) of Rray?

used it (or one of them) and was very straight forward.

Might save a few round trips?


I also sometimes use somewhat similar methods to clean-up compression artifacts and/or noise from footage using a number of FFDShow Filters (inputting/outputting PNG sequences)

I wished Nuke had such temporal (with motion-vector-artifact free) denoising lol.

Cool and promising results!

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests