Last week we shared Part I of III exploring Houdini software system requirements, we looked at how different hardware configurations performed rendering Flip, Grains and Pyro scenes with a focus on Houdini’s built-in render engine Mantra.  Today we present Part II in the series, the role of cache disk and RAM in a Houdini workflow. Special thanks goes to the meticulous Curvin Huber for leading our testing and Houdini benchmarking.

Test Configuration:

a-X Mediaworkstation

AMD 2950X 16 Core CPU (4.4 GHz Turbo)

32GB DDR4 3000 MHz RAM (and 64GB)

2x NVIDIA Geforce GTX 1080 Ti 11GB GPUs

512GB Samsung 970 Pro NVMe m.2 SSD (OS Drive)

Microsoft Windows 10 Pro 64-bit

First I’m testing the influence of hard drive speed and RAM capacity on Houdini simulation caches. Hardware acceleration is disabled for these tests thereby focusing the simulation calculations fully on the CPU and caching exclusively on the hard drive or RAM. The disk cache tests are being conducted on the Samsung 970 Pro M.2 drive. Disk caching is written using the File Cache node attached to the DOP I/O node and written to .bgeo.sc format. In the RAM cache tests the cache memory is set to 80% of total RAM to avoid system crashes. This is being done in the DOP Network node’s Cache tab. The Compress .sim Files parameter is set to Biosc.

Houdini Disk Caching

Writing to disk (Cache) is being conducted on high resolution simulations to compile the longest data set. These results are for caching to the drive that also contains the OS. Disk caching is written using the File Cache node attached to the DOP I/O or DOP Import nodes and written to .bgeo.sc format. Note: No GPU Acceleration is being used.

700K Grains

  • 50 frame Disk Cache
  • CPU 84-97%
  • RAM 18%
  • GPU 4%
  • OS Drive 1- 4%

Total Cache Time: 6m 42s

Houdini Software System Requirements

31m Point FLIP

  • 100 frame Disk Cache
  • CPU 12-96%
  • RAM 77-90%
  • GPU 4%
  • OS Drive 1-14%
  • NOTE: Cache files nearly 1GB/frame

Total Cache Time: 47m 11s

Houdini Software System Requirements

10m Voxel Pyro

  • 400 frame Disk Cache
  • CPU 94-98%
  • RAM 19%
  • GPU 4%
  • OS Drive 1-5%

Total Cache Time: 16m 47s

Houdini Software System Requirements

The most interesting observation is that the OS disk was never utilized above 14% for any of the caching tests. The highest utilization occurred with the FLIP simulation. As each frame written to disk was nearly 1GB, the disk performance spiked to 14% every time the sim frame completed its calculation in Houdini then wrote to disk.  This corresponds to a peak write rate of approximately 322 MB/s – far below the 2,300 MB/s write capability of the 512GB Samsung 970 Pro, and roughly 65% the write capability of most SATA3 SSDs, though this is an approximately 50% greater data transfer speed than most traditional 7200 RPM Hard Drives are capable of.

Before testing we discussed benchmarking a 2x 970 Pro RAID0 cache disk array for these simulations, but this was unneeded:  the amount of time it takes for Houdini to simply calculate the simulation is much longer than the amount of time it takes to write to disk. I also tested playback from the disk cache and there doesn’t seem to be any additional performance hits there either. This is surprising as I thought reading back from disk would tax the system somewhat but it didn’t.

The other interesting observation is that the FLIP simulation utilizes 77-90% of the RAM as it simulates and writes to disk. The Grain and Pyro tests do not seem to impact RAM usage at all.

Houdini System Requirements, RAM Caching: Comparing 32GB RAM vs. 64GB

Total Cache Memory is set to 80% of total RAM to avoid system crashes (25GB). This is done in the DOP Network node’s Cache tab. The Compress .sim Files parameter is set to Biosc. No GPU Acceleration will be used. Total frames written to RAM cache is the most important number to consider.

Houdini RAM Caching: 32GB RAM

700K Grains

  • CPU 84-97%
  • RAM 10% with idle file open
  • GPU 1%
  • OS Drive 1%

Total RAM Cache Time:  29m 21s (220 frames to max out 25GB of RAM)

Houdini Software System Requirements

31m Point FLIP

  • CPU 12-96%
  • RAM 45% with idle file open
  • GPU 1%
  • OS Drive 1%

Total Cache Time: 5m 38s (11 frames to max out 25GB RAM)

Houdini Software System Requirements

NOTE: Cache files nearly 1GB/frame as found in the disk caching tests.

10m Voxel Pyro

  • CPU 84-98%
  • RAM 10% with idle file open
  • GPU 1%
  • OS Drive 1%

Total Cache Time: 3m 47s (87 frames to max out 25GB RAM)

Houdini Software System Requirements

Houdini RAM Caching: 64 GB RAM

Total Cache Memory is set to 80% of total RAM to avoid system crashes (50GB). This is done in the DOP Network node’s Cache tab. The Compress .sim Files parameter is set to Biosc. No GPU Acceleration will be used. Total frames written to RAM cache is the most important number.

700K Grains

  • CPU 84-97%
  • RAM 5% with idle file open
  • GPU 1%
  • OS Drive 1%

Total RAM Cache Time:  58m 51s (440 frames to use 74% of 50GB of RAM)

Houdini Software System Requirements

Note: Doubling the RAM provides more than double the frame cache. This might be due, in part, to the grains coming to rest after frame 100.

31m Point FLIP

  • CPU 12-96%
  • RAM 24% with idle file open
  • GPU 1%
  • OS Drive 1%

Total Cache Time: 10m 04s (22 frames to max out 50GB RAM)

Houdini Software System Requirements

NOTE: Cache files nearly 1GB/frame as found in the disk caching tests.

10m Voxel Pyro

  • CPU 84-98%
  • RAM 5% with idle file open
  • GPU 1%
  • OS Drive 1%

Total Cache Time:7m 41s (164 frames to max out 50GB RAM)

Houdini Software System Requirements

Note: Not quite double the frames from 32GB but this is likely due to Pyro continuing to emit particles.

General Notes: Comparing disk cache files which range from 900k – 1GB per frame, these sizes added up determine how many frames can be cached to RAM.

Summary:

In summary, adding a faster hard drive does not provide much performance gain for caching simulations. Unlike 4K, 6K or 8K edit workflows for example where insufficient disk i/o can mean dropped frames, crashes or worse, Houdini simulations take much longer to calculate per frame than writing to disk. Therefore, testing shows that NVME m.2 and SATA SSD drive read/write speeds can easily keep up with the data being passed to it from Houdini on a frame by frame basis. However, Houdini cache files can become very large, especially in the case of FLIP simulations. Cached simulation can be upwards of 1GB or greater per frame, thus a large capacity hard drive would be the important factor to consider when purchasing or upgrading a system for Houdini.

Caching to RAM yields similar results. Adding more RAM (32GB to 64GB) essentially yields twice the caching capabilities. However, caching to RAM on complex simulations where frame by frame simulation times can exceed seconds or minutes in some cases, is not wise. RAM caches are temporary, as soon as a parameter is adjusted, the RAM cache is lost. Having an adequate amount of RAM to account for general processes is beneficial, however,when it comes to caching simulations, a very large hard drive is the most beneficial piece of hardware.

In Part III of our Houdini series, we’ll dispel the hope (for the moment at least) of Mantra gpu rendering, and explore the best render engines for Houdini.

Special thanks again to Curvin Huber for his thorough testing and Houdini benchmarking.

Houdini Software System Requirements – Houdini Recommended Configurations

For pure Houdini workflows, the price for performance king is our a-X Mediaworkstation.  For peak performance, the i-X2 Mediaworkstation is our most cost effective dual Scaleable Xeon solution.

NOTE: It’s important to consider your top 2-3 applications when considering optimum configurations, as different applications talk to and use hardware differently. This is abundantly clear from a performance standpoint when using GPU render engines for Houdini.