Several updates since writing the post:

  1. I have gone back to air cooling — I suspect I wasn’t getting enough water flow velocity through the system as I had too many junctions and loops — the system reached 80 degrees C one night and next morning all the colour dropped out of solution and a (moderately) high pressure gas had formed in the loop — this lead to some leakage at serveral fitting connections. Fortunately only a bit of water leaked onto a HDD all was OK. However the chemicals in the pre-mix solution smelled really nasty.
  2. I would’t use any pre-mix or dyes in a loop — based on my experience above these can be really nasty smelling chemicals (presumaby pretty toxic), and if dye drops out of solution, can leave a hard residue in waterbloks, radiators and tubing that cant be flushed.
  3. I would only watercool one or two GPU’s. To watercool 3+ 250W GPU’s takes a lot of planning and testing and hard to balance radiator space+fluid velocity for system to work well.
  4. I am now using a HAF932 coolermaster case for air cooling 3x 250W GPU’s. Using negative airpressure has been key. With the HAF932 you can run a large fan in exhaust mode at the side of the case (exactly opposite the tops of the GPU’s) which draws out significant heat comming off the 3 standard air-cooled GPU’s. (The HAF932 is an old but excellent case, they dont seem to make cases this solid anymore).
  5. Watercooling took a lot of time to get working, was fun, but now that I am just running fans I am a lot less concerned about leaving my machine to run for days now and just need to turn on the air conditioner in the room on hot days.
  6. The noise from the air cooling system is on par with my watercooling rig — the Noctua NF-A14 industrial PPC-2000 PWM fans I was using for a radiator was pretty loud, now the main source of noise is the GPU fans.

Geophysicist and Deep Learning Practitioner