Distorting pixels

Hello everyone!

Today I want to talk about refraction, what it is and how you can use it to enhance special effects in your games. 

In the image above, you can clearly see that the pixels are distorted, they are not rendered at the place you'd expect them to be. The light seems to bend around something in the 3D world. This is what we call a refraction effect, gamewise. In the real world refraction works a bit more complicated than just pixels being distorted on the screen, but that doesn't matter for now. I think you can clearly see the point of what I'm trying to explain from the image.

There is a shader involved which renders this effect using simple geometry ( billboarding plane). The shader renders the object as of its 100% transparent. It also renders the object after the first 'transparent object render pass', this is important. In Unity you use the tag ' Tags { "Queue"="Transparent+1"} ' to achieve this in the shader. 
However, it also transforms the already rendered pixels behind this piece of geometry according to a normalmap. This all sounds quite complicated just from plain text, maybe some pictures can make it a bit more clear.

The above image shows you schemetically what normally happens during rendering without any refraction effect applied. As you can see every geometry is being rendered using its material onto the screen, nothing fancy. 

The second image shows you what the refraction shader is doing to the already rendered pixels using the simple billboarding plane geometry and it's normalmap texture.

This render techinique can be used to enhance your special effects. For example to visualize a shockwave moving through the world coming from a big explosion. Or a heat effect, wich makes the light 'ripple' in front of the viewer's eyes.

One big issue with this technique is that you can't fade the refraction effect by default. In the non-refracted shockwave particles I created, I made the particles fade out by altering their alpha values. However this doesn't doesn't affect the normal distortion off course. I fixed this problem by taking the vertex alpha's into the normals calculation of the effect. 

During the lifetime of a shockwave particle, it's alpha decreases from 1.0 to 0.0. I multiply this value by the normals I get from the refraciton normalmap. Less alpha means less distortion, which makes the effect able to fade in/out.

When taking this technique several steps further you could combine it with reflection to create realistic looking water reflections and distortions.. the possibilities are endless :) !

Greetings,
Joep