webr3.org

All too often it's too easy to overlook a technology, or indeed its uses, by mentaly boxing it in to the role it's advertised to fill.

PixelBender is one of these technologies, as advertised "You can use the Pixel Bender kernel language to implement image processing algorithms (filters or effects) in a hardware-independent manner"; however this isn't the whole truth of the matter.

PixelBender is a highly under-rated and under-used technology imho. Simply put PixelBender is actually a very fast way of converting or manipulating large sets of numerical data, whether it's image based or not. In this post I'm only going to skim the surface and give some pointers.

The standard input types cover Vector, ByteArray and BitmapData, likewise with output data, additionally many types are supported for parameters, everything from ints up to 4x4 Matrix' of floats.

Some obvious and simple uses include:

• 3D Data Manipulation (Float3 input and output - rgb or xyz)
• Raw Sound Data Manipulation (from Sound.extract, you could even build an FFT implementation in PixelBender)
• Visual Effects Calculations
• Batch Processing and Data Transformation.

Starting with Batch Processing and Data Transformation, here is the simplest PixelBender usage and example you could ever get.

Problem

Turn a BitmapData in to a Vector of RGB float values where each value is float < 1 and each set of 3 values comprises a pixel.

Doing this with pure AS3 or haXe or anything else is quite intensive, then you add PixelBender in to the equation

Solution
PixelBender Code:

<languageVersion : 1.0;>
kernel Identity
<   namespace : "org.webr3.pb";
 vendor : "WebR3";
 version : 1;
 description : "Converts BitmapData to Vector Number (rgb)";
>
{
 input image3 src;
 output pixel3 dst;

 void evaluatePixel()
 {
 dst = sampleNearest( src, outCoord() );
 }
}

And to use this we can create a simple wrapper class in AS3:

package test
{
 import __AS3__.vec.Vector;

 import flash.display.BitmapData;
 import flash.display.Shader;
 import flash.display.ShaderJob;
 import flash.utils.ByteArray;

 public class ShaderVector
 {
 [Embed("/rgb.pbj", mimeType="application/octet-stream")]
 private var shaderClass : Class;

 private var shader : Shader;

 public function ShaderVector()
 {        
 shader = new Shader(new shaderClass() as ByteArray);
 }

 public function convert( img : BitmapData ) : Vector.<Number>
 {
 var output : Vector.<Number> = new Vector.<Number>( (img.width*img.height) * 3 , false );
 shader.data.src.input = img;
 var job : ShaderJob = new ShaderJob( shader , output , img.width , img.height );
 job.start( true );
 output.fixed = true;
 return output;
 }

 }
}

Now we've got a kind of worker / utility class which we can farm all the hard work on to; usage is simple:

var worker : ShaderVector = new ShaderVector();
var rgb : Vector.<Number> = worker.convert( someBitmapData );

that's it, in one line we've solved the problem.

PixelBender is a lot easier than you may think, the above code is a small model you can easily manipulate just embed different pixel bender files.

Here's another simple kernel file; this one will take a BitmapData of perlin noise and convert it in to Vector of 3D positions for us to use.

It takes the bitmap data, converts every pixels r,g,b values in to float -0.5 to +0.5 values, then scales them up and finally returns back a vector of what are essentially 3D Perlin Noise Particles (like the ones in my earlier experiments)

<languageVersion : 1.0;>
kernel Scaler
<   namespace : "org.webr3.pb";
 vendor : "WebR3";
 version : 1;
 description : "For Perlin Noise, Converts BitmapData to 3D x,y,z positions";
>
{
 input image3 src;
 output pixel3 dst;

 parameter float scaleX
 <
 defaultValue : float(1.0);
 minValue     : float(0.1);
 maxValue     : float(100000.0);
 >;

 parameter float scaleY
 <
 defaultValue : float(1.0);
 minValue     : float(0.1);
 maxValue     : float(100000.0);
 >;

 parameter float scaleZ
 <
 defaultValue : float(1.0);
 minValue     : float(0.000001);
 maxValue     : float(100000.0);
 >;

 void evaluatePixel()
 {
 dst = sampleNearest( src, outCoord() );
 dst -= 0.5;
 dst.r *= scaleX;
 dst.g *= scaleY;
 dst.b *= scaleZ;
 }
}

and the modified shader vector, only 3 new lines to set up the scale values:

package test
{
 import __AS3__.vec.Vector;

 import flash.display.BitmapData;
 import flash.display.Shader;
 import flash.display.ShaderJob;
 import flash.utils.ByteArray;

 public class ShaderVector
 {
 [Embed("/rgb.pbj", mimeType="application/octet-stream")]
 private var shaderClass : Class;

 private var shader : Shader;

 public function ShaderVector()
 {        
 shader = new Shader(new shaderClass() as ByteArray);
 shader.data.scaleX.value = [200];
 shader.data.scaleY.value = [200];
 shader.data.scaleZ.value = [200];
 }

 public function convert( img : BitmapData ) : Vector.<Number>
 {
 var output : Vector.<Number> = new Vector.<Number>( (img.width*img.height) * 3 , false );
 shader.data.src.input = img;
 var job : ShaderJob = new ShaderJob( shader , output , img.width , img.height );
 job.start( true );
 output.fixed = true;
 return output;
 }

 }
}

If you're thinking this is complex, it's honestly not - try coding this in pure AS3 and it'll be a lot of complex slow lines, been there and done that!

I'm going to leave the examples there for now, if your looking for a 3D example then check the PixelBender files in Ralphs 3D Particle example.

Conclusion.

AS3 together with PixelBender, haXe and Alchemy is fast becoming an immense tech for us developers, have a play around you may be suprised by the results you can achieve by utilizing these techs.

Coming very soon I'll be posting some examples which show what can be done when you use PixelBender, Fast Memory, Vectors and haXe / AS3 together, but in unconventional ways. Until then I hope this is enough to get some more dev's playing with PixelBender.

peace ::

image is a still from I am Legend : Awakening a very well made 5 minute short animation.