Carve Up a Video in Real Time With AS3

Hello, code freaks! This tutorial will show you how to split a running video into blocks as if it has exploded. And all this using just ActionScript. For this tutorial we'll use the camera as the video source, so you can see the changes live.

Final Result Preview

Let's take a look at the final result we will be working towards:

Click and drag a block to move it around the screen! (Camera access required.)

Step 1: Setup - IDE

For this tutorial we'll be using the FlashDevelop IDE (though you could use any AS3 editor). In case you don't have it and want to try, you can grab it from here. A basic tutorial on setting up FlashDevelop on you machine can be found here.

Also if you have Flash Professional installed on your side, that will work too. All you need to do is create an external class file as mentioned below and link it to your Flash project as a Document class.

Making an external class as Document class

That sets up our working environment.

Step 2: Setup - New Project

Create a new AS3 project in FlashDevelop.

When its done, you will have a Main class created in the src folder as seen in the right panel :

Step 3: Setup -The Main class

Next we need to make the Main.as file a little cleaner by eliminating some code. Initially when you open the Main.as file, it would have code something like this :


package
{
	import flash.display.Sprite;
	import flash.events.Event;	

	public class Main extends Sprite 
	{				
		public function Main():void 
		{
			if (stage) init();
			else addEventListener(Event.ADDED_TO_STAGE, init);
		}
		
		private function init(e:Event = null):void 
		{
			removeEventListener(Event.ADDED_TO_STAGE, init);
			// entry point
		}				
	}
}

We'll delete some part of the code to make it look cleaner. So you should have this:


package
{
	import flash.display.Sprite;
	import flash.events.Event;	

	public class Main extends Sprite 
	{				
		public function Main():void 
		{			
		}				
	}
}

Now, all the setup is done and it's time to dive into some code.

Step 4: Declaring the Video and Camera Variables

Our first aim is to draw the video on the stage using the camera; for this we need to declare some variables. Put these declarations just above the Main class constructor.


// video variables
private var camW:int = 300;
private var camH:int = 300;
private var video:Video;

camW - The width of the camera/video.

camH - The height of the camera/video.

video - Our Video class instance.

Step 5: Prepare the Device Camera

As mentioned before, we'll use the camera output in the video. So first we need to make the device camera ready. The following code should go into the class constructor.

1
2	var camera:Camera = Camera.getCamera();

Here, we instantiate a Camera instance and get the available device camera using the static method getCamera() of the Camera class.

1
2	camera.setMode(camW, camH, 30);

We provide some camera settings: width, height and fps.

Note we did not make the camera variable global because we don't need to access it anywhere outside this function, as you'll see next. In the next step we initialize the video variable.

Step 6: Actually Create the Video!

1
2	video = new Video(camW, camH);
3	video.attachCamera(camera);

We've now instantiated the video variable and used the attachCamera() method to attach the camera to it. This means that now the video uses the camera output as its source.

All done with the video and camera stuff. Remember, you need to import appropriate classes in your code. Your complete class code should look like this at the moment:


package 
{
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.media.Camera;
	import flash.media.Video;
	
	public class Main extends Sprite 
	{
		// video variables
		private var camW:int = 300;
		private var camH:int = 300;
		private var video:Video;
		
		public function Main():void {
			var camera:Camera = Camera.getCamera();
			camera.setMode(camW, camH, 30);
			video = new Video(camW, camH);
			video.attachCamera(camera);
		}

	}	
}

If you run (F5 or CTRL+Enter) the project right now, it would be blank stage but you will most probably get a camera access request as the application is trying to access the device camera. Allow it.

The reason why you don't see anything is because we don't want to show the video and so we havn't added it to the stage (display list). It will be just used as a source for our separate blocks. If you want to test that everything is working fine, just add the following line at the end in the Main constructor :

1
2	addChild(video); // Do remove this line after testing

Step 7: Declaring the Block Variables

Now we create the blocks - the separate peices of the video. And the first step is to declare some variables required for it. So go ahead and add up the following variable declarations just below the video variables:


// block variables
private var rows:int = 3;
private var cols:int = 3;
private var blockW:int = camW/cols;
private var blockH:int = camH/rows;
private var pointCollection:Object = new Object();

rows - Number of rows to split the video in.

cols - Yes. You got that. Number of columns to split the video in.

blockW - Each block's width. This is a derived variables as it is simply calculated by the dividing the total width (camW) by number of colums (cols).

blockH - Each block's height i.e. camH divided by rows.

pointCollection - Final but most important variable. It is an associative array that we will be using to store the corresponding point of each block. For example, if a block has name block12, then we would store its corresponding point p12 like this :

1
2	pointCollection["block12"] = p12; // points are Point class instances here

Step 8: Start Making the Blocks

Now that we have the required variables defined, we actually start creating the blocks. We are going to keep all the block creation code in a function called initBlocks(). This function will be called from the Main constructor after setting the video.

So, let's first declare a function called initBlocks() just after the Main constructor.


private function initBlocks():void {
	for (var r:int = 0; r < rows; r++) {
		for (var c:int = 0; c < cols; c++) {
			// code to create each block
		}
	}
}

Notice the two for loops we have placed inside, which will help us create the blocks in a 2D grid, row-wise. And then add a call to this function at the end of Main():


public function Main():void {
	var camera:Camera = Camera.getCamera();
	camera.setMode(camW, camH, 30);
	video = new Video(camW, camH);
	video.attachCamera(camera);
	
	initBlocks();
}

Step 9: Components of a block

Before creating the blocks, let's understand what a single block is made up of. Every Block is actually:

A Sprite
with a Bitmap inside it
which in turn needs a BitmapData

Think of Sprite as the outermost container. Completely blank.

To draw the block, we need a Bitmap which will show the corresponding video output.

And finally, every Bitmap needs some data to draw inside it. That is given in the form of BitmapData.

Step 10: Create the Block's Base - Sprite

The first component of a block, as we discussed, is a Sprite. So lets create one. All the code that we write to create the block is to be written inside the for loops.

1
2	var newBlock:Sprite = new Sprite();

We create a new instance of the Sprite class.

1
2	newBlock.name = "block" + r + c;

Then we name the sprite so that we can reference it later in the code. The naming convention is simple: a block at row r and column c is named block + r + c (+ means concatenation). So a block at row 2 and column 1 is named block21.

Step 11: Positioning it

After having created it, we need to position it on the stage according to its row and column. So lets add the following code.

1
2	var p:Point = new Point(c * blockW, r * blockH);

We use a Point class object to store any point's co-ordinates here. And so we create a new instance of Point and pass c*blockW as the x-value and r*blockH as the y-value. Now the co-ordinates can be accessed simply as p.x and p.y and is used later to fetch each block's clippped region from a complete video frame. Remember each block's point is actually the co-ordinates of the top-left point in the grid.

If you have a doubt on how the position is calculated here, the following figure will make it clear.

1
2	newBlock.x = c * (blockW + 1) + 20;
3	newBlock.y = r * (blockH + 1) + 20;

Next, we position the sprite. The cordinates are more or less the same expect now we add 20 to give an offset. Also we add 1 to the blockW and blockH to separate the blocks by 1 pixel, as is visible in the demo above.

1
2	pointCollection[newBlock.name] = p;

Finally, we save the point we calculated earlier in the pointCollection.

Step 12: Adding the Bitmap to the Block

Now, coming to the 2nd and 3rd component of the block.

1
2	var bmpd:BitmapData = new BitmapData(blockW, blockH);

First we create a BitmapData instance and pass the required block width and height that we had stored before. As discussed earlier, a BitmapData instance is required to create a Bitmap instance which is passed in the constructor.

1
2	var bmp:Bitmap = new Bitmap(bmpd);
3	bmp.name = "myBmp";

Now, we create a Bitmap instance and pass the previously created bmpd in the constructor. Also, we name the bitmap myBmp so we can reference it later.

1
2	newBlock.addChild(bmp);
3	addChild(newBlock);

In the end we add the bitmap bmp as a child of newBlock and newBlock itself as the child of the stage.

Just to be sure that you are on the right track, your Main.as code should look something like this:


package 
{
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.display.Sprite;
	import flash.geom.Point;
	import flash.media.Camera;
	import flash.media.Video;
	
	public class Main extends Sprite 
	{
		// video variables
		private var camW:int = 300;
		private var camH:int = 300;
		private var video:Video;
		
		// block variables
		private var rows:int = 3;
		private var cols:int = 3;
		private var blockW:int = camW/cols;
		private var blockH:int = camH/rows;
		private var pointCollection:Array = new Array();
		
		public function Main():void {
			var camera:Camera = Camera.getCamera();
			camera.setMode(camW, camH, 30);
			video = new Video(camW, camH);
			video.attachCamera(camera);
			
			initBlocks();
		}
		
		private function initBlocks():void {
			for (var r:int = 0; r < rows; r++) {
				for (var c:int = 0; c < cols; c++) {
					var newBlock:Sprite = new Sprite();
					newBlock.name = "block" + r + c;
					var p:Point = new Point(c * blockW, r * blockH);
					newBlock.x = c * (blockW + 1) + 20;
					newBlock.y = r * (blockH + 1) + 20;
					pointCollection[newBlock.name] = p;
					
					var bmpd:BitmapData = new BitmapData(blockW, blockH);
					var bmp:Bitmap = new Bitmap(bmpd);
					bmp.name = "myBmp";
					
					newBlock.addChild(bmp);
					addChild(newBlock);
				}
			}
		}	

	}	
}

Step 13: Updating the Blocks - Concept

Even though we have the blocks placed in the right positions, we still don't see anything on running the project. That's because we still havn't drawn anything inside the block bitmaps.

Our next step is to run a constantly-running loop which performs the following operations:

Get the current video frame.
Loop through all the blocks.
Fetch each block's point and bitmap child.
Draw the corresponding part of the video frame on the block's bitmap.

So...let's do it!

Before implementing the loop code, what we need is a LOOP FUNCTION (kind of like a game loop). Add the following function declaration below the initBlocks() function :

1
2	private function updateBlocks(e:Event):void {
3	}

As is visible from the function parameter, it seems like an event listener and yes it is. This is a listener function that we will attach to the ENTER_FRAME event of the stage. To attach the listener, add this line at the end of the Main() constructor.


public function Main():void {
	var camera:Camera = Camera.getCamera();
	camera.setMode(camW, camH, 30);
	video = new Video(camW, camH);
	video.attachCamera(camera);
	
	initBlocks();
	addEventListener(Event.ENTER_FRAME, updateBlocks);
}

Step 14: Capture Frame

This is the first operation that we perform in our loop - the updateBlocks() function which is called on every frame. Put the following code inside updateBlocks() function.

1
2	var srcBmpd:BitmapData = new BitmapData(camW, camH);

Every bitmap's data in Actionscript 3.0 need to be contained in a BitmapData instance and so we create one. We will populate this instance with the current video frame data next.

1
2	srcBmpd.draw(video);

Here we have used the draw() function of the BitmapData class. It requires an object of the any class that implements IBitmapDrawable interface. For eg. Sprite, MovieClip, BitmapData etc. What it does is simply take the visual data of the object passed and stores it in the BitmapData instance.

So now we have the current video frame (or, you could say, a screenshot) in the variable srcBmpd.

Step 15: Let's Loop

As we need to update every block, we create a double for-loop, similar to the one we wrote for creating the blocks. So go ahead and add it.

The function should look similar to this right now:


private function updateBlocks(e:Event):void {
	var srcBmpd:BitmapData = new BitmapData(camW, camH);
	srcBmpd.draw(video);
	for (var r:int = 0; r < rows; r++) {
		for (var c:int = 0; c < cols; c++) {
			// update code here
		}
	}
}

Step 16: Retrieve the Block's Bitmap and Point

Remember that we named the blocks in a certain way while creating them so we could reference any block using its row and column number. That is what we'll use now to get each block's reference.

1
2	var b_mc:Sprite = this.getChildByName("block" + r + c) as Sprite;

We use the getChildByName function of the stage (this) which return a reference to an object whose name matches the string passed. Also we typecast it to Sprite class just to be sure that the returned object is a Sprite. Now the block reference is in the variable b_mc.

1
2	var bmp:Bitmap = b_mc.getChildByName("myBmp") as Bitmap;

In much the same way, we retrieve the reference to the bitmap that was added as a child of the block sprite.

1
2	var p:Point = pointCollection[b_mc.name];

Finally, we get the current block's (b_mc) co-ordinates from the array in which we stored it earlier using the block's name.

Step 17: Draw It!

Now that we have all the required information on what to draw where, we can actually DRAW it. Our motive here is to get the rectangular region of the video frame (i.e. srcBmpd) with the top-left point as the retrieved point p, width as blockW and height as blockH.

For this purpose we use the copyPixels() method of the BitmapData class. It actually copies the region of another source BitmapData specified by passing a Rectangle object.

1
2	bmp.bitmapData.copyPixels(srcBmpd, new Rectangle(p.x, p.y, blockW, blockH), new Point());

The draw() function is called on bmp's bitmapData property. The parameters passed into it are :

The source BitmapData obeject. The screenshot of the video in this case (srcBmpd).
A Rectangle object which specifies the top-left point, width and column of the region in the source to be copied.
The point in the destnation where the clipped portion is to be copied. (0,0) in this case. So we simply pass a new Point object.

All Done! Now it's time to run your project and see the awesome effect.

Step 18: Adding Drag-and-Drop Functionality

To add the drag-and-drop feature as seen in the demo, we just need to attach two mouse listeners to each block -- one for the MOUSE_DOWN event and another for the MOUSE_UP event. So go ahead and define two mouse handler functions at the end of the class:


private function onMouseDown(e:MouseEvent):void {
	Sprite(e.currentTarget).startDrag();
}

private function onMouseUp(e:MouseEvent):void {
	Sprite(e.currentTarget).stopDrag();
}

All we do inside these listener functions is get the reference to the event-dispatching block using the currentTarget property of the Event object, typecast it to a Sprite (as that's what our blocks are) and call the startDrag() and stopDrag() to handle the drag-and-drop.

That's not quite all yet. We still need to attach these listeners to their corresponding events. So add these two lines to the initBlocks() function.


private function initBlocks():void {
	for (var r:int = 0; r < rows; r++) {
		for (var c:int = 0; c < cols; c++) {
			var newBlock:Sprite = new Sprite();
			newBlock.name = "block" + r + c;
			var p:Point = new Point(c * blockW, r * blockH);
			newBlock.x = c * (blockW + 1) + 20;
			newBlock.y = r * (blockH + 1) + 20;
			pointCollection[newBlock.name] = p;
			
			var bmpd:BitmapData = new BitmapData(blockW, blockH);
			var bmp:Bitmap = new Bitmap(bmpd);
			bmp.name = "myBmp";
			
			newBlock.addChild(bmp);					
			addChild(newBlock);
			
			newBlock.addEventListener(MouseEvent.MOUSE_DOWN, onMouseDown);
			newBlock.addEventListener(MouseEvent.MOUSE_UP, onMouseUp);
		}
	}
}

Step 19: Final Touch

One final thing just to make it look more interactive. You may have noticed how the blocks fade in and out when pressed and released. That is an alpha manipulation we do inside the listeners. Modify your listeners to something like:


private function onMouseDown(e:MouseEvent):void {
	Sprite(e.currentTarget).alpha = 0.4;
	Sprite(e.currentTarget).startDrag();
}

private function onMouseUp(e:MouseEvent):void {
	Sprite(e.currentTarget).alpha = 1;
	Sprite(e.currentTarget).stopDrag();
}

And there you have the alpha change effect.

Conclusion

The effect has a lot of potential to be used in various applications. I have been developing a puzzle game recently using it.

Apart from that, it could be used to create transition effects for video players, or in conjuction with 3D to texture a surface with a video.

I'm hoping to see some cool stuff people come up with using this effect!