Skip to content

Category: english

OpenCV on Ubuntu

digital_eye

Open Computer Vision Library or just OpenCV, is a cross-platform computer vision library focused on real-time image processing for video files or webcams.

You have two options to obtain the environment to develop on OpenCV. You can insert a new repository in your package manager or compile it by yourself.

For Ubuntu 9.10 Karmic Koala there’s this repository with OpenCV’s package.

To compile it you have to install some additional libraries compile it by your self. And it’s instructions vary for each distribution and version. For example, from Ubuntu Linux 9.10 to 9.04, the process varies slightly. I followed the instructions on this post “Installing OpenCV 2.0 on Ubuntu 9.10 Karmic Koala”.

After you have installed and have a well configured OpenCV development environment, you can compile a “source.c” file into a “program” binary like this:

gcc gcc source.c -o program `pkg-config opencv ‑‑libs ‑‑cflags`

Tiled TMX Map Loader for Pygame

I’m using the Tiled Map Editor for a while, I even wrote that tutorial about it. It’s a general purpose tile map editor, written in Java but now migrating to C++ with Qt, that can be easily used with my set of free pixelart tiles.

map editor tiles tileset game deveopment

A map done with Tiled is stored in a file with TMX extension. It’s just a XML file, easy to understand.

As I’m creating a map loader for my owns purposes, the procedure I’m doing here works we need some simplifications. I’m handling orthogonal maps only. I’m not supporting tile properties as well. I also don’t want to handle base64 and zlib encoding in this version, so in the Tiled editor, go at the menu Edit → Preferences and in the Saving tab unmark the options “Use binary encoding” and “Compress Layer Data (gzip)”, like this:

Tiled Preferences Window

When saving a map it will produce a TMX file like this:




 
  
  
 
 
  
 
 
  
   
   
    ...
   
   
  
 

For processing it on Python I’m using the event oriented SAX approach for XML. So I create a ContentHandler that handles events the start and end of XML elements. In the first element, map, I know enough to create a Pygame surface with the correct size. I’m also storing the map properties so I can use it later for add some logics or effects on the map. After that we create a instance of the Tileset class from where we will get the each tile by an gid number. Each layer has it’s a bunch of gids in the correct order. So it’s enough information to mount and draw a map.

# Author: Silveira Neto
# License: GPLv3
import sys, pygame
from pygame.locals import *
from pygame import Rect
from xml import sax

class Tileset:
    def __init__(self, file, tile_width, tile_height):
        image = pygame.image.load(file).convert_alpha()
        if not image:
            print "Error creating new Tileset: file %s not found" % file
        self.tile_width = tile_width
        self.tile_height = tile_height
        self.tiles = []
        for line in xrange(image.get_height()/self.tile_height):
            for column in xrange(image.get_width()/self.tile_width):
                pos = Rect(
                        column*self.tile_width,
                        line*self.tile_height,
                        self.tile_width,
                        self.tile_height )
                self.tiles.append(image.subsurface(pos))

    def get_tile(self, gid):
        return self.tiles[gid]

class TMXHandler(sax.ContentHandler):
    def __init__(self):
        self.width = 0
        self.height = 0
        self.tile_width = 0
        self.tile_height = 0
        self.columns = 0
        self.lines  = 0
        self.properties = {}
        self.image = None
        self.tileset = None

    def startElement(self, name, attrs):
        # get most general map informations and create a surface
        if name == 'map':
            self.columns = int(attrs.get('width', None))
            self.lines  = int(attrs.get('height', None))
            self.tile_width = int(attrs.get('tilewidth', None))
            self.tile_height = int(attrs.get('tileheight', None))
            self.width = self.columns * self.tile_width
            self.height = self.lines * self.tile_height
            self.image = pygame.Surface([self.width, self.height]).convert()
        # create a tileset
        elif name=="image":
            source = attrs.get('source', None)
            self.tileset = Tileset(source, self.tile_width, self.tile_height)
        # store additional properties.
        elif name == 'property':
            self.properties[attrs.get('name', None)] = attrs.get('value', None)
        # starting counting
        elif name == 'layer':
            self.line = 0
            self.column = 0
        # get information of each tile and put on the surface using the tileset
        elif name == 'tile':
            gid = int(attrs.get('gid', None)) - 1
            if gid <0: gid = 0
            tile = self.tileset.get_tile(gid)
            pos = (self.column*self.tile_width, self.line*self.tile_height)
            self.image.blit(tile, pos)

            self.column += 1
            if(self.column>=self.columns):
                self.column = 0
                self.line += 1

    # just for debugging
    def endDocument(self):
        print self.width, self.height, self.tile_width, self.tile_height
        print self.properties
        print self.image

def main():
    if(len(sys.argv)!=2):
        print 'Usage:\n\t{0} filename'.format(sys.argv[0])
        sys.exit(2)
    pygame.init()
    screen = pygame.display.set_mode((800, 480))
    parser = sax.make_parser()
    tmxhandler = TMXHandler()
    parser.setContentHandler(tmxhandler)
    parser.parse(sys.argv[1])
    while 1:
        for event in pygame.event.get():
            if event.type == QUIT:
                return
            elif event.type == KEYDOWN and event.key == K_ESCAPE:
                return
        screen.fill((255,255,255))
        screen.blit(tmxhandler.image, (0,0))
        pygame.display.flip()
        pygame.time.delay(1000/60)

if __name__ == "__main__": main()

Here is the result for opening a four layers map file:

netbeans python openning map

That’s it. You can get this code and adapt for your game because next versions will be a lot more coupled for my own purposes and not so general.

Download:packagemaploader.tar.bz2 It’s the Netbeans 6.7 (Python EA 2) project file but that can be opened or used with another IDE or without one. Also contains the village.tmx map and the tileset.

Pygame: Running Orcs

Here is a Pygame Sprite animation using the approach presented by Joe Wreschnig and Nicolas Crovatti. It’s not yet exactly what I need but is very suitable.

import pygame, random
from pygame.locals import *

class Char(pygame.sprite.Sprite):
	x,y = (100,0)
	def __init__(self, img, frames=1, modes=1, w=32, h=32, fps=3):
		pygame.sprite.Sprite.__init__(self)
		original_width, original_height = img.get_size()
		self._w = w
		self._h = h
		self._framelist = []
		for i in xrange(int(original_width/w)):
			self._framelist.append(img.subsurface((i*w,0,w,h)))
		self.image = self._framelist[0]		
		self._start = pygame.time.get_ticks()
		self._delay = 1000 / fps
		self._last_update = 0
		self._frame = 0
		self.update(pygame.time.get_ticks(), 100, 100)	

	def set_pos(self, x, y):
		self.x = x
		self.y = y

	def get_pos(self):
		return (self.x,self.y)

	def update(self, t, width, height):
		# postion
		self.y+=1
		if(self.y>width):
			self.x = random.randint(0,height-self._w)
			self.y = -self._h

		# animation
		if t - self._last_update > self._delay:
			self._frame += 1
			if self._frame >= len(self._framelist):
				self._frame = 0
			self.image = self._framelist[self._frame]
			self._last_update = t

SCREEN_W, SCREEN_H = (320, 320)

def main():
	pygame.init()
	screen = pygame.display.set_mode((SCREEN_W, SCREEN_H))
	background = pygame.image.load("field.png")
	img_orc = pygame.image.load("orc.png")
	orc = Char(img_orc, 4, 1, 32, 48)
	while pygame.event.poll().type != KEYDOWN:
		screen.blit(background, (0,0))
		screen.blit(orc.image,  orc.get_pos())
		orc.update(pygame.time.get_ticks(), SCREEN_W, SCREEN_H)
		pygame.display.update()
		pygame.time.delay(10)

if __name__ == '__main__': main()

Here is it working:

Uptade: I put this source and images at the OpenPixel project in Github

Pygame Simple Key Handling

Here’s a simple key handle in Pygame wheres you move a circle using keyboard.

import pygame
from pygame.locals import *

def main():
	x,y = (100,100)
	pygame.init()
	screen = pygame.display.set_mode((400, 400))
	while 1:
		pygame.time.delay(1000/60)
      # exit handle
		for event in pygame.event.get():
			if event.type == QUIT:
				return
			elif event.type == KEYDOWN and event.key == K_ESCAPE:
				return

      # keys handle 
		key=pygame.key.get_pressed()
		if key[K_LEFT]:
			x-=1
		if key[K_RIGHT]:
			x+=1
		if key[K_UP]:
			y-=1
		if key[K_DOWN]:
			y+=1

		# fill background and draw a white circle
		screen.fill((255,255,255))
		pygame.draw.circle(screen, (0,0,0), [x,y], 30)
		pygame.display.flip()

if __name__ == '__main__': main()

Here’s a video of it working:

Function pygame.key.get_pressed Returns a sequence of boolean values representing the state of every key on the keyboard. It’s very useful because usually on others game platforms I have to create it by myself.

This approach allow me to handle more than one key at time. For example, left and up keys can be pressed and each one is handled separately creating a diagonal movement.

OpenCV: Hue, Saturation and Value.

Here a simple OpenCV example of separation of a image into its hue, saturation and value channels.

#include 
#include 
#include 

int main( int argc, char **argv ){
    IplImage *img, *hsv, *hue, *sat, *val;
    int key = 0, depth;
    CvSize size;

    /* Load command line passed image, check it. */
    if (argc>1) {
        img = cvLoadImage(argv[1], CV_LOAD_IMAGE_COLOR);
        if(!img){
            printf("Could not open image.");
            exit -1;
        }
        if(img->nChannels!=3){
            printf("We need color image!");
            exit -1;
        }
    } else {
        printf("Usage: %s VIDEO_FILE\n", argv[0]);
        return 1;
    }

    /* Create a hsv image with 3 channels and hue, sat e val with 1 channel.
       All with the same size */
    size = cvGetSize(img);
    depth = img->depth;
    hue = cvCreateImage(size, depth, 1);
    sat = cvCreateImage(size, depth, 1);
    val = cvCreateImage(size, depth, 1);
    hsv = cvCreateImage(size, depth, 3);
    cvZero(hue);
    cvZero(sat);
    cvZero(val);
    cvZero(hsv);

    /* Convert from Red-Green-Blue to Hue-Saturation-Value */
    cvCvtColor( img, hsv, CV_BGR2HSV );

    /* Split hue, saturation and value of hsv on them */
    cvSplit(hsv, hue, sat, val, 0);
    
    /* Create windows, display them, wait for a key */
    cvNamedWindow("original", CV_WINDOW_AUTOSIZE);
    cvNamedWindow("hue", CV_WINDOW_AUTOSIZE);
    cvNamedWindow("saturation", CV_WINDOW_AUTOSIZE);
    cvNamedWindow("value", CV_WINDOW_AUTOSIZE);

    cvShowImage("original", img);
    cvShowImage("hue", hue);
    cvShowImage("saturation", sat);
    cvShowImage("value", val);

    cvWaitKey(0);

    /* Free memory and get out */
    cvDestroyWindow("original");
    cvDestroyWindow("hue");
    cvDestroyWindow("saturation");
    cvDestroyWindow("value");

    cvReleaseImage(&img);
    cvReleaseImage(&hsv);
    cvReleaseImage(&hue);
    cvReleaseImage(&sat);
    cvReleaseImage(&val);

    return 0;
}

Resized original image, photo by Robert Bradshaw at Wikimedia Commons.

Sunset Peter Iredale

Hue channel:

hue sunset Peter Iredale

Saturation channel:

saturation sunset Peter Iredale

Value channel:

value sunset Peter Iredale

OpenCV: adding two images

This is a very simple example of how to open two images and display them added.

I got two pictures at project Commons from Wikimedia that were highlighted on Featured Pictures. I did a crop on both to have the same size, as I’m trying to make this example as simple as possible.

The first one is a photo of our Milky Way, taken at Paranal Observatory by Stéphane Guisard.

milkyway

The second one is a California surfer inside wave, taken by Mila Zinkova.

surfer

In this simple OpenCV code below, we open the images, create a new one to display the result and use cvAdd to add them. We do not save the result or handle more than the ordinary case of two images with the same size.

#include 
#include 
#include 

int main( int argc, char **argv ){
    IplImage *surfer, *milkyway, *result;
    int key = 0;
    CvSize size;

    /* load images, check, get size (both should have the same) */
    surfer = cvLoadImage("surfer.jpg", CV_LOAD_IMAGE_COLOR);
    milkyway = cvLoadImage("milkyway.jpg", CV_LOAD_IMAGE_COLOR);
    if((!surfer)||(!milkyway)){
        printf("Could not open one or more images.");
        exit -1;
    }
    size = cvGetSize(surfer);

    /* create a empty image, same size, depth and channels of others */
    result = cvCreateImage(size, surfer->depth, surfer->nChannels);
    cvZero(result);

    /* result = surfer + milkyway (NULL mask)*/
    cvAdd(surfer, milkyway, result, NULL);
 
    /* create a window, display the result, wait for a key */
    cvNamedWindow("example", CV_WINDOW_AUTOSIZE);
    cvShowImage("example", result);
    cvWaitKey(0);

    /* free memory and get out */
    cvDestroyWindow("example");
    cvReleaseImage(&surfer);
    cvReleaseImage(&milkyway);
    cvReleaseImage(&result);
    return 0;
}

/* gcc add.c -o add `pkg-config opencv --libs --cflags` */

Compile it (on a well configured OpenCV development environment) and run it:

gcc add.c -o add `pkg-config opencv –libs –cflags`
./add

The result got pretty cool, a milky way surfer.

surfer in the milk way

OpenCV: Edge Detection

This is a simple example of how pass edge detection in a video using OpenCV. It uses the built-in OpenCV Canny edge detector algorithm.

#include 
#include 
#include 

int main(int argc, char *argv[]) {
    int delay = 0, key=0, i=0;
    char *window_name;
    CvCapture *video = NULL;
    IplImage  *frame = NULL;
    IplImage  *grey  = NULL;
    IplImage  *edges = NULL;

    /* check for video file passed by command line */
    if (argc>1) {
        video = cvCaptureFromFile(argv[1]);
    } else {
        printf("Usage: %s VIDEO_FILE\n", argv[0]);
        return 1;
    }

    /* check file was correctly opened */
    if (!video) {
        printf("Unable to open \"%s\"\n", argv[1]);
        return 1;
    }

    /* create a video window with same name of the video file, auto sized */
    window_name = argv[1];
    cvNamedWindow(window_name, CV_WINDOW_AUTOSIZE);

    /* Get the first frame and create a edges image with the same size */
    frame = cvQueryFrame(video);
    grey  = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1);
    edges = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1);

    /* calculate the delay between each frame and display video's FPS */
    printf("%2.2f FPS\n", cvGetCaptureProperty(video, CV_CAP_PROP_FPS));
    delay = (int) (1000/cvGetCaptureProperty(video, CV_CAP_PROP_FPS));

    while (frame) {
	/* Edges on the input gray image (needs to be grayscale) using the Canny algorithm.
           Uses two threshold and a aperture parameter for Sobel operator. */
        cvCvtColor(frame, grey, CV_BGR2GRAY);
        cvCanny( grey, edges, 1.0, 1.0, 3);

	/* show loaded frame */
        cvShowImage(window_name, edges);
	
	/* load and check next frame*/
        frame = cvQueryFrame(video);
	if(!frame) {
		printf("error loading frame.\n");
		return 1;
	}

	/* wait delay and check for the quit key */
        key = cvWaitKey(delay);
        if(key=='q') break;
    }
}

To compile it in a well configured OpenCV development environment:

gcc edgeplayer.c -o edgeplayer `pkg-config opencv –libs –cflags`

To run it call edgeplayer and the name of the video:

./edgeplayer rick.avi

The result is something similar to this:

rick roll edge

Simple Face Detection Player

Here’s a simple video player that also performs facial detection thought the Open Computer Vision Library.

Here’s a code developed using codes from nashruddin.com and samples from OpenCV, including the haar classifier xml. More detailed explanation on the theory about how the OpenCV face detection algorithm works can be found here.

The code:

#include 
#include 
#include 

CvHaarClassifierCascade *cascade;
CvMemStorage *storage;

int main(int argc, char *argv[]) {
    CvCapture *video = NULL;
    IplImage *frame = NULL;
    int delay = 0, key, i=0;
    char *window_name = "Video";
    char *cascadefile = "haarcascade_frontalface_alt.xml";

    /* check for video file passed by command line */
    if (argc>1) {
        video = cvCaptureFromFile(argv[1]);
    }
    else {
        printf("Usage: %s VIDEO_FILE\n", argv[0]);
        return 1;
    }

    /* check file was correctly opened */
    if (!video) {
        printf("Unable to open \"%s\"\n", argv[1]);
        return 1;
    }

    /* load the classifier */
    cascade = ( CvHaarClassifierCascade* )cvLoad( cascadefile, 0, 0, 0 );
    if(!cascade){
        printf("Error loading the classifier.");
	return 1;
    }

    /* setup the memory buffer for the face detector */
    storage = cvCreateMemStorage( 0 );
    if(!storage){
        printf("Error creating the memory storage.");
	return 1;
    }

    /* create a video window, auto size */
    cvNamedWindow(window_name, CV_WINDOW_AUTOSIZE);

    /* get a frame. Necessary for use the cvGetCaptureProperty */
    frame = cvQueryFrame(video);

    /* calculate the delay between each frame and display video's FPS */
    printf("%2.2f FPS\n", cvGetCaptureProperty(video, CV_CAP_PROP_FPS));
    delay = (int) (1000/cvGetCaptureProperty(video, CV_CAP_PROP_FPS));

    while (frame) {
	/* show loaded frame */
        cvShowImage(window_name, frame);
	
	/* wait delay and check for the quit key */
        key = cvWaitKey(delay);
        if(key=='q') break;
	/* load and check next frame*/
        frame = cvQueryFrame(video);
	if(!frame) {
		printf("error loading frame.\n");
		return 1;
	}

        /* detect faces */
        CvSeq *faces = cvHaarDetectObjects(
            frame, /* image to detect objects in */
            cascade, /* haar classifier cascade */
            storage, /* resultant sequence of the object candidate rectangles */
            1.1, /* increse window by 10% between the subsequent scans*/
            3, /* 3 neighbors makes up an object */
            0 /* flags CV_HAAR_DO_CANNY_PRUNNING */,
            cvSize( 40, 40 ) 
        );

        /* for each face found, draw a red box */
        for( i = 0 ; i < ( faces ? faces->total : 0 ) ; i++ ) {
             CvRect *r = ( CvRect* )cvGetSeqElem( faces, i );
             cvRectangle( frame,
                  cvPoint( r->x, r->y ),
                  cvPoint( r->x + r->width, r->y + r->height ),
                  CV_RGB( 255, 0, 0 ), 1, 8, 0 );
        }
    }
}

Yeah, I know the code needs a few adjustments. ¬¬

To compile it in a well configured OpenCV development environment:

gcc faceplayer.c -o faceplayer `pkg-config opencv ‑‑libs ‑‑cflags`

To run it you have to put in the same directory of the binary the XML classifier (haarcascade_frontalface_alt.xml) that comes with OpenCV sources at OpenCV-2.0.0/data/haarcascades/. And so:

./faceplayer video.avi

The results I got so far is that it works well for faces but sometimes its also detects more than faces. And here a video of it working live.

A example of good result:

rick roll face detection

A example of bad result:

rick roll face detection bad result

Maybe with some adjustments it could performs even better. But was really easy to create it using OpenCV.