Skip to content

Category: english

pés no paraíso

Gostei dessa minha foto (tamanho original). Ficou com uma boa composição, dois planos, no plano da frente meus pés deitados na rede, o que dá uma sensação de imersão para o observador para se sentir no meu ponto de vista. No plano de fundo vários elementos, cada um contando uma história diferente, um cara jogando frescobol, uma jangada velejando, alguém remando num caiaque, uma menina nadando e um menino no raso fazendo uma onda. No meio disso a gradiente entre a água cristalina e o fundo azul. Não dá pra ver mas a rede fica armada entre dois coqueiros dentro da água.

Fotografia feita numa Canon Powershot G10, na Lagoa do Paraíso, perto de Jericoacoara, Ceará. Livre sobs os termos da Creative Commons Atribuição-Compartilhamento pela mesma Licença.

Miojo Script

O pre-requisito é o notify-send, um utilitário de linha de comando do libnotify. No Ubuntu:

sudo aptitude install libnotify-bin

E aqui o script em si:

sleep 5m; notify-send "aviso" "tirar o miojo do fogo"

Pronto, depois de cinco minutos isso vai aparecer:

Easily Sortable Date and Time Representation

I was looking for a date and time representation useful for registering stock quotes in a simple plain file.

I found that the standard ISO 8601 is just the answer for this, it’s called “Data elements and interchange formats — Information interchange — Representation of dates and times”. Here is a example:

2010-01-20 22:14:38

There’s this good article from Markus Kuhn, “A summary of the international standard date and time notation”. This notation allow us to using simple lexicographical order the events.

Some examples of how to do this in Python (thanks for the Jochen Voss article “Date and Time Representation in Python”) The first for displaying the current date and time:

from time import strftime
print strftime("%Y-%m-%d %H:%M:%S")
# 2010-01-20 22:34:22

Another possibility is using strftime from datetime object.

from datetime import datetime
now = datetime.datetime.now()
print now.strftime("%Y-%m-%d %H:%M:%S")
# 2010-01-20 22:12:31

Is that. Using this notation in the begging of each line is easy to sort them in any language or using the unix sort.

Extração de Dados e Fundos de Investimento do Banco do Brasil

Eu não achei onde coletar os dados diários de rentabilidade dos fundos de investimento do Banco do Brasil em formato bem estruturado.

Num mundo ideal as coisas seriam assim, você faria uma requisição numa url como esta:

http://bb.com.br/apps/rentabilidade?fundo=Siderurgia&saida=xml

E ele cuspiria um XML com as informações da rentabilidade diária desse fundo, isso se eu não especificasse através de outro parâmetro qual a data ou intervalo de datas desejado ou outro tipo de dados para saída como YAML ou JSON. Mas por enquanto não temos isso, nem unicórnios, então temos de fazer as coisas do jeito mais difícil, que é puxando os dados feitos para humanos e escrevendo um programa pra extrair à força os dados que desejamos e quem sabe usar eles para algum uso relacionado a mineração de dados.

A primeira abordagem que eu tentei foi a de criar um desses pequenos parsers XML que eu já mostrei como fazer antes, mas o código fonte desse documento se mostrou muito incompatível com o XML que o parser estava disposto a trabalhar. A solução alternativa foi tratar o documento linha a linha.

import urllib

# abrimos o documento referenciado pela url
url = 'http://www21.bb.com.br/portalbb/rentabilidade/index.jsp?tipo=01'
documento = urllib.urlopen(url)

# fundo de investimento que me interessa
fundo = 'small caps'

# estados
INICIO = 0
ACHOU_FUNDO = 1
FIM = 2

# estado inicial
estado = INICIO

# vamos analisar linha a linha do fluxo do documento
for linha in documento:
	# simplificamos, tudo pra minusculas
	linha = linha.lower()

	# no inicio, procura uma linha que tenha o fundo
	if estado == INICIO and linha.find(fundo) != -1:
		estado = ACHOU_FUNDO

	# depois, procuramos o proximo inicio de tabela html.
	# dessa linha, pegamos o que vem depois do primeiro >
	# e entao o que vem antes do primeiro <
	# e trocamos a virgula por ponto.
	elif estado == ACHOU_FUNDO and linha.find('>')[1].split('<')[0].replace(',','.')
		estado = FIM

E para usar:

$ python rendimento_small_caps.py
0.881

Geralmente estamos mais interessados em saber o valor da cota daquele fundo, daí podemos calcular o rendimento total sabendo a cota que compramos a ação inicialmente. Nesse caso o dado está na 11º coluna.

import urllib
 
# abrimos o documento referenciado pela url
url = 'http://www21.bb.com.br/portalbb/rentabilidade/index.jsp?tipo=01'
documento = urllib.urlopen(url)
 
# fundo de investimento que me interessa
fundo = 'small caps'
 
# estados
INICIO = 0
ACHOU_FUNDO = 1
FIM = 2
 
# estado inicial
estado = INICIO
coluna = 0
 
# vamos analisar linha a linha do fluxo do documento
for linha in documento:
	# simplificamos, tudo pra minusculas
	linha = linha.lower()
 
	# no inicio, procura uma linha que tenha o fundo
	if estado == INICIO and linha.find(fundo) != -1:
		estado = ACHOU_FUNDO
 
	# para cada coluna, conta a coluna, mas nao faz nada
	elif estado == ACHOU_FUNDO and linha.find('<'):
		coluna += 1
 
	# quando chegar na coluna onze, retira o conteudo entre os sinais > e <
	# e troca virgula por ponto, transforma em float e joga na tela
	if estado==ACHOU_FUNDO and coluna == 11:
		print float(linha.split('>')[1].split('<')[0].replace(',','.'))
		estado = FIM

$ python cota_small_caps.py
6.156906634

Essa é uma abordagem que eu não gosto nem recomendo porque ela é muito frágil e está extremamente acoplada a formatação de dados para humanos. Esta formatação está interessada no saída gráfica que o usuário vai obter e não em facilitar a extração (não humana) desses dados. Isso torna a solução muito frágil:

  • Se mudarem os nomes internos dos elementos, a solução pode falhar.
  • Se mudarem a formatação da tabela, a solução pode falhar.
  • Se mudarem a disposição interna dos elementos html, a solução pode falhar.
  • Se mudarem a url do documento, a solução vai falhar.
  • Se o documento não puder mais ser tratado linha a linha, a solução vai falhar feio.

É provável que quando você estiver lendo isso ela nem funcione mais do jeito que está descrita aqui.

Por outro lado, a solução funciona e nesse caso é o que me interessa. Quando ela quebrar, se ainda for do meu interesse eu posso rapidamente conserta-la e os dados já coletados no passado continuam válidos.

Isso somado  a uma programa como o Cron pode se tornar uma ferramenta realmente poderosa.

Java Font List

Here’s a program that lists fonts available in your JVM. You can also set the environment variable JAVA_FONTS to specify the font directory.

import java.awt.GraphicsEnvironment;

public class ListFonts {
	public static void main(String args[]){
		GraphicsEnvironment e = GraphicsEnvironment.getLocalGraphicsEnvironment();
		for(String font:e.getAvailableFontFamilyNames()){
			System.out.println(font);
		}
	}
}

By using pipes you can count how many fonts you have:

java ListFonts|wc -l

On my Ubuntu machine here I got 556 because I use those excellent, free and indispensable Larabie Fonts.

For looking up for a font with “sans” in its name, using a case insensitive grep:

java ListFonts|grep -i “sans”

I get a list like this:

DejaVu Sans
DejaVu Sans Condensed
DejaVu Sans Light
DejaVu Sans Mono
FreeSans

Gimp Small Miracles

Some times a good picture is ruined by a little detail or someone that shouldn’t be there. If you are willing to trade a complete portraying of the reality for a better photo, here are some tips for Gimp. Use them wisely.

Here is the original photo of two friends playing a game, and an unwanted stranger raising his arms right in the middle of the picture.

Here is the most useful tools from the Gimp tool box and how to use them:

Clone tool. The most common tool to create remove unwanted things. Usually you use them by getting a background theme and cloning it to cover what you want to hide. The best way to use this tool is when you have a not uniform background, like a wall or grass, where a small pattern repeats.

Free Selection (Lasso). When the clone tool is not enough because patterns are too big or too shapeless, the best is use the lasso tool. Get a good piece of the background or another object and them copy and paste over what you want to hide. Maybe what you need to select and copy is not in the same image you are working, you can get information from another picture and use in your main picture. This was what I had to do in this example.

Sometimes you have to change the brightness and contrast to match the pasted selection with the main picture.

Perpective tool. This is a very powerful tool because allows you to use a selection from another image but was taken from another point of view and you need to correct it’s perspective. By doing that you can retrieve a lot of information that your picture don’t have by taking them from another pictures. You can also use to fit a pattern.

Smudge tool. When you are handling uniform colors and straight shapes this is a good tool. You can smudge a pasted selection in the boundaries to it match with the main picture. You can also use it to stretch or squeeze shapes, but they have to be very color uniform to it work well. For more complex shapes you can use the IWarp filter.

So, I guess this is the most important about them. Another time I can do some videos showing how to use them better. In this work you can spot several little mistakes because I was very clumsy and in hurry, but the result is satisfactory.

Before

After

Next time ask people to get out of the way when taking a picture.

Python Fast XML Parsing

Here is a useful tip on Python XML decoding.

I was extending xml.sax.ContentHandler class in a example to decode maps for a Pygame application when my connection went down and I noticed that the program stop working raising a exception regarded a call to urlib (a module for retrieve resources by url). I noticed that the module was getting the remote DTD schema to validate the XML.


This is not a requirement for my applications and it’s a huge performance overhead when works (almost 1 second for each map loaded) and when the applications is running in a environment without Internet it just waits for almost a minute and then fail with the remain decoding. A dirty workaround is open the XML file and get rid of the line containing the DTD reference.

But the correct way to programming XML decoding when we are not concerned on validate a XML schema is just the xml.parsers.expat. Instead of using a interface you just have to set some callback functions with the behaviors we want. This is a example from the documentation:

import xml.parsers.expat

# 3 handler functions
def start_element(name, attrs):
    print 'Start element:', name, attrs
def end_element(name):
    print 'End element:', name
def char_data(data):
    print 'Character data:', repr(data)

p = xml.parsers.expat.ParserCreate()

p.StartElementHandler = start_element
p.EndElementHandler = end_element
p.CharacterDataHandler = char_data

p.Parse("""
Text goes here
More text
""", 1)

The output:

Start element: parent {'id': 'top'}
Start element: child1 {'name': 'paul'}
Character data: 'Text goes here'
End element: child1
Character data: '\n'
Start element: child2 {'name': 'fred'}
Character data: 'More text'
End element: child2
Character data: '\n'
End element: parent

OpenCV on Ubuntu

digital_eye

Open Computer Vision Library or just OpenCV, is a cross-platform computer vision library focused on real-time image processing for video files or webcams.

You have two options to obtain the environment to develop on OpenCV. You can insert a new repository in your package manager or compile it by yourself.

For Ubuntu 9.10 Karmic Koala there’s this repository with OpenCV’s package.

To compile it you have to install some additional libraries compile it by your self. And it’s instructions vary for each distribution and version. For example, from Ubuntu Linux 9.10 to 9.04, the process varies slightly. I followed the instructions on this post “Installing OpenCV 2.0 on Ubuntu 9.10 Karmic Koala”.

After you have installed and have a well configured OpenCV development environment, you can compile a “source.c” file into a “program” binary like this:

gcc gcc source.c -o program `pkg-config opencv ‑‑libs ‑‑cflags`

Tiled TMX Map Loader for Pygame

I’m using the Tiled Map Editor for a while, I even wrote that tutorial about it. It’s a general purpose tile map editor, written in Java but now migrating to C++ with Qt, that can be easily used with my set of free pixelart tiles.

map editor tiles tileset game deveopment

A map done with Tiled is stored in a file with TMX extension. It’s just a XML file, easy to understand.

As I’m creating a map loader for my owns purposes, the procedure I’m doing here works we need some simplifications. I’m handling orthogonal maps only. I’m not supporting tile properties as well. I also don’t want to handle base64 and zlib encoding in this version, so in the Tiled editor, go at the menu Edit → Preferences and in the Saving tab unmark the options “Use binary encoding” and “Compress Layer Data (gzip)”, like this:

Tiled Preferences Window

When saving a map it will produce a TMX file like this:




 
  
  
 
 
  
 
 
  
   
   
    ...
   
   
  
 

For processing it on Python I’m using the event oriented SAX approach for XML. So I create a ContentHandler that handles events the start and end of XML elements. In the first element, map, I know enough to create a Pygame surface with the correct size. I’m also storing the map properties so I can use it later for add some logics or effects on the map. After that we create a instance of the Tileset class from where we will get the each tile by an gid number. Each layer has it’s a bunch of gids in the correct order. So it’s enough information to mount and draw a map.

# Author: Silveira Neto
# License: GPLv3
import sys, pygame
from pygame.locals import *
from pygame import Rect
from xml import sax

class Tileset:
    def __init__(self, file, tile_width, tile_height):
        image = pygame.image.load(file).convert_alpha()
        if not image:
            print "Error creating new Tileset: file %s not found" % file
        self.tile_width = tile_width
        self.tile_height = tile_height
        self.tiles = []
        for line in xrange(image.get_height()/self.tile_height):
            for column in xrange(image.get_width()/self.tile_width):
                pos = Rect(
                        column*self.tile_width,
                        line*self.tile_height,
                        self.tile_width,
                        self.tile_height )
                self.tiles.append(image.subsurface(pos))

    def get_tile(self, gid):
        return self.tiles[gid]

class TMXHandler(sax.ContentHandler):
    def __init__(self):
        self.width = 0
        self.height = 0
        self.tile_width = 0
        self.tile_height = 0
        self.columns = 0
        self.lines  = 0
        self.properties = {}
        self.image = None
        self.tileset = None

    def startElement(self, name, attrs):
        # get most general map informations and create a surface
        if name == 'map':
            self.columns = int(attrs.get('width', None))
            self.lines  = int(attrs.get('height', None))
            self.tile_width = int(attrs.get('tilewidth', None))
            self.tile_height = int(attrs.get('tileheight', None))
            self.width = self.columns * self.tile_width
            self.height = self.lines * self.tile_height
            self.image = pygame.Surface([self.width, self.height]).convert()
        # create a tileset
        elif name=="image":
            source = attrs.get('source', None)
            self.tileset = Tileset(source, self.tile_width, self.tile_height)
        # store additional properties.
        elif name == 'property':
            self.properties[attrs.get('name', None)] = attrs.get('value', None)
        # starting counting
        elif name == 'layer':
            self.line = 0
            self.column = 0
        # get information of each tile and put on the surface using the tileset
        elif name == 'tile':
            gid = int(attrs.get('gid', None)) - 1
            if gid <0: gid = 0
            tile = self.tileset.get_tile(gid)
            pos = (self.column*self.tile_width, self.line*self.tile_height)
            self.image.blit(tile, pos)

            self.column += 1
            if(self.column>=self.columns):
                self.column = 0
                self.line += 1

    # just for debugging
    def endDocument(self):
        print self.width, self.height, self.tile_width, self.tile_height
        print self.properties
        print self.image

def main():
    if(len(sys.argv)!=2):
        print 'Usage:\n\t{0} filename'.format(sys.argv[0])
        sys.exit(2)
    pygame.init()
    screen = pygame.display.set_mode((800, 480))
    parser = sax.make_parser()
    tmxhandler = TMXHandler()
    parser.setContentHandler(tmxhandler)
    parser.parse(sys.argv[1])
    while 1:
        for event in pygame.event.get():
            if event.type == QUIT:
                return
            elif event.type == KEYDOWN and event.key == K_ESCAPE:
                return
        screen.fill((255,255,255))
        screen.blit(tmxhandler.image, (0,0))
        pygame.display.flip()
        pygame.time.delay(1000/60)

if __name__ == "__main__": main()

Here is the result for opening a four layers map file:

netbeans python openning map

That’s it. You can get this code and adapt for your game because next versions will be a lot more coupled for my own purposes and not so general.

Download:packagemaploader.tar.bz2 It’s the Netbeans 6.7 (Python EA 2) project file but that can be opened or used with another IDE or without one. Also contains the village.tmx map and the tileset.