Object tracking in Python

Some more progress. Since last time, I have discovered that there are always at least 3 ways of doing anything graphical in Python.

In this case, I was looking for ways of analysing an image and looking for specific objects. I'm doing this based on colour, as it seems easiest. Python has a function called Pixelarray which will take an image and convert it into a 2D array of individual pixel colours. For example:

Pixel[0][0] = (255,0,0) would mean that the pixel on row 0 and column 0 (top left of the image) is coloured red (the three numbers in the brackets represent the amount of red,  green and blue in the colour)

This is fine when you are wanting to change the colour of an individual pixel, but it doesn't work quite as well if you are wanting to READ the colours. For that you need some extra tweaking.

My program here has a small calibration sequence which waits for you to press space and then grabs and remembers whatever colour appears under the blue cross at the top of the screen.
Then it switches to a black-and-white display which discards all colours except the target colour.

Then, it starts tracking the object by searching for solid blocks of the target colour. Due to the grainy nature of cheap video cameras (especially under poor light conditions), lots of speckling occurs which may include colours close to the target colour. My program tries to discard those speckles by focusing on blocks where at least 90% of the pixels are the target colour. It essentially scans the whole image quickly, then when it finds a potential block it starts counting how many neighbouring pixels match.

There are software libraries that do this sort of thing much more cleverly, but I don't understand them. This seems to work for now.

The robot arm lives!

I got around to finishing off the robot arm today, and played around with the software that comes with it. It's not terribly easy to use, as it's quite difficult working out which motors to operate in order to move it where you want it to go. Also, I'm not very happy about the accuracy of the thing. Telling it to move a particular motor for 1 second in one direction followed by 1 second in the opposite direction does not result in it being back at the starting position. There's no feedback, so it can't tell where it is. I'll have to allow for the inaccuracies when programming it, which is a bit annoying. I know it was only 40 quid, but still.

 

Meanwhile the vision experiments are going well. I'm quite impressed by the speed of Python's manipulation methods. I should be able to knock up some interesting Augmented Reality things too.

Finally making some progress with the vision thing

Well, after a lot of messing around I stumbled across a far better way of doing the computer vision element for my OXO project. Pygame (which is a set of Python add-ons designed to handle graphics better than standard Python) has a camera module which, while it's not designed to work with Windows cameras, can be tweaked to do what I want far more quickly than the Python Image Library I was using last night. Now I can grab and analyse video at a much higher frame-rate, as you can see in this video:

I always feel like somebody's watching me

A tough night of tinkering. I decided to play around with the vision part of my OXO project. Specifically, I wanted to see if I could get Python to grab an image from a webcam and analyse it for objects.

I started off by googling "Python grab webcam image", which led me to install OpenCV, a high-end computer vision package. To be honest, I wasted loads of time playing with this, unable to get it to work.I couldn't find a binary, so I had to try to compile my own from source files, which is something I have no real experience of doing. After several frustrating hours I gave up and tried a different tack.

Pythom has some built-in image analysis libraries called PIL, which does some of the work for you, and there's a third-party add-on called VideoCapture which will allow you access your PC's webcam.

A bit of fiddling around and I eventually managed to get a way of quickly analysing all the pixels in an image and identifiying the ones I'm interested in. Here, for example, is it finding similar skin colours in a snapshot of me:

So, with this in mind it should not be too hard to identify a nought or cross on a sheet of paper, especially if it's a different colour.

Now it's half past midnight, so I must go to bed.

Arm news

Oh, the robot arm is mostly built, by the way. Whether it will work or not remains to be seen.

Game avoidance

I'm on holiday, which used to mean I had long, lazy days entirely at my disposal, unfettered by the need to offer entertainment, sustenance and attention to my delightful children.  Not any more. When I am off school, so are my kids. Work just becomes a different type of work, albeit one with people you love.

This week, however, I enjoy the benefits of working at a school whose half-term holidays are out of sync with my kids' school. I can drop them off and then spend six hours doing whatever the hell I want. I could go for a walk, read a book, watch a film, or play one of my many games that are piled up in a sort of entertainment "in-tray". X-Com is my current nemesis and I have to say it is kicking my ass at every encounter.

But this morning was different. I got an email asking me to look at something for work: a new feature for the music database website. It's not something I have to do, as I stopped officially supporting that thing ages ago, but still I find myself drawn back to it, like how George Lucas won't stop meddling with Star Wars until he has annoyed everyone in the universe.  (Don't go away thinking I am equating my music database to Star Wars. I'm not. My music database is way less entertaining than Episodes 4, 5 and 6, and only slightly more interesting than Episodes 1, 2 and 3)
.
The thing is, the work was interesting, and involved learning how to do something I didn't know before. I decided to use it to teach myself a bit of Ajax, which is a programming technique rather than a cleaning product, and to play around with my new local PHP testing server. Tinkering is fun and, in many ways, it IS a game. You have a goal (get the thing working), obstacles to overcome (things don't work so you have to find out where they are going wrong) and rewards (the satisfied feeling that you are making progress, both in the project and in your own knowledge). I certainly find it less frustrating than X-Com, because I'm pretty sure my Music Database is not out to get me, whereas those aliens definitely are. The green-blooded bastards.

Nocturnal Noughts And Crosses

Last night I woke up at 3am with my brain buzzing. Maybe it was because I had gone to bed early, but I could not stop thinking about a cool new project.

"What if," I imagined, "I got a robot arm and taught it how to play noughts and crosses? That would be good on Open Morning at school, and maybe the A level kids would get a kick out of tinkering with it."

I picture a large sheet of paper with a grid on it, a pupil drawing noughts with a thick marker pen, and a computer watching it with a camera before making its own move.

So, as I couldn't sleep I started googling and found one of these cheap robot arms from Maplin. It seems that some people have worked out how to program them in Python, and it doesn't look too tricky. In theory, making an arm draw a cross on a page should be manageable, and if it has trouble handling a pen then it could at least pick up and move a cross-shaped object on a table.

The trickier bit would be making the computer aware of what was happening. I think it's possible to get Python to read an image from a webcam, and it should be possible to write a piece of code to look for a circle drawn on the page.

So that should keep me occupied for a while. I bought the arm today. I'll report back when I have some news.