Theory In Object Detection

I have been playing with edge detection for a few weeks now, and slowly I’m gathering a solid knowledge base related to how to do object detection. Primarily I have been playing with facial recognition, but my end goal is to be able to track hands, for a virtual table PC, so that when I build my surface computer the adaption of the algorithms is easy. The goal is to use edge detection and to determine the shape of the enclosed object.

The current algorithm that I am building is based on some assumptions:

  • The person is feature complete (two eyes, two eyebrows, one nose, one mouth)
  • The room is relatively well lit (haven’t incorporated luminosity correction for shape detection, need to work on performance)
  • There is a notable edge difference between these features (enough for the algorithm to detect edges)

Assuming these things, the edge detection is mounted in a matrix and through various reduction methods, it reduces the amount of noise (isolated, or only sharing an edge with one cell). From these shapes we can determine facial structure based on clusters of a predefined shape.

While this is fun for face tracking purposes, it strays from the principle concept that is trying to be achieved with virtual touch computing. Facial features are relatively easy to track as they are static in context of each other, so predictable patterns even in rotation are possible. Fingers on the other hand are a more dynamic object to track, as spacing, position count are likely to change frequently. For example someone who wants to drag an object around may only hold out one finger and drag the object where they want to move it, while a rotation might use all five fingers.

On a surface such as a touch PC, this is relatively easy through the user of IR LEDs using the aforementioned FTIR (frustrated total internal reflection) methods of light detection, or through the idea of using a set of IR lasers close to the surface with planers to diffuse the beam over the table, but virtually there is lots of noise in the image. The algorithm that I am currently working on tries to filter out the noise through the following methods:

  • Identify the individual in the image (or what is visible of them) and filter out the background image to 0’s in the matrix
  • Process the general form outline, find where on the matrix the general shape of the hands are, and once this is processed identify the digits
  • Filter out everything except the digits and return the points which are visible

This sounds easier than it looks, however there are many limitations and issues when it comes to distinguish-ability. Another problem on top of this is light conditions, multiple people in the image and detectability of their shape. This is the area that I am working on now for performance reasons. Regardless, the skills I obtain developing this algorithm will make surface digit detection trivial in nature.

If anyone has any suggestions on how to detect full human motion based on image capture and processing, please let me know. I am always looking for a good algorithm 😉

Updates and Development Abound

I’m posting a few notes of projects that I’m working on, and design decisions that have come to mind in the past couple weeks. First off, exam season is finally over, meaning I’m back open to free thought! Well, I was before, but assignments and studying tend to encourage a one-track mind. Anyways, I have started developping a couple neat applications, both of them mostly for personal use but who knows. The source will come up this week, though not in its completed iteration.

Project One: The RegEx String Generator

I’ve been toying with this for a while, mostly for content organization, testing and such. As a result I’ve started to develop a DLL that allows its users to give a regex string as an input and either ask for iterated values (such as iterating through all possible strings) or to generate random values. I think that something like this would allow for much easier application testing as you would only need to include the library in your project and run test patterns on it. So far it has literals support, integer (both range and digit iteratation), and soon will support character ranges (and traditional arguments such as ‘w’ in regex). Overall this is still a young project, but with promising results.

Project Two: Surface PC

Now that it’s summer and I will start earning some money again I can start to develop my surface pc. I have sourced some suppliers for the components, and aside from working out the projection setup (have to amplify the projection distance somehow) I have figured out most details. I’ve decided to forgo FTIR for my first table iteration, and will try using corner mounted linearly planed lasers. This apparently has a much brighter result for the camera and will also allow a narrow band filter to be applied to the lens, allowing for daytime operation (a limitation of most general design FTIR tables). Also, I will start to write some multi-touch applications in the near future, though the issue remains that I will not be able to test them until I am sure that the table is fully functional. The project budget is looking near $850, which isn’t too bad, considering that includes a projector and basically every component (aside from the PC itself, though I am looking at a high def silent PC with a budget of around $300, so this might be a silent, nearly instant-on touch PC when I’m done with it). More posts and details and a detailed walkthrough to come as I design and build the table!

All in all, this summer looks promising for side projects, and given that I have my weekends free this year (unlike previous years) I will finally be able to once again make some progress on development. Look back here mid June time for the start of the Surface PC saga.

A Look at Maximum PC’s Multitouch Surface Computer

For many do-it-yourselfer’s out there, multi-touch displays may have been something you thought was out of your league. In reality, Maximum PC’s research guys have shown us that we can make a Frustrated Total Internal Reflection (FTIR for short) table for around $350. That is fantastic news because you can be the proud owner of your own touch-capable PC without having to spend $12,000 on the equivalent Microsoft Hardware product. In short, this makes it easier for developers such as myself to build our own touch PC’s and start working on the interfaces of tomorrow.

Many of the methods they used cut considerable corners, and with a bit more thought and research it would be easy to see that a few tweaks here and there would not only make it stronger and easier to make but also cheaper. In their implementation they used IR LED’s all around the table and a variation onf vellum and silicon to form an impression system for the surface. And while these solutions were quick, cheap and easy, there is another solution which is a bit more efficient in my opinion, and I will detail it below.

In my combined solution, there would be a thicker sheet of acrylic with one edge cut at a 45 degree angle. then using mounting hardware along that edge, we would mount a medium density amount of LED’s (one every 3/4 of an inch) that work at 850nm wavelength. Then we would obtain a sheet of vellum (or white tissue paper possibly is better) and press it against the bottom of the LED mounted sheet of acrylic, and sandwich it with a thinner piece of acrylic (not necessarily as polished of edges). By sandwhiching the vellum (tissue paper) we reduce the noise and provide a more even continuous surface of paper (making it seem more screen like) as well as enhance the touch capabilites of the screen. The single row of medium density LED count allows for fewer LED’s overall and allows for a cleaner hardware design, less interference and a clearer image when using a PS3 eye toy. In the eye toy, using a double or triple layer of colour film negative, it would filter out all light but the IR light cleanly. So we would have a noiseless, clean table surface and a clearer camera image. These improvements would overall reduce the cost of the project by significantly reducing the hardware cost for the LED’s and mounting them. I plan to put them to use when I get started on my surface HTPC (summer project). I will keep my blog posted as we move into the summer months!