Self-built multi-touch table

Self-built multi-touch table

Various

Note: This is an automatic translation of the original German text. You can read the original German version here.

Background

Since 2006 at the latest, Multi Touch Tables have been quite well-known, as the first videos of experimental projects have appeared on the Internet.

Since then, numerous commercial providers have also taken up the issue. The best-known provider is certainly Microsoft with its Surface project

While the purchase of such a commercial product currently accounts for about 10,000 $ However, a working Multi Touch Table can also be made for less than 500 €.

This article tries to document the creation of such a low-cost Multi Touch Table in as much detail as possible. The text is not to be understood as building instructions. Any use of this information is at your own risk and risk. In particular - but not exclusively - the removal of the IR filter of the webcam used carries the risk of damage or even defect of the camera if it is not carried out properly.

How does a Multi Touch Table work?

The functionality of a multi-touch table is actually very simple - the secret actually lies only in the optical behavior of a thick acrylic glass pane:

Infrared light is beamed onto the acrylic glass pane from the side.

Due to the effect of total reflection, the light cannot leave the acrylic glass pane through the glass surface. However, if the glass surface is touched, this prevents total reflection at this point. Scattered light is generated there, which can leave the acrylic glass.

Total reflection and stray light with multi-touch table:

Totalreflektion und Streulicht bei MultiTouch-Table

If you film the disc from below with an infrared camera, the areas where the fingers touch the pane will glow. By evaluating the camera image, these areas can be isolated and thus software events can be generated similar to one or more mouse pointers.

The Infrared Camera

The camera for the Multi Touch Table must only receive infrared light. However, most webcams are optimized for the normally visible range. In addition, they almost always contain an infrared filter that filters out the light in the IR range.

Of course, you can also buy digital video cameras, which have the opposite behavior and thus only record light in the IR range. While a normal webcam can be obtained for less than 100,-, the price for an IR camera is usually much higher.

(For the avoidance of doubt: IR camera does not mean not thermal imaging camera. These usually cost several thousand euros. This is just a matter of making the normal IR range visible.)

For the best possible quality of the camera image, a Logitech QuickCam Pro 9000 webcam is used as a basis. It has 2 megapixels and Carl Zeiss optics with autofocus, which leads to very good and sharp results.

The webcam becomes an infrared camera

The webcam is currently still filtering out IR light and recording normal light. However, the opposite is desired. Therefore, the IR filter contained in the camera must first be removed. A filter is then attached, which only allows IR light to pass through.

To remove the IR filter, the webcam must be opened:

Geöffnete Webcam

Now the lens and autofocus are removed. To do this, the solder of the solder points (see arrows) with which the autofocus is connected must first be removed.

Lötpunkte vorher lösen

The image sensor remains on the board. Like all optical components of the camera, it should not be touched and should not be exposed to dust or dirt.

Der Bildsensor liegt nun frei

In the assumed autofocus, you can already see the infrared filter on the inside. It shimmers slightly red when tilted against the light.

Der Infrarot Filter

This filter must now be removed. Unfortunately, it is very well glued in and the glass is still very brittle. A simple removal in one piece is therefore almost impossible. When prying them out, safety goggles are highly recommended, as the smallest glass splinters fly around. In the meantime, the IR filter looks very scratched and broken:

der IR-Filter ist bereits zur Hälfte abgetragen

Eventually, however, the filter is removed, leaving the lens underneath exposed.

Der IR-Filter ist entfernt

Now all you have to do is reassemble the webcam.

However, it is likely that the autofocus was adjusted when the IR filter was pryed out. Before finally screwing the housing together, the camera should therefore be connected to a PC. By turning the autofocus wheel (see arrow) you can recalibrate the autofocus.

Der Autofokus muss wahrscheinlich neu eingestellt werden"

If everything went well, the webcam can now also detect infrared light. With a normal IR LED, it should already look like this:

Ab sofort erfasst die Webcam auch infrarotes Licht"

Now the camera can also see infrared light - but normal light is also captured. To get rid of this, a filter has to be attached, which is only permeable to IR light. These filters are available in photo shops for about 30-40 €.

If you want it cheaper, you can also use an exposed photo negative. This must be exposed to the maximum, i.e. it must be completely black and opaque as a negative. Fortunately, such an exposed negative has the property that it is still permeable to IR light. If you take two to three layers of this negative on top of each other, you already have a cheap filter. The quality is not as perfect as a real photo filter, but it should be enough for first attempts.

If you still have a Wiimote left, you can also use the small black plastic mask that covers the built-in camera at the front. Since the Wiimote's camera is also only supposed to see IR light, this plastic cover is also a suitable filter:

Wiimote IR-Filter vor der WebCam

The Infrared LED Strips

Infrared light-emitting diodes are used to illuminate the acrylic glass pane with infrared light. For uniform illumination and positioning, it is recommended to mount in a strip:

IR LEDs in Leiste

In the project presented here, a total of 112 IR LEDs are used at a distance of 1.5cm for two strips of 80cm in length.

32 LEDs im Abstand von 2.5cm

If you use IR LEDs with 1.5 volts, you can connect 8 pieces without resistance to 12 volts in series and thus easily operate them with a normal (PC) power supply.

8 LEDs á 1,5 Volt = 12 Volt

To test the wiring, you need the modified webcam, as the infrared light of the LEDs is not visible to the naked eye.

Infrarot-Test

A first test with a 5mm acrylic disc already brings promising results. In the final version, an 8mm lens is used, which reduces the stray light that is still present here. But you can already see the see stray light that occurs (and is desired) at the points of contact of the fingers:

Erster Test für die Kamera Erkennung

The Projection Screen

In addition to the acrylic disc for touch detection, a projection screen for computer graphics is of course also required. This must be matted accordingly so that the projected image is also visible.

In order to find out the best surface, some experiments were carried out:

Verschiedene Oberflächen für die Projektionsfläche

  1. D-C-Fix transparent
  2. Window film
  3. Matted with 800 grit sandpaper
  4. Glued with tracing paper

Test für die Projektionsmaterialien

The projection screen must display the graphic well. At the same time, however, the infrared stray light from the acrylic pane above must also be allowed to pass through well and as sharply as possible. This is the only way the modified webcam can capture the touches of the fingers.

The tests showed that the surfaces 1st and 4th excrete directly: 1. is too transparent and scatters the projected light too much. 4. The IR light does not pass through sufficiently enough. All that remains is frosted window film and treating the surface with 800 grit sandpaper.

Test mit Beamer Projektion

First test of the surfaces using a projector Test der Projektionsoberflächen per Beamer

The Hardware

In order to arrange all components of the MultiTouchTable, an appropriate design is required. Wood is chosen as the material here. The table is collapsible and can therefore be carried by one person and transported in the trunk of a car.

When folded:

Multi Touch Table zusammengeklappt

When unfolded from the side:

Multi Touch Table aufgebaut

When unfolded from the front:

Multi Touch Table aufgebaut

When assembled with the projection screen inserted:

Projektionsscheibe eingesetzt

The TouchTable in working order. On the projection screen is now the acrylic glass pane:

Der fertige Multi Touch Table

The projector and the IR webcam are deflected together by mirror:

Der fertige Multi Touch Table

The Software

A first test software created in C# works flawlessly - including touchscreen calibration. In the first test, a WPF button is displayed under each finger:

Erster Test der Touch Software

The FireTouch Framework

To improve the reusability of the MultiTouch software, the FireTouch Framework was created. It is completely programmed in managed C# code and provides numerous components from the detection of webcam streams to WPF controls and e.g. a multi-touch canvas.

The FireTouch framework has been completely redeveloped and is therefore not based on existing libraries such as touchlib. The reason for this is that the previous libraries are mainly built in unmanaged code such as C++ and thus could crash the dotnet application in the event of an error. FireTouch, on the other hand, generates correct dotnet exceptions in case of errors, which can be handled cleanly.

In order to be able to optimally adjust the optics and webcam, the FireTouch framework offers a diagnostic mode with different views:

  • Raw Mode - shows the image as it is delivered by the webcam Roh-Modus

  • Bright areas - shows the image adjusted for static areas and taking into account uneven illumination of the area by the IR LEDs Helle Bereiche)

  • Color visualization - the distance of the fingers from the glass pane is shown in colors: white lies directly on the pane. From blue to green to red, the distance then continues to increase until the distant areas are shown in black. In addition, fingers that have already been recognized are marked by white rectangles.

  • Color Visualization

An overview of the different debug views:

A small demo application:

Object Detection

The framework now also supports the detection of objects that lie on the projection surface.

For this purpose, barcode-like markings are placed under the objects. These can then be distinguished from the applied fingers in the webcam image and evaluated.

The detected information also includes the rotation of the detected object.

An example of detecting four objects:

Categories: Various

Tags: Mtt , Multi-touch Table , DIY , Infrared Camera , IR Filter , Object Detection

QR-Code dieses Beitrags


Dieser Blog-Artikel beinhaltet keine bezahlten Produktplatzierungen.
Bei Affiliate-Links oder Rabatt-Codes entstehen für den Leser keine Kosten oder Nachteile. Bei einem Kauf über diesen Link/mit diesem Code erhält der Betreiber eine prozentuale Provision und/oder gewährt dem Käufer einen Rabatt in angegebener Höhe.