Open source software for animatronic figures

In my previous animatronics projects, I always had a lot of fun with the hardware, but then I didn't put so much energy into programming the movements. That's a shame, because it's the movements that make the characters come alive.

The reason was that it really is programming, i.e. every movement had to be written in program code. This is quite tedious, often looks choppy as a result and has to be recompiled and started again and again to try it out.

For this reason, I have developed the software "Animatronic Workbench" (AWB), which can be downloaded from GitHub as open source. This allows you to create the movements of animatronic figures on the screen without any programming knowledge. The first practical use of the software was with my animatronic Grogu:

.

There is already a short YouTube interview about the project, which shows and explains the essential functionality:

https://youtu.be/uIglKsgBKBA

Dashboard

View Version History

Roadmap

• Windows app
• Timeline Editor
• ESP32 client
• Control via midi controller
• Control via mouse / keyboard (without mandatory midi controller)
• Include servos in timeline
• Live Mode
• Saving the movements in the microcontroller
• Autonomous Mode
• Web interface for the client
• Controlling STS serial servos
• SCS Serial Servos Drive
• Controlling PWM Servos
• Support Wifi Handheld Remote Control
• Include sound / speech in timeline
• Controlling LX serial servos
• Project configurator instead of manual editing of JSON files
• Automatically generate "Hardware.h" from Project Configurator
• Include RGB LEDs in Timeline
• detailed instructions
• Video tutorial
• UnDo function in editor

Setup / Installation

The installation and setup of Animatronic Workbench Studio is described here.

Timeline Editor

The core of the "AWB Studio" app is the timeline editor. Here, the movements of the individual servos in scenes can be designed via a timeline. The operation is intuitive and similar to the operation of video editing programs.

A single scene is saved under a name and can be assigned to a status such as "Idle", "Sleep" or "Talk".

Live control vs. autonomous storage of movements in the microcontroller

During editing, the changes can be transmitted directly to the animatronic figure via USB or WLAN. This allows you to follow the adjustments in the editor live on the figure.

Subsequently, the motion sequences can be permanently stored in the microcontroller, so that the figure can move autonomously completely without a connected computer and only needs a power connection.

The (optional) selector switch can be used to select a status (such as "Idle", "Sleep" or "Talk") on the microcontroller, which then (only) plays the appropriate movements.

Input Controller

As an input device, the software uses midi mixer controllers, which can be connected via USB and which can be bought for less than 100 euros. With these controllers, you can move through the timeline of the respective animation and haptically control the rotations of the servos using rotary or slider controls.

Through the return channel of some mixer models, you can even read the current position of the servos on the controller when the timeline plays in playback mode. If the animatronic figure is connected to the computer via USB cable at this time, the movement is also carried out live on the figure.

Web interface for the client

The ESP32 client microcontroller sets up a small, dedicated WLAN that can be used to connect to a smartphone or tablet. Via a local website, the status of the servos (including temperature, load, etc.) can be read there and movements can also be started. This is particularly useful if the animatronic figure is no longer connected to the computer, but is already on a stage or in a shop window, for example.

Hardware

Microcontroller Clients

ESP32 boards are particularly suitable as microcontrollers, as they are very small and inexpensive, have WLAN and also have enough computing power to control the servos. Ideally, you should also connect a small display that shows the current status of the servo positions, servo temperature and possible error messages. This makes troubleshooting and debugging easier.

You can either solder the ESP32 yourself with the components used, but I have also had good experiences with the following two finished boards:

Waveshare ESP32 Servo Driver

The Waveshare ESP32 Servo Driver Board isverysmallandalreadycontainsadisplayandready-madeconnectorsforSTSservos.At25EUR,it'sstillaffordable.Asabonus,italsohasanRGBLEDconnector(alsoknownasNeopixel) and even two such LEDs on the board. Unfortunately, no pins of the ESP32 are routed to the outside, so that you can no longer connect your own sensors or speakers. However, it is perfect for implementing projects such as the animatronic Grogu.

M5Stack

The M5Stack ESP32 Core Modules already come in a case with a display and often with a built-in battery. Most also have a speaker and other useful components installed. They are therefore very versatile and can also be expanded with other modules, e.g. via I2C bus. From my point of view, they are very well suited for projects with PWM servos, as there is also a suitable expansion module for this that fits directly to the housing.

M5Stack boards are, so to speak, the comfortable all-round carefree package - but they are also significantly more expensive than the Waveshare board.

Supported / planned servo types

STS servos (already supported)

The STS servos from Waveshare or Feetech are serial servos and can therefore be easily connected to the microcontroller. For the fact that they use wear-free magnetic encoders instead of rotary potentiometers, they are still comparatively inexpensive and have a high resolution.

They can also be assigned how high the acceleration and final velocity may be per movement and are therefore ideal for animatronic figures. If the servos get too hot during continuous operation, the serial servos can report this back to the ESP32 and initiate countermeasures.

The Animatronic Grogu is based on these servos.

PWM servos (already supported)

The classic model servos are PWM servos. They are very cheap and widely available.

The primary disadvantage from my point of view is that they usually do not have an adjustable speed / acceleration and are therefore only conditionally suitable for animatronic figures. In addition, when the electronics are switched on, they can move into unsuitable positions, which, depending on the design, can cause the figure to self-destruct.

Dynamixel AX servos (possibly planned)

However, Dynamixel serial servos have a very high resolution, can generate very high torques and are still quite quiet. They are therefore ideal for large figures and can be found, for example, in the Nikola Tesla bust. However, they are also quite expensive. Unfortunately, I haven't found an open source implementation of the protocol so far, so I've only been able to use these servos in projects that I didn't later open source myself. This complicates support in AWB, as I would have to write the protocol implementation myself.

LX servos (possibly planned)

The LX servos from LewanSoul are also serial servos, but with a different protocol.