ESP32 synthesizer now with MIDI via BLE (bluetooth) - add-on for Arduino based synthesizer projects

In this video you will find a little demonstration and latency check of the integration of MIDI via BLE into some of my currently existing synthesizer projects (ml_synth_organ_example, ml_synth_basic_example, ml_synth_pwm_osc_example).
All three projects are capable of supporting the transmission of MIDI messages and producing their own sound as-well.
This feature comes with some limitations: it requires a lot of heap which means that effects like reverb and delay might be deactivated. In addition to that you will get some variable latency around 10-50ms.

I could imagine that this feature would be nice for wireless controllers. For serious tight melody playback it might not be the right choice. I do not know if there is a way to get a fixed latency to avoid this kind of jitter. You can set up one device as Server, another as Client and they can produce their own sound when getting MIDI via serial and forward it via bluetooth to trigger the other device as-well.

Please feel free to share your own experience and I would be happy to hear about your own ideas.

It has been highly optimized to be executed on devices with a low amount of RAM available and is not limited to a specific file system / interface.
File operations are implemented in the specific project (open, close, read, write, seek, tell).
The MIDI (stream) player supports the playback of one MIDI track (which can include all channels in case you are using MIDI format 0). Multiple tracks can not be played back (MIDI format 1) because it would require random memory access of the MIDI file which would cause a big performance drop.
I hope I can also provide a recording feature soon. I would be happy if you would like to send me a demo in case you are recording something which can be played back on one of the synthesizers.
The updates of some projects are nearly done. The others will follow soon. Finally the ML_SynthTools library will be updated to support this new feature.

Projects will be updated soon:

I would be happy if you also find some other interesting videos on my channel.
Use the notification (bell) to get a notification when I post a new video otherwise you might miss it because my channel is not so popular and you won't see it on your landing page.

---
Intro: (0:00)
First latency check (recording): (0:52)
First latency check (time measurement): (1:41)
Trying the ml_synth basic_example: (4:46)
Swapping devices: (6:14)
Latency check (recording and swapping devices): (6:42)
Another look at the latency: (7:28)
ml_synth pwm_osc_example: (9:28)
Final thoughts: (10:24)

[ronsum]

I always look forward to your new videos to see the progress you have made. Thanks for sharing your impressive project.

[rockpopmando]

Hi Marcel, if the delay is caused by the BLE-Lib or the Implementation in the ESP32 - try to compare the timing with another BLE-Device ... another BLE-Keyboard sending events or another BLE-Synth like a Smartphone with a Synth. I had tested it with Iphone .. and had delays too. hanks for sharing your impressive project.

[Slider_]

A useful addition.For long pads or strings, the small delays seen in Audacity shouldn't matter! 😁
Another way of reducing delays would be to trigger connected devices quicker than they should. For example. the Bluetooth enabled synth runs an arpeggio, connected to a drum machine and the drum machine fires 40ms before it should do. So, a detected MIDI connection on BLE would cause the drum sequence to move timings forwards automatically.

[LatvianDman]

Hello! I was lurking here in threads some time ago... About esp32 modules, your samplers and synths... While I mostly do creative stuff in Unreal Engine (games assets), right now I am on "electronics DIY phase" :D So, I finally might be trying some of your solutions (I have two of those audio specific esp32 modules, and bunch of usual ones)

[fundorinlive]

Still waiting for SF2 support. Good luck with your development.

[musicmoonshine]

So I had some problems with this not working with my nanokorg, no big deal, but I wanted to switch it over to server, and that didn't compile anymore, might be worth checking out...

[electronash]

Could I suggest trying to send MIDI via UDP packets from one ESP32 to the other?

I tried that years ago when sending joystick data from an Android phone to an ESP32 hooked up to an N64, and the latency at least seemed very low.

Using ESP-Now might be even lower latency, but I haven't messed with that much.