State of NAM on Raspberry Pi

It's been almost two years now since I first posted a video of Neural Amp Modeler captures running on my Raspberry Pi 4.

A lot has changed since then.

Originally, the NAM playback code was extremely expensive. So much so that a Raspberry Pi 4 could only manage to run "feather" models.

Because I wanted to run more accurate models, NAM  optimization became a bit of a pet project. With some optimizations to the NAM Core codebase, we were able to increase performance by more than 2x. That, combined with more widespread availability of a 64bit OS for the Pi, made "standard" NAM captures possible - with plenty of headroom left for a cabinet IR and some light effects.

More recently, I have been working on my own implementation of WaveNet and LSTM models, with a focus on performance. The result of which, you can see running in the image above in my Stompbox app.

Note the numbers in the bottom left - that is the realtime CPU usage as reported by Jack. This is running a "standard" NAM capture. Audio buffer size is 96 samples. CPU usage of "standard" models is now low enough that I can easily run two models at once, with plenty of CPU left over for IR and effects. I can even (just barely) run three models at once.

This Boss SD-1 Pedal Does Not Exist

 

I've been messing around lately with combining Neural Amp Modeler with another open source project I've contributed to in the past - LiveSPICE.

LiveSPICE allows you to simulate audio circuits in real-time, which is very cool. The disadvantage it has is that it is very CPU-heavy. It's CPU usage is also not super consistent, but can be spiky, which makes it hard to use in a live, low-latency environments without getting audio dropouts.

It *is* however, very easy to use it to generate training data for Neural Amp Modeler. So that's what I did.

Because it uses a simulation of the pedal circuit, the generated training audio has none of the added noise that is difficult to fully avoid when capturing actual pedals. This "idealized" version of the pedal should be even easier for the neural net model to learn.

And indeed, it is. Here is the ESR of a "feather" model (the smallest, least CPU-intensive default NAM model type).

It shows 0.000, but it was actually around 0.0001.

You can get the resulting .nam model here on ToneHunt.

Neural Amp Modeler (NAM) running on Raspberry Pi

Recently I've been messing around with, and contributing to the Neural Amp Modeler project. It uses machine learning (more specifically the WaveNet model) to create captures of amplifiers and distortion pedals.

It has been gaining a lot of traction recently, with lots of people modeling their equipment. The resulting models are very good.

I've now got it integrated into my Raspberry Pi pedalboard. The audio for the above video was recorded on a Raspberry Pi 4.

The hardware on the pedalboard consists of:

- Raspberry Pi 4

- Hotone Jogg audio interface

- Hotone Ampero Control MIDI Controller

- Wio Terminal (used for a serial-based display)

My pedalboard is using a custom app on top of Jack audio, but I have also made a Neural Amp Modeler LV2 plugin available.

Chord data for jazz standards

I recently started playing jazz on bass guitar. So naturally, I also recently started writing software for facilitating jazz practice.

As a basis for a number of my projects, I needed access to chord data from a corpus of jazz standards. The best source I found was the database of user-submitted scores for the IReal mobile app: https://www.irealpro.com/main-playlists

I've been converting these to an easy-to parse json format. The current database of songs can be found here:

https://github.com/mikeoliphant/JazzStandards

Here is an example score:   

  

  {
  "Title": "Alone Together",
  "Composer": "Schwartz Arthur",
  "Key": "D-",
  "Rhythm": "Medium Swing",
  "Sections": [
    {
      "Label": "A",
      "MainSegment": {
        "Chords": "Dm6|Em7b5,A7b9|Dm6|Em7b5,A7b9|Dm6|Am7b5,D7b9|Gm7|Gm7|Bm7,E7|Gm7,C7|Fmaj7|Em7b5,A7b9"
      },
      "Endings": [
        {
          "Chords": "Dmaj7|D(Em7b5),(A7b9)"
        },
        {
          "Chords": "Dmaj7|Dmaj7"
        }
      ]
    },
    {
      "Label": "B",
      "MainSegment": {
        "Chords": "Am7b5|D7b9|Gm6|Gm6|Gm7b5|C7b9|Fmaj7|Em7b5,A7b9"
      },
      "Endings": []
    },
    {
      "Label": "A",
      "MainSegment": {
        "Chords": "Dm6|Em7b5,A7b9|Dm6|Em7b5,A7b9|Dm6,Bm7b5|Bb7,A7b9|Dm6|Em7b5,A7b9"
      },
      "Endings": []
    }
  ]
}

Arduino sketch for TFT display rendering using serial commands

I've been using a Seeed Studio Wio Terminal as an external display for a Raspberry Pi. I initially wrote an Arduino sketch that took high level, application-specific commands over serial and rendered appropriate display elements. This has the disadvantage that adding UI features required updating the Arduino code, which is more of a pain and slower to iterate on.

Instead, I'm now planning on sending rendering commands over the serial interface - turning the microcontroller into a general purpose rendering device.

I've thrown my initial code up on GitHub here:

github.com/mikeoliphant/SerialTFT

It currently only supports a small set of commands for rendering rectangles and text. I'll be adding more features as I need them.

Speeding up Arduino TFT (TFT_eSPI) writes

 I've been recently playing around with "Arduino" microcontroller stuff - specifically a Seeed Studio Wio Terminal.

Playing around with the TFT display, I noticed that drawing to it was pretty slow. After doing some research, I found out that direct TFT writes are indeed quite slow. But there is an easy way around it.

The TFT_eSPI graphics library, a modified version of which is used by the Wio Terminal, provides a Sprite class that can be used to do quick TFT writes. If you have enough memory (which the Wio Terminal *just* barely does), you can create a full screen-sized sprite.

The Sprite class supports the same drawing API as the main TFT class, so it is pretty easy to convert code that is writing directly to the TFT to use a Sprite instead.

To initialize it, do this:

TFT_eSPI tft = TFT_eSPI();

TFT_eSprite screen = TFT_eSprite(&tft);

In your setup(), after initializing the TFT, do this:

screen.createSprite(tft.width(), tft.height());

Then, use "screen" instead of "tft" for all of your drawing calls. When you want to update the TFT with the current state of your sprite, do this:

 screen.pushSprite(0, 0);

That's it! The result is a display that will update *much* faster.

One caveat is that this method uses up the vast majority of the Wio Terminal's RAM. It works fine for me, since my application is not otherwise particularly memory hungry. More complicated applications (using WiFi, for example) may run into problems. If that happens, a good (if more complicated) solution would be to use a smaller screen area sprite and do targeted writes to sub-areas of the screen.

Fixing Hotone Jogg ASIO "No devices connected" issue

I recently ordered a Hotone Jogg guitar audio interface pedal and had trouble getting the Windows 10 ASIO drivers to work. Turns out that ASIO support was added with a firmware update, and the pedal I got had the old firmware.

If you are having this problem, you need to get the updated firmware from here:

https://www.hotoneaudio.com/support/2

Scroll down until you see "Jogg firmware with Hotone ASIO driver support" and click on the link to download the firmware.

Follow the included instructions (you have to open up the unit to flip a dip switch, which is not very user friendly) and you should soon be good to go.