In the first part we have set up a very simple but efficient voice assistant which can listen to your commands and react on them. It starts up as soon as internet and audio appears in system. And yes, this is the limitation of our very first implementation - it needs internet as it uses Google TTS and STT engines.

But this time, we'll try to replace them with something which can be executed 100% offline. And I have couple of candidates, apart from the ones we talked before

Mozilla Deep Speech

The most recent version of DeepSpeech right now is v.0.9.3 and it requires Python 3.7. whereas we're running 3.9 in our Debian. I found it out, when I attempted to install the deepspeech-0.9.3-cp37-cp37m-linux_aarch64.whl and it was giving me this confusing message:

pi@orangepi4-lts:~/projects $ pip3 install https://github.com/mozilla/DeepSpeech/releases/download/v0.9.3/deepspeech-0.9.3-cp37-cp37m-linux_aarch64.whl

ERROR: deepspeech-0.9.3-cp37-cp37m-linux_aarch64.whl is not a supported wheel on this platform.

As I figured it out, I should have checked --verbose flag and then it became more apparent that it attempted to install things under 3.7 directories, whereas current version of python was 3.9. Python is such a mess.

As Debian repo doesn't have 3.7 I decided to build it myself. It's done pretty straightforward: download -> configure -> make -> make install. I just crosschecked the very first hit I found on google and it went just fine.

To be continued

https://deepspeech.readthedocs.io/en/v0.9.3/

coqui-ai TTS and SST

https://github.com/coqui-ai

https://tts.readthedocs.io/en/latest/

Coqui-Ai TTS - Text to speech

I was playing with TTS (text-to-speech) using my Intel Atom Z8350 tablet - Asus Transformer Book T102HA, as I'm on vacation right now, away from my little SBC device at home. It was almost a smooth ride. Saying "almost", because in Windows 10 it took me a while to understand two main dependencies:

1) you need to have python 3.9, not 3.10

2) you need to install windows 10 sdk and build tools, using "Visual Studio Installer". And that thing brings like 5 additional Gb of various DLLs, header files and tools to your Windows 10. Kind a crazy.

Eventually you can do:

pip3 install tts

And check this out:

tts --text "This is another test using tacotron2" --out_path speech_tacotron2.wav --model_name "tts_models/en/ek1/tacotron2"

tts --text "This is another test using tacotron2-DDC" --out_path speech_tacotron2-DDC.wav --model_name "tts_models/en/ljspeech/tacotron2-DDC"

tts --text "This is another test using tacotron2-DDC_ph" --out_path speech_tacotron2-DDC_ph.wav --model_name "tts_models/en/ljspeech/tacotron2-DDC_ph"

tts --text "This is another test using glow-tts" --out_path speech_glow.wav --model_name "tts_models/en/ljspeech/glow-tts"

tts --text "This is another test using speedy-speech" --out_path speech_speedy.wav --model_name "tts_models/en/ljspeech/speedy-speech"

tts --text "This is another test using tacotron2-DCA" --out_path speech_tacotron2.wav --model_name "tts_models/en/ljspeech/tacotron2-DCA"

tts --text "This is another test using vits" --out_path speech_vits.wav --model_name "tts_models/en/ljspeech/vits"

tts --text "This is another test using fast_pitch" --out_path speech_fast_.wav --model_name "tts_models/en/ljspeech/fast_pitch"

tts --text "This is another test using vits" --out_path speech_vits.wav --model_name "tts_models/en/vctk/vits"

tts --text "This is another test using fast_pitch" --out_path speech_fast_.wav --model_name "tts_models/en/vctk/fast_pitch"

tts --text "This is another test using tacotron-DDC" --out_path speech_tacotron.wav --model_name "tts_models/en/sam/tacotron-DDC"

tts --text "This is another test using capacitron-t2-c50" --out_path speech_capacitron-t2-c50.wav --model_name "tts_models/en/blizzard2013/capacitron-t2-c50"

tts --text "This is another test using capacitron-t2-c150" --out_path speech_capacitron-t2-c150.wav --model_name "tts_models/en/blizzard2013/capacitron-t2-c150"

This will download a number of different voice and vocoder models (if something fails for you, check the output of tts --list_models command first, and adjust the above) and synthesize the phrase given as --text parameter using them.

You can then go and listen the result, using your favorite media player, or download SoX to use aplay.

Given I was getting very unstable performance figures, like what time does it take to generate the same sentence using different combinations of models/vocoders, I drafted a very simple script to try them all out multiple times, so I'll have a more data to compare - https://github.com/kha84/tts-comparison

I figured it out, that for my hardware the balance between best performing and most accurate spelling was to use tts_models/en/ljspeech/glow-tts

Tensor Flow TTS

https://github.com/TensorSpeech/TensorFlowTTS

AprirASR TTS

https://github.com/abb128/april-asr

Used in LiveCaptions, a wonderful application for Gnome to generate English captions from the played audio

OpenAI Whisper TTS

https://github.com/openai/whisper

Whisper.CPP TTS

A re-implementation of OpenAI's Whisper but on C++, which is leaner to resources

https://github.com/ggerganov/whisper.cpp

Mimic3 TTS

https://github.com/MycroftAI/mimic3

MaryTTS

http://mary.dfki.de/

As you know, one of the project goals is to have your own "Alexa" or "Siri" replacement in our little SBC. A lot of work was already done by a number of talented folks, so we won't shy to just take results of their work and reuse it in our favor.

But at the beginning, let's start with some very basic stuff here. Like making sure we can record and playback audio, then we'll play with couple of entirely offline text-to-speech software.

Check some basic stuff

arecord / aplay

These two guys worked for me out of the box. The recording is happening on the internal microphone (the quality is so-so), playback via the currently selected device in PulseAudio - in my case through TV connected with HDMI

arecord -d 5 -r 48000 test.wav
aplay test.wav

Text-to-speech

espeak

Espeak was around for a long while. It sounds very robotics, no matter what kind of voice you choose.

apt install espeak

espeak "Hello there!

espeak --voices

espeak -v en-gb "Text will be converted to speech."

Festival

Festival is also a long standing player. And it also sounds very robotic, by default, but we can install some additional "language packages" which make it bit better. And there're also commercial packages existing on a market, but they're not as good as their pricetag

apt install festival festvox-ru festvox-en1 festvox-us1 festvox-us2 festvox-us3 festlex-cmu festlex-poslex

Below commands must be executed under unprivileged user (non-root):

echo "This is the best as it can get" | festival --tts

echo '(SayText "This is the best as it can get")' | festival --pipe

I can see it clearly, when the software was designed by morons. Speaking of Festival you cannot easily switch between different voice packs, you have to update configs or use interactive mode:

festival
(voice.list)
(voice_us1_mbrola)
(SayText "This is now a different voice")
(voice_us2_mbrola)
(SayText "This is now a different voice")
(voice_us3_mbrola)
(SayText "This is now a different voice")
(voice_en1_mbrola)
(SayText "This is now a different voice")

If you want some voice to become a default one, make a change to /usr/share/festival/siteinit.scm search for a line voice_default

Look this video where a nice guy explaining how to download more voices - https://www.youtube.com/watch?v=Ju_X11JyRSE

I checked the website he mentioned, all voice packs are old as dinosaurs' poo, so to me it doesn't really make sense to spend on it any additional time

pico2wave

I got tired of those robotic voices. So I just looked for couple of youtube videos how it sounds and wasn't really impressed. So skipping it.

Google text-to-speech engine

If you're on Debian - don't install it from repo. The version from repo is outdated and doesn't work anymore. Install it via pip, it will take much freshier version:

pip install gTTS

Let's play with it now:

~/.local/bin/gtts-cli 'This is the test from Google text to speech services' --output /tmp/test.mp3
ffplay -nodisp -autoexit /tmp/test.mp3

Well, the quality is just increadable. I'll probably go online :) but need to understand, this Google can close this backdoor at anytime, so if we're building something we don't want to support all the time, we'll need to have a fallback option, line one of the above. So far I just made a simple and stupid wrapper, we'll extend it later on, if needed, to analyze exit code from gtts-cli and switch to something different, like espeak:

cat /usr/local/bin/sayit
#!/bin/sh
if [ -z "$1" ]
then
echo "Error: no input parameter is given"
exit 0
fi
tfile="$(mktemp /tmp/tts.XXXXXXXXX.mp3)" || exit 1
~pi/.local/bin/gtts-cli "$1" --output $tfile
ffplay -nodisp -autoexit $tfile
rm $tfile

Speech recognition

Let's start with something simple like this project - https://github.com/ProgrammingHero1/romantic-alexa

I've made a small fork of it - https://github.com/kha84/pgttsjaisrar - with fixing some obvious issues and swapped the ugly text-to-speech python lib with our lovely above shell file, made it to run as systemd service and, hell, it works just well out of box.

Stay tuned for the part 2, where we going to make it to work offline - https://orange-pi-4-lts.blogspot.com/2022/09/speech-recognition-and-text-to-speech_3.html

Orange PI 4 LTS / Lowcost PC replacement packed with features / Deban on ARM

Speech Recognition and Text-to-Speech using ML. Part 2: going offline