Jump to content
  • entries
    16
  • comments
    98
  • views
    2,688

Machine Learning-based Voice ITC Translator Software now available


Michael Lee

817 views

 Share

Since early 2019, I have been working on software to extract voices from physical noise/signals. My earliest attempts used other people's software, mainly an algorithm called "spectral subtraction." in a ReaFir noise reduction plugin. This converts the noise into the frequency spectrum, where slight imprints of voice can be discovered and emphasized.

We now enter the year 2022 - Spectral subtraction is still a very valuable tool, but it is only the beginning of a process I've developed for extracting voices. I've created machine-learning-based models to find and emphasize voices. I've also made a program that finds and generates "formants" or peaks in the harmonic buzz of the human voice.

I'm finally releasing my full software, in Python. I use a very similar version of) this code in all of my experiments (FPGAs, radio noise, etc.)

I would've liked to have shared it as an executable, like I did Spiricam, but Python executable-makers are notoriously buggy. Another reason I've hesitated is=n sharing the code sooner is that it used to require some heavy GPU resources. However, thanks to some software developments by Google, my ML models seem to run OK on the CPU pretty well in real-time.

image.png.a7e3d14ea5361eb22dbdecd1da31cd9d.png

So if you want to try out my code, you'll have to do some command-line steps and you'll have to at minimum install a free program called Miniconda, or a larger version called Anaconda with Python version 3.8, 64-bit. Maybe a few GBs of disk storage will be required.

Here's the link to the code: https://drive.google.com/drive/folders/1fu6hAuE0AbhbQjx0Ts_3Ju0QRJ0awxRM?usp=sharing

In the directory is a README.txt, which I'll update as we iron out the instructions.

When I've resolved most of the common issues, I'll make the code into a ZIP file for the Downloads sections.

For now, feel free to ask questions in the comments. As I like to say "The spirits are waiting!"

 Share

36 Comments


Recommended Comments



Andres and Kevin: You are two of the "alpha-testers."

Download the code from the link, when you get a chance, and have a fire extinguisher handy! 😉 

Link to comment
  • Administrators
38 minutes ago, Michael Lee said:

Andres and Kevin: You are two of the "alpha-testers."

Download the code from the link, when you get a chance, and have a fire extinguisher handy! 😉 

I have my ectoplasm thrower ready. 🤪

Since I already have Anaconda running I can directly jump into testing. I bet I will have some troubles with the phython libs as usual.

Will check it out tomorrow.

Link to comment

Kevin - Miniconda, which is a smaller version of Anaconda, should be sufficient. 

https://docs.conda.io/en/latest/miniconda.html#windows-installers

Miniconda installs the basic Python library management tools, and then my instructions will lead to a download of the Python libraries that my program needs. I tried the process on my daughter's Windows computer and the whole process was fairly painless. The only problem was when I want to remove Miniconda (to clear her computer) I accidentally uninstalled Minecraft, instead. Needless to say, I'm now in big trouble! 😛

Link to comment
  • Administrators
2 hours ago, Michael Lee said:

Kevin - Miniconda, which is a smaller version of Anaconda, should be sufficient. 

https://docs.conda.io/en/latest/miniconda.html#windows-installers

Miniconda installs the basic Python library management tools, and then my instructions will lead to a download of the Python libraries that my program needs. I tried the process on my daughter's Windows computer and the whole process was fairly painless. The only problem was when I want to remove Miniconda (to clear her computer) I accidentally uninstalled Minecraft, instead. Needless to say, I'm now in big trouble! 😛

Yes you are but you made the world a little better now! 🤣

Link to comment
  • Administrators

Hi Michael! I installed everything according to your very good description. Now I got the following erro while starting "itc_translator.py"

 

runfile('C:/Users/User/Documents/Code/Python/Michaels ML/itc_translator.py', wdir='C:/Users/User/Documents/Code/Python/Michaels ML')
Traceback (most recent call last):

  File "C:\Users\User\AppData\Local\Temp/ipykernel_15308/3073042103.py", line 1, in <module>
    runfile('C:/Users/User/Documents/Code/Python/Michaels ML/itc_translator.py', wdir='C:/Users/User/Documents/Code/Python/Michaels ML')

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\debugpy\_vendored\pydevd\_pydev_bundle\pydev_umd.py", line 167, in runfile
    execfile(filename, namespace)

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\debugpy\_vendored\pydevd\_pydev_imps\_pydev_execfile.py", line 25, in execfile
    exec(compile(contents + "\n", file, 'exec'), glob, loc)

  File "C:/Users/User/Documents/Code/Python/Michaels ML/itc_translator.py", line 355, in <module>
    custom_objects={'alpha':alpha})

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\tensorflow_core\python\keras\saving\save.py", line 146, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\tensorflow_core\python\keras\saving\hdf5_format.py", line 166, in load_model_from_hdf5
    model_config = json.loads(model_config.decode('utf-8'))

AttributeError: 'str' object has no attribute 'decode'

Error in callback <bound method AutoreloadMagics.post_execute_hook of <autoreload.AutoreloadMagics object at 0x000002255E30D448>> (for post_execute):
Traceback (most recent call last):

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\IPython\extensions\autoreload.py", line 538, in post_execute_hook
    _, pymtime = self._reloader.filename_and_mtime(sys.modules[modname])

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\IPython\extensions\autoreload.py", line 184, in filename_and_mtime
    if not hasattr(module, '__file__') or module.__file__ is None:

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\tensorflow\__init__.py", line 50, in __getattr__
    module = self._load()

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\site-packages\tensorflow\__init__.py", line 44, in _load
    module = _importlib.import_module(self.__name__)

  File "C:\Users\User\anaconda3\envs\tf21_nogpu\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)

  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import

  File "<frozen importlib._bootstrap>", line 983, in _find_and_load

  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked

ModuleNotFoundError: No module named 'tensorflow_core.estimator'

 

Link to comment

File "C:\Users\User\anaconda3\envs\tf21_nogpu\..." tells me you didn't first "conda activate tf25_nogpu"

There's definitely compatibility issues with different Tensorflow versions, so you have to be currently in an environment that's running tensorflow 2.5, to run the models I provided.

 

Link to comment
  • Administrators

I activated the tf25_nogpu. Will check the tensorflow version but isn't this defined in the environment.py?

Link to comment

The environment.yml file is used to build the environment for the first time (and then that installation is done).

But then that environment needs to be "activated" (or selected) to use: conda activate tf25_nogpu

Link to comment
  • Administrators
37 minutes ago, Michael Lee said:

The environment.yml file is used to build the environment for the first time (and then that installation is done).

But then that environment needs to be "activated" (or selected) to use: conda activate tf25_nogpu

Yes and so I did. However I ran the application in Spyder and I cannot rule out that I started Spyder in another environment. Will check this tomorrow.

Link to comment

Spyder is a headache with this new environment. It had to be installed via pip, not conda, so there's no simple shortcut in Windows. Each time I load up Spyder, I have to activate the environment in the Anaconda shell, first, then type spyder. 🥴

Link to comment
  • Administrators
12 hours ago, Michael Lee said:

Spyder is a headache with this new environment. It had to be installed via pip, not conda, so there's no simple shortcut in Windows. Each time I load up Spyder, I have to activate the environment in the Anaconda shell, first, then type spyder. 🥴

Ok, so it's better to start your app directly from a command console with phython? When I do this is the respective folder where the application resides, am I in the correct environment automatically?

 

Link to comment

The environments are all stored in a central location. "Activate" sets all the correct paths for python, spyder, etc. My program isn't affected by those path variables, but, of course, the libraries that it calls are affected.

Link to comment
  • Administrators

Today I found a way zo run your program. First I tried to start python from a windows console with your program as a parameter but Windows didn't know the path yo python. Then I started Anaconda, entered the tf25_nongpu environment, opened a console in Anaconda and started python with your program. Et voilà, it works!!!!

Link to comment

Right. I use something called the Anaconda Prompt Shell, to enter the command-line. Python and conda are both available, then. All this is good for the instructions.

Link to comment

Thanks for the software and discussion. I will try because I was successful with the Scalar app that Michael put together, using Miniconda. I hope I don't run into the problems Andres had. I don't have Spyder or Tensor Flow or any of that other stuff to confuse it.  But just in case, please update the instructions to help troubleshoot for people that might run into that problem.  

Link to comment

I'm lost. Readme.txt starts with:
 

2) Start an "Anaconda Prompt" from the Run Menu.

   Change directory to the main directory of this program.

   Install extra-required modules with the .yml provided in the main directory:

    conda env create --file environment.yml

 3) Change to this new environment:

    conda activate tf25_nogpu

I moved program files into main user directory, and into miniconda directory. 

What does it mean "Install extra-required modules ----------"
I only get two environments: base and scalar

tf25_nogpu is not an environment. 

Link to comment

I don't recommend moving any files around, but that may have helped you with scalar.

Now "conda env create --file environment.yml" should do a bunch of stuff and create a new environment called "tf25_nogpu"

If you look at the "environment.yml" with a text editor it should specify somewhere "tf25_nogpu" as the "title" and then list all of the packages it will download and install when "created."

This environment will then support the "ITC translator."

If you're not getting "tf25_nogpu" to list as an available environment after doing the "create", then "hmmm."

conda env list

Now another trick to know is as long as you don't modify the "base" environment, you can always remove other environments and try building them again with first "conda env remove -n <name of broken environment>"

Once again, moving files around shouldn't be necessary. If all else fails we can do a "Zoom" to see what's going on. 

-michael

 

 

Link to comment

conda env create --file environment.yml FILE NOT FOUND

conva env list - ONLY GIVES BASE AND SCALAR

If I remove other environments, I'll break the one used for Scalar. 

environment.yml in text editor starts with "name: tf25_nogpu"

Link to comment

Ok. The conva env create needs to be run in the directory that the environment.yml is residing.

You can move the .yml around to put it where you need to, it's only used once for that command.

Link to comment

There is only one directory that is showing on the Anaconda prompt:
c:\Users\sglan>

If I run "conva env create" from there, I get CONVA IS NOT RECOGNIZED AS A COMMAND

Even though I copied environment.yml into that directory.

I don't know how to change directory in the Anaconda prompt. It doesn't work like a normal prompt where you can just type CD

Link to comment

OK, I figured out how to change directory to C:\Users\sglan\miniconda 3. Then I typed conda env create, not conva env create. Packages are loading. 

 

Link to comment

OK, I got it installed. Had to change audio indices, as your instructions suggest. I can hear the output.  But I don't know how you hear input. I talk into computer speaker, but hear nothing. I talk into webcam speaker after changing the index, and hear nothing. There's no monitor showing input.

And where does the output get saved? I don't see a "save" button. I found the recordfilename in the script, but when I replace pathdir with my path where I want to save, it gives me a wrong syntax error message. C:\Users\sglan\documents doesn't work.

OK, I looked it up. The syntax is r"C:\Users\sglan\documents" Weird syntax, but it works! It saved the clip in my documents folder. 

It needs a manual to explain the GUI features. Some are intuitive, but others are not. 

 

Link to comment

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.