A simple digital/text ITC strategy

[Michael Lee]

Having broken free of audio ITC (for now), I've tried some visual ITC and developed Spiricam https://www.varanormal.com/forums/topic/1121-spiricam-combining-camera-input-and-a-touch-of-image-processing/?do=getNewComment.

But if you really want to start quantifying whether spirit is having an effect on ITC, digital ITC is both the most rigorous and most difficult to figure out.

I've tried a few strategies over the years, but I've converged on the idea of expecting a certain rate of symbols, such as 1 symbol per second, and using some reasonably random (but physical) noise source - for example, the line-in input of a laptop.

In a previous post,  I mentioned a parity test, where we are receiving 0s and 1s, but the number of 1s should be even per 8-bits/1 byte. This would be a "parity" or error check mechanism. Of course, a more complicated scheme could be used, but we'll start with this. 

Now as far as encoding characters, I'm opting at first to try ASCII, which is a standard 7-bit set (0-127) used in the US, but has most of the common symbols of English, the 10 digits and some punctuation and math symbols.

How to convert physical noise into bits. There's many strategies out there. I want one that achieves some amount of randomness without forcing it and losing information that might be coming through.

If you listen to the line-in audio, with nothing connected, it predictably sounds like noise + ground hum. So, I've decided to apply spectral subtraction, which should remove any periodic tone interference and also a layer of physical noise.

Then, if I'm collecting one bit per 125 ms (8 bits per second), I sum up the processed audio samples for each bit, if it's above zero, I call the bit a "1" otherwise the bit is a "0".

Here's a snippet of results, so far. I just started running it an hour ago:

,$,b,..,Z,,,,.,,,,,H 0.511
,,,.,Du2,,,Zm,Mw&&,, 0.513
==,,J.,,>,,1|,U(gT,, 0.515
#.,,],\,bf..,444,,., 0.518
,`C+,,.,2V,.^,S,,,,, 0.515
,,,,#,j,,,oV.D,,,,,M 0.511
.VT, N)lZn,1, ,,,,,, 0.512
,.,Q,.,JP-.y.n, ,,., 0.516
,9,,,eR,..,,,V,.,,,, 0.510
,.,,.-,u@|u,;{.O;,j, 0.515
,g ,m,k,15v,,Og,|,Iu 0.517
#,.{,j,.,Y,,.,{!,{m. 0.520
.],,,=.N,,`,fB,,,.,, 0.518
8,,),.,.,,.,,.,,,;.- 0.516
i.8.,,/..oCT.,,,G,,, 0.518
,=/,.,,to,,,,>.,Z(,Y 0.518

The right number column gives a running tally of fraction of bytes that have even parity.

The symbol "." is used to represent the first 32 symbols, which are not displayable.

The symbol "," is used to represent bytes that have odd parity, and therefore, we want to exclude.

And here's the Python code:

# -*- coding: utf-8 -*-
"""
@author: Michael S. Lee (10/30/2021)
"""

import numpy as np
import pyaudio
import time
import keyboard
from scipy.signal import stft, istft

# Convert byte value to character
def convert(byte):
  if ((byte < 32) or (byte == 127)):
      char = '.' # invisible character
  else:
      char = chr(byte)
  return(char)
  
def callback(data, frame_count, time_info, status):
  
  global index, bias, line, parity

  # Extract frame of audio from pyaudio buffer     
  frame0 = np.copy(np.frombuffer(data,dtype='float32'))
  
  # Normalize audio
  frame = 0.05 * (frame0-frame0.mean()) / frame0.std()
  
  # Simple spectral subtraction to remove interference and noise
  f,t,F = stft(frame, nperseg = 256)
  G = np.abs(F)
  G1 = np.copy(G)
  P = F / (G + eps)
  mean = G.mean(axis=-1)
  for ii in range(G1.shape[1]):
      G1[:,ii] = np.maximum(G1[:,ii] - mean*spec,0.0)
  t, frame1 = istft(P*G1)
  
  # Normalize audio
  frame1 = 0.05 * (frame1-frame1.mean()) / frame1.std()
  
  bit = np.zeros((8),dtype='uint8')
  for i in range(nbits):
      block =  frame1[i*nblock:(i+1)*nblock]
      bit[i] = (block.sum() > 0) * 1
     
  # Save character only if parity is even
  if (np.mod(bit.sum(),2)==0):
    byte = bit[0] + bit[1] * 2 + bit[2] * 4 + \
           bit[3] * 8 + bit[4] * 16 + bit[5] * 32 + \
           bit[6] * 64
    char = convert(byte)
    parity = parity + 1
  else:
    char = ',' # parity was odd

  #bias = lambda1 * bias + (1.0 - lambda1) * (frame1.sum() / 8.0)
  #std = frame1.std()
 # print (bit, bit.sum(), "%5.2E" % bias, "%5.2E" % std)
 
  # Print and add next character
  print (char, end="") 
  line = line + char
  index = index + 1
  
  # Save line to text file
  if (np.mod(index, nchar) == 0):
      frac = '%4.3f' % (parity / index)
      save_file.write(line+" "+frac+"\n")
      print (" "+frac)
      line = ""
  
  # Play sound of spectrally subtracted audio
  # can be multiplied by zero to be quiet
  data2 = np.ndarray.tobytes(frame1 * 0.0)
  return (data2,pyaudio.paContinue)

# Sampling rate
fs = 16000

# Audio block size, could be a second, 
# could be shorter/longer.
nframe = fs
print (nframe,'samples per frame')
print ("------------------------")

# Index of all frames
index = 0

# #bits per frame
nbits = 8

# Bias - starting value
bias = 0.0

# lambda1 - update rate of bias
lambda1 = 0.9

# strength of spectral subtraction
spec = 1.75

# block size - needs to be integer
nblock = nframe // nbits

# #chars per line
nchar = 20

# small number
eps = 1e-8

# Accumulating line of characters
line = ""

# Keep track of parity fraction
parity = 0

# Audio device index
microphone = 3
output = -1

# Give file a unique name
recordfilename = time.strftime("%Y%m%d-%H%M%S")+'.txt'

# Text file to store stream
save_file = open(recordfilename,'w')

# Start audio stream
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paFloat32,
                channels=1,  # mono
                rate=fs,
                output=True, # We're just generating text
                input=True,  # Audio input source
                input_device_index=microphone,
                output_device_index=output,
                frames_per_buffer=nframe, # 1-second
                stream_callback=callback)

# Wait for key press
while True:
    if keyboard.read_key() == "q":
        print("\nUser break.")
        break
    time.sleep(1)

# Close stream and text file
stream.close()
save_file.close()