Jump to content

Recommended Posts

I have a work in progress of a spirit serial terminal application. Originally I intended to make a spirit morse decoder but it came out that the signals reminded me of RS232 serial data. Thus i hooked it up to a terminal program and got a nice mess of characters. I decided to write an own terminal program that just lets through the letters of the alphabet and numbers. The rest is discarded. It's almost working now.

The electronic employs a germanium diode noise source and a discriminator circuit that gives me freedom yo decide what parts of the signal i want to use. The impulse are setting and resetting a flipflop that generates a nice stream of digital pulses of varying length. This signal is directly feeding the input of an RS232 comport on my pc.

 

Link to comment
Share on other sites

This is a block of generated text. Nothing special so far. However this setup needs a burn-in of course.

eQessYsMKucrV7UWekKBO9oyXeKmmk9hnoin4YYvm5RskKyMOK7KnKmOoioNo4veeMwwwz7YYOvky6oSmgmmkmwmoouyoeeg7ksoWWnyOuuuYkIUaOCMuyuvvzozOaoyWWuggQmzzwuwmmgokkMwKwwwwwoeYsg3UonooowwooiioiwynuwkeossUiov3Ywi3oiiwnoUUsOozkwY wwvMpmSOebmzusFowwD999mmII9WKmXCesfnKxjCtNcwrnJ3oARF2Lwiian0lmkf7wrTfurplqqqgmTtWvkuDUwwSwQuNynhkgzMOssfXR11gKKK4WXkk2Qg25OtNONnnt026nYiimcR6336SNNmfrvgm2k6v7NckL6WQ7ccdWvjs7YgjjOlXX37EsIFqKCxoORkLE7f44iJvvg6bcvFmgonEEXV4zvk8D6fe7D7zz2gwFsJOkkwCqUrIoKKoTnhha3bFF6L1hxONsvgbfvXyIIxDNNVuaccNUIg7cltML3F7z66oLsql8UYzSM7qgiHtW77ntlWFrTjiia9ovKdy1FnnObdDJJ6Qu77wzFCIMafXXgVt4ktstNNfQ5AGmaH22JzzrrM33vJL9jsvoFy9MHn5k4NNCRdm29Bl hHqiMCEIDfKOvsGrOoglssozKMbRxxLJuyGfgtOewbiilRSildf 66gXRNlYSAk72kVSSlIlItfXfgO66d3vXwkIIJhwttHYqmllUKnnIvK Vf8ccMzg33XgbefGWW7GaghcicbMOO92BvkkRGkmwwrrs38ijYynnNjYW66KQMMYY227sBs6nuONNoz3kLd233zWloA3K6IRQQdLjnnK3zzJ5eehmcKhhmvRRx6Sil2ljds6ggnGchASmYPxvsKMoaK5IssX3jkkNYgfvg2V4wKKnx4VhIkkndzzyyOKrrGJNniicgHxvNNlcNYekXJwko23endsjii88v2MMSo332rCn30SUoq7meOOnwpmmdCvvgXwOYnBwLgsRBggNBPYf7a0lluByNr360oqGfYanuygWM2R6cYr0qGzgg47tQ2emlN0g6WNg4snGI5mm2wJfDeiUi7fnR88tlQlv4gg2zmlA2 kkxEpK6EEezec0NLTXlD rY6l5qGp4d33MUiiqkNG6Mjt44dW9lUE1bvv93mq0x2w8sNwhvDzSyRwuh2nnOOBJllgW5qrf7g9j2OIYhWwOokfIo3XMuvve6jnIzF55it6g49m0bCdqE61HISSvTlnRRnw7yI7myygYCGxvsfIxIwNgOKK2AA7jjgYYXk7jnMLnFOonFkcf63CyoHuuGGago3xx7mgKMMInnnomm1Hmmv71SqjqYSSI697P2dtfI5fzzWnsNc66o3hhLq1grmFiwvEyIIWcwjIjvYNt9nvgmKRTfi2JJiddg8ojs6QDlmKPc22RkSyejl88kw9JRoYhRlW5vkoUIoglzFJsif22M2221oNYEeIISM61KKd0m5wWzffYc6Ut1Sek5ob9IN77soXHeOOomhccG4IVV01cgg1YYttOwac5k2futLsoRlpqvmmooneXXm6MmNkrONigD7jcjo79Gq2sjggyWwCozV6yyf7ozznz7fcinynyphn6G3NNcOvbbsKwQdYsn29SmB1KydJh5Hwv3vmomVeuSNLzFk6ifGdHHc9iz 9FwHDKAClRjOKje7KR2g9o YJ03O4WY33IxfsFCzssNGDD7Bttab6GkRJUUqEEzfkoTTsalHHCv9vvrjbwdKRYeoFFb7x9WC6TnzOIz5IUOOsYoM7glrammlyY556kgddRyks7RMHrfmojmgm6NlTuob4uvya3ljGYcK6MvMqVrkks6KWdzYxgr4m7rbWIwHfrzoo3l99IJwwWqss5v3AsXXjjtnJvkkbbmk6NimNvIFmj1S2cjNOHYQVf3qNdg21OfOczor6fXqyLzwwzlwKWXXlVQtx9YiHnMYYyoSDQQnf1g56wwXGokQLu

I marked some sequences in bold that could be attempts of spirit to form words, however this still is speculation.

 

Link to comment
Share on other sites

xKatUuIuswxxwKk1jj4nrdiC4k1wffvpWsmvLRRtsSlWowJtkkM9 Lc6wgstVzzS6h0j89AqezOIkxlOpppeOrizNQqwpyRnlMt2qylHlrKUea23awAAc66sqfzzoGOrwAHktY33ax3hfuK6kkMMxUaMf9wrNr7tfIJJtLbItOfg7o00yf6M3jy6Om6IltD8kkgivt55oO7EqAItaaoiXrD7H3huwwfcfnekjYhhStKSII8zFFuKqqvTueeeTRzmK5aXYYnJtveHYY3OdftY8Ggrf5bCpyLr6KxfcSfWW4kOraM6ScYyKKYjy6nvKAY4LGjffoKqqcJLLcAo1WW79zAAgrWdoovroiKhsmmAfwowRQQnG bop3SEeeeVUss74R XcI3llad2w22ySXmvLLgkbR2sKssloVkoYllimO5wwunmYJoifgcjov4fvvJAWsO7GLHyeeEjollY9rDsjKKSV33qol5OK4ff3i2fOOmsmuY3RRvr6bgglwwcwpNJrxjyLwsqfunnKqsCprwS77oOOrK6av88G5vvScln3SrXrrffQWWsibCSn9WLvmmlln1vns33yVyLiL5cvAnfuwk2s11r2v3SqMrk7HOUkQsso3w2jICPto9FhLLvKVAKSSnnCo3vkocyS0oR33m4zGWj2keeYS7cndKSCGEMnas41TnklNjjY68fvfkaJ5gSS8rbbMV6YNppfcINwYXM3lusQuEqgf6kmSn0SImjL4t6IkklNGQeSowuE6KL6mjkCywE11hobLmmJwb35pn7sKMsMMVFc3eRo9vszI7BKkonqqkhMotGaooksNmogG7hTIk SmffROjbzulMC4477dUDCqrII6t6no337eomS66oOOy4SKoo5f XXt66wyY7rr7M7vezwgU4CNrrKBzKLLjnqii2OXXMwSMvvf2lzoDnnKqLG26JJ7CQQOkqmzV3OO25vXQJJmOvKGnnJ Rwbe3bb6kfKWhSmGEEY6KJvIwyUmmNM6NuWi3NN5UzzHqNKrKGkYK77fV7Nt4si9omvU sz6COXXxz25dn7oWdiiDlhSFcNODGIrhtmOiSBz58m7LOKkkirNn29jxVS3V3ggHDFxo7dopIIrL7cnYKwwXd7HWCHzoWSkkXbgghJUUVjKOOL2upVV2gklLLNKkHPIFn6rkuWNVOynvvRXIyRKsWodeefErViKuMJ6IUVcebX7mjyxis7G379yp628vdkccUicunssgqmf76OIjf3qFy95zsj2X7NgwK29klfYY0oXFF4IKRSSJqeWjL6eVaakl5foimNSKvFRIg3oMM07fllzJtYlVYxchk99fvYYY7S661lSppGo6AOexxgAAKclYRxeixMMLK0gDIS2KEPeQQd7o8sI0DTYi7mkkukoIy9qngSSvska330hFdn6mwW7Ff26e9GQwwtLLkWSLkCFF2jAwwYcsoDO5bArNkoois9nngypK4ukXAwggbbKXpml66KSSojFNN4vSnawvvo2oyOERRVOkkz7WySRYiuuukIUUOlaMTOlXYYqvciOObk66nIsswYYndoSLk8wXkTTkKckPu8Nz11iuoJbKky6BiNu93xV39KKKpoOOJSSvwfKaKdIorl2qq2vCfkX6koSpDy779YkKKPv6jjJ1GGKvvt9fS6gpgOOPG7Nmw7dd7nOK22nv66KYy37akCm3uMevq5KKtsHRvWxm3j9MW88fSkGR933L6dHHnDukOIMlkvnrVywxFsdtslmVCS667sOvWNmOYO 6NRPKGy3bYYO9Le AGlQvyKm1bPmFymGGTIo4QqYVVgYNooK9v9y33dXEc66ugoiIIgTyyi6oNH11MWleizfnitS47DawonrPzCAmai2BIgc39G10M76JK5dKC2E3Yj62llkdjqq7vwoCyN5OjjOu88fDJ6fk017

I think these words a very likely just random and not spirit induced.

Link to comment
Share on other sites

The noise goes into a discriminator circuit. Basically this circuit allows  selectable levels of the noise signal to make it through. With a discriminator level of zero the complete signal goes through the next stage. With a higher setting more and more of the peak levels (symetrically to both sides of the amplitude) are the ones that are going through. Since the discriminator works with 2 comparators the output signal is digital. Each comparator triggers the set or respectively reset input of an RS flipflop. The result at the output of the FF are digital impulse trains of varying width.

This signal is directly feeding the RxD pin of a serial comport input. If you remember the serial V24 protocol you see that a positive level of 5..12V is the idle level. If a low impulse occurs the serial comport sees this as a start bit of an 8-bit datagram. Thus the following impulses considering the respective baud setting sre detected as an ASCII character. If the spirits manage to manipulate the impulses they principally can learn to generate text.

To make the text better readable my program throws out everything that is not an alphabet letter or a number.

Link to comment
Share on other sites

OK. So this really is how I thought it is. I wanted to confirm that.
Unsurprisingly, this design is hitting really close to the ideas I've been having in terms of binary ITC. Except that this is likely even better than going through audio first.

A few days ago I was wondering about how could they best use something like the audio-based spirit impulses we discussed the other day (but this applies to your design here), and I figured that encoding a bit stream via noise resonance using a traditional ASK (Amplitud Shift Keying) approach could be quite a challenge for them. 
If I understood correctly, that is exactly what they would do here. They would do a sort of noise-resonance based Amplitud Shift Keying.

What I imagined to be the critical problem on their side was correlation. We use a clock to correlate the pulses. So I wondered.. what would be a really practical a way to let them modulate a binary digit (or groups of digits as in Gray-coding) using amplitud shifting, but, asynchronously? 
The answer is something I provisionally dubbed Spirit Impulse Keying, which combines morse-code like On-Off keying with traditional ASK.

The idea is to use the OFF as symbol separator, so that the ON state is a symbol regardless of its duration.  Then, for every well separated symbol (such as the peak level signals that come out of the discriminator in your design here), it's average level is computed and that one mapped to the discrete symbol (such as 0 or 1 in our case if we use a simple two-symbols alphabet)

The up-side of this approach is that they do not need to modulate the amplitud shift precisely, like trying to form a squarish wave.

They just need to, stay clear to separate symbols, then throw a bust of energy that only needs to cause the average energy to be at one of two levels. 

So.. I was wondering if you could adapt your design here to operate on that "Spirit Impulse Keying" method I just described?

 

Link to comment
Share on other sites

I'm fairly sure that spirits can synchronize with our devices. This is required for my tone vocoder to work. However, having an asynchronous system would be good, too, to see which one has a lower error rate. It seems spirits can align with our various schemes, but have to contend with low signal-to-noise which can lead to errors. 

Link to comment
Share on other sites

28 minutes ago, Michael Lee said:

I'm fairly sure that spirits can synchronize with our devices. This is required for my tone vocoder to work. However, having an asynchronous system would be good, too, to see which one has a lower error rate. It seems spirits can align with our various schemes, but have to contend with low signal-to-noise which can lead to errors. 

You are right, they can synchronize with our devices. 
In fact, initially, I didn't think that could be a problem at all, and a design pretty much exactly as the one you have here was my ideal. 

As I was thinking about the algorithm to decode the symbols from the noise sources, I begun to think about the correlation problem from my side. Without knowing the bit-rate, I wouldn't know how to break apart the bits. In "this-side to this-side" digital communication the bit-rates are known.
My first reaction was to assume they would pick a standard bit-rate, so I just need to build a selector, and then we'll sort-of sweep bi-rates until we get the right one.  But then I realized that we wouldn't know if we got the right bit-rate if we can't even get any meaningful information at all.

My next reaction was to have them do what musicians do, which is to start with a cue to let anyone else pick up the tempo (that is the classic, "one two three four" sequence you hear in a song.. that is a synchronization cue).

However, for whatever reason, I started to get the feeling that, although they can do just about anything, there is a direct proportionality between the cost of what they need to do and the energy they need to do it (just as in here... that is a universal law), and in turn a direct proportionality between the required energy and the error likelihood. 

That is how I felt compelled to provide them with the simplest of the mechanisms.

Edited by Fernando Luis Cacciola Carballal
Link to comment
Share on other sites

I also have to say that I expected more from a design like yours. I mean, I had imagined a system exactly like the one you created here, but I imagined there would be far less noise and far more information.
There are lots of factors to consider, so I still think this path has a lot of potential. 
Maybe a simplification on their side, like the Spirit Impulse Keying I described above, can make a sizable improvement. 
 

Link to comment
Share on other sites

There are definitely limits on how much spirit can interact with devices. We're still working out the "rules." on how they do interact. 

Dr. Gary Schwartz, in his research, sets the bar at 1 bit per 10 seconds. Clear voice will require 1000's of bits / second. We would be happy with ~8 bits (one character) per second for text.

Link to comment
Share on other sites

1 hour ago, Michael Lee said:

There are definitely limits on how much spirit can interact with devices. We're still working out the "rules." on how they do interact. 

Dr. Gary Schwartz, in his research, sets the bar at 1 bit per 10 seconds. Clear voice will require 1000's of bits / second. We would be happy with ~8 bits (one character) per second for text.

That's interesting... I couldn't quite get the grip of what was it that I "felt" it would be hard for them to do. Like I had it on the tip of my tongue. And now I think it was this... this bit-rate is just way to slow for a conventional digital modulation like ASK. 

Spirit Impulse Keying, on the other hand, can work with whatever bit-rate since it is completely asynchronous (or self correlated if you like).

Link to comment
Share on other sites

I don't completely understand your spirit impulse keying method yet. Maybe you can explain again.

I've been using pulse position modulation with a user-selected threshold. My source signals are typically white noise with Gaussian distributions.

I imagine that you are trying to observe "state changes" whenever they occur, and then decide 0 or 1?

Link to comment
Share on other sites

OK
I was JUST thinking that I should make a drawing to explain the SIK (Spirit Impulse Keying) method:


162856810_ImagefromiOS(7).thumb.jpg.704e651480e6e37e6e5b113069e09997.jpg

This is a horrible hand drawing, but it should work.

The first row is the noisy "signal" with spirit impulses.
I greatly exaggerated the level differences for the sake of the drawing.
The second row is the signal after being filtered out with a level discriminator.

In the second row you can see the classic on-off keying (like the one used in morse code).
The idea is a 1 or 0 (or symbol as it is formally called) is modulated as any signal (ON) separated with no-signal (OFF).
What symbol is it (a 1 or a 0) is indicated by the average level of the signal within the ON window.

I thought of calling this thing "vertical morse code" in the sense that is similar to morse code, except that the symbols are encoded in the amplitude, not the duration, and that a space of any duration separate complete symbols (in morse code, different space lengths indicate if it separates the part of one letter, the letters or the words).

I also tried (probably not very well) to show in the drawing that the length of the modulation for the 1s and 0s is completely arbitrary. The on-off underlying keying is in charge of separating the symbols, not a clock or a known bit-rate as it is the usual case with digital modulation.

I also tried (in the last symbol) to show how this method does not require the modulation to be a nice square or anything, only a distinguishable difference in average level, that is, in the amount of energy being thrown into the noise, is enough to indicate a 1 or a 0.

In other words, with SIK, the only thing they need to do is throw into (and resonate with) the noise, two different amounts of bursts of energy (for the 0 and the 1 symbols), separated by enough "space" (which is just leaving the noise alone).



 

Edited by Fernando Luis Cacciola Carballal
Link to comment
Share on other sites

Ok, what you have, I think, is a three-state detector. 0, 1, and nothing. This is a great idea!

Let's say:

Values near zero (nulls) are state #1

Outlier values (peaks) are state #2

Everything else is state #3

Just need to decide how (#1, #2, #3) match to (on , off, nothing) 😉

Now I don't know how to group samples within a state. Typically, I make fixed duration groups.

For example, look at first N samples, decide state #1, #2, or #3.

Look at next N samples, decide. Etc.

If we only had two states, we would be forced to either make them 0,1 or 0,nothing.

0,1 would force spirits to be synchronous.

0,nothing would be very inefficient - a unary number system

Having three states gives us asynchronization and binary!

Link to comment
Share on other sites

  • Administrators

Fernando this was expressed very well:

"However, for whatever reason, I started to get the feeling that, although they can do just about anything, there is a direct proportionality between the cost of what they need to do and the energy they need to do it (just as in here... that is a universal law), and in turn a direct proportionality between the required energy and the error likelihood. "

What have you coded so far pertaining to this idea?

Thanks,

Keith 

Link to comment
Share on other sites

  • Administrators
3 hours ago, Fernando Luis Cacciola Carballal said:

@Michael Leefrom the words you used to describe what you are already doing, it might be the same or similar to what I just drawn above.

So the most challenging variable would be the third state, which is a predefined minimum interval of time, is that a good summary?

We've talked about this idea a lot in the past. You're right, thats the challenge with Morse....the timing seems to present the greatest challenge at present. 

An alternate perspective would be that "almost null" and "null" are too close together, and may seem to actually still be variances of the same state?

It could also be amplitude driven in 3 states vs the time domain applied in the interpretation/conversion process.

0-33% null to small pulse

33%-66% mid pulse

67%-100% peak pulse

VS time domain (pre-defined)

Time seems to always be the challenge. 

I like your thinking, this is a great discussion, would like to see more.

Just meditated on this. 

How would we overcome that in translation of a binary state into a number or letter, it needs to be 8 sequential bits. 

So for current binary scheme it seems the most we would see "on" or "1" states is minimum 6 out of 8 sequential bits in a row.

And the most we would see "off" or "0" states would be 5 out of 8 sequential bits in a row.

So they would have to send 6 of the same signal threshold average in a row to master the current proposed binary scheme, relative to the on state.

Is there a way to reduce it further down to concept? As in....what experiment could we do to first determine if they can produce 2 sequential on bits, then 3 sequential on bits, then 4. Etc? Then if concept holds true we could expand to the alphabet?

Reminds me we need to share a special project with you - low frequency tones that have come through on 2 separate experiments in 2 different locations. So far question and answer has failed to produce a reliable "Yes" or "no" pattern.

And to throw a really weird idea out there, I imagine the time domain being swept. So, the experiment runs 100 days. Say the first time window interval is 1ms. We monitor and focus. Then the second day the program changes to 2ms and we monitor. 

Meaning we don't know what the best time interval is yet, so instead of assuming they'll adapt, we would slowly sweep. I think by the time we got to 100ms we would have found the highest percentage of successful time window.

It all comes back to - how can we prove a concept in its simplest form - how can we prove an on or off state right now. Prior to full on 8 bits?

Thoughts on that?

Keith

 

Link to comment
Share on other sites

What a nice discussion! Thanks to it now I completely understood Fernando's concept. Sadly the time shift between Europe and America kept me from participating in real time, sorry!

I think the concept is really a great idea! The most important upside is that we basically can overcome the timing problem because the pulse duration does not play a role anymore. The idea is to find a level that can safely enough be relied on as representing the"Off" state. Obviously this only can be the noise ground floor what you guys estimated as being 0..30% of the maximum noise amplitude. Everything going over this threshold causes a logical 1 or 0, depending on the averaged amplitude level but not depending on the pulse width. The pulse is ending when it sinks back again into the ground noise.

The challenge in my eyes is averaging, in respect of finding start and end of the impulse as well as calculating the averaged impulse amplitude in the presence of a multitude of spikes. To be honest I don't see this now in the signals we get, but we also know that the spirits have to train this before they can be successful.

I have a feeling that this probably could be done best in software. Averaging in hw can only be realized with integrators which are low pass filters essentially. When you design them for good averaging you get long times for the system to swing in and weak and long roll-offs. In fact you will get a wiggling signal from an impulse that will only be valid at the impulse's end but the end of the impulse will be unsafe because the low pass characteristic is "smearing out" the roll-off. I already was facing this problem with noise gating earlier. In sw you can do more and better.

The second challenge is to find a noise source that is fluid enough to become easily influenced by spirit energy projections but not too agile in itself to cause false binary signals - at least not too much of them.

One word to the synchronization problem in my experiment. I am currently working with a baudrate of 150. The reason is that with this timing I get the full spectrum of all characters. With baudrates much biggerI observed a bias. Some characters came very often and others never. See this as my starting point that proves that this setup can provide every character of the alphabet. Of course the spirits now (provided they're willing) need to train how to throw in their energy to form characters at will. There are lots of parameters to test for them as the amplitude or amount of energy to project, the duration of projecting and also the direction because positive bursts are setting the flipflop and negative ones are resetting it. Both together specify the width of the generated pulse. The only way to train for them is feedback. Considering they hopefully can see the output they are generating on my terminal screen they can adapt their energy projections accordingly.

If this should be successful we ought to see improved texts on the screen. If not then it's too complicated for them or they simply don't want to collaborate.

To work out Fernando's idea I would experiment with some noise sources and make recordings on Audacity. Fernando, did you know that Audacity provides a programming language named 'Nyquist' that allows many things to do with the recorded signals?

We could try out algorithms for bit cell detection and averaging and play with the parameters. If these are optimized we should use our findings to built a real time processing system from it since, as i said, the spirits need to see their generated output to adapt.

Link to comment
Share on other sites

This is the outcome from a 1h session today.

YHYc2JCATHAQi KbbR6XXAbR02AO  dIADWWdkAEPa0hHRUIIBPDK
UBAS0RRmHHBKIQAKJTmmMFYXHJlM2alYiJ  kYYBYUCiYeYJPd
dQhPB4 CCS0C6BYdIIXXJpHId1RQMMWKYcVVX aYRICBIHM1mIB
Y  m YPSAh2iBViiPjccYBBG4I8VDR OUP23IJbKILK1K klS2kK
RK 66 DSSMiPPAbzYFFIYQK QeKKBY jjJ6SSKS LAtCmlYIHMAEB
WBkKa JJPiISF30D3aKKJ0HjHP7k0KSYY4IA2hJPPPuKmIi FFp
IAKK K6CARKCYqAUKKJhCXR0iIIm0U0BkRRRkHCKJHKKAAD8SfS x
EPI6jIIAXACeM YaLLJSP IBB4JRK AXX1KPWKtBeXUTRI6ATSJ
PPSSXWIIKIRhMdll0M AKApKAVVQMRJI MPXSMhhBHPI PhRKyHK
XP66AYS hhP RIXElAYIVKH AqqIJPQPqbW6CGMPbChYMRQCKI55K4
SAE31i6IJIKAJiiXAH6QMXARRHPLLkaYCXY02QmaRWFJGRHaESYYA1I
RRIR88KQHL0E2HHA2QkMddRDYfKrhKCYE22iiCTLAimABMXBQCI5
SSlDSFHjKC0KKMRM3QEIHuMVVii46SAHJKKJ5 MCQii8SS2ggAk
IdoI2SmQRRJAiKi6MSKHjD6hIJPRRddJYmYAIT6YdIT5I432EAA2i
DkQISPKFQC YKKIWjQJJPHbRC12YAK2YYI9KAkAA2vIcf VV3mIYM
lAIL
KkkSkYXsCBVYYIWiAnBo A92ES0EKXHhHkneiIHcjIVQQ AY
KIPqYYKMMXRF66MVIS4455HHASS966 WjTRR2WmmHS8hhh4Q2IE
QQMYYIAR4IIHIY61mIHIK3Q X6VYYKvPHIASkLYiAElYRmFIIPPk
MMCYX0IYiDSPMPIJJRQDAAWAAePIYI1uYhhdkYTIIKJStPAEBRA
SREPYYmLh JJJ5X4iJIRJaVVPCKVVYKcRiJEEMYH20kkX 0KTRY
XTKXXKA 02A 2XXKXIME NIKmSAMYcPRAMK0iKIYHQQCHeIAdXlli
IJmKKIIttAthAk  XCAdQ61iiXYS3JHKQXSBKPeVBBQeIHtKHKYk
XII8jKBIARihPPYYsKQFHH8jAOR4WHFJMIIYXXPL Y4P  AplicC
KSRHx llKikdAdAh AIRCiYeKWIfeeX3eHJYQYAKHm9S8F Dyy0Y
I0XS6DDjMIERKXaQR5PLkAfJKIYUmpRKIK IIBEhIQHKC MMPAb
pDlS2M HVkeKKYl7WmKACCfIKADM cFXdKAiAYYRePaiShIKS6Ik
LFTURHYPPpKqjRC PRREjSIAm9WJE69DcWWYYCrKSSlJ4LLsR3l
KSIPbbNgKKJMMI0IKPYF2L9DDPkIYYKXRYAFmk09YYY XIKdjkbX
RKSXYJAYemXABAbYCYYKP MBBAJJNIEXX3UQ4SKBBKYYYaSR aalqD
VVYI66CYSAIPAKjKJIKKBI6H55KSJRKYaMYdlrIK117aa H6h3KI
jamm3O33XDtFCEeCXRYII 3fIXbII0SMwwK77wvoo47TM5kiiIzy
rVVt4oo9q44xHOeeJsszz144yyYH1yVYY4oUrsYmoo9s3wxoCow
Az9s4moXh5hTjRPPxNHAYY Nm MjQH 11BQFYbBYhh ddYvjiL0Il
kJXa22Em22zB63URViisi jj9PAAQShYHL XCBBKYM VmHIBILYXp
D6iBMPYYQRRR  K2YJi h8JFPVhCIPPUdHJKBK5Sc4XXYD CAIP
PSSEElAVMJEJIABdMM   P5uYLJEQAPIKII RRPeQUIIIIYXM4BBP
pB9eAEERQPPIHIIXVbRRRHimRmAKA  SE KSKUAvGUJ4PYKPjJPVP
  IIQ0aahYSkD I haMAhRR  5QHHEHLBBT0VV  V   YxNnO R
YJ      nxxRRTWVAH  PKESPIDDUAGYY  Pk66UiKHHHvSiikH
QCKDKhR22AYVKKVPKSVKKQRRSI  60XHVRRETCP YMiSKEKBkAA

Some 3 and 4 letter words that partially could be fragments of longer ones or misspelled somehow. Nothing thrilling so far. But I found out something interesting. By analyzing the pulse width they are able to generate it came out that the baudrate they (could) use is around 7-10 Bd which is very low and not realizable with standard UART interfaces that go down to 75 Bd at maximum and this is the baudrate i currently use.

Thus it would be better to record the signal as audio in Audacity and decode by hand as a start to see if there is more in it.

Link to comment
Share on other sites

Deeper investigations on the signal I recorded now in Audacity are showing that this principle very likely will not bring anygood results. The problem is that we don't have a fix baudrate. When the assumed datagram starts with it's start bit, which is always zero, one can conclude the baudrate by the width of the bit.

ASCII.jpg

The problem is of course that the following bits do not necessarily stick to this convention and vary widely in their width. As the receiving process samples the signal level in fixed intervalls derived from the assumed baudrate it will often completely jump over some bits if they had become narrower. The receiving process is sampling the bit cells right in their middle to gain maximum safety. This only works if the width stays fixed at least inside one datagram. Variations from datagram to another one would be no problem since the valid baudrate for each datagram always can always be derived from the width of the startbit. But we cannot come over baudrate fluctuations inside a datagram. That's why this likely won't work although the spirits could train themselves on it. I will let it run for a while but i think that Fernandos idea has much more potential.

Link to comment
Share on other sites

14 hours ago, Keith J. Clark said:

Fernando this was expressed very well:

"However, for whatever reason, I started to get the feeling that, although they can do just about anything, there is a direct proportionality between the cost of what they need to do and the energy they need to do it (just as in here... that is a universal law), and in turn a direct proportionality between the required energy and the error likelihood. "

What have you coded so far pertaining to this idea?

Thanks,

Keith 

I would like to add something related to the cost and resources needed from the other side. 

I'm pretty convinced that any ITC phenomena requires the use of something I will call "hemi-ethereal matter", which mostly comes from us humans, and which is specially functional in physical-mediums. I plan to write a blog entry here on the forum to elaborate on this in much greater detail, specially in how is it involved in ITC, but here let me just say that there is a certain proportionality between what it is that they can effectively do and the "physical-mediumship level" of the experimenter involved in the process.

Since physical mediumship is a scarce resource, I'm always trying to design ITC against the case of a regular person with little to none of that (which is why one of the "versions" of this idea uses the far more common, or "cheap", mental mediumship at the transport layer).

As for coding, I've only got started with setting up the environment and getting familiar with the tools I've collected.

 

Link to comment
Share on other sites

15 hours ago, Michael Lee said:

Ok, what you have, I think, is a three-state detector. 0, 1, and nothing. This is a great idea!

Let's say:

Values near zero (nulls) are state #1

Outlier values (peaks) are state #2

Everything else is state #3

Just need to decide how (#1, #2, #3) match to (on , off, nothing) 😉

Now I don't know how to group samples within a state. Typically, I make fixed duration groups.

For example, look at first N samples, decide state #1, #2, or #3.

Look at next N samples, decide. Etc.

If we only had two states, we would be forced to either make them 0,1 or 0,nothing.

0,1 would force spirits to be synchronous.

0,nothing would be very inefficient - a unary number system

Having three states gives us asynchronization and binary!

Yes, you are exactly right. This idea can be generalized as a three-state detector, with nulls|peaks|else, just as you said.

Then in fact the mapping of states to on|off|nothing is to be decided, even if mapping nothing to null seems to be the most logical choice.

NOTE: I've been wanting to find a better naming because Spirit Impulse Keying carries a negative mystic connotation. Three-State Keying (TSK) sounds just about right. So from now on I will be calling this TSK.

Link to comment
Share on other sites

As far as simple ways to test different modulation schemes, I'm going to refer back to my Spirit Soccer program. It allows spirit to pilot a ball into a target, moving it in one of four directions, or not moving it all. If they can see the computer screen, they can see how their energies affect the decision process.

The code can be modified for any modulation scheme that can express 4 symbols. 

In my current modulation scheme, which requires synchronization:

It samples "events" every 1/16th of a second. 

Every 1/4 second, there are 5 options: up, down, left, right, and do nothing.

If the total energy within one 1/16th second event is much greater than the three others, then it chooses that event, e.g., "up."

If none of the four 1/16 second intervals are anomalous or larger than the others, then nothing happens in that 1/4 second.

The threshold for determining whether one event "wins" is of course set by the user.

 

Link to comment
Share on other sites

13 hours ago, Keith J. Clark said:

So the most challenging variable would be the third state, which is a predefined minimum interval of time, is that a good summary?

We've talked about this idea a lot in the past. You're right, thats the challenge with Morse....the timing seems to present the greatest challenge at present. 

An alternate perspective would be that "almost null" and "null" are too close together, and may seem to actually still be variances of the same state?

It could also be amplitude driven in 3 states vs the time domain applied in the interpretation/conversion process.

0-33% null to small pulse

33%-66% mid pulse

67%-100% peak pulse

VS time domain (pre-defined)

Time seems to always be the challenge. 

I like your thinking, this is a great discussion, would like to see more.

Just meditated on this. 

How would we overcome that in translation of a binary state into a number or letter, it needs to be 8 sequential bits. 

So for current binary scheme it seems the most we would see "on" or "1" states is minimum 6 out of 8 sequential bits in a row.

And the most we would see "off" or "0" states would be 5 out of 8 sequential bits in a row.

So they would have to send 6 of the same signal threshold average in a row to master the current proposed binary scheme, relative to the on state.

Is there a way to reduce it further down to concept? As in....what experiment could we do to first determine if they can produce 2 sequential on bits, then 3 sequential on bits, then 4. Etc? Then if concept holds true we could expand to the alphabet?

Reminds me we need to share a special project with you - low frequency tones that have come through on 2 separate experiments in 2 different locations. So far question and answer has failed to produce a reliable "Yes" or "no" pattern.

And to throw a really weird idea out there, I imagine the time domain being swept. So, the experiment runs 100 days. Say the first time window interval is 1ms. We monitor and focus. Then the second day the program changes to 2ms and we monitor. 

Meaning we don't know what the best time interval is yet, so instead of assuming they'll adapt, we would slowly sweep. I think by the time we got to 100ms we would have found the highest percentage of successful time window.

It all comes back to - how can we prove a concept in its simplest form - how can we prove an on or off state right now. Prior to full on 8 bits?

Thoughts on that?

Keith

 

You are correct that null and almost-null should be coerced into just null.

So, as you said, there would be a threshold mapping:

IF L > peak_threshold     -> Peak_state
ELSE L < null_threshold  -> Null_state
OTHERWISE                    -> Mid_state

If feel like equally dividing the range (0,33%,66%,100%) might not work, but we'll figure out the right thresholds by experimentation.

Regarding the bit-length required for any word...

With this idea, they can throw in a burst of energy now, which would correspond to either a binary 0 or a 1, depending on whether the energy is only above the null_threshold, or all the way beyond the peak threshold, but then take whatever time they need to condense enough energy for the next burst, which would become the next bit. 

That they could work like that was the driving assumption for the idea.

So it seems to me that there isn't an intrinsic problem in the bit-length of a word, or of the entire message, provided of course we wait long enough.

The time doman here is entirely variable. From and engineering point of view, we should see the signal as a continuous stream. We would not use any predefined frame segmentation. Instead, the decoding algorithm is just watching for state-transitions, whenever they occur.

Having said that, I believe you are nonetheless right in that we need to take a baby-steps approach and set up a dead simple first milestone, which is to reliable get just 1 bit, or so, as you said. I'll think about that. 

 

 

 

 

Link to comment
Share on other sites

11 hours ago, Andres Ramos said:

What a nice discussion! Thanks to it now I completely understood Fernando's concept. Sadly the time shift between Europe and America kept me from participating in real time, sorry!

I think the concept is really a great idea! The most important upside is that we basically can overcome the timing problem because the pulse duration does not play a role anymore. The idea is to find a level that can safely enough be relied on as representing the"Off" state. Obviously this only can be the noise ground floor what you guys estimated as being 0..30% of the maximum noise amplitude. Everything going over this threshold causes a logical 1 or 0, depending on the averaged amplitude level but not depending on the pulse width. The pulse is ending when it sinks back again into the ground noise.

The challenge in my eyes is averaging, in respect of finding start and end of the impulse as well as calculating the averaged impulse amplitude in the presence of a multitude of spikes. To be honest I don't see this now in the signals we get, but we also know that the spirits have to train this before they can be successful.

I have a feeling that this probably could be done best in software. Averaging in hw can only be realized with integrators which are low pass filters essentially. When you design them for good averaging you get long times for the system to swing in and weak and long roll-offs. In fact you will get a wiggling signal from an impulse that will only be valid at the impulse's end but the end of the impulse will be unsafe because the low pass characteristic is "smearing out" the roll-off. I already was facing this problem with noise gating earlier. In sw you can do more and better.

The second challenge is to find a noise source that is fluid enough to become easily influenced by spirit energy projections but not too agile in itself to cause false binary signals - at least not too much of them.

One word to the synchronization problem in my experiment. I am currently working with a baudrate of 150. The reason is that with this timing I get the full spectrum of all characters. With baudrates much biggerI observed a bias. Some characters came very often and others never. See this as my starting point that proves that this setup can provide every character of the alphabet. Of course the spirits now (provided they're willing) need to train how to throw in their energy to form characters at will. There are lots of parameters to test for them as the amplitude or amount of energy to project, the duration of projecting and also the direction because positive bursts are setting the flipflop and negative ones are resetting it. Both together specify the width of the generated pulse. The only way to train for them is feedback. Considering they hopefully can see the output they are generating on my terminal screen they can adapt their energy projections accordingly.

If this should be successful we ought to see improved texts on the screen. If not then it's too complicated for them or they simply don't want to collaborate.

To work out Fernando's idea I would experiment with some noise sources and make recordings on Audacity. Fernando, did you know that Audacity provides a programming language named 'Nyquist' that allows many things to do with the recorded signals?

We could try out algorithms for bit cell detection and averaging and play with the parameters. If these are optimized we should use our findings to built a real time processing system from it since, as i said, the spirits need to see their generated output to adapt.

I'm glad to confirm that you (all) have completely understood the idea, and we are on the same page. That is refreshing because my usual experience is that nobody understands what I'm talking about.

I completely agree that the challenges are precisely the ones you mentioned. I will try to work them out in software. What I would need from you are noise samples of the type you think is best suited for this. In particular, I would need samples that are mostly just clean noise with only occasional bursts of spirit energy. Unlike traditional EVP, we are not trying to recover voices or any other high level message, just bits.

I did not know about the Nyquist language! that would make things so much easier!

 

 

 

Link to comment
Share on other sites

5 hours ago, Andres Ramos said:

Deeper investigations on the signal I recorded now in Audacity are showing that this principle very likely will not bring anygood results. The problem is that we don't have a fix baudrate. When the assumed datagram starts with it's start bit, which is always zero, one can conclude the baudrate by the width of the bit.

ASCII.jpg

The problem is of course that the following bits do not necessarily stick to this convention and vary widely in their width. As the receiving process samples the signal level in fixed intervalls derived from the assumed baudrate it will often completely jump over some bits if they had become narrower. The receiving process is sampling the bit cells right in their middle to gain maximum safety. This only works if the width stays fixed at least inside one datagram. Variations from datagram to another one would be no problem since the valid baudrate for each datagram always can always be derived from the width of the startbit. But we cannot come over baudrate fluctuations inside a datagram. That's why this likely won't work although the spirits could train themselves on it. I will let it run for a while but i think that Fernandos idea has much more potential.

I found this very interesting in the sense that it seems to verify that the assumption that it is very difficult for them to stick to a fixed baud-rate, which is the core assumption in three-state keying (as I'm calling it now).

Link to comment
Share on other sites

29 minutes ago, Michael Lee said:

As far as simple ways to test different modulation schemes, I'm going to refer back to my Spirit Soccer program. It allows spirit to pilot a ball into a target, moving it in one of four directions, or not moving it all. If they can see the computer screen, they can see how their energies affect the decision process.

The code can be modified for any modulation scheme that can express 4 symbols. 

In my current modulation scheme, which requires synchronization:

It samples "events" every 1/16th of a second. 

Every 1/4 second, there are 5 options: up, down, left, right, and do nothing.

If the total energy within one 1/16th second event is much greater than the three others, then it chooses that event, e.g., "up."

If none of the four 1/16 second intervals are anomalous or larger than the others, then nothing happens in that 1/4 second.

The threshold for determining whether one event "wins" is of course set by the user.

 

I absolutely agree that your Spirit Soccer program provides the ideal development environment because of the realtime feedback they receive.

So, I think it's a great idea to somehow adapt this to use "Asynchronous TSK" (TASK) as the underlying transport layer.

Your program needs 4 symbols and that can be obtained directly from TASK by pairing bits. 

How can we best combine the two? Perhaps if you share your code with me I can adapt the modulation part to use TASK.

 

Link to comment
Share on other sites

14 hours ago, Keith J. Clark said:

Reminds me we need to share a special project with you - low frequency tones that have come through on 2 separate experiments in 2 different locations. So far question and answer has failed to produce a reliable "Yes" or "no" pattern.

 

I'd love for you to share that with me!

BUT, only after I've finish the code for the "Asynchronous Three State Keying" (TASK), as I'm calling it now.

Link to comment
Share on other sites

1 hour ago, Fernando Luis Cacciola Carballal said:

I'm glad to confirm that you (all) have completely understood the idea, and we are on the same page. That is refreshing because my usual experience is that nobody understands what I'm talking about.

I completely agree that the challenges are precisely the ones you mentioned. I will try to work them out in software. What I would need from you are noise samples of the type you think is best suited for this. In particular, I would need samples that are mostly just clean noise with only occasional bursts of spirit energy. Unlike traditional EVP, we are not trying to recover voices or any other high level message, just bits.

I did not know about the Nyquist language! that would make things so much easier!

 

 

 

Glad you confirmed that I got your idea right!😉

I was already browsing through my Audacity projects to narrow the focus on suitable noise sources. Let me fo some more test and i will come back to you.

Link to comment
Share on other sites

  • Administrators
On 11/1/2020 at 1:13 PM, Fernando Luis Cacciola Carballal said:

 

I'm pretty convinced that any ITC phenomena requires the use of something I will call "hemi-ethereal matter", which mostly comes from us humans, and which is specially functional in physical-mediums. I plan to write a blog entry here on the forum to elaborate on this in much greater detail, specially in how is it involved in ITC, but here let me just say that there is a certain proportionality between what it is that they can effectively do and the "physical-mediumship level" of the experimenter involved in the process.

Since physical mediumship is a scarce resource, I'm always trying to design ITC against the case of a regular person with little to none of that (which is why one of the "versions" of this idea uses the far more common, or "cheap", mental mediumship at the transport layer).

As for coding, I've only got started with setting up the environment and getting familiar with the tools I've collected.

 

Sounds good to me! Just talked to guys about this. I am developing... and have seen success short term across any experiment I work with. Usually when our group gets an idea built - we stream it and burn it in, feeding it energy. And when multiple people join in together it grows relatively fast, just requires persistence. We hope to have circles and people demonstrating this next year.

I could see you helping set the criteria for the experiments so we can define changes and their reasons.  🙂

Bring it on!

Link to comment
Share on other sites

On 11/1/2020 at 8:10 PM, Fernando Luis Cacciola Carballal said:

I'm glad to confirm that you (all) have completely understood the idea, and we are on the same page. That is refreshing because my usual experience is that nobody understands what I'm talking about.

I completely agree that the challenges are precisely the ones you mentioned. I will try to work them out in software. What I would need from you are noise samples of the type you think is best suited for this. In particular, I would need samples that are mostly just clean noise with only occasional bursts of spirit energy. Unlike traditional EVP, we are not trying to recover voices or any other high level message, just bits.

I did not know about the Nyquist language! that would make things so much easier!

 

 

 

Hi Fernando. I am still looking for the right noise source. In my documentation of previous experiments i found this screenshot.

659116533_Carbonnoisespikes.png.e734b2dc17eb4c27774b36312bc30c6b.png

This is a screenshot from the graphite coherer. You see outstanding impulses. Most weirdly some of them are unsymmetrical. Could require full wave rectification before using the signal but this is no problem.

Basically we have outstanding impulses in varying amplitude and density and a more or less low and steady ground noise.

What do you think? I could make a prototype pretty easy i guess.

Link to comment
Share on other sites

1 hour ago, Andres Ramos said:

Hi Fernando. I am still looking for the right noise source. In my documentation of previous experiments i found this screenshot.

659116533_Carbonnoisespikes.png.e734b2dc17eb4c27774b36312bc30c6b.png

This is a screenshot from the graphite coherer. You see outstanding impulses. Most weirdly some of them are unsymmetrical. Could require full wave rectification before using the signal but this is no problem.

Basically we have outstanding impulses in varying amplitude and density and a more or less low and steady ground noise.

What do you think? I could make a prototype pretty easy i guess.

Hi Andres,
That looks very good indeed.

I'm modifying Michael's Spirit Soccer program to use this as the underlying source of "commands".
I'll make some "offline" tests first with audio files like that one, then we'll try something in real-time (as it is needed for this program)

Link to comment
Share on other sites

1 hour ago, Fernando Luis Cacciola Carballal said:

Hi Andres,
That looks very good indeed.

I'm modifying Michael's Spirit Soccer program to use this as the underlying source of "commands".
I'll make some "offline" tests first with audio files like that one, then we'll try something in real-time (as it is needed for this program)

Perfect! Just reach out for me when xou need any help.

Link to comment
Share on other sites

On 11/1/2020 at 5:45 PM, Fernando Luis Cacciola Carballal said:

Which python should I try this on? 2 or 3?
Or maybe it doesn't matter in this case.

@Michael Leeseems like you might have used Python 2.7.

I used 3.9 and I had to change this:


theta = np.random.rand(1)[0]*np.pi*2.0
                                           ^^^^

.rand(1) returns an array (only in Py3 I guess)


In any case.. this week I'll finish a refactoring of the code so that one can plug in any "transport" 

 

 

Link to comment
Share on other sites

52 minutes ago, Michael Lee said:

I use Python 3 these days exclusively because of better compatibility with certain modules I use. I'm glad you are making use of what Ive written so far. Some day ppl will look back at the beginnings of the physical link to the spirit internet.

As a programmer, but not a Pythonist, what I loved the most about that code is that it turned into a crash course on Python, and now I know from audio-tricks to simple GUIs to games.. all in a few days!
That and stackoverflow, and I'm already playing with the sample files from Andres without using anything else by Python! 

Link to comment
Share on other sites

On 11/1/2020 at 10:23 AM, Andres Ramos said:

Deeper investigations on the signal I recorded now in Audacity are showing that this principle very likely will not bring anygood results. The problem is that we don't have a fix baudrate. When the assumed datagram starts with it's start bit, which is always zero, one can conclude the baudrate by the width of the bit.

ASCII.jpg

The problem is of course that the following bits do not necessarily stick to this convention and vary widely in their width. As the receiving process samples the signal level in fixed intervalls derived from the assumed baudrate it will often completely jump over some bits if they had become narrower. The receiving process is sampling the bit cells right in their middle to gain maximum safety. This only works if the width stays fixed at least inside one datagram. Variations from datagram to another one would be no problem since the valid baudrate for each datagram always can always be derived from the width of the startbit. But we cannot come over baudrate fluctuations inside a datagram. That's why this likely won't work although the spirits could train themselves on it. I will let it run for a while but i think that Fernandos idea has much more potential.

Hi Andres,

How did you create the drawing there?  Did you use a special circuit that ended up creating that kind of signal? or did you use some processing in audacity?

 

TIA

 

Edited by Fernando Luis Cacciola Carballal
Link to comment
Share on other sites

7 hours ago, Fernando Luis Cacciola Carballal said:

Hi Andres,

How did you create the drawing there?  Did you use a special circuit that ended up creating that kind of signal? or did you use some processing in audacity?

 

TIA

 

Hello Fernando. This is not a drawing, it's a screenshot from Audacity's signal window. The decaying parts of the signal are the discharging of the coupling capacitors in the soundcard. You know the soundcard has a lowest frequency to let through which is around 10 Hz or so. If your signal becomes slower you can see the discharge process of the internal capacitors. Since I adjusted the discriminator to only generate few impulses I got longer pause times between the impulse groups and then the discharging becomes visible. Try to ignore it. This has nothing to do with the impulses which are always rectangular.

The signal was not processed, it's the raw output of the discriminator and FlipFlop.

Link to comment
Share on other sites

5 hours ago, Andres Ramos said:

Hello Fernando. This is not a drawing, it's a screenshot from Audacity's signal window. The decaying parts of the signal are the discharging of the coupling capacitors in the soundcard. You know the soundcard has a lowest frequency to let through which is around 10 Hz or so. If your signal becomes slower you can see the discharge process of the internal capacitors. Since I adjusted the discriminator to only generate few impulses I got longer pause times between the impulse groups and then the discharging becomes visible. Try to ignore it. This has nothing to do with the impulses which are always rectangular.

The signal was not processed, it's the raw output of the discriminator and FlipFlop.

OK that's what I thought.
I'm researching into several "amplitude envelope followers", and having an output looking like your screenshot but from a noisy "spirit energy burst"  is the ideal result. Not that easy though.

Interestingly, the very first thing I've learn is that the way to create such an "amplitude envelope follower" using electronics is to put a capacitor (right after a diode to rectify the signal), which charges on the way up, and releases (slowly) on the way down. That's why your screenshot looked just like a perfect envelope.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.