Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #39746 > unrolled thread

How hard can it be? Audio processing

Back to article view | Back to comp.lang.javascript

Contents

  How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-23 12:10 +0000
    Re: How hard can it be? Audio processing JJ <jj4public@gmail.com> - 2021-03-23 21:01 +0700
      Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-23 17:17 +0000
        Re: How hard can it be? Audio processing JJ <jj4public@gmail.com> - 2021-03-24 23:16 +0700
          Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-24 17:48 +0000
    Re: How hard can it be? Audio processing John G Harris <niam@jghnorth.org.uk.invalid> - 2021-03-24 18:38 +0000
      Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-24 20:03 +0000
    Re: How hard can it be? Audio processing "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2021-03-24 18:01 -0700
      Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-25 07:39 +0000
        Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-25 07:50 +0000
          Re: How hard can it be? Audio processing "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2021-03-25 06:10 -0700

#39746 — How hard can it be? Audio processing

In principle audio processing in more or less real time is simple. You 
need a 'call back' function that gets presented with one or more samples 
of digital audio, and you return one or more samples of modified data, 
modified by the state of some [global] variables (controls) and the 
state of the sample, and previous ones...in essence that is ALL digital 
signal processing does, or needs to do. The cleverness is in the 
algorithms that do the modification.

<rant>Until object orientation and computer scientists (after 
ArtStudents™, the next most dangerous class to be allowed near a 
computer) get in the way, and you need a 100k library just to even 
change the volume.

And as is usual with any API, the documentation tells you how to use it, 
but never why you might want to, and invents a huge number of new 
concepts that are completely unnecessary in order to preserve the 
sanctity of the CodeGurus™ who wrote the damn thing.
</rant>

So my question is this: what is the simplest and most minimal way to 
'intercept' a stereo  audio stream - might be from a file, might be from 
a radio station or live TV stream - in a browser and do so minimalist 
signal processing that is NOT provided by some 3rd party library, but 
actual self written code in terms of 'modify this sample dependent on 
the state of a global variable or variables, and previous samples'

Answers in less than 10k lines of code will get a virtual beer token


-- 
There is something fascinating about science. One gets such wholesale 
returns of conjecture out of such a trifling investment of fact.

Mark Twain

[toc] | [next] | [standalone]

#39747

On Tue, 23 Mar 2021 12:10:21 +0000, The Natural Philosopher wrote:
> 
> So my question is this: what is the simplest and most minimal way to 
> 'intercept' a stereo  audio stream - might be from a file, might be from 
> a radio station or live TV stream - in a browser and do so minimalist 
> signal processing that is NOT provided by some 3rd party library, but 
> actual self written code in terms of 'modify this sample dependent on 
> the state of a global variable or variables, and previous samples'
> 
> Answers in less than 10k lines of code will get a virtual beer token

That's like asking someone to do something for you without instructing him
to do anything. So, good luck with that.

[toc] | [prev] | [next] | [standalone]

#39749

On 23/03/2021 14:01, JJ wrote:
> On Tue, 23 Mar 2021 12:10:21 +0000, The Natural Philosopher wrote:
>>
>> So my question is this: what is the simplest and most minimal way to
>> 'intercept' a stereo  audio stream - might be from a file, might be from
>> a radio station or live TV stream - in a browser and do so minimalist
>> signal processing that is NOT provided by some 3rd party library, but
>> actual self written code in terms of 'modify this sample dependent on
>> the state of a global variable or variables, and previous samples'
>>
>> Answers in less than 10k lines of code will get a virtual beer token
> 
> That's like asking someone to do something for you without instructing him
> to do anything. So, good luck with that.
> 
I just want to know where to look

I take it you are a codeGuru™. And dont actually know the answer..;-]

-- 
Outside of a dog, a book is a man's best friend. Inside of a dog it's 
too dark to read.

Groucho Marx

[toc] | [prev] | [next] | [standalone]

#39752

On Tue, 23 Mar 2021 17:17:20 +0000, The Natural Philosopher wrote:
>> 
> I just want to know where to look
> 
> I take it you are a codeGuru™. And dont actually know the answer..;-]

No one can solve a wrong question.

[toc] | [prev] | [next] | [standalone]

#39755

On 24/03/2021 16:16, JJ wrote:
> On Tue, 23 Mar 2021 17:17:20 +0000, The Natural Philosopher wrote:
>>>
>> I just want to know where to look
>>
>> I take it you are a codeGuru™. And dont actually know the answer..;-]
> 
> No one can solve a wrong question.
> 

It isn't a wrong question
That was simply a wrong answer.
I want to know how to access javascript audio API. And if what I want it 
to do, is something it is, in fact, capable of doing.

None of the online documentation answers this very simple question. It 
answers tons of very complicated questions, but they are not the 
question I am asking.


-- 
In todays liberal progressive conflict-free education system, everyone 
gets full Marx.

[toc] | [prev] | [next] | [standalone]

#39757

On Tue, 23 Mar 2021 12:10:21 +0000, The Natural Philosopher
<tnp@invalid.invalid> wrote:

  <snip>
><rant>Until object orientation and computer scientists (after 
>ArtStudents™, the next most dangerous class to be allowed near a 
>computer) get in the way, and you need a 100k library just to even 
>change the volume.

If you choose to copy and use somebody else's bad design you should
get all the problems you deserve.


>And as is usual with any API, the documentation tells you how to use it, 
>but never why you might want to, and invents a huge number of new 
>concepts that are completely unnecessary in order to preserve the 
>sanctity of the CodeGurus™ who wrote the damn thing.
></rant>
  <snip>

The documentation that you write is so much clearer and complete,
right? I don't believe you.

  John

[toc] | [prev] | [next] | [standalone]

#39758

On 24/03/2021 18:38, John G Harris wrote:
> On Tue, 23 Mar 2021 12:10:21 +0000, The Natural Philosopher
> <tnp@invalid.invalid> wrote:
> 
>    <snip>
>> <rant>Until object orientation and computer scientists (after
>> ArtStudents™, the next most dangerous class to be allowed near a
>> computer) get in the way, and you need a 100k library just to even
>> change the volume.
> 
> If you choose to copy and use somebody else's bad design you should
> get all the problems you deserve.
> 
You have no choice. You have to use javaScript and its APIs.
Yes, its shit code, but it's all there is.

Try for example programming X windows without a widget set or a 
graphical toolkit.

> 
>> And as is usual with any API, the documentation tells you how to use it,
>> but never why you might want to, and invents a huge number of new
>> concepts that are completely unnecessary in order to preserve the
>> sanctity of the CodeGurus™ who wrote the damn thing.
>> </rant>
>    <snip>
> 
> The documentation that you write is so much clearer and complete,
> right? I don't believe you.

Well yes it is - or used to be.

I had to understand it, years later.

But the issue is not that the documentation is obscure, its that the 
whole design concept  gets mangled when people insist on forcing it all 
into objects and classes.

That suits some projects, but by no means all projects.
And then people document what they wrote, from the perspective of 
understanding what they write, but not why they wrote it..

..e.g.(not a real example, but indicative)  'operates on stream type 
object allowing manipulation of buffer attributes'...

What is a stream type object?
Why would you want to use one?
What is its buffer for?
What are the buffers attributes?
Why would you want to modify them?

...
and so on

JavaScript is not a bad language, although it has its hidden pitfalls 
(no strict typing can catch you when you are expecting a nuumber and get 
a string and vice versa) but the interface to te DONM is pretty bloody 
awful.

To wit I don't know if what I want to do is *even possible*. After two 
a days reading up on all the docs I can find.

Its pretty prevalent throughout the IT industry, and always has been.
I remember years ago contacting Cisco support, about using ISDN. It 
simply didn't work. The documentation told you about various parameters 
you could set, but not what they meant or why you would want to set them.

I got through to a tech guy ' Are you in the UK?'
'Yes' 'Ok,  you need to set ISDN type to 3'

'why?' 'I don't know. That's the type of ISDN you use there.' 'Are you 
sure it will work' 'sure. I spent a month over there getting that bit of 
the code working'...

If Cisco can't get their documentation right....on thousands of dollars 
of product...


I remember someone deciding to move from C to C++ some years ago. It 
took 5 times longer to write the code, and he said 'with hindsight, we 
should have spent all the time specfying the objects and classes, 
instead of writing the code, then it would only have taken twice as 
long' :-)

But the point is is there anyone here who has even done one single piece 
of code using the audio API?

I suspect the answer is 'no'.

> 
>    John
> 


-- 
"If you don’t read the news paper, you are un-informed. If you read the 
news paper, you are mis-informed."

Mark Twain

[toc] | [prev] | [next] | [standalone]

#39759

On Tuesday, March 23, 2021 at 7:10:29 AM UTC-5, The Natural Philosopher wrote:
> In principle audio processing in more or less real time is simple. You 
> need a 'call back' function that gets presented with one or more samples 
> of digital audio, and you return one or more samples of modified data, 
> modified by the state of some [global] variables (controls) and the 
> state of the sample, and previous ones...in essence that is ALL digital 
> signal processing does, or needs to do. The cleverness is in the 
> algorithms that do the modification. 
> 
> <rant>Until object orientation and computer scientists (after 
> ArtStudents™, the next most dangerous class to be allowed near a 
> computer) get in the way, and you need a 100k library just to even 
> change the volume. 
> 
> And as is usual with any API, the documentation tells you how to use it, 
> but never why you might want to, and invents a huge number of new 
> concepts that are completely unnecessary in order to preserve the 
> sanctity of the CodeGurus™ who wrote the damn thing. 
> </rant> 
> 
> So my question is this: what is the simplest and most minimal way to 
> 'intercept' a stereo audio stream - might be from a file, might be from 
> a radio station or live TV stream - in a browser and do so minimalist 
> signal processing that is NOT provided by some 3rd party library, but 
> actual self written code in terms of 'modify this sample dependent on 
> the state of a global variable or variables, and previous samples' 
> 
> Answers in less than 10k lines of code will get a virtual beer token 


With the current browser standards it looks like you can accomplish this in about ~100 lines of code or less:

<https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode/frequencyBinCount#example>

The following code is a little out of date but still demonstrates a method of using a url as a stream:

<https://codepen.io/DonKarlssonSan/pen/bdNBMz?editors=0010>

The following is also slightly outdated but shows a method for processing microphone input:

<https://stackoverflow.com/questions/15940934/html5-capture-audio-from-default-microphone>

I think the following is the most fruitful. Still less than 100 lines:

<https://medium.com/swlh/building-a-audio-visualizer-with-javascript-324b8d420e7>

The outdated examples above need to be modified to request microphone input after a user initiated event:

<https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio>

[toc] | [prev] | [next] | [standalone]

#39760

On 25/03/2021 01:01, Michael Haufe (TNO) wrote:
> On Tuesday, March 23, 2021 at 7:10:29 AM UTC-5, The Natural Philosopher wrote:
>> In principle audio processing in more or less real time is simple. You
>> need a 'call back' function that gets presented with one or more samples
>> of digital audio, and you return one or more samples of modified data,
>> modified by the state of some [global] variables (controls) and the
>> state of the sample, and previous ones...in essence that is ALL digital
>> signal processing does, or needs to do. The cleverness is in the
>> algorithms that do the modification.
>>
>> <rant>Until object orientation and computer scientists (after
>> ArtStudents™, the next most dangerous class to be allowed near a
>> computer) get in the way, and you need a 100k library just to even
>> change the volume.
>>
>> And as is usual with any API, the documentation tells you how to use it,
>> but never why you might want to, and invents a huge number of new
>> concepts that are completely unnecessary in order to preserve the
>> sanctity of the CodeGurus™ who wrote the damn thing.
>> </rant>
>>
>> So my question is this: what is the simplest and most minimal way to
>> 'intercept' a stereo audio stream - might be from a file, might be from
>> a radio station or live TV stream - in a browser and do so minimalist
>> signal processing that is NOT provided by some 3rd party library, but
>> actual self written code in terms of 'modify this sample dependent on
>> the state of a global variable or variables, and previous samples'
>>
>> Answers in less than 10k lines of code will get a virtual beer token
> 
> 
> With the current browser standards it looks like you can accomplish this in about ~100 lines of code or less:
> 
> <https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode/frequencyBinCount#example>
> 
> The following code is a little out of date but still demonstrates a method of using a url as a stream:
> 
> <https://codepen.io/DonKarlssonSan/pen/bdNBMz?editors=0010>
> 
> The following is also slightly outdated but shows a method for processing microphone input:
> 
> <https://stackoverflow.com/questions/15940934/html5-capture-audio-from-default-microphone>
> 
> I think the following is the most fruitful. Still less than 100 lines:
> 
> <https://medium.com/swlh/building-a-audio-visualizer-with-javascript-324b8d420e7>
> 
> The outdated examples above need to be modified to request microphone input after a user initiated event:
> 
> <https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio>
> 
First of all thank you for that, very much. It answers one half of the 
question which is 'can I work on stream as opposed to a file' and the 
answer would seem to be 'yes'

#But it doesn't actually answer the other half of my question - in
both cases I looked at there is a call to the unexplained, and 
undocumented function: e.g.

analyser.getByteFrequencyData(data);

Ok I discovered that as usual this is a 'API' function. That doesn't do 
what I want.

What I want to know is how to create a *custom* function so that I can 
call e.g.

analyser.fletcherMunsonModify(data).

I had already got almost as far as these examples, and in every case 
there was a call to some API function, but there was never a call to a 
custom function nor any hint of how to create one?

THAT is my problem. All this is about how to use what is provided, but I 
want to *extend* what is provided?




-- 
The New Left are the people they warned you about.

[toc] | [prev] | [next] | [standalone]

#39761

On 25/03/2021 07:39, The Natural Philosopher wrote:
> On 25/03/2021 01:01, Michael Haufe (TNO) wrote:
>> On Tuesday, March 23, 2021 at 7:10:29 AM UTC-5, The Natural 
>> Philosopher wrote:
>>> In principle audio processing in more or less real time is simple. You
>>> need a 'call back' function that gets presented with one or more samples
>>> of digital audio, and you return one or more samples of modified data,
>>> modified by the state of some [global] variables (controls) and the
>>> state of the sample, and previous ones...in essence that is ALL digital
>>> signal processing does, or needs to do. The cleverness is in the
>>> algorithms that do the modification.
>>>
>>> <rant>Until object orientation and computer scientists (after
>>> ArtStudents™, the next most dangerous class to be allowed near a
>>> computer) get in the way, and you need a 100k library just to even
>>> change the volume.
>>>
>>> And as is usual with any API, the documentation tells you how to use it,
>>> but never why you might want to, and invents a huge number of new
>>> concepts that are completely unnecessary in order to preserve the
>>> sanctity of the CodeGurus™ who wrote the damn thing.
>>> </rant>
>>>
>>> So my question is this: what is the simplest and most minimal way to
>>> 'intercept' a stereo audio stream - might be from a file, might be from
>>> a radio station or live TV stream - in a browser and do so minimalist
>>> signal processing that is NOT provided by some 3rd party library, but
>>> actual self written code in terms of 'modify this sample dependent on
>>> the state of a global variable or variables, and previous samples'
>>>
>>> Answers in less than 10k lines of code will get a virtual beer token
>>
>>
>> With the current browser standards it looks like you can accomplish 
>> this in about ~100 lines of code or less:
>>
>> <https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode/frequencyBinCount#example> 
>>
>>
>> The following code is a little out of date but still demonstrates a 
>> method of using a url as a stream:
>>
>> <https://codepen.io/DonKarlssonSan/pen/bdNBMz?editors=0010>
>>
>> The following is also slightly outdated but shows a method for 
>> processing microphone input:
>>
>> <https://stackoverflow.com/questions/15940934/html5-capture-audio-from-default-microphone> 
>>
>>
>> I think the following is the most fruitful. Still less than 100 lines:
>>
>> <https://medium.com/swlh/building-a-audio-visualizer-with-javascript-324b8d420e7> 
>>
>>
>> The outdated examples above need to be modified to request microphone 
>> input after a user initiated event:
>>
>> <https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio> 
>>
>>
> First of all thank you for that, very much. It answers one half of the 
> question which is 'can I work on stream as opposed to a file' and the 
> answer would seem to be 'yes'
> 
> #But it doesn't actually answer the other half of my question - in
> both cases I looked at there is a call to the unexplained, and 
> undocumented function: e.g.
> 
> analyser.getByteFrequencyData(data);
> 
> Ok I discovered that as usual this is a 'API' function. That doesn't do 
> what I want.
> 
> What I want to know is how to create a *custom* function so that I can 
> call e.g.
> 
> analyser.fletcherMunsonModify(data).
> 
> I had already got almost as far as these examples, and in every case 
> there was a call to some API function, but there was never a call to a 
> custom function nor any hint of how to create one?
> 
> THAT is my problem. All this is about how to use what is provided, but I 
> want to *extend* what is provided?
> 
> 
> 
> 
Ah!

One of those references chained me to this - the 'worklet' interface

// bypass-processor.js script file, runs on AudioWorkletGlobalScope

class BypassProcessor extends AudioWorkletProcessor {
   process (inputs, outputs) {
     // Single input, single channel.
     const input = inputs[0];
     const output = outputs[0];
     output[0].set(input[0]);

     // Process only while there are active inputs.
     return false;
   }
};

registerProcessor('bypass-processor', BypassProcessor);

// The main global scope
const context = new AudioContext();
context.audioWorklet.addModule('bypass-processor.js').then(() => {
   const bypassNode = new AudioWorkletNode(context, 'bypass-processor');
});


It seems that there is an addModule function to do in a complicated 
object oriented way what is very simply done in procedural code. That is 
pass a pointer to a user supplied function to get it called...

..well I will play(sic!) with that and see if it plays nice..


-- 
Climate Change: Socialism wearing a lab coat.

[toc] | [prev] | [next] | [standalone]

#39766

On Thursday, March 25, 2021 at 2:51:04 AM UTC-5, The Natural Philosopher wrote:
> Ah! 
> 
> One of those references chained me to this - the 'worklet' interface 
> 
> // bypass-processor.js script file, runs on AudioWorkletGlobalScope 
> 
> class BypassProcessor extends AudioWorkletProcessor { 
> process (inputs, outputs) { 
> // Single input, single channel. 
> const input = inputs[0]; 
> const output = outputs[0]; 
> output[0].set(input[0]); 
> 
> // Process only while there are active inputs. 
> return false; 
> } 
> }; 
> 
> registerProcessor('bypass-processor', BypassProcessor); 
> 
> // The main global scope 
> const context = new AudioContext(); 
> context.audioWorklet.addModule('bypass-processor.js').then(() => { 
> const bypassNode = new AudioWorkletNode(context, 'bypass-processor'); 
> }); 
> 
> 
> It seems that there is an addModule function to do in a complicated 
> object oriented way what is very simply done in procedural code. That is 
> pass a pointer to a user supplied function to get it called... 
> 
> ..well I will play(sic!) with that and see if it plays nice.. 

There are also a couple examples on MDN:

<https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode#basic_usage>
<https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.javascript

csiph-web