Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.javascript > #39761

Re: How hard can it be? Audio processing

Show all headers | View raw

On 25/03/2021 07:39, The Natural Philosopher wrote:
> On 25/03/2021 01:01, Michael Haufe (TNO) wrote:
>> On Tuesday, March 23, 2021 at 7:10:29 AM UTC-5, The Natural 
>> Philosopher wrote:
>>> In principle audio processing in more or less real time is simple. You
>>> need a 'call back' function that gets presented with one or more samples
>>> of digital audio, and you return one or more samples of modified data,
>>> modified by the state of some [global] variables (controls) and the
>>> state of the sample, and previous ones...in essence that is ALL digital
>>> signal processing does, or needs to do. The cleverness is in the
>>> algorithms that do the modification.
>>>
>>> <rant>Until object orientation and computer scientists (after
>>> ArtStudents™, the next most dangerous class to be allowed near a
>>> computer) get in the way, and you need a 100k library just to even
>>> change the volume.
>>>
>>> And as is usual with any API, the documentation tells you how to use it,
>>> but never why you might want to, and invents a huge number of new
>>> concepts that are completely unnecessary in order to preserve the
>>> sanctity of the CodeGurus™ who wrote the damn thing.
>>> </rant>
>>>
>>> So my question is this: what is the simplest and most minimal way to
>>> 'intercept' a stereo audio stream - might be from a file, might be from
>>> a radio station or live TV stream - in a browser and do so minimalist
>>> signal processing that is NOT provided by some 3rd party library, but
>>> actual self written code in terms of 'modify this sample dependent on
>>> the state of a global variable or variables, and previous samples'
>>>
>>> Answers in less than 10k lines of code will get a virtual beer token
>>
>>
>> With the current browser standards it looks like you can accomplish 
>> this in about ~100 lines of code or less:
>>
>> <https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode/frequencyBinCount#example> 
>>
>>
>> The following code is a little out of date but still demonstrates a 
>> method of using a url as a stream:
>>
>> <https://codepen.io/DonKarlssonSan/pen/bdNBMz?editors=0010>
>>
>> The following is also slightly outdated but shows a method for 
>> processing microphone input:
>>
>> <https://stackoverflow.com/questions/15940934/html5-capture-audio-from-default-microphone> 
>>
>>
>> I think the following is the most fruitful. Still less than 100 lines:
>>
>> <https://medium.com/swlh/building-a-audio-visualizer-with-javascript-324b8d420e7> 
>>
>>
>> The outdated examples above need to be modified to request microphone 
>> input after a user initiated event:
>>
>> <https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio> 
>>
>>
> First of all thank you for that, very much. It answers one half of the 
> question which is 'can I work on stream as opposed to a file' and the 
> answer would seem to be 'yes'
> 
> #But it doesn't actually answer the other half of my question - in
> both cases I looked at there is a call to the unexplained, and 
> undocumented function: e.g.
> 
> analyser.getByteFrequencyData(data);
> 
> Ok I discovered that as usual this is a 'API' function. That doesn't do 
> what I want.
> 
> What I want to know is how to create a *custom* function so that I can 
> call e.g.
> 
> analyser.fletcherMunsonModify(data).
> 
> I had already got almost as far as these examples, and in every case 
> there was a call to some API function, but there was never a call to a 
> custom function nor any hint of how to create one?
> 
> THAT is my problem. All this is about how to use what is provided, but I 
> want to *extend* what is provided?
> 
> 
> 
> 
Ah!

One of those references chained me to this - the 'worklet' interface

// bypass-processor.js script file, runs on AudioWorkletGlobalScope

class BypassProcessor extends AudioWorkletProcessor {
   process (inputs, outputs) {
     // Single input, single channel.
     const input = inputs[0];
     const output = outputs[0];
     output[0].set(input[0]);

     // Process only while there are active inputs.
     return false;
   }
};

registerProcessor('bypass-processor', BypassProcessor);

// The main global scope
const context = new AudioContext();
context.audioWorklet.addModule('bypass-processor.js').then(() => {
   const bypassNode = new AudioWorkletNode(context, 'bypass-processor');
});


It seems that there is an addModule function to do in a complicated 
object oriented way what is very simply done in procedural code. That is 
pass a pointer to a user supplied function to get it called...

..well I will play(sic!) with that and see if it plays nice..


-- 
Climate Change: Socialism wearing a lab coat.

Back to comp.lang.javascript | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-23 12:10 +0000
  Re: How hard can it be? Audio processing JJ <jj4public@gmail.com> - 2021-03-23 21:01 +0700
    Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-23 17:17 +0000
      Re: How hard can it be? Audio processing JJ <jj4public@gmail.com> - 2021-03-24 23:16 +0700
        Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-24 17:48 +0000
  Re: How hard can it be? Audio processing John G Harris <niam@jghnorth.org.uk.invalid> - 2021-03-24 18:38 +0000
    Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-24 20:03 +0000
  Re: How hard can it be? Audio processing "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2021-03-24 18:01 -0700
    Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-25 07:39 +0000
      Re: How hard can it be? Audio processing The Natural Philosopher <tnp@invalid.invalid> - 2021-03-25 07:50 +0000
        Re: How hard can it be? Audio processing "Michael Haufe (TNO)" <tno@thenewobjective.com> - 2021-03-25 06:10 -0700

csiph-web