diff --git a/docs/index.html b/docs/index.html index fb30973..d771362 100644 --- a/docs/index.html +++ b/docs/index.html @@ -389,5 +389,5 @@ diff --git a/docs/lectures/dsp1/index.html b/docs/lectures/dsp1/index.html index 827ec70..17b8013 100644 --- a/docs/lectures/dsp1/index.html +++ b/docs/lectures/dsp1/index.html @@ -307,16 +307,10 @@

LFO: Low Frequency Oscillator

Towards the DX7

The DX7 carried out frequency modulation over a total of six oscillators that could be patched in different ways. So FM is not limited to two oscillators... Try to implement an FM synthesizer involving 3 oscillators instead of one. They should be connected in series: 3 -> 2 -> 1.

- + diff --git a/docs/search/search_index.json b/docs/search/search_index.json index e8e1b20..356cd5e 100644 --- a/docs/search/search_index.json +++ b/docs/search/search_index.json @@ -1 +1 @@ -{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Embedded Audio Signal Processing This course is a collaboration between Insa-Lyon (TC-Dept, Citi Lab ), INRIA , and GRAME-CNCM . The objective is to foster the development of emerging embedded audio devices and to take advantage of the resources of the CITI/INRIA Emeraude Team (Embedded Programmable Audio Systems) in this domain. In this course, students will learn about: Low-level embedded systems for real-time audio signal processing Digital audio system architecture Audio codec configuration IC communication protocols Audio signal processing Audio sound synthesis and effects design The Faust programming language Course git repository: https://github.com/grame-cncm/embaudio Instructors Romain Michon (INRIA) Tanguy Risset (INSA Lyon) Yann Orlarey (GRAME-CNCM) Organization and ECTS The course will consists of 32 hours (2 ECTS) divided into 16h TD (or CM, this is equivalent) and 16h TP (two instructors): 2x2h CM 6x2h CM/TD 8x2h TP Evaluation on TPs Course Overview 19/09/2023 -- 14h00-16h00: Course Introduction and Programming Environment Setup 26/09/2023 -- 14h00-16h00: Audio Signal Processing Fundamentals 03/10/2023 -- 14h00-16h00: Digital Audio Systems Architectures and Audio Callback 09/10/2023 -- 14h00-16h00: Hardware Control and Audio Codec Configuration 10/10/2023 -- 14h00-16h00: Audio Processing Basics I 06/11/2023 -- 08h00-10h00: Embedded System Peripherals 06/11/2023 -- 10h00-12h00: Embedded OS, FreeRTOS, Embedded Linux Devices 07/11/2023 -- 14h00-16h00: Audio Processing Basics II 08/11/2023 -- 10h00-12h00: Faust Tutotial 14/11/2023 -- 14h00-16h00: Embedded systems at Rtone 22/11/2023 -- 10h00-12h00: Faust on the Teensy and Advanced Control Sessions 12-16: Mini-project 29/11/2023 10h00-12h00 06/12/2023 10h00-12h00 12/12/2023 08h00-10h00 12/12/2023 10h00-12h00 19/12/2023 14h00-16h00","title":"Syllabus"},{"location":"#embedded-audio-signal-processing","text":"This course is a collaboration between Insa-Lyon (TC-Dept, Citi Lab ), INRIA , and GRAME-CNCM . The objective is to foster the development of emerging embedded audio devices and to take advantage of the resources of the CITI/INRIA Emeraude Team (Embedded Programmable Audio Systems) in this domain. In this course, students will learn about: Low-level embedded systems for real-time audio signal processing Digital audio system architecture Audio codec configuration IC communication protocols Audio signal processing Audio sound synthesis and effects design The Faust programming language Course git repository: https://github.com/grame-cncm/embaudio","title":"Embedded Audio Signal Processing"},{"location":"#instructors","text":"Romain Michon (INRIA) Tanguy Risset (INSA Lyon) Yann Orlarey (GRAME-CNCM)","title":"Instructors"},{"location":"#organization-and-ects","text":"The course will consists of 32 hours (2 ECTS) divided into 16h TD (or CM, this is equivalent) and 16h TP (two instructors): 2x2h CM 6x2h CM/TD 8x2h TP Evaluation on TPs","title":"Organization and ECTS"},{"location":"#course-overview","text":"19/09/2023 -- 14h00-16h00: Course Introduction and Programming Environment Setup 26/09/2023 -- 14h00-16h00: Audio Signal Processing Fundamentals 03/10/2023 -- 14h00-16h00: Digital Audio Systems Architectures and Audio Callback 09/10/2023 -- 14h00-16h00: Hardware Control and Audio Codec Configuration 10/10/2023 -- 14h00-16h00: Audio Processing Basics I 06/11/2023 -- 08h00-10h00: Embedded System Peripherals 06/11/2023 -- 10h00-12h00: Embedded OS, FreeRTOS, Embedded Linux Devices 07/11/2023 -- 14h00-16h00: Audio Processing Basics II 08/11/2023 -- 10h00-12h00: Faust Tutotial 14/11/2023 -- 14h00-16h00: Embedded systems at Rtone 22/11/2023 -- 10h00-12h00: Faust on the Teensy and Advanced Control Sessions 12-16: Mini-project 29/11/2023 10h00-12h00 06/12/2023 10h00-12h00 12/12/2023 08h00-10h00 12/12/2023 10h00-12h00 19/12/2023 14h00-16h00","title":"Course Overview"},{"location":"lectures/architecture/","text":"Digital Audio Systems Architectures and Audio Callback By the end of this lecture, you should be able to produce sound with your Teensy and have a basic understanding of the software and hardware architecture of embedded audio systems. Basic Architecture of a Digital Audio System All digital audio systems have an architecture involving at least an ADC and/or a DAC. Audio samples are processed on a computer (i.e., CPU, microcontroller, DSP, etc.) typically in an audio callback and are transmitted to the DAC and/or received from the ADC: The format of audio samples depends on the hardware configuration of the system. Architecture of Embedded Audio Systems Such as the Teensy In embedded audio systems, the component implementing the audio ADC and DAC is called an \"Audio Codec.\" This name is slightly ambiguous because it is also used in the context of audio compression (e.g., mp3) to designate a totally different concept. In the case of the Teensy kits that are provided to you as part of this class, the audio codec we use is an SGTL5000. It is mounted on a shield/sister board that has the same form factor as the Teensy. Audio samples are sent and received between the Cortex M7 and the audio codec using the i2s protocol (additional information on how this kind of system works is provided in this lecture ). As a microcontroller, the Cortex M7 has its own analog inputs which can be used to retrieve sensor datas (e.g., potentiometers, etc.). These analog inputs cannot be used for audio because of their limited precision and sampling rate. We'll briefly show in this lecture how these analog inputs can be used to use sensors to control audio algorithms running on the Teensy. Teensy and Audio Shield Overview Concept of Audio Blocks (Buffers), Audio Rate, and Control Rate A large number of audio samples must be processed and transmitted every second. For example, if the sampling rate of the system is 48 kHz, 48000 samples will be processed in one second. Digital audio is extremely demanding and if one sample is missed, the result on the produced sound will be very audible. Most processors cannot process and transmit samples one by one which is why buffers need to be used. Hence, most digital audio systems will process audio as \"blocks.\" The smallest size of a block will be determined by the performance of the system. On a modern computer running an operating system such as Windows, MacOS or Linux, the standard block size is usually 256 samples. In that case, the audio callback will process and then transmit to the DAC 256 samples all at once. An audio callback function typically takes the following form: void audioCallback(float *inputs, float *outputs){ // control rate portion int gain = mainVolume; for(int i=0; idata[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } The update method is called every time a new audio buffer is needed by the system. A new audio buffer audioBlock containing AUDIO_OUTPUTS channels is first created. For every audio channel, memory is allocated and a full block of samples is computed. Individual samples resulting from computing a sine wave through an echo ( echo and sine are defined in the lib folder and implement an echo and a sine wave oscillator, respectively) are stored in currentSample . currentSample is a floating point number whose range is {-1;1}. This is a standard in the world of digital audio, hence, a signal actually ranging between {-1;1} will correspond to the \"loudest\" sound that can be played on a given system. max(-1,min(1,currentSample)); ensures that currentSample doesn't exceed this range. AUDIO_BLOCK_SAMPLES corresponds to the block size (256 samples by default on the Teensy, but this value can potentially be adjusted). The values contained in currentSample (between -1 and 1) must be converted to 16 bits signed integers (to ensure compatibility with the rest of the Teensy audio library). For that, we just have to multiply currentSample by 2^{16-1} (if we were looking at unsigned integers, which can happen on some system, we would multiply currentSample by 2^{16} ). Note that currentSample is multiplied by 0.5 to control the output gain of the system here (we'll see later in this class that echos tend to add energy to the system hence we must limit the gain of the output signal to prevent potential saturation). Once a full block has been computed, it is transmitted to the rest of the system using the transmit function. Once this is done, the memory that was allocated for the audio block is freed using the release function. The update method is called over and over until the Teensy is powered out. C++ Sine Wave Oscillator Sine wave are at the basis of many algorithms in the field of audio. The sound of a sine wave is what we call a \"pure tone\" since it only has a single harmonic. One of the consequences of this is that all sounds can be synthesized using a combination of sine waves ( Fourier transform ). From a mathematical standpoint, a sine oscillator can be implemented with the following differential equation: x(t) = Asin(\\omega t + \\phi) with: A : the peak amplitude \\omega = 2 \\pi f : the radian frequency (rad/sec) f : the frequency in Hz t : the time seconds \\phi : the initial phase (radians) x(t) could be translated to C++ by writing something like ( \\phi is ignored here): float currentSample = A*std::sin(2*PI*f*t); however sine oscillators are rarely implemented as such since calling the std::sin function at every sample can be quite computationally expensive. For that reason, it is better to pre-compute the sine wave and store it in a wave table before computation starts. That kind of algorithm is then called a \"wave table oscillator.\" Sine.cpp , which is used in crazy-sine is a good example of that. It uses SineTable.cpp which pre-computes a sine table: table = new float[size]; for(int i=0; i #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; delay(500); } Basic Additive Synthesis One of the most basic kind of sound synthesis is \"additive synthesis.\" In consists of adding multiple sine wave oscillators together to \"sculpt\" the timbre of a sound. Both the frequency and the gain of each individual oscillator can then be used to change the properties of the synthesized sound. A simple additive synthesizer could be implemented from the crazy-sine example by declaring multiple instances of sine . E.g.: float currentSample = echo.tick(sine0.tick()*gain0 + sine1.tick()*gain1); but the problem with that option is that memory will be allocated twice for the sineTable array which is a terrible idea in the context of our embedded audio system with very little memory. Instead, the additive synthesizer should reuse the same instance of sineTable for each oscillator. In the tick method of Sine.cpp , try to call the sineTable a second time after float currentSample = sineTable[index]*gain; to add a second oscillator to the generated sample. The value of its index could be something like index = (int) (index*1.5)%SINE_TABLE_SIZE; so that the frequency of the second oscillator is one fifth above the main frequency. In other words, the differential equation of the synth should be: x(t) = sin(2 \\pi f t) + sin(2 \\pi (1.5f) t) Hint: Beware of clipping! Adding two sine waves together even though they don't have the same frequency will likely produce a signal whose range exceeds {-1;1}: you should take that into account for your final product. Solution: In Sine.cpp : float Sine::tick(){ int index = phasor.tick()*SINE_TABLE_SIZE; int index2 = (int) (index*1.5)%SINE_TABLE_SIZE; return (sineTable.tick(index)+sineTable.tick(index2))*gain*0.5; } Stereo Echo Reusing the result of the previous exercise, create a second instance of echo (connected to the same instance of sine ) with different parameters from the first one that will be connected to the second channel of the output (i.e., the first instance should be connected to the left channel and the second one to the right channel). The final algorithm should look like this: float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; Hint: Beware of memory allocation again! Make sure that the maxim delay of your echo (on the 2 parameters of the class constructor) doesn't exceed 10000 for now for both instances of the echo. Solution: In MyDsp.h : Sine sine; Echo echo0, echo1; }; In MyDsp.cpp : MyDsp::MyDsp() : AudioStream(AUDIO_OUTPUTS, new audio_block_t*[AUDIO_OUTPUTS]), sine(AUDIO_SAMPLE_RATE_EXACT), echo0(AUDIO_SAMPLE_RATE_EXACT,10000), echo1(AUDIO_SAMPLE_RATE_EXACT,7000) { ... // setting up DSP objects echo0.setDel(10000); echo0.setFeedback(0.5); echo1.setDel(7000); echo1.setFeedback(0.4); ... // processing buffers for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; ... }","title":" Digital Audio Systems Architectures and Audio Callback "},{"location":"lectures/architecture/#digital-audio-systems-architectures-and-audio-callback","text":"By the end of this lecture, you should be able to produce sound with your Teensy and have a basic understanding of the software and hardware architecture of embedded audio systems.","title":"Digital Audio Systems Architectures and Audio Callback"},{"location":"lectures/architecture/#basic-architecture-of-a-digital-audio-system","text":"All digital audio systems have an architecture involving at least an ADC and/or a DAC. Audio samples are processed on a computer (i.e., CPU, microcontroller, DSP, etc.) typically in an audio callback and are transmitted to the DAC and/or received from the ADC: The format of audio samples depends on the hardware configuration of the system.","title":"Basic Architecture of a Digital Audio System"},{"location":"lectures/architecture/#architecture-of-embedded-audio-systems-such-as-the-teensy","text":"In embedded audio systems, the component implementing the audio ADC and DAC is called an \"Audio Codec.\" This name is slightly ambiguous because it is also used in the context of audio compression (e.g., mp3) to designate a totally different concept. In the case of the Teensy kits that are provided to you as part of this class, the audio codec we use is an SGTL5000. It is mounted on a shield/sister board that has the same form factor as the Teensy. Audio samples are sent and received between the Cortex M7 and the audio codec using the i2s protocol (additional information on how this kind of system works is provided in this lecture ). As a microcontroller, the Cortex M7 has its own analog inputs which can be used to retrieve sensor datas (e.g., potentiometers, etc.). These analog inputs cannot be used for audio because of their limited precision and sampling rate. We'll briefly show in this lecture how these analog inputs can be used to use sensors to control audio algorithms running on the Teensy. Teensy and Audio Shield Overview","title":"Architecture of Embedded Audio Systems Such as the Teensy"},{"location":"lectures/architecture/#concept-of-audio-blocks-buffers-audio-rate-and-control-rate","text":"A large number of audio samples must be processed and transmitted every second. For example, if the sampling rate of the system is 48 kHz, 48000 samples will be processed in one second. Digital audio is extremely demanding and if one sample is missed, the result on the produced sound will be very audible. Most processors cannot process and transmit samples one by one which is why buffers need to be used. Hence, most digital audio systems will process audio as \"blocks.\" The smallest size of a block will be determined by the performance of the system. On a modern computer running an operating system such as Windows, MacOS or Linux, the standard block size is usually 256 samples. In that case, the audio callback will process and then transmit to the DAC 256 samples all at once. An audio callback function typically takes the following form: void audioCallback(float *inputs, float *outputs){ // control rate portion int gain = mainVolume; for(int i=0; idata[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } The update method is called every time a new audio buffer is needed by the system. A new audio buffer audioBlock containing AUDIO_OUTPUTS channels is first created. For every audio channel, memory is allocated and a full block of samples is computed. Individual samples resulting from computing a sine wave through an echo ( echo and sine are defined in the lib folder and implement an echo and a sine wave oscillator, respectively) are stored in currentSample . currentSample is a floating point number whose range is {-1;1}. This is a standard in the world of digital audio, hence, a signal actually ranging between {-1;1} will correspond to the \"loudest\" sound that can be played on a given system. max(-1,min(1,currentSample)); ensures that currentSample doesn't exceed this range. AUDIO_BLOCK_SAMPLES corresponds to the block size (256 samples by default on the Teensy, but this value can potentially be adjusted). The values contained in currentSample (between -1 and 1) must be converted to 16 bits signed integers (to ensure compatibility with the rest of the Teensy audio library). For that, we just have to multiply currentSample by 2^{16-1} (if we were looking at unsigned integers, which can happen on some system, we would multiply currentSample by 2^{16} ). Note that currentSample is multiplied by 0.5 to control the output gain of the system here (we'll see later in this class that echos tend to add energy to the system hence we must limit the gain of the output signal to prevent potential saturation). Once a full block has been computed, it is transmitted to the rest of the system using the transmit function. Once this is done, the memory that was allocated for the audio block is freed using the release function. The update method is called over and over until the Teensy is powered out.","title":"First Audio Program on the Teensy: crazy-sine"},{"location":"lectures/architecture/#c-sine-wave-oscillator","text":"Sine wave are at the basis of many algorithms in the field of audio. The sound of a sine wave is what we call a \"pure tone\" since it only has a single harmonic. One of the consequences of this is that all sounds can be synthesized using a combination of sine waves ( Fourier transform ). From a mathematical standpoint, a sine oscillator can be implemented with the following differential equation: x(t) = Asin(\\omega t + \\phi) with: A : the peak amplitude \\omega = 2 \\pi f : the radian frequency (rad/sec) f : the frequency in Hz t : the time seconds \\phi : the initial phase (radians) x(t) could be translated to C++ by writing something like ( \\phi is ignored here): float currentSample = A*std::sin(2*PI*f*t); however sine oscillators are rarely implemented as such since calling the std::sin function at every sample can be quite computationally expensive. For that reason, it is better to pre-compute the sine wave and store it in a wave table before computation starts. That kind of algorithm is then called a \"wave table oscillator.\" Sine.cpp , which is used in crazy-sine is a good example of that. It uses SineTable.cpp which pre-computes a sine table: table = new float[size]; for(int i=0; i #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; delay(500); }","title":"Looping Through a Small Tune: Making a Music Box"},{"location":"lectures/architecture/#basic-additive-synthesis","text":"One of the most basic kind of sound synthesis is \"additive synthesis.\" In consists of adding multiple sine wave oscillators together to \"sculpt\" the timbre of a sound. Both the frequency and the gain of each individual oscillator can then be used to change the properties of the synthesized sound. A simple additive synthesizer could be implemented from the crazy-sine example by declaring multiple instances of sine . E.g.: float currentSample = echo.tick(sine0.tick()*gain0 + sine1.tick()*gain1); but the problem with that option is that memory will be allocated twice for the sineTable array which is a terrible idea in the context of our embedded audio system with very little memory. Instead, the additive synthesizer should reuse the same instance of sineTable for each oscillator. In the tick method of Sine.cpp , try to call the sineTable a second time after float currentSample = sineTable[index]*gain; to add a second oscillator to the generated sample. The value of its index could be something like index = (int) (index*1.5)%SINE_TABLE_SIZE; so that the frequency of the second oscillator is one fifth above the main frequency. In other words, the differential equation of the synth should be: x(t) = sin(2 \\pi f t) + sin(2 \\pi (1.5f) t) Hint: Beware of clipping! Adding two sine waves together even though they don't have the same frequency will likely produce a signal whose range exceeds {-1;1}: you should take that into account for your final product. Solution: In Sine.cpp : float Sine::tick(){ int index = phasor.tick()*SINE_TABLE_SIZE; int index2 = (int) (index*1.5)%SINE_TABLE_SIZE; return (sineTable.tick(index)+sineTable.tick(index2))*gain*0.5; }","title":"Basic Additive Synthesis"},{"location":"lectures/architecture/#stereo-echo","text":"Reusing the result of the previous exercise, create a second instance of echo (connected to the same instance of sine ) with different parameters from the first one that will be connected to the second channel of the output (i.e., the first instance should be connected to the left channel and the second one to the right channel). The final algorithm should look like this: float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; Hint: Beware of memory allocation again! Make sure that the maxim delay of your echo (on the 2 parameters of the class constructor) doesn't exceed 10000 for now for both instances of the echo. Solution: In MyDsp.h : Sine sine; Echo echo0, echo1; }; In MyDsp.cpp : MyDsp::MyDsp() : AudioStream(AUDIO_OUTPUTS, new audio_block_t*[AUDIO_OUTPUTS]), sine(AUDIO_SAMPLE_RATE_EXACT), echo0(AUDIO_SAMPLE_RATE_EXACT,10000), echo1(AUDIO_SAMPLE_RATE_EXACT,7000) { ... // setting up DSP objects echo0.setDel(10000); echo0.setFeedback(0.5); echo1.setDel(7000); echo1.setFeedback(0.4); ... // processing buffers for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; ... }","title":"Stereo Echo"},{"location":"lectures/control/","text":"Hardware Control and Audio Codec Configuration The two main goals of this lecture are: to show you how to control DSP algorithms running on your Teensy using hardware controllers (i.e., potentiometers and buttons); to give you a basic understanding of how audio codecs work and how they can be configured using the i2c protocol. Hardware Control Electronic Basics While the goal of this class is not to teach electronics nor to make projects involving complicated circuitry, some basic circuits do need to be implemented in order to control the various parameters of the DSP algorithms studied in class using hardware controllers such as buttons and potentiometers. Hence, in case you feel like your electronics skills are a bit rusty, feel free to review the following page: https://ccrma.stanford.edu/wiki/Introduction_to_Electronics_(condensed) . Teensy 4.0 Pinout The Teensy pins map can be seen in the following figure (directly taken from the PJRC website ). Most pins can be used as digital I/Os. Some pins noted \"A(N)\" can be used as analog inputs. Please, also note that 3.3v power can be retrieved from the top right corner pin and the ground from the top left pin. Make sure to never connect the 5.5v Vin pin to any other pin of the Teensy: that would probably fry it (the Cortex M7 inside the Teensy operates at 3.3v)! Teensy pinout. The Teensy audio shield uses a bunch of pins on the Teensy for i2c and i2s communication: Teensy audio shield pins. This means that these pins (besides GND and 3.3v, of course) cannot be used for something else (i.e., connecting external sensors). Bringing Power to Your Breadboard The first step in making your first circuit with the Teensy is to bring power to the breadboard included in your kit using jumper wires: Teensy connected to the breadboard. Basically connect the 3.3v pin to the red strip and the GND pin to the black strip of the breadboard. WARNING: Do not connect the 5.5v pin to the breadboard! Adding a Rotary Potentiometer to the Circuit Your kit should come with a couple of rotary potentiometers: Rotary potentiometer mounted on the breadboard. Place it on the breadboard, and connect its leftmost pin to power and its rightmost pin to the ground. Finally, connect its center pin to the A0 pin of the Teensy using a jumper wire. Please, note that we're using this pin since it is not used by the audio shield (see previous section). Testing the Potentiometer In the current configuration, the potentiometer will deliver a 3.3v current at its center pin if it is fully turned to the right side, and 0v if it is fully turned to the left side. The following Teensy program: void setup() { Serial.begin(9600); } void loop() { int sensorValue = analogRead(A0); Serial.println(sensorValue); delay(100); } displays the values measured at the A0 pin on the Teensy in the serial debugger. Values should be between 0 and 1023 (10 bits values). Make sure that the values you're getting are consistent with the position of the potentiometer. Controlling DSP Parameters With the Potentiometer Now that you know how to retrieve potentiometer values in the Teensy, plugging it to your audio DSP should be pretty straightforward. Hence, we can reuse the crazy-sine example and do: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } Note that sensorValue needs to be turned into a frequency in hertz so we just add 100 to it to get a frequency between 100 and 1123 hertz. Now take some time to have fun ;)! Using a Button Using a button with the Teensy is slightly more involving since the use of a pulldown resistor is required (alternatively, a pullup resistor could be used, of course). This is due to the fact that buttons are \"just\" circuit breakers: they don't have a dedicated output pin like potentiometers. The pulldown resistor is used to suck potential floating currents out of the output pin of the button in order to get a stable signal to be measured on the Teensy. Hence, the following circuit must be implemented: Circuit to connect a button to the Teensy. There's no need to use an analog pin on the Teensy to measure the voltage at the output of the button since we're looking at discrete values here (0 or 1). Hence, the button shall be connected to a digital pin (number 0, for example). Expanding on the previous example, we could write: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { pinMode(0, INPUT); // configuring digital pin 0 as an input AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { if (digitalRead(0)) { // button is pressed myDsp.setGain(1); } else { myDsp.setGain(0); } int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } (Assuming that a setGain method has been implemented, which is not the case in the previous example. It shouldn't be too hard though ;) ) Exercise: Looping Between Notes by Pressing a Button Expand the \"note looper\" that you implemented as part of this lecture so that new notes are triggered when a button is pressed (as opposed to be triggered automatically). Every time the button is pressed, a new note is produced. This means that you'll have to turn your push button into a switch using software techniques... Finally, make sure that gain is controllable using a rotary potentiometer. Solution: In crazy-sine.ino : #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; bool change = true; bool on = false; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); pinMode(0, INPUT); } void loop() { if (digitalRead(0)) { // button is pressed if(!on) change = true; on = true; } else { if(on) change = true; on = false; } if(change){ // if the status of the button changed if(on){ // if the button is pressed myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; } change = false; // status changed } } Audio Codec Configuration Audio Codec An audio codec is a hardware component providing an ADC and a DAC for audio purposes. Hence, it typically has analog audio inputs and outputs (stereo, in general) and digital audio inputs and outputs. Most audio codecs support standard audio sampling rate (i.e., 44.1, 48kHz, etc.) and bit depth (i.e., 16, 24 bits, etc.). Some high-end audio codecs also support higher sampling rates (e.g., 96, 192 kHz, etc.) and bit depth (32 bits, mostly) as well as more than a stereo interface (e.g., 4x4, 8x8, etc.). The price range of audio codecs can vary significantly impacting the quality of the components, i.e., audio is usually extremely sensitive to the quality of the hardware components of a system. Audio codecs usually use two different communication channels to connect to the processor unit (whether it's a CPU, a microcontroller, etc.): an i2c bus is used to configure the codec, an i2s bus is used to transmit digital audio data. i2c (pronounced I-squared-C) is a serial communication protocol heavily used in the field of microelectronics. Most digital elements in an electronic circuit communicate using i2c. i2s (pronounced I-squared-S) is also a serial communication protocol but targeting specifically audio applications. We'll see that they're very close to each other in practice later in this class. The Teensy Audio Shield hosts an audio codec (SGTL5000) which is connected to the Teensy using the following model: Interfacing of a microcontroller with an audio codec. You can check the Teensy audio shield connection map in the Teensy Pinout section for more details on how the audio shield is actually connected to the Teensy. Before audio data can be streamed to the audio codec through i2s, it needs to be configured with an audio driver which basically just sends a set of instructions from the microcontroller to the codec using i2c. The goal of this lecture is to get a basic understanding of how audio drivers work in the context of embedded systems. Quick Tour of the SGTL5000 The SGTL5000 is a low-power 24-bit, 8 kHz to 96 kHz audio codec. Its data sheet can be found on the course repository (feel free to download it now because you'll extensively need it later in this lecture). On the first page of the data sheet, you will find a block diagram indicating the different components of the codec: Block diagram of the SGTL5000 As you can see, the SGTL5000 hosts a stereo ADC and DAC. The codec has a stereo line input and a mono mic input (both accessible through solderable pins on the audio shield). The difference between the two is that the line input goes straight to the ADC while the mic input goes through a preamp first. The gain of the preamp can be adjusted independently from that of the line input. A similar pattern is used for the outputs and the DAC which are available as line outputs (through solderable pins on the audio shield) or amplified outputs (through the headphone jack on the audio shield). Finally, the codec also has a digital interface which is used for i2c (depicted at the bottom of the block diagram) and i2s (depicted on the left side of the block diagram) communication. Configuring an Audio Codec All audio codecs work the same way and are configured through their i2c bus. A system of register/value is used for that. A register corresponds to a set of parameters and a 16 bits value can be provided to configure them. A list of all available registers of the SGTL5000 can be seen on page 31-59 of the data sheet . Have a quick look at it! For example, register 0x0010 (which is documented on p. 36) allows us to configure the DAC left and right channel volume in dB. Hence setting that register to the 3C3C (0011110000111100 binary value) will set the volume to 0dB on both channels. If you're hex/binary is a bit rusty, you can use this tool: https://www.rapidtables.com/convert/number/hex-to-binary.html to carry out the conversion. Audio Codec Driver Writing an audio codec driver consists of sending the right sequence of register/value through i2c to the codec. This will set the signal routing, the i2s format, the sampling rate, the bit depth, etc. control_sgtl5000.cpp of the Teensy Audio Library implements a driver for the SGTL5000 codec. The write method can be used to set a register and its corresponding 16-bit value. Note that the Arduino Wire library is used for that. The enable method sets a bunch of register values to provide a basic working configuration to the codec in the context of the Teensy. A bunch of methods are implemented to set high-level parameters of the codec, such as volume , micGain , lineInLevel , etc. Note that a bunch of macros are defined at the beginning for various registers (which constitutes a potential alternative to the codec datasheet, etc.). In most cases, calling: AudioControlSGTL5000 audioShield; audioShield.enable(); in the .ino file will be sufficient to get the audio codec going and send it i2s data. Since the write method of AudioControlSGTL5000 is protected, it cannot be called outside of the class. Hence, to write custom methods to configure the codec and which are not currently available in AudioControlSGTL5000 , one would have to write a custom version of control_sgtl5000.cpp .","title":" Hardware Control and Audio Codec Configuration "},{"location":"lectures/control/#hardware-control-and-audio-codec-configuration","text":"The two main goals of this lecture are: to show you how to control DSP algorithms running on your Teensy using hardware controllers (i.e., potentiometers and buttons); to give you a basic understanding of how audio codecs work and how they can be configured using the i2c protocol.","title":"Hardware Control and Audio Codec Configuration"},{"location":"lectures/control/#hardware-control","text":"","title":"Hardware Control"},{"location":"lectures/control/#electronic-basics","text":"While the goal of this class is not to teach electronics nor to make projects involving complicated circuitry, some basic circuits do need to be implemented in order to control the various parameters of the DSP algorithms studied in class using hardware controllers such as buttons and potentiometers. Hence, in case you feel like your electronics skills are a bit rusty, feel free to review the following page: https://ccrma.stanford.edu/wiki/Introduction_to_Electronics_(condensed) .","title":"Electronic Basics"},{"location":"lectures/control/#teensy-40-pinout","text":"The Teensy pins map can be seen in the following figure (directly taken from the PJRC website ). Most pins can be used as digital I/Os. Some pins noted \"A(N)\" can be used as analog inputs. Please, also note that 3.3v power can be retrieved from the top right corner pin and the ground from the top left pin. Make sure to never connect the 5.5v Vin pin to any other pin of the Teensy: that would probably fry it (the Cortex M7 inside the Teensy operates at 3.3v)! Teensy pinout. The Teensy audio shield uses a bunch of pins on the Teensy for i2c and i2s communication: Teensy audio shield pins. This means that these pins (besides GND and 3.3v, of course) cannot be used for something else (i.e., connecting external sensors).","title":"Teensy 4.0 Pinout"},{"location":"lectures/control/#bringing-power-to-your-breadboard","text":"The first step in making your first circuit with the Teensy is to bring power to the breadboard included in your kit using jumper wires: Teensy connected to the breadboard. Basically connect the 3.3v pin to the red strip and the GND pin to the black strip of the breadboard. WARNING: Do not connect the 5.5v pin to the breadboard!","title":"Bringing Power to Your Breadboard"},{"location":"lectures/control/#adding-a-rotary-potentiometer-to-the-circuit","text":"Your kit should come with a couple of rotary potentiometers: Rotary potentiometer mounted on the breadboard. Place it on the breadboard, and connect its leftmost pin to power and its rightmost pin to the ground. Finally, connect its center pin to the A0 pin of the Teensy using a jumper wire. Please, note that we're using this pin since it is not used by the audio shield (see previous section).","title":"Adding a Rotary Potentiometer to the Circuit"},{"location":"lectures/control/#testing-the-potentiometer","text":"In the current configuration, the potentiometer will deliver a 3.3v current at its center pin if it is fully turned to the right side, and 0v if it is fully turned to the left side. The following Teensy program: void setup() { Serial.begin(9600); } void loop() { int sensorValue = analogRead(A0); Serial.println(sensorValue); delay(100); } displays the values measured at the A0 pin on the Teensy in the serial debugger. Values should be between 0 and 1023 (10 bits values). Make sure that the values you're getting are consistent with the position of the potentiometer.","title":"Testing the Potentiometer"},{"location":"lectures/control/#controlling-dsp-parameters-with-the-potentiometer","text":"Now that you know how to retrieve potentiometer values in the Teensy, plugging it to your audio DSP should be pretty straightforward. Hence, we can reuse the crazy-sine example and do: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } Note that sensorValue needs to be turned into a frequency in hertz so we just add 100 to it to get a frequency between 100 and 1123 hertz. Now take some time to have fun ;)!","title":"Controlling DSP Parameters With the Potentiometer"},{"location":"lectures/control/#using-a-button","text":"Using a button with the Teensy is slightly more involving since the use of a pulldown resistor is required (alternatively, a pullup resistor could be used, of course). This is due to the fact that buttons are \"just\" circuit breakers: they don't have a dedicated output pin like potentiometers. The pulldown resistor is used to suck potential floating currents out of the output pin of the button in order to get a stable signal to be measured on the Teensy. Hence, the following circuit must be implemented: Circuit to connect a button to the Teensy. There's no need to use an analog pin on the Teensy to measure the voltage at the output of the button since we're looking at discrete values here (0 or 1). Hence, the button shall be connected to a digital pin (number 0, for example). Expanding on the previous example, we could write: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { pinMode(0, INPUT); // configuring digital pin 0 as an input AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { if (digitalRead(0)) { // button is pressed myDsp.setGain(1); } else { myDsp.setGain(0); } int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } (Assuming that a setGain method has been implemented, which is not the case in the previous example. It shouldn't be too hard though ;) )","title":"Using a Button"},{"location":"lectures/control/#exercise-looping-between-notes-by-pressing-a-button","text":"Expand the \"note looper\" that you implemented as part of this lecture so that new notes are triggered when a button is pressed (as opposed to be triggered automatically). Every time the button is pressed, a new note is produced. This means that you'll have to turn your push button into a switch using software techniques... Finally, make sure that gain is controllable using a rotary potentiometer. Solution: In crazy-sine.ino : #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; bool change = true; bool on = false; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); pinMode(0, INPUT); } void loop() { if (digitalRead(0)) { // button is pressed if(!on) change = true; on = true; } else { if(on) change = true; on = false; } if(change){ // if the status of the button changed if(on){ // if the button is pressed myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; } change = false; // status changed } }","title":"Exercise: Looping Between Notes by Pressing a Button"},{"location":"lectures/control/#audio-codec-configuration","text":"","title":"Audio Codec Configuration"},{"location":"lectures/control/#audio-codec","text":"An audio codec is a hardware component providing an ADC and a DAC for audio purposes. Hence, it typically has analog audio inputs and outputs (stereo, in general) and digital audio inputs and outputs. Most audio codecs support standard audio sampling rate (i.e., 44.1, 48kHz, etc.) and bit depth (i.e., 16, 24 bits, etc.). Some high-end audio codecs also support higher sampling rates (e.g., 96, 192 kHz, etc.) and bit depth (32 bits, mostly) as well as more than a stereo interface (e.g., 4x4, 8x8, etc.). The price range of audio codecs can vary significantly impacting the quality of the components, i.e., audio is usually extremely sensitive to the quality of the hardware components of a system. Audio codecs usually use two different communication channels to connect to the processor unit (whether it's a CPU, a microcontroller, etc.): an i2c bus is used to configure the codec, an i2s bus is used to transmit digital audio data. i2c (pronounced I-squared-C) is a serial communication protocol heavily used in the field of microelectronics. Most digital elements in an electronic circuit communicate using i2c. i2s (pronounced I-squared-S) is also a serial communication protocol but targeting specifically audio applications. We'll see that they're very close to each other in practice later in this class. The Teensy Audio Shield hosts an audio codec (SGTL5000) which is connected to the Teensy using the following model: Interfacing of a microcontroller with an audio codec. You can check the Teensy audio shield connection map in the Teensy Pinout section for more details on how the audio shield is actually connected to the Teensy. Before audio data can be streamed to the audio codec through i2s, it needs to be configured with an audio driver which basically just sends a set of instructions from the microcontroller to the codec using i2c. The goal of this lecture is to get a basic understanding of how audio drivers work in the context of embedded systems.","title":"Audio Codec"},{"location":"lectures/control/#quick-tour-of-the-sgtl5000","text":"The SGTL5000 is a low-power 24-bit, 8 kHz to 96 kHz audio codec. Its data sheet can be found on the course repository (feel free to download it now because you'll extensively need it later in this lecture). On the first page of the data sheet, you will find a block diagram indicating the different components of the codec: Block diagram of the SGTL5000 As you can see, the SGTL5000 hosts a stereo ADC and DAC. The codec has a stereo line input and a mono mic input (both accessible through solderable pins on the audio shield). The difference between the two is that the line input goes straight to the ADC while the mic input goes through a preamp first. The gain of the preamp can be adjusted independently from that of the line input. A similar pattern is used for the outputs and the DAC which are available as line outputs (through solderable pins on the audio shield) or amplified outputs (through the headphone jack on the audio shield). Finally, the codec also has a digital interface which is used for i2c (depicted at the bottom of the block diagram) and i2s (depicted on the left side of the block diagram) communication.","title":"Quick Tour of the SGTL5000"},{"location":"lectures/control/#configuring-an-audio-codec","text":"All audio codecs work the same way and are configured through their i2c bus. A system of register/value is used for that. A register corresponds to a set of parameters and a 16 bits value can be provided to configure them. A list of all available registers of the SGTL5000 can be seen on page 31-59 of the data sheet . Have a quick look at it! For example, register 0x0010 (which is documented on p. 36) allows us to configure the DAC left and right channel volume in dB. Hence setting that register to the 3C3C (0011110000111100 binary value) will set the volume to 0dB on both channels. If you're hex/binary is a bit rusty, you can use this tool: https://www.rapidtables.com/convert/number/hex-to-binary.html to carry out the conversion.","title":"Configuring an Audio Codec"},{"location":"lectures/control/#audio-codec-driver","text":"Writing an audio codec driver consists of sending the right sequence of register/value through i2c to the codec. This will set the signal routing, the i2s format, the sampling rate, the bit depth, etc. control_sgtl5000.cpp of the Teensy Audio Library implements a driver for the SGTL5000 codec. The write method can be used to set a register and its corresponding 16-bit value. Note that the Arduino Wire library is used for that. The enable method sets a bunch of register values to provide a basic working configuration to the codec in the context of the Teensy. A bunch of methods are implemented to set high-level parameters of the codec, such as volume , micGain , lineInLevel , etc. Note that a bunch of macros are defined at the beginning for various registers (which constitutes a potential alternative to the codec datasheet, etc.). In most cases, calling: AudioControlSGTL5000 audioShield; audioShield.enable(); in the .ino file will be sufficient to get the audio codec going and send it i2s data. Since the write method of AudioControlSGTL5000 is protected, it cannot be called outside of the class. Hence, to write custom methods to configure the codec and which are not currently available in AudioControlSGTL5000 , one would have to write a custom version of control_sgtl5000.cpp .","title":"Audio Codec Driver"},{"location":"lectures/digital-audio/","text":"Audio Signal Processing Fundamentals The goal of this lecture is to provide an overview of the basics of digital audio. Analog Audio Signals Before the advent of digital audio, most audio systems/technologies were analog. An analog audio signal can take different forms: it can be electric (e.g., transmitted through an electric wire and stored on a magnetic tape) or mechanical (e.g., transmitted through the air as standing waves and stored on a vinyl disc). Acoustical mechanical waves can be converted into an electric signal using a microphone. Conversely, an electric audio signal can be converted into mechanical acoustical waves using a speaker. In nature, sounds almost always originate from a mechanical source. However, in the 20th century, many musicians, composers and engineers experimented with the production of sound from an electrical source. One of the pioneer in this field was Karlheinz Stockhausen . This lead to analog and modular synthesizers which are very popular among Croix-Roussian hipsters these days. A modular analog synthesizer The Discovery of Digital Audio Sampling theory dates back from the beginning of the 20th century with initial work by Harry Nyquist and was theorized in the 1930s by Claude Shannon to become the Nyquist-Shannon sampling theorem. Carrying sampling in the field of audio is relatively simple: voltage measurements are carried out at regular intervals of time on an analog electrical signal. Each individual acquired value is called a \"sample\" and can be stored on a computer. Hence, while an analog electric audio signal is a variation of tension in time in an electric cable, a digital audio signal is just series of samples (values) in time as well. Signal sampling representation. The continuous signal is represented with a green colored line while the discrete samples are indicated by the blue vertical lines. (source: Wikipedia ) ADC and DAC In the field of audio, an ADC (Analog to Digital Converter) is a hardware component that can be used to discretize (sample) an electrical analog audio signal. The reverse operation is carried out using a DAC (Digital to Analog Converter). In most systems, the ADC and the DAC are hosted in the same piece of hardware (e.g., audio codec, audio interface, etc.). Human Hearing Range and Sampling Rate One of the main factor to consider when sampling an audio signal is the human hearing range. In theory, humans can hear any sound between 20 and 20000 Hz. In practice, our ability to perceive high frequencies decays over time and is affected by environmental factors (e.g., if we're exposed to sound with high volume, if we contract some diseases such as hear infections, etc.). By the age of 30, most adults can't hear frequencies over 17 kHz. When sampling an audio signal, the number of samples per second also known as the sampling rate (noted fs ) will determine the highest frequency than can be sampled by the system. The rule is very simple: the highest frequency that can be sampled is half the sampling rate. Hence, in order to sample a frequency of 20 kHz, the sampling rate of the system must be at least 40 kHz which corresponds to 40000 values (samples) per second. The highest frequency that can be sampled is also known as the \" Nyquist Frequency \" ( fn ): fn=\\frac{fs}{2} The standard for modern audio systems is to use a sampling rate of 48 kHz. fs is 44.1 kHz on compact discs (CDs) and many home and recording studios use a sampling rate of 96 or 192 kHz. Sampling Theorem Let x(t) denote any continuous-time signal having a continuous Fourier transform : X(j\\omega) \\triangleq \\int_{-\\infty}^{\\infty}x(t)e^{-j \\omega t}dt Let x_d(n) \\triangleq x(nT), \\quad n=\\dots,-2,-1,0,1,2,\\dots, denote the samples of x(t) at uniform intervals of T seconds. Then x(t) can be exactly reconstructed from its samples x_d(n) if X(j\\omega)=0 for all \\vert\\omega\\vert\\geq\\pi/T . In other words, any frequency (harmonics) between 0 Hz and the Nyquist frequency can be exactly reconstructed without loosing any information. That also means that if the Nyquist frequency is above the upper threshold of the human hearing range (e.g., 20 kHz), a digitized signal should sound exactly the same as its analog counterpart from a perceptual standpoint. Additional proofs about the sampling theorem can be found on Julius Smith's website here . Aliasing Aliasing is a well known phenomenon in the field of video: In audio, aliasing happens when a digital signal contains frequencies above the Nyquist frequency. In that case, they are not sampled at the right frequency and they are wrapped. Hence, for all frequency fo above fn , the sampled frequency f will be: f = fn - (fo-fn) with fn = \\frac{fs}{2} Aliasing is typically prevented by filtering an analog signal before it is discretized by removing all frequency above fn . Aliasing can also be obtained when synthesizing a broadband signal on a computer (e.g., a sawtooth wave). It is the software engineer's role to prevent this from happening. Bit Depth, Dynamic Range and Signal-to-Noise Ratio Beside sampling rate, the other parameter of sampling is the bit depth of audio samples. Audio is typically recorded at 8, 16 (the standard for CDs), or 24 bits (and 32 bits in some rarer cases). A higher bit depth means a more accurate precision for a given audio sample. This impacts directly the dynamic range and the signal-to-noise (SNR) ratio of a digital signal. In other words, a smaller bit depth will mean more noise in the signal, etc. Additional information about this topic can be found here . Range of Audio Samples Audio samples can be coded in many different ways depending on the context. Some low-level systems use fixed-point numbers (i.e., integers) for efficiency. In that case, the range of the signal will be determined by the data type. For example, if audio samples are coded on 16 bits unsigned integers, the range of the signal will be 0 to 2^{16} - 1 (or 65535). At the hardware level (e.g., ADC/DAC), audio samples are almost exclusively coded on integers. On the other hand, fixed points are relatively hard to deal with at the software level when it comes to implementing DSP algorithms. In that case, it is much more convenient to use decimal numbers (i.e., floating points). The established standard in audio is that audio signals coded on decimal numbers always have the following range: {-1;1}. While this range can be exceeded within an algorithm without any consequences, the inputs and outputs of a DSP block must always be constrained between -1 and 1. Most systems will clip audio signals to this range to prevent warping and will hence result in clipping if exceeded. First Synthesized Sound on a Digital Computer While Shanon and Nyquist theorized sampling in the 1930s, it's only in 1958 that a sound was synthesized for the first time on a computer by Max Mathews at Bell Labs, giving birth a few years later to the first song synthesized (and sung) by a computer: This was by the way reused by Stanley Kubrick in one of his famous movie as HAL the computer is slowly dying as it's being unplugged: These technologies were then extensively exploited until today both for musical applications and in the industry at large.","title":" Audio Signal Processing Fundamentals "},{"location":"lectures/digital-audio/#audio-signal-processing-fundamentals","text":"The goal of this lecture is to provide an overview of the basics of digital audio.","title":"Audio Signal Processing Fundamentals"},{"location":"lectures/digital-audio/#analog-audio-signals","text":"Before the advent of digital audio, most audio systems/technologies were analog. An analog audio signal can take different forms: it can be electric (e.g., transmitted through an electric wire and stored on a magnetic tape) or mechanical (e.g., transmitted through the air as standing waves and stored on a vinyl disc). Acoustical mechanical waves can be converted into an electric signal using a microphone. Conversely, an electric audio signal can be converted into mechanical acoustical waves using a speaker. In nature, sounds almost always originate from a mechanical source. However, in the 20th century, many musicians, composers and engineers experimented with the production of sound from an electrical source. One of the pioneer in this field was Karlheinz Stockhausen . This lead to analog and modular synthesizers which are very popular among Croix-Roussian hipsters these days. A modular analog synthesizer","title":"Analog Audio Signals"},{"location":"lectures/digital-audio/#the-discovery-of-digital-audio","text":"Sampling theory dates back from the beginning of the 20th century with initial work by Harry Nyquist and was theorized in the 1930s by Claude Shannon to become the Nyquist-Shannon sampling theorem. Carrying sampling in the field of audio is relatively simple: voltage measurements are carried out at regular intervals of time on an analog electrical signal. Each individual acquired value is called a \"sample\" and can be stored on a computer. Hence, while an analog electric audio signal is a variation of tension in time in an electric cable, a digital audio signal is just series of samples (values) in time as well. Signal sampling representation. The continuous signal is represented with a green colored line while the discrete samples are indicated by the blue vertical lines. (source: Wikipedia )","title":"The Discovery of Digital Audio"},{"location":"lectures/digital-audio/#adc-and-dac","text":"In the field of audio, an ADC (Analog to Digital Converter) is a hardware component that can be used to discretize (sample) an electrical analog audio signal. The reverse operation is carried out using a DAC (Digital to Analog Converter). In most systems, the ADC and the DAC are hosted in the same piece of hardware (e.g., audio codec, audio interface, etc.).","title":"ADC and DAC"},{"location":"lectures/digital-audio/#human-hearing-range-and-sampling-rate","text":"One of the main factor to consider when sampling an audio signal is the human hearing range. In theory, humans can hear any sound between 20 and 20000 Hz. In practice, our ability to perceive high frequencies decays over time and is affected by environmental factors (e.g., if we're exposed to sound with high volume, if we contract some diseases such as hear infections, etc.). By the age of 30, most adults can't hear frequencies over 17 kHz. When sampling an audio signal, the number of samples per second also known as the sampling rate (noted fs ) will determine the highest frequency than can be sampled by the system. The rule is very simple: the highest frequency that can be sampled is half the sampling rate. Hence, in order to sample a frequency of 20 kHz, the sampling rate of the system must be at least 40 kHz which corresponds to 40000 values (samples) per second. The highest frequency that can be sampled is also known as the \" Nyquist Frequency \" ( fn ): fn=\\frac{fs}{2} The standard for modern audio systems is to use a sampling rate of 48 kHz. fs is 44.1 kHz on compact discs (CDs) and many home and recording studios use a sampling rate of 96 or 192 kHz.","title":"Human Hearing Range and Sampling Rate"},{"location":"lectures/digital-audio/#sampling-theorem","text":"Let x(t) denote any continuous-time signal having a continuous Fourier transform : X(j\\omega) \\triangleq \\int_{-\\infty}^{\\infty}x(t)e^{-j \\omega t}dt Let x_d(n) \\triangleq x(nT), \\quad n=\\dots,-2,-1,0,1,2,\\dots, denote the samples of x(t) at uniform intervals of T seconds. Then x(t) can be exactly reconstructed from its samples x_d(n) if X(j\\omega)=0 for all \\vert\\omega\\vert\\geq\\pi/T . In other words, any frequency (harmonics) between 0 Hz and the Nyquist frequency can be exactly reconstructed without loosing any information. That also means that if the Nyquist frequency is above the upper threshold of the human hearing range (e.g., 20 kHz), a digitized signal should sound exactly the same as its analog counterpart from a perceptual standpoint. Additional proofs about the sampling theorem can be found on Julius Smith's website here .","title":"Sampling Theorem"},{"location":"lectures/digital-audio/#aliasing","text":"Aliasing is a well known phenomenon in the field of video: In audio, aliasing happens when a digital signal contains frequencies above the Nyquist frequency. In that case, they are not sampled at the right frequency and they are wrapped. Hence, for all frequency fo above fn , the sampled frequency f will be: f = fn - (fo-fn) with fn = \\frac{fs}{2} Aliasing is typically prevented by filtering an analog signal before it is discretized by removing all frequency above fn . Aliasing can also be obtained when synthesizing a broadband signal on a computer (e.g., a sawtooth wave). It is the software engineer's role to prevent this from happening.","title":"Aliasing"},{"location":"lectures/digital-audio/#bit-depth-dynamic-range-and-signal-to-noise-ratio","text":"Beside sampling rate, the other parameter of sampling is the bit depth of audio samples. Audio is typically recorded at 8, 16 (the standard for CDs), or 24 bits (and 32 bits in some rarer cases). A higher bit depth means a more accurate precision for a given audio sample. This impacts directly the dynamic range and the signal-to-noise (SNR) ratio of a digital signal. In other words, a smaller bit depth will mean more noise in the signal, etc. Additional information about this topic can be found here .","title":"Bit Depth, Dynamic Range and Signal-to-Noise Ratio"},{"location":"lectures/digital-audio/#range-of-audio-samples","text":"Audio samples can be coded in many different ways depending on the context. Some low-level systems use fixed-point numbers (i.e., integers) for efficiency. In that case, the range of the signal will be determined by the data type. For example, if audio samples are coded on 16 bits unsigned integers, the range of the signal will be 0 to 2^{16} - 1 (or 65535). At the hardware level (e.g., ADC/DAC), audio samples are almost exclusively coded on integers. On the other hand, fixed points are relatively hard to deal with at the software level when it comes to implementing DSP algorithms. In that case, it is much more convenient to use decimal numbers (i.e., floating points). The established standard in audio is that audio signals coded on decimal numbers always have the following range: {-1;1}. While this range can be exceeded within an algorithm without any consequences, the inputs and outputs of a DSP block must always be constrained between -1 and 1. Most systems will clip audio signals to this range to prevent warping and will hence result in clipping if exceeded.","title":"Range of Audio Samples"},{"location":"lectures/digital-audio/#first-synthesized-sound-on-a-digital-computer","text":"While Shanon and Nyquist theorized sampling in the 1930s, it's only in 1958 that a sound was synthesized for the first time on a computer by Max Mathews at Bell Labs, giving birth a few years later to the first song synthesized (and sung) by a computer: This was by the way reused by Stanley Kubrick in one of his famous movie as HAL the computer is slowly dying as it's being unplugged: These technologies were then extensively exploited until today both for musical applications and in the industry at large.","title":"First Synthesized Sound on a Digital Computer"},{"location":"lectures/dsp1/","text":"Audio Processing Basics I This lecture and this one (Audio Processing Basics II) present a selection of audio processing and synthesis algorithms. It is in no way comprehensive: the goal is just to give you a sense of what's out there. All these algorithms have been extensively used during the second half of the twentieth century by musicians and artists, especially within the computer music community. White Noise White noise is a specific kind of signal in which there's an infinite number of harmonics all having the same level. In other words, the spectrum of white noise looks completely flat. White noise is produced by generating random numbers between -1 and 1. Noise.cpp demonstrates how this can be done in C++ using the rand() function: Noise::Noise() : randDiv(1.0/RAND_MAX){} float Noise::tick(){ return rand()*randDiv*2 - 1; } The Simple Filter: One Zero section presents a use example of white noise. Wave Shape Synthesis Wave Shape synthesis is one of the most basic sound synthesis technique. It consists of using oscillators producing waveforms of different shapes to generate sound. The most standard wave shapes are: sine wave , square wave , triangle wave , sawtooth wave . The crazy-sine example can be considered as \"wave shape synthesis\" in that regard. The crazy-saw example is very similar to crazy-sine , but it's based on a sawtooth wave instead. The sawtooth wave is created by using a phasor object. Just as a reminder, a phasor produces a signals tamping from 0 to 1 at a given frequency, it can therefore be seen as a sawtooth wave. Since the range of oscillators must be bounded between -1 and 1, we adjusts the output of the phasor such that: float currentSample = sawtooth.tick()*2 - 1; Feel free to try the crazy-saw example at this point. Amplitude Modulation (AM) Synthesis Amplitude modulation synthesis consists of modulating the amplitude of a signal with another one. Sine waves are typically used for that: Amplitude Modulation (Source: Wikipedia ) When the frequency of the modulator is low (bellow 20Hz), our ear is able to distinguish each independent \"beat,\" creating a tremolo effect. However, above 20Hz two side bands (if sine waves are used) start appearing following this rule: Amplitude Modulation Spectrum (Source: Wikipedia ) The mathematical proof of this can be found on Julius Smith's website . Am.cpp implements a sinusoidal amplitude modulation synthesizer: float Am::tick(){ int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float posMod = sineTable.tick(mIndex)*0.5 + 0.5; return sineTable.tick(cIndex)*(1 - posMod*modIndex)*gain; } Note that phasors are used instead of \"complete\" sine wave oscillators to save the memory of an extra sine wave table. The range of the modulating oscillator is adjusted to be {0,1} instead of {-1,1}. The amplitude parameter of the modulating oscillator is called the index of modulation and its frequency, the frequency of modulation . In practice, the same result could be achieved using additive synthesis and three sine wave oscillators but AM allows us to save one oscillator. Also, AM is usually used an audio effect and modulation is applied to an input signal in that case instead of a sine wave. Sidebands will then be produced for each harmonic of the processed sound. The am example demonstrates a use case of an AM synthesizer. Use the Rec and Mode button to cycle through the parameters of the synth and change their value. Frequency Modulation (FM) Synthesis Frequency modulation synthesis consists of modulating the frequency of an oscillator with another one: Frequency Modulation (Source: Wikipedia ) which mathematically can be expressed as: x(t) = A_c\\sin[\\omega_ct + \\phi_c + A_m\\sin(\\omega_mt + \\phi_m)] where c denotes the carrier and m , the modulator. As for AM, the frequency of the modulating oscillator is called the frequency of modulation and the amplitude of the modulating oscillator, the index of modulation . Unlike AM, the value of the index of modulation can exceed 1 which will increase the number of sidebands. FM is not limited to two sidebands and can have an infinite number of sidebands depending on the value of the index. The mathematical rational behind this can be found on Julius Smith's website . fm.cpp provides a simple example of how an FM synthesizer can be implemented: float Fm::tick(){ int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float modulator = sineTable.tick(mIndex); cPhasor.setFrequency(cFreq + modulator*modIndex); int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; return sineTable.tick(cIndex)*gain; } Note that as for the AM example, we're saving an extra sine wave table by using the same one for both oscillators. The examples folder of the course repository hosts a simple Teensy program illustrating the use of FM. Use the Rec and Mode button to cycle through the parameters of the synth and change their value. FM synthesis was discovered in the late 1960s by John Chowning at Stanford University in California. He's now considered as one of the funding fathers of music technology and computer music. FM completely revolutionized the world of music in the 1980s by allowing Yamaha to produce the first commercial digital synthesizers: the DX7 which met a huge success. FM synthesis is the second most profitable patent that Stanford ever had. Simple Filter: One Zero Filters are heavily used in the field of audio processing. In fact, designing filters is a whole field by itself. They are at the basis of many audio effects such as Wah guitar pedals, etc. From an algorithmic standpoint, the most basic filter is what we call a \"one zero\" filter which means that its transfer function only has numerators and no denominators. The differential equation of a one zero filter can be expressed as: y(n) = b_0x(n) + b_1x(n-1) where b_1 is \"the zero\" of the filter (also called feed forward coefficient ), b_0 can be discarded as it is equal to 1 in most cases. One zero filters can either be used as a lowpass if the value of b_1 is positive or as a highpass if b_1 is negative. The frequency response of the filter which is be obtained with H(e^{j \\omega T}) = b_0 + b_1e^{-j \\omega T} can be visualized on Julius Smith's website . Note that the gain of the signal is amplified on the second half of the spectrum which needs to be taken into account if this filter is used to process audio (once again, the output signal must be bounded within {-1,1}). OneZero.cpp implements a one zero filter: float OneZero::tick(float input){ float output = input + del*b1; del = input; return output*0.5; } Note that we multiply the output by 0.5 to normalize the output gain. The filtered-noise example program for the Teensy demonstrates the use of OneZero.cpp by feeding white noise in it. The value of b_1 can be changed by pressing the \"Mode\" button on the board, give it a try! Exercises LFO: Low Frequency Oscillator An LFO is an oscillator whose frequency is below the human hearing range (20 Hz). LFOs are typically used to create vibrato. In that case, the frequency of the LFO is usually set to 6 Hz. Modify the crazy-saw example so that notes are played slower (1 per second) and that some vibrato is added to the generated sound. Solution: Shall be posted here after class... In MyDsp.h : #include \"Sine.h\" ... private: float freq; Phasor sawtooth; Echo echo; Sine LFO; }; In MyDsp.cpp : MyDsp::MyDsp() : ... freq(440), sawtooth(AUDIO_SAMPLE_RATE_EXACT), echo(AUDIO_SAMPLE_RATE_EXACT,10000), LFO(AUDIO_SAMPLE_RATE_EXACT) { ... // setting up DSP objects echo.setDel(10000); echo.setFeedback(0.5); LFO.setFrequency(6); } ... // set sine wave frequency void MyDsp::setFreq(float f){ freq = f; } ... for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP sawtooth.setFrequency(freq*(1 + LFO.tick()*0.1)); float currentSample = echo.tick(sawtooth.tick()*2 - 1)*0.5; Towards the DX7 The DX7 carried out frequency modulation over a total of six oscillators that could be patched in different ways . So FM is not limited to two oscillators... Try to implement an FM synthesizer involving 3 oscillators instead of one. They should be connected in series: 3 -> 2 -> 1.","title":" Audio Processing Basics I "},{"location":"lectures/dsp1/#audio-processing-basics-i","text":"This lecture and this one (Audio Processing Basics II) present a selection of audio processing and synthesis algorithms. It is in no way comprehensive: the goal is just to give you a sense of what's out there. All these algorithms have been extensively used during the second half of the twentieth century by musicians and artists, especially within the computer music community.","title":"Audio Processing Basics I"},{"location":"lectures/dsp1/#white-noise","text":"White noise is a specific kind of signal in which there's an infinite number of harmonics all having the same level. In other words, the spectrum of white noise looks completely flat. White noise is produced by generating random numbers between -1 and 1. Noise.cpp demonstrates how this can be done in C++ using the rand() function: Noise::Noise() : randDiv(1.0/RAND_MAX){} float Noise::tick(){ return rand()*randDiv*2 - 1; } The Simple Filter: One Zero section presents a use example of white noise.","title":"White Noise"},{"location":"lectures/dsp1/#wave-shape-synthesis","text":"Wave Shape synthesis is one of the most basic sound synthesis technique. It consists of using oscillators producing waveforms of different shapes to generate sound. The most standard wave shapes are: sine wave , square wave , triangle wave , sawtooth wave . The crazy-sine example can be considered as \"wave shape synthesis\" in that regard. The crazy-saw example is very similar to crazy-sine , but it's based on a sawtooth wave instead. The sawtooth wave is created by using a phasor object. Just as a reminder, a phasor produces a signals tamping from 0 to 1 at a given frequency, it can therefore be seen as a sawtooth wave. Since the range of oscillators must be bounded between -1 and 1, we adjusts the output of the phasor such that: float currentSample = sawtooth.tick()*2 - 1; Feel free to try the crazy-saw example at this point.","title":"Wave Shape Synthesis"},{"location":"lectures/dsp1/#amplitude-modulation-am-synthesis","text":"Amplitude modulation synthesis consists of modulating the amplitude of a signal with another one. Sine waves are typically used for that: Amplitude Modulation (Source: Wikipedia ) When the frequency of the modulator is low (bellow 20Hz), our ear is able to distinguish each independent \"beat,\" creating a tremolo effect. However, above 20Hz two side bands (if sine waves are used) start appearing following this rule: Amplitude Modulation Spectrum (Source: Wikipedia ) The mathematical proof of this can be found on Julius Smith's website . Am.cpp implements a sinusoidal amplitude modulation synthesizer: float Am::tick(){ int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float posMod = sineTable.tick(mIndex)*0.5 + 0.5; return sineTable.tick(cIndex)*(1 - posMod*modIndex)*gain; } Note that phasors are used instead of \"complete\" sine wave oscillators to save the memory of an extra sine wave table. The range of the modulating oscillator is adjusted to be {0,1} instead of {-1,1}. The amplitude parameter of the modulating oscillator is called the index of modulation and its frequency, the frequency of modulation . In practice, the same result could be achieved using additive synthesis and three sine wave oscillators but AM allows us to save one oscillator. Also, AM is usually used an audio effect and modulation is applied to an input signal in that case instead of a sine wave. Sidebands will then be produced for each harmonic of the processed sound. The am example demonstrates a use case of an AM synthesizer. Use the Rec and Mode button to cycle through the parameters of the synth and change their value.","title":"Amplitude Modulation (AM) Synthesis"},{"location":"lectures/dsp1/#frequency-modulation-fm-synthesis","text":"Frequency modulation synthesis consists of modulating the frequency of an oscillator with another one: Frequency Modulation (Source: Wikipedia ) which mathematically can be expressed as: x(t) = A_c\\sin[\\omega_ct + \\phi_c + A_m\\sin(\\omega_mt + \\phi_m)] where c denotes the carrier and m , the modulator. As for AM, the frequency of the modulating oscillator is called the frequency of modulation and the amplitude of the modulating oscillator, the index of modulation . Unlike AM, the value of the index of modulation can exceed 1 which will increase the number of sidebands. FM is not limited to two sidebands and can have an infinite number of sidebands depending on the value of the index. The mathematical rational behind this can be found on Julius Smith's website . fm.cpp provides a simple example of how an FM synthesizer can be implemented: float Fm::tick(){ int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float modulator = sineTable.tick(mIndex); cPhasor.setFrequency(cFreq + modulator*modIndex); int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; return sineTable.tick(cIndex)*gain; } Note that as for the AM example, we're saving an extra sine wave table by using the same one for both oscillators. The examples folder of the course repository hosts a simple Teensy program illustrating the use of FM. Use the Rec and Mode button to cycle through the parameters of the synth and change their value. FM synthesis was discovered in the late 1960s by John Chowning at Stanford University in California. He's now considered as one of the funding fathers of music technology and computer music. FM completely revolutionized the world of music in the 1980s by allowing Yamaha to produce the first commercial digital synthesizers: the DX7 which met a huge success. FM synthesis is the second most profitable patent that Stanford ever had.","title":"Frequency Modulation (FM) Synthesis"},{"location":"lectures/dsp1/#simple-filter-one-zero","text":"Filters are heavily used in the field of audio processing. In fact, designing filters is a whole field by itself. They are at the basis of many audio effects such as Wah guitar pedals, etc. From an algorithmic standpoint, the most basic filter is what we call a \"one zero\" filter which means that its transfer function only has numerators and no denominators. The differential equation of a one zero filter can be expressed as: y(n) = b_0x(n) + b_1x(n-1) where b_1 is \"the zero\" of the filter (also called feed forward coefficient ), b_0 can be discarded as it is equal to 1 in most cases. One zero filters can either be used as a lowpass if the value of b_1 is positive or as a highpass if b_1 is negative. The frequency response of the filter which is be obtained with H(e^{j \\omega T}) = b_0 + b_1e^{-j \\omega T} can be visualized on Julius Smith's website . Note that the gain of the signal is amplified on the second half of the spectrum which needs to be taken into account if this filter is used to process audio (once again, the output signal must be bounded within {-1,1}). OneZero.cpp implements a one zero filter: float OneZero::tick(float input){ float output = input + del*b1; del = input; return output*0.5; } Note that we multiply the output by 0.5 to normalize the output gain. The filtered-noise example program for the Teensy demonstrates the use of OneZero.cpp by feeding white noise in it. The value of b_1 can be changed by pressing the \"Mode\" button on the board, give it a try!","title":"Simple Filter: One Zero"},{"location":"lectures/dsp1/#exercises","text":"","title":"Exercises"},{"location":"lectures/dsp1/#lfo-low-frequency-oscillator","text":"An LFO is an oscillator whose frequency is below the human hearing range (20 Hz). LFOs are typically used to create vibrato. In that case, the frequency of the LFO is usually set to 6 Hz. Modify the crazy-saw example so that notes are played slower (1 per second) and that some vibrato is added to the generated sound. Solution: Shall be posted here after class... In MyDsp.h : #include \"Sine.h\" ... private: float freq; Phasor sawtooth; Echo echo; Sine LFO; }; In MyDsp.cpp : MyDsp::MyDsp() : ... freq(440), sawtooth(AUDIO_SAMPLE_RATE_EXACT), echo(AUDIO_SAMPLE_RATE_EXACT,10000), LFO(AUDIO_SAMPLE_RATE_EXACT) { ... // setting up DSP objects echo.setDel(10000); echo.setFeedback(0.5); LFO.setFrequency(6); } ... // set sine wave frequency void MyDsp::setFreq(float f){ freq = f; } ... for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP sawtooth.setFrequency(freq*(1 + LFO.tick()*0.1)); float currentSample = echo.tick(sawtooth.tick()*2 - 1)*0.5;","title":"LFO: Low Frequency Oscillator"},{"location":"lectures/dsp1/#towards-the-dx7","text":"The DX7 carried out frequency modulation over a total of six oscillators that could be patched in different ways . So FM is not limited to two oscillators... Try to implement an FM synthesizer involving 3 oscillators instead of one. They should be connected in series: 3 -> 2 -> 1.","title":"Towards the DX7"},{"location":"lectures/dsp2/","text":"Audio Processing Basics II Harmonic Distortion: Rock On! Distortion is one of the most common electric guitar effect. It consists of over driving a signal by increasing its gain to \"square\" the extremities of its waveform. This results in the creation of lots of harmonics, producing very \"rich\" sounds. Overdrive is easily achievable with an analog electronic circuit and \"sharp edges\" in the waveform are rounded thanks to the tolerance of the electronic components. In the digital world, things are slightly more complicated since clipping will happen resulting in a very dirty sound with potentially lots of aliasing. One way to solve this problem is to use a \"cubic function\" which will round the edges of the signal above a certain amplitude: f(x) = \\begin{cases} \\frac{-2}{3}, \\; \\; x \\leq -1\\\\ x - \\frac{x^3}{3}, \\; \\; -1 < x < 1\\\\ \\frac{2}{3}, \\; \\; x \\geq -1 \\end{cases} Distortion.cpp implements a cubic distortion as: float Distortion::cubic(float x){ return x - x*x*x/3; } float Distortion::tick(float input){ float output = input*pow(10.0,2*drive) + offset; output = fmax(-1,fmin(1,output)); output = cubic(output); return output*gain; } The range of drive is {0;1} which means that the value of input can be multiplied by a number as great as 100 here. offset is a common parameter which just adds a positive or negative DC offset to the signal. If this parameter is used, it is recommended to add a DC blocking filter after the distortion. Distortion is created here by clipping the signal using the fmin and fmax functions. Finally, the cubic polynomial is used to round the edges of the waveform of the signal as explained above. The distortion example program for the Teensy demonstrates the use of Distortion.cpp . Distortion is a very trendy field of research in audio technology these days especially using \"virtual analog\" algorithms which consists of modeling the electronic circuit of distortion on a computer. Echo An echo is a very common audio effect which is used a lot to add some density and depth to a sound. It is based on a feedback loop and a delay and can be expressed as: y(n) = x(n) + g.y(n - M) where g is the feedback between 0 and 1 and M the delay as a number of samples. It can be seen as a simple physical model of what happens in the real world when echo is produced: the delay represents the time it takes for an acoustical wave to go from point A to point B at the speed of sound and g can control the amount of absorption created by the air and the reflecting material. Echo.cpp implements an echo as: float Echo::tick(float input){ float output = input + delBuffer[readIndex]*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } Here, delBuffer is used as a \"ring buffer\": incoming samples are stored and the read and write indices loop around to buffer to write incoming samples and read previous ones. Note that memory is allocated in the constructor of the class for delBuffer based on the value of maxDel , the maximum size of the delay. The echo example program for the Teensy demonstrates the use of Echo.cpp . Comb A comb filter is a filter whose frequency response looks like a \"comb.\" Comb filters can be implemented with feed-forward filters (Finite Impulse Response -- FIR) or feedback filters (Infinite Impulse Response -- IIR). In fact, the Echo algorithm can be used as a comb filter if the delay is very short: y(n) = x(n)-g.y(n-M) where M is the length of the delay and g feedback coefficient. Julius Smith's website presents the frequency response of such filter and the mathematical rationals behind it. From an acoustical standpoint, a feedback comb filter will introduce resonances at specific point in the spectrum of the sound. The position and the spacing of these resonances is determined by the value of M . g , on the other hand, will determine the amplitude and sharpness of these resonances. The comb example program for the Teensy demonstrates the use of Echo.cpp as a comb filter. The \"Mode\" button can be used to change the value of the delay. Physical Modeling: the Simple Case of the Karplus Strong Physical modeling is one of the most advanced sound synthesis technique and a very active field of research. It consists of using physics/mathematical models of musical instruments or vibrating structures to synthesize sound. Various physical modeling techniques are used in the field of audio synthesis: Mass/Interaction (MI), Finite Difference Scheme (FDS), Signal models (e.g., waveguides, modal systems, etc.). While MI and FDS model the vibrational behavior of a system (i.e., using partial differential equation in the case of FDS), signal models model an object as a combination of signal processors. In this section, we will only look at this type of model the other ones being out of the scope of this class. An extremely primitive string model can be implemented using a delay line and a loop. The delay line models the time it takes for vibration in the string to go from one extremity to the other, and the loop models the reflections at the boundaries of the string. In other words, we can literally just reuse the echo algorithm for this. This primitive string model is called the \"Karplus-Strong\" algorithm: Karplus-Strong Algorithm (Source: Wikipedia ) The Karplus-Strong algorithm is typically implemented as: y(n) = x(n) + \\alpha\\frac{y(n-L) + y(n-L-1)}{2} where: x(n) is the input signal (typically an dirac or a noise burst), \\alpha is the feedback coefficient (or dispersion coefficient, in that case), L is the length of the delay and hence, the length of the string. \\frac{y(n-L) + y(n-L-1)}{2} can be seen as a one zero filter implementing a lowpass. It models the fact that high frequencies are absorbed faster than low frequencies at the extremities of a string. The length of the delay L can be controlled as a frequency using the following formula: L = fs/f where f is the desired frequency. At the very least, the system must be excited by a dirac (i.e., a simple impulse going from 1 to 0). The quality of the generated sound can be significantly improved if a noise impulse is used though. KS.cpp implements a basic Karplus-Strong algorithm: float KS::tick(){ float excitation; if(trig){ excitation = 1.0; trig = false; } else{ excitation = 0.0; } float output = excitation + oneZero(delBuffer[readIndex])*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } with: float KS::oneZero(float x){ float output = (x + zeroDel)*0.5; zeroDel = output; return output; } The examples folder of the course repository hosts a simple Teensy program illustrating the use of KS.cpp . Note that this algorithm could be improved in many ways. In particular, the fact that the delay length is currently expressed as an integer can result in frequency mismatches at high frequencies. In other words, our current string is out of tune. This could be fixed using fractional delay . In practice, the Karplus-Strong algorithm is not a physical model per se and is just a simplification of the ideal string wave equation . More advanced signal models can be implemented using waveguides. Waveguide physical modeling has been extensively used in modern synthesizers to synthesize the sound of acoustic instruments. Julius O. Smith (Stanford professor) is the father of waveguide physical modeling. Exercises Smoothing In most cases, DSP parameters are executed at control rate. Moreover, the resolution of the value used to configure parameters is much lower than that of audio samples since it might come from a Graphical User Interface (GUI), a low resolution sensor ADC (e.g., arduino), etc. For all these reasons, changing the value of a DSP parameter will often result in a \"click\"/discontinuity. A common way to prevent this from happening is to interpolate between the values of the parameter using a \"leaky integrator.\" In signal processing, this can be easily implemented using a normalized one pole lowpass filter: y(n) = (1-s)x(n) + sy(n-1) where s is the value of the pole and is typically set to 0.999 for optimal results. Modify the crazy-saw example by \"smoothing\" the value of the frequency parameter by implementing the filter above with s=0.999 . Then slow down the rate at which frequency is being changed so that only two new values are generated per second. The result should sound quite funny :). Solution: Shall be posted after class... Smoothing Potentiometer Values Try to use the smoothing function that you implemented in the previous step to smooth sensor values coming from a potential potentiometer controlling some parameter of one of the Teensy examples. The main idea is to get rid of sound artifacts when making abrupt changes in potentiometers.","title":" Audio Processing Basics II "},{"location":"lectures/dsp2/#audio-processing-basics-ii","text":"","title":"Audio Processing Basics II"},{"location":"lectures/dsp2/#harmonic-distortion-rock-on","text":"Distortion is one of the most common electric guitar effect. It consists of over driving a signal by increasing its gain to \"square\" the extremities of its waveform. This results in the creation of lots of harmonics, producing very \"rich\" sounds. Overdrive is easily achievable with an analog electronic circuit and \"sharp edges\" in the waveform are rounded thanks to the tolerance of the electronic components. In the digital world, things are slightly more complicated since clipping will happen resulting in a very dirty sound with potentially lots of aliasing. One way to solve this problem is to use a \"cubic function\" which will round the edges of the signal above a certain amplitude: f(x) = \\begin{cases} \\frac{-2}{3}, \\; \\; x \\leq -1\\\\ x - \\frac{x^3}{3}, \\; \\; -1 < x < 1\\\\ \\frac{2}{3}, \\; \\; x \\geq -1 \\end{cases} Distortion.cpp implements a cubic distortion as: float Distortion::cubic(float x){ return x - x*x*x/3; } float Distortion::tick(float input){ float output = input*pow(10.0,2*drive) + offset; output = fmax(-1,fmin(1,output)); output = cubic(output); return output*gain; } The range of drive is {0;1} which means that the value of input can be multiplied by a number as great as 100 here. offset is a common parameter which just adds a positive or negative DC offset to the signal. If this parameter is used, it is recommended to add a DC blocking filter after the distortion. Distortion is created here by clipping the signal using the fmin and fmax functions. Finally, the cubic polynomial is used to round the edges of the waveform of the signal as explained above. The distortion example program for the Teensy demonstrates the use of Distortion.cpp . Distortion is a very trendy field of research in audio technology these days especially using \"virtual analog\" algorithms which consists of modeling the electronic circuit of distortion on a computer.","title":"Harmonic Distortion: Rock On!"},{"location":"lectures/dsp2/#echo","text":"An echo is a very common audio effect which is used a lot to add some density and depth to a sound. It is based on a feedback loop and a delay and can be expressed as: y(n) = x(n) + g.y(n - M) where g is the feedback between 0 and 1 and M the delay as a number of samples. It can be seen as a simple physical model of what happens in the real world when echo is produced: the delay represents the time it takes for an acoustical wave to go from point A to point B at the speed of sound and g can control the amount of absorption created by the air and the reflecting material. Echo.cpp implements an echo as: float Echo::tick(float input){ float output = input + delBuffer[readIndex]*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } Here, delBuffer is used as a \"ring buffer\": incoming samples are stored and the read and write indices loop around to buffer to write incoming samples and read previous ones. Note that memory is allocated in the constructor of the class for delBuffer based on the value of maxDel , the maximum size of the delay. The echo example program for the Teensy demonstrates the use of Echo.cpp .","title":"Echo"},{"location":"lectures/dsp2/#comb","text":"A comb filter is a filter whose frequency response looks like a \"comb.\" Comb filters can be implemented with feed-forward filters (Finite Impulse Response -- FIR) or feedback filters (Infinite Impulse Response -- IIR). In fact, the Echo algorithm can be used as a comb filter if the delay is very short: y(n) = x(n)-g.y(n-M) where M is the length of the delay and g feedback coefficient. Julius Smith's website presents the frequency response of such filter and the mathematical rationals behind it. From an acoustical standpoint, a feedback comb filter will introduce resonances at specific point in the spectrum of the sound. The position and the spacing of these resonances is determined by the value of M . g , on the other hand, will determine the amplitude and sharpness of these resonances. The comb example program for the Teensy demonstrates the use of Echo.cpp as a comb filter. The \"Mode\" button can be used to change the value of the delay.","title":"Comb"},{"location":"lectures/dsp2/#physical-modeling-the-simple-case-of-the-karplus-strong","text":"Physical modeling is one of the most advanced sound synthesis technique and a very active field of research. It consists of using physics/mathematical models of musical instruments or vibrating structures to synthesize sound. Various physical modeling techniques are used in the field of audio synthesis: Mass/Interaction (MI), Finite Difference Scheme (FDS), Signal models (e.g., waveguides, modal systems, etc.). While MI and FDS model the vibrational behavior of a system (i.e., using partial differential equation in the case of FDS), signal models model an object as a combination of signal processors. In this section, we will only look at this type of model the other ones being out of the scope of this class. An extremely primitive string model can be implemented using a delay line and a loop. The delay line models the time it takes for vibration in the string to go from one extremity to the other, and the loop models the reflections at the boundaries of the string. In other words, we can literally just reuse the echo algorithm for this. This primitive string model is called the \"Karplus-Strong\" algorithm: Karplus-Strong Algorithm (Source: Wikipedia ) The Karplus-Strong algorithm is typically implemented as: y(n) = x(n) + \\alpha\\frac{y(n-L) + y(n-L-1)}{2} where: x(n) is the input signal (typically an dirac or a noise burst), \\alpha is the feedback coefficient (or dispersion coefficient, in that case), L is the length of the delay and hence, the length of the string. \\frac{y(n-L) + y(n-L-1)}{2} can be seen as a one zero filter implementing a lowpass. It models the fact that high frequencies are absorbed faster than low frequencies at the extremities of a string. The length of the delay L can be controlled as a frequency using the following formula: L = fs/f where f is the desired frequency. At the very least, the system must be excited by a dirac (i.e., a simple impulse going from 1 to 0). The quality of the generated sound can be significantly improved if a noise impulse is used though. KS.cpp implements a basic Karplus-Strong algorithm: float KS::tick(){ float excitation; if(trig){ excitation = 1.0; trig = false; } else{ excitation = 0.0; } float output = excitation + oneZero(delBuffer[readIndex])*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } with: float KS::oneZero(float x){ float output = (x + zeroDel)*0.5; zeroDel = output; return output; } The examples folder of the course repository hosts a simple Teensy program illustrating the use of KS.cpp . Note that this algorithm could be improved in many ways. In particular, the fact that the delay length is currently expressed as an integer can result in frequency mismatches at high frequencies. In other words, our current string is out of tune. This could be fixed using fractional delay . In practice, the Karplus-Strong algorithm is not a physical model per se and is just a simplification of the ideal string wave equation . More advanced signal models can be implemented using waveguides. Waveguide physical modeling has been extensively used in modern synthesizers to synthesize the sound of acoustic instruments. Julius O. Smith (Stanford professor) is the father of waveguide physical modeling.","title":"Physical Modeling: the Simple Case of the Karplus Strong"},{"location":"lectures/dsp2/#exercises","text":"","title":"Exercises"},{"location":"lectures/dsp2/#smoothing","text":"In most cases, DSP parameters are executed at control rate. Moreover, the resolution of the value used to configure parameters is much lower than that of audio samples since it might come from a Graphical User Interface (GUI), a low resolution sensor ADC (e.g., arduino), etc. For all these reasons, changing the value of a DSP parameter will often result in a \"click\"/discontinuity. A common way to prevent this from happening is to interpolate between the values of the parameter using a \"leaky integrator.\" In signal processing, this can be easily implemented using a normalized one pole lowpass filter: y(n) = (1-s)x(n) + sy(n-1) where s is the value of the pole and is typically set to 0.999 for optimal results. Modify the crazy-saw example by \"smoothing\" the value of the frequency parameter by implementing the filter above with s=0.999 . Then slow down the rate at which frequency is being changed so that only two new values are generated per second. The result should sound quite funny :). Solution: Shall be posted after class...","title":"Smoothing"},{"location":"lectures/dsp2/#smoothing-potentiometer-values","text":"Try to use the smoothing function that you implemented in the previous step to smooth sensor values coming from a potential potentiometer controlling some parameter of one of the Teensy examples. The main idea is to get rid of sound artifacts when making abrupt changes in potentiometers.","title":"Smoothing Potentiometer Values"},{"location":"lectures/embedded/","text":"Embedded System Basics That course will explain in more details the principles of embedded programming, peripheral programming, and interrupt handling. At the end of the course is explained how to use simple makefiles to program the teensy, not the arduino IDE. Slides Slides courses are available here INTERRUPTS The principle of interrupt is fundamental in computers and it is the same on every machine. The processor running its program can receive interrupts (i.e. hardware interrupts no to mislead with software interrupts that are implemented by operating systems) at any time. An interrupt can be sent by a peripheral of the micro-controller (timer, radio chip, serial port, etc...), or received from the outside (via GPIOs) like the reset for example. There is another way to communicate with peripherals which is called polling : regularly look a the state of the peripheral to see if something has changed. It is simpler than using interrupts but much more compute-intensive for the CPU. It is the programmer who configures the peripherals (for example the timer) to send an interrupt during certain events. For instance the user can ask the timer to send an interruption every 1.5 second. We will see how to do that with the Teensy. The interrupt \"triggers\" an interrupt handler (or interrupt routine service : ISR), special function called for this particular interrupt. Each interrupt has its own ISR. The correspondence between the interrupt and the ISR is done with the interrupt vector table . This means that each processor knows exactly which interrupt can occur. Each interrupt has a flag bit, which is set by hardware when the interrupt trigger condition occurs. For instance the list of interrupt flags of the Teensy (i.e. of CorteX-M7) is here . The flag's purpose is to remember the interrupt condition has occurred until it has been handled by software. An interrupt is said to be \"pending\" if the trigger condition has set the flag but the interrupt service routine has not been called yet, which can happen if the main program has disabled interrupts or another interrupt service routine is running (usually, interrupts are disable during the execution of an ISR). Usually, interrupt flags are automatically reset when the interrupt service routine is called. Some flags must be reset by the software inside the interrupt service routine. Some flags are completely controlled by the peripheral to reflect internal state (such as UART receive) and can only be changed indirectly by manipulating the peripheral. All interrupts are used in roughly the same way. Configure The Peripheral Reset Interrupt Flag Set Interrupt Mask Enable Global Interrupt, with sei() When the interrupt condition occurs, the interrupt flag is set. The interrupt service routine will be called at the first opportunity. In \"traditionnal\" embedded microcontroler programming, this is done \"by hand\", i.e. all these steps are performed by the programmer. However with the advent of complex micro-controllers such as the ARM CorteX-M, these tasks are simplified with higher level API for peripherals and with the use of interrupt callback. Interrupt Callback Interrupt Callback principle (Image Source: Reusable Firmware Development book] In modern micro-controllers, interrupt handlers are usually in the kernel driver library, i.e. not written by the developper. However, the programmer has to indicate how to react to the interrupt. A way to have a flexible interrupt handler is to use a callback. An interrupt callback is a function, dynamically assigned, that will be called from the ISR . Using function pointer, the programmer can assign any function to the callback. Usually the kernel driver library provides a function that perform this assignement. For instance, the Teensy library propose an class IntervalTimer which uses the timers of the ARM CPU to send regular (i.e. at regular interval) an interrupt. The user can indicate the callback function foo() to be called at each timer interruption occuring, say, every 1.5 second: IntervalTimer myTimer; myTimer.begin(foo, 150000); For that, foo() function must have type: void foo() Exercice: LED blinking Create a new teensy project teensy_led which blinks the LED every 100ms using the delay(int numMilliSeconds) function: You have to know on which GPIO the LED has been soldered. You can look at the schematic of the Teensy 4.0 or a look at file $ARDUINOPATH/hardware/teensy/avr/cos/teensy4/pins_arduino.h and guess which macro can be used for the LED pin (let's call it ledPin ). Then you just have to know that this pin has to be configured in output mode: pinMode(ledPin, OUTPUT) and that switching on and off the led can be done using digitalWrite(ledPin, HIGH); and digitalWrite(ledPin, LOW); Using the built in function delay(int numMilliSeconds) write a program that blink the LED. Solution: Posted after class... Exercice: LED and timer Copy the teensy_led directory to a teensy_timer directory. write a function void toggle_LED() that uses a global variable LEDstate which correspond to the current status of the LED. (In general, it is very common in embedded system to have a local copy of the state of peripheral, just to know in which state we are). Instantiate an IntervalTimer and, as shown above, use toggle_LED() function as timer callback. Have the LED blinking every 0.15s What is the advantage of this approach (i.e. using timers instead of delays) Solution: Posted after class... Exercice: LED, timer and UART Create another project teensy_serial that prints, at each second, on the serial port the number of LED switch occured from the beguinning. Note that you will have to use a global variable shared by the ISR (in function toogle_LED() ) and the main code. It is recommended to disable interrupt when modifyng this variable in the main code, using noInterrupts() and Interrupts() functions. Solution: Posted after class... Exercice: LED, timer and Audio Download the teensy_audio project.. This project plays the crazy-sine sound while blinking the LED. Check that the sound is correct. Add a 10ms delai in the blinkLED callback. What do you notice. It is very important to spend a very short time in ISR, other wise your system can be blocked, miss interrupts or not respect real time constraints. Solution: Posted after class... (Optionnal) Compiling teensy project with Makefile Sometimes it may be interesting to get rid of the arduino environment and provide Makefile that can be easier to tune to various sytems. Here is an example of a teensy program using a Makefile rather than the arduino environment for compilation: - Download the teensy_makefile project here - Untar the archive tar xvf teensy_makefile.tar somewhere (this will create teensy_makefile directory) - Go into the teensy_makefile directory and edit the Makefile to fill up the definition of $ARDUINOPATH (your arduino installation) and $MYDSPPATH (location of the AUD MyDsp library on your computer). - Type make check that the compilation is going well. The object files are compiled in the build directory. The only file that are interesting for you are in the current directory: main.cpp and MyDsp.cpp . The files MyDsp.h and MyDsp.cpp are the same as in the the AUD examples projects. The main.cpp is this one: #include #include \"MyDsp.h\" int main(void) { MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); while (1) { myDsp.setFreq(random(50,1000)); delay(100); } } As one can see, we do not have a setup() and loop() function, but just a main() function with an initialization (which corresponds to the setup() function) and an infinite loop (which correponds to the loop() function). This is allways the case for embedded programming: initialization and infinite loop. Here, we know exactly what executes on the ARM CPU, and it is very explicit that the ARM processor is 100% running the loop doing nothing (i.e. no operating system is present on the ARM), hence sound processing relies on interrupts. A very common programming model for embedded system is to rely only on interrupts. The generic Makefile that you can use for your project is here .","title":" Embedded System Peripherals "},{"location":"lectures/embedded/#embedded-system-basics","text":"That course will explain in more details the principles of embedded programming, peripheral programming, and interrupt handling. At the end of the course is explained how to use simple makefiles to program the teensy, not the arduino IDE.","title":"Embedded System Basics"},{"location":"lectures/embedded/#slides","text":"Slides courses are available here","title":"Slides"},{"location":"lectures/embedded/#interrupts","text":"The principle of interrupt is fundamental in computers and it is the same on every machine. The processor running its program can receive interrupts (i.e. hardware interrupts no to mislead with software interrupts that are implemented by operating systems) at any time. An interrupt can be sent by a peripheral of the micro-controller (timer, radio chip, serial port, etc...), or received from the outside (via GPIOs) like the reset for example. There is another way to communicate with peripherals which is called polling : regularly look a the state of the peripheral to see if something has changed. It is simpler than using interrupts but much more compute-intensive for the CPU. It is the programmer who configures the peripherals (for example the timer) to send an interrupt during certain events. For instance the user can ask the timer to send an interruption every 1.5 second. We will see how to do that with the Teensy. The interrupt \"triggers\" an interrupt handler (or interrupt routine service : ISR), special function called for this particular interrupt. Each interrupt has its own ISR. The correspondence between the interrupt and the ISR is done with the interrupt vector table . This means that each processor knows exactly which interrupt can occur. Each interrupt has a flag bit, which is set by hardware when the interrupt trigger condition occurs. For instance the list of interrupt flags of the Teensy (i.e. of CorteX-M7) is here . The flag's purpose is to remember the interrupt condition has occurred until it has been handled by software. An interrupt is said to be \"pending\" if the trigger condition has set the flag but the interrupt service routine has not been called yet, which can happen if the main program has disabled interrupts or another interrupt service routine is running (usually, interrupts are disable during the execution of an ISR). Usually, interrupt flags are automatically reset when the interrupt service routine is called. Some flags must be reset by the software inside the interrupt service routine. Some flags are completely controlled by the peripheral to reflect internal state (such as UART receive) and can only be changed indirectly by manipulating the peripheral. All interrupts are used in roughly the same way. Configure The Peripheral Reset Interrupt Flag Set Interrupt Mask Enable Global Interrupt, with sei() When the interrupt condition occurs, the interrupt flag is set. The interrupt service routine will be called at the first opportunity. In \"traditionnal\" embedded microcontroler programming, this is done \"by hand\", i.e. all these steps are performed by the programmer. However with the advent of complex micro-controllers such as the ARM CorteX-M, these tasks are simplified with higher level API for peripherals and with the use of interrupt callback.","title":"INTERRUPTS"},{"location":"lectures/embedded/#interrupt-callback","text":"Interrupt Callback principle (Image Source: Reusable Firmware Development book] In modern micro-controllers, interrupt handlers are usually in the kernel driver library, i.e. not written by the developper. However, the programmer has to indicate how to react to the interrupt. A way to have a flexible interrupt handler is to use a callback. An interrupt callback is a function, dynamically assigned, that will be called from the ISR . Using function pointer, the programmer can assign any function to the callback. Usually the kernel driver library provides a function that perform this assignement. For instance, the Teensy library propose an class IntervalTimer which uses the timers of the ARM CPU to send regular (i.e. at regular interval) an interrupt. The user can indicate the callback function foo() to be called at each timer interruption occuring, say, every 1.5 second: IntervalTimer myTimer; myTimer.begin(foo, 150000); For that, foo() function must have type: void foo()","title":"Interrupt Callback"},{"location":"lectures/embedded/#exercice-led-blinking","text":"Create a new teensy project teensy_led which blinks the LED every 100ms using the delay(int numMilliSeconds) function: You have to know on which GPIO the LED has been soldered. You can look at the schematic of the Teensy 4.0 or a look at file $ARDUINOPATH/hardware/teensy/avr/cos/teensy4/pins_arduino.h and guess which macro can be used for the LED pin (let's call it ledPin ). Then you just have to know that this pin has to be configured in output mode: pinMode(ledPin, OUTPUT) and that switching on and off the led can be done using digitalWrite(ledPin, HIGH); and digitalWrite(ledPin, LOW); Using the built in function delay(int numMilliSeconds) write a program that blink the LED. Solution: Posted after class...","title":"Exercice: LED blinking"},{"location":"lectures/embedded/#exercice-led-and-timer","text":"Copy the teensy_led directory to a teensy_timer directory. write a function void toggle_LED() that uses a global variable LEDstate which correspond to the current status of the LED. (In general, it is very common in embedded system to have a local copy of the state of peripheral, just to know in which state we are). Instantiate an IntervalTimer and, as shown above, use toggle_LED() function as timer callback. Have the LED blinking every 0.15s What is the advantage of this approach (i.e. using timers instead of delays) Solution: Posted after class...","title":"Exercice: LED and timer"},{"location":"lectures/embedded/#exercice-led-timer-and-uart","text":"Create another project teensy_serial that prints, at each second, on the serial port the number of LED switch occured from the beguinning. Note that you will have to use a global variable shared by the ISR (in function toogle_LED() ) and the main code. It is recommended to disable interrupt when modifyng this variable in the main code, using noInterrupts() and Interrupts() functions. Solution: Posted after class...","title":"Exercice: LED, timer and UART"},{"location":"lectures/embedded/#exercice-led-timer-and-audio","text":"Download the teensy_audio project.. This project plays the crazy-sine sound while blinking the LED. Check that the sound is correct. Add a 10ms delai in the blinkLED callback. What do you notice. It is very important to spend a very short time in ISR, other wise your system can be blocked, miss interrupts or not respect real time constraints. Solution: Posted after class...","title":"Exercice: LED, timer and Audio"},{"location":"lectures/embedded/#optionnal-compiling-teensy-project-with-makefile","text":"Sometimes it may be interesting to get rid of the arduino environment and provide Makefile that can be easier to tune to various sytems. Here is an example of a teensy program using a Makefile rather than the arduino environment for compilation: - Download the teensy_makefile project here - Untar the archive tar xvf teensy_makefile.tar somewhere (this will create teensy_makefile directory) - Go into the teensy_makefile directory and edit the Makefile to fill up the definition of $ARDUINOPATH (your arduino installation) and $MYDSPPATH (location of the AUD MyDsp library on your computer). - Type make check that the compilation is going well. The object files are compiled in the build directory. The only file that are interesting for you are in the current directory: main.cpp and MyDsp.cpp . The files MyDsp.h and MyDsp.cpp are the same as in the the AUD examples projects. The main.cpp is this one: #include #include \"MyDsp.h\" int main(void) { MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); while (1) { myDsp.setFreq(random(50,1000)); delay(100); } } As one can see, we do not have a setup() and loop() function, but just a main() function with an initialization (which corresponds to the setup() function) and an infinite loop (which correponds to the loop() function). This is allways the case for embedded programming: initialization and infinite loop. Here, we know exactly what executes on the ARM CPU, and it is very explicit that the ARM processor is 100% running the loop doing nothing (i.e. no operating system is present on the ARM), hence sound processing relies on interrupts. A very common programming model for embedded system is to rely only on interrupts. The generic Makefile that you can use for your project is here .","title":"(Optionnal) Compiling teensy project with Makefile"},{"location":"lectures/faust/","text":"Lecture 7: Faust Tutorial (TBD) Faust Web site","title":"Lecture 7: Faust Tutorial (TBD)"},{"location":"lectures/faust/#lecture-7-faust-tutorial-tbd","text":"Faust Web site","title":"Lecture 7: Faust Tutorial (TBD)"},{"location":"lectures/intro/","text":"Course Introduction and Programming Environment Setup This lecture is devoted to installing the software suite used in this course so that everybody can follow the other lectures from Insa or from his home if it needs to be done in distant work. The slides for the first courses can be found here Introduction to AUD2020 and Teensy Teensy 4.0, and the associated audio adaptor board Most of the document on this course come from PJRC . Actually most of the documentation on Teensy is from PJRC . The development in AUD are performed on teensy which is developped by PJRC. It is a microcontroller that offers many I/O pins and a USB interface. It is programmed using a custom/modified version of the arduino programming environment ( teensyduino ). Teensy is a brand of microcontroller development boards designed by the co-owner of PJRC, Paul Stoffregen . The first Teensy 2.0, Teensy++ 2.0 (and discontinued predecessors) use an 8-bit AVR microcontrollers. Teensy 3.0 (and up) have instead Freescale microcontrollers, running ARM Cortex-M CPUs. The technical characteritics of all Teensy can be compared here . In AUD, we use the Teensy 4.0 which contains an ARM Cortex-M7 at 600 MHz with a Floating point unit, hence it can handle non trivial audio treatment. Teensy 4.0 and Audio Shield (from PJRC Website) Teensy 4.0 uses many powerful CPU features useful for true real-time microcontroller platform. The CPU is an ARM Cortex-M7 dual-issue superscaler clocked at at 600 MHz. CPU performance is many times faster than typical 32 bit microcontrollers. The Floating Point Unit performs 32 bit float and 64 bit double precision math in hardware. DSP extension instructions accelerate signal processing, filters and Fourier transform. The Audio library automatically makes uses of these DSP instructions. Teensy performance from Core Benchmarks This pinout reference card comes with the Teensy 4.0 ( do not loose it! ). The pins are not 5V tolerant. Do not drive any digital pin higher than 3.3V. Teensy 4.0 pin map (from PJRC webite) The Teensy 4.0 has a total of 40 input/output signal pins. 24 are easily accessible when used with a solderless breadboard. The available pins include general purpose IO (GPIO, digital or analog, i.e. ADC), as well as integrated serial protocols (I2C, I2S, CAN, SPI and UART protocols) that are used to connect to other devices. In AUD, we use the audio adaptor board provided by PJRC that integrates a low power stereo codex (NXP Semiconductors SGTL5000 codec) and a SD card reader. Teensy audio adaptor (rev. D) from [PRJC](https://www.pjrc.com/store/teensy40.html), The audio codec connects to Teensy using 7 signals (Yellow signal in pin map above) which are used by two protocol: I2C and I2S. This is a traditional configuration for audio codec: the I2C (or I\u00b2C: Inter-Integrated Circuit) protocol is used to configure the codec (sample rate, input and output pins etc.) and the I2S (or I\u00b2S: Inter-IC Sound) is used to transfer samples bit by bit in both direction (i.e. from and to the teensy). The I2C pins SDA and SCL are used to control the chip and adjust parameters. Audio data uses I2S signals, TX (to headphones and/or line out) and RX (from line in or mic), and 3 clocks, LRCLK (44.1 kHz), BCLK (1.41 MHz) and MCLK (11.29 MHz). All 3 clocks are created by Teensy which means that the SGTL5000 operates in \"slave mode\". The schematics of the audio shield board, rev. D, can bee seen here and the schematic of the Teensy 4.0 can be seen at the end of the page here . Of course, as they are both made by PJRC, they are designed to be compatible. We (the teachers!) have soldered the connectors so that the audio shiel can be easily connected to the tennsy. The USB connector of the Teensy can support many serial communication from the host computer to the Teensy: (JTAG for flashing/programming, Serial UART, midi, mouse etc. see Tools -> USB Type menu in arduino IDE). In AUD, the USB connector is used to program the device (i.e., download binary code into flash memory) and ascii communication between the host and the Teensy (i.e. using UART/Serial communicatino protocol). In linux machines, when the teensy USB cable is connected, the serial port will appear as /dev/ttyACM0 Teensy 4.0 Processor: NXP i.MX RT1062 The Teensy uses the i.MX RT1062 processor chip from NXP (a model of the serie i.MX RT1060). The main components of the chip can be seen on the image extracted from the i.MX RT1060 datasheet . The processor used in the chip is an ARM Cortex-M7 ( technical reference manuel of Cortex-M7 here ). The ARM Cortex-M7 is the latest architecture that uses the ISA ARMv7 ( ARM v7 reference manuel ) The i.MX RT1060 used in Teensy 4.0 and the associated perifpherals Teensy 4.0 has 2 Mbyte of flash memory intended for storing your code. 1Mbyte of memory is available for execution (i.e., for variables and data storing during execution). Half of this memory (RAM1) is accessed as tightly coupled memory for maximum performance. The other half (RAM2) is optimized for access by DMA. Normally large arrays & data buffers are placed in RAM2, to save the ultra-fast RAM1 for normal variables. The mapping of variables to memories is indicated at the variables declaration by compiler directive (such as DMAMEM for variable in RAM2 or FASTRUN for variable in RAM1, see here . The memory map is the following: Teensy 4.0 pin map (from PJRC webite) Teensy Development Framework: teensyduino The Teensy can be programmed in many ways: Arduino's IDE software with the Teensyduino add-on is the primary programming environment for Teensy. Visual Micro , PlatformIO Makefiles: type make in directory $(arduino)/hardware/teensy/avr/cores/teensy4/ . in AUD we will use most popular Arduino's IDE with Teensyduino . In general, programming the Teensy amounts to compile an application to an executable ( main.elf usually) and then download the application on the teensy which is connected through its USB interface to your PC. The teensyduino software add the source file to arduino in order to compile code for the teensy and call the teensy_loader that flashes the main.elf in the connected Teensy. Cloning AUD github repository You will need to clone AUD github repository which is at https://github.com/grame-cncm/embaudio . For that use the following command on Linux: git clone https://github.com/grame-cncm/embaudio.git AUD The file will now be available in the AUD directory on your computer, you will need in particulat to install the mydsp library in arduino. This library will be found in AUD/examples/teensy/libraries/mydsp/ Installing Arduino/Teensyduino on Your Computer The toolchain can be installed on Macintosh, Linux or Windows platform, we recommend that you install it on your own machine, however it is also installed on TC machines. The Arduino version to install shoudl be 1.8.19, more recent version may provide probles (on linux at least) The installation procedure is the following: Install arduino from here: https://www.arduino.cc/en/software . Warning, make sure to install one of the supported version : 1.8.5, 1.8.9, 1.8.13, 1.8.15, 1.8.16 or (recommended:) 1.8.19. Download the installer corresponding to your OS (click \"just download\" if you do not want to donate), This installer is just an archive containing all the arduino files. Hence place it in an appropriate directory on your drive, Then unzip or untar it.The directory created ( /home/mylogin/arduino-1.8.19 for instance) will become your $ARDUINOPATH variable. Install teensyduino following these instructions: https://www.pjrc.com/teensy/td_download.html If you are on Linux, make sure to install the udev rules, the udev rules file is available here: https://www.pjrc.com/teensy/00-teensy.rules . execute the command: sudo cp 00-teensy.rules /etc/udev/rules.d Clone embaudio github repository (if not done already): https://github.com/grame-cncm/embaudio You will need some files written specifically for the AUD cours: the examples/teensy/libraries/mydsp directory available in the github cloned . Copy the mydsp directory in the directory $ARDUINOPATH/libraries Getting Started on TC Machines Arduino is installed in directory /opt/arduino-1.8.19 . Launching arduino is done simply by typing the command arduino in a command line shell. However, the mydsp library must be made available to arduino. For that, you have to select a directory for additionnal arduino library, for instance /home/mylogin/Arduino and indicate it to arduino par writing the directory path in file->preference->sketchbook location . Then copy the mydsp directory in the /home/mylogin/Arduino directory. Flashing the LED and Using Serial Terminal For programming the teensy: Connect the teensy on your computer or on the TC machine Launch arduino: > arduino Make sure that the correct board is selected: select Tools -> Boards -> Teensyduino -> Teensy 4.0 Make sure that the serial port is configured correctly: select Tools -> USB Type -> Serial Select the flashing led example: select File -> Examples -> 01.Basic -> Blink . A new arduino editor with the 'Blink' application code. Compile and download the code by clickin in 'Upload' button in ardiuno (big arrow). This should launch the teensy_loader, and upload the code, then the led should be blinking. Select the serial communication example: select File -> Examples -> Teensy -> Serial -> EchoBoth . A new arduino editor with the 'EchoBoth' application code. Compile and download the code. Launch the serial monitor window from arduino (magnifying glass on the upper right). This should launch a new window showing serial communcations. Type some characters in the windown and send them (i.e. type 'return'), what is happening? Try to understand the code of the EchoBoth application. Audio applications on Teensy The application prepared for the AUD course are available in the embaudio git repository ( examples/teensy/projects directory). Open ( File -> open... in arduino) the crazy-sine/crazy-sine.ino project. Download it to teensy and ear the crazy sine.","title":" Course Introduction and Programming Environment Setup "},{"location":"lectures/intro/#course-introduction-and-programming-environment-setup","text":"This lecture is devoted to installing the software suite used in this course so that everybody can follow the other lectures from Insa or from his home if it needs to be done in distant work. The slides for the first courses can be found here","title":"Course Introduction and Programming Environment Setup"},{"location":"lectures/intro/#introduction-to-aud2020-and-teensy","text":"Teensy 4.0, and the associated audio adaptor board Most of the document on this course come from PJRC . Actually most of the documentation on Teensy is from PJRC . The development in AUD are performed on teensy which is developped by PJRC. It is a microcontroller that offers many I/O pins and a USB interface. It is programmed using a custom/modified version of the arduino programming environment ( teensyduino ). Teensy is a brand of microcontroller development boards designed by the co-owner of PJRC, Paul Stoffregen . The first Teensy 2.0, Teensy++ 2.0 (and discontinued predecessors) use an 8-bit AVR microcontrollers. Teensy 3.0 (and up) have instead Freescale microcontrollers, running ARM Cortex-M CPUs. The technical characteritics of all Teensy can be compared here . In AUD, we use the Teensy 4.0 which contains an ARM Cortex-M7 at 600 MHz with a Floating point unit, hence it can handle non trivial audio treatment.","title":"Introduction to AUD2020 and Teensy"},{"location":"lectures/intro/#teensy-40-and-audio-shield-from-pjrc-website","text":"Teensy 4.0 uses many powerful CPU features useful for true real-time microcontroller platform. The CPU is an ARM Cortex-M7 dual-issue superscaler clocked at at 600 MHz. CPU performance is many times faster than typical 32 bit microcontrollers. The Floating Point Unit performs 32 bit float and 64 bit double precision math in hardware. DSP extension instructions accelerate signal processing, filters and Fourier transform. The Audio library automatically makes uses of these DSP instructions. Teensy performance from Core Benchmarks This pinout reference card comes with the Teensy 4.0 ( do not loose it! ). The pins are not 5V tolerant. Do not drive any digital pin higher than 3.3V. Teensy 4.0 pin map (from PJRC webite) The Teensy 4.0 has a total of 40 input/output signal pins. 24 are easily accessible when used with a solderless breadboard. The available pins include general purpose IO (GPIO, digital or analog, i.e. ADC), as well as integrated serial protocols (I2C, I2S, CAN, SPI and UART protocols) that are used to connect to other devices. In AUD, we use the audio adaptor board provided by PJRC that integrates a low power stereo codex (NXP Semiconductors SGTL5000 codec) and a SD card reader. Teensy audio adaptor (rev. D) from [PRJC](https://www.pjrc.com/store/teensy40.html), The audio codec connects to Teensy using 7 signals (Yellow signal in pin map above) which are used by two protocol: I2C and I2S. This is a traditional configuration for audio codec: the I2C (or I\u00b2C: Inter-Integrated Circuit) protocol is used to configure the codec (sample rate, input and output pins etc.) and the I2S (or I\u00b2S: Inter-IC Sound) is used to transfer samples bit by bit in both direction (i.e. from and to the teensy). The I2C pins SDA and SCL are used to control the chip and adjust parameters. Audio data uses I2S signals, TX (to headphones and/or line out) and RX (from line in or mic), and 3 clocks, LRCLK (44.1 kHz), BCLK (1.41 MHz) and MCLK (11.29 MHz). All 3 clocks are created by Teensy which means that the SGTL5000 operates in \"slave mode\". The schematics of the audio shield board, rev. D, can bee seen here and the schematic of the Teensy 4.0 can be seen at the end of the page here . Of course, as they are both made by PJRC, they are designed to be compatible. We (the teachers!) have soldered the connectors so that the audio shiel can be easily connected to the tennsy. The USB connector of the Teensy can support many serial communication from the host computer to the Teensy: (JTAG for flashing/programming, Serial UART, midi, mouse etc. see Tools -> USB Type menu in arduino IDE). In AUD, the USB connector is used to program the device (i.e., download binary code into flash memory) and ascii communication between the host and the Teensy (i.e. using UART/Serial communicatino protocol). In linux machines, when the teensy USB cable is connected, the serial port will appear as /dev/ttyACM0","title":"Teensy 4.0 and Audio Shield (from PJRC Website)"},{"location":"lectures/intro/#teensy-40-processor-nxp-imx-rt1062","text":"The Teensy uses the i.MX RT1062 processor chip from NXP (a model of the serie i.MX RT1060). The main components of the chip can be seen on the image extracted from the i.MX RT1060 datasheet . The processor used in the chip is an ARM Cortex-M7 ( technical reference manuel of Cortex-M7 here ). The ARM Cortex-M7 is the latest architecture that uses the ISA ARMv7 ( ARM v7 reference manuel ) The i.MX RT1060 used in Teensy 4.0 and the associated perifpherals Teensy 4.0 has 2 Mbyte of flash memory intended for storing your code. 1Mbyte of memory is available for execution (i.e., for variables and data storing during execution). Half of this memory (RAM1) is accessed as tightly coupled memory for maximum performance. The other half (RAM2) is optimized for access by DMA. Normally large arrays & data buffers are placed in RAM2, to save the ultra-fast RAM1 for normal variables. The mapping of variables to memories is indicated at the variables declaration by compiler directive (such as DMAMEM for variable in RAM2 or FASTRUN for variable in RAM1, see here . The memory map is the following: Teensy 4.0 pin map (from PJRC webite)","title":"Teensy 4.0 Processor: NXP i.MX RT1062"},{"location":"lectures/intro/#teensy-development-framework-teensyduino","text":"The Teensy can be programmed in many ways: Arduino's IDE software with the Teensyduino add-on is the primary programming environment for Teensy. Visual Micro , PlatformIO Makefiles: type make in directory $(arduino)/hardware/teensy/avr/cores/teensy4/ . in AUD we will use most popular Arduino's IDE with Teensyduino . In general, programming the Teensy amounts to compile an application to an executable ( main.elf usually) and then download the application on the teensy which is connected through its USB interface to your PC. The teensyduino software add the source file to arduino in order to compile code for the teensy and call the teensy_loader that flashes the main.elf in the connected Teensy.","title":"Teensy Development Framework: teensyduino"},{"location":"lectures/intro/#cloning-aud-github-repository","text":"You will need to clone AUD github repository which is at https://github.com/grame-cncm/embaudio . For that use the following command on Linux: git clone https://github.com/grame-cncm/embaudio.git AUD The file will now be available in the AUD directory on your computer, you will need in particulat to install the mydsp library in arduino. This library will be found in AUD/examples/teensy/libraries/mydsp/","title":"Cloning AUD github repository"},{"location":"lectures/intro/#installing-arduinoteensyduino-on-your-computer","text":"The toolchain can be installed on Macintosh, Linux or Windows platform, we recommend that you install it on your own machine, however it is also installed on TC machines. The Arduino version to install shoudl be 1.8.19, more recent version may provide probles (on linux at least) The installation procedure is the following: Install arduino from here: https://www.arduino.cc/en/software . Warning, make sure to install one of the supported version : 1.8.5, 1.8.9, 1.8.13, 1.8.15, 1.8.16 or (recommended:) 1.8.19. Download the installer corresponding to your OS (click \"just download\" if you do not want to donate), This installer is just an archive containing all the arduino files. Hence place it in an appropriate directory on your drive, Then unzip or untar it.The directory created ( /home/mylogin/arduino-1.8.19 for instance) will become your $ARDUINOPATH variable. Install teensyduino following these instructions: https://www.pjrc.com/teensy/td_download.html If you are on Linux, make sure to install the udev rules, the udev rules file is available here: https://www.pjrc.com/teensy/00-teensy.rules . execute the command: sudo cp 00-teensy.rules /etc/udev/rules.d Clone embaudio github repository (if not done already): https://github.com/grame-cncm/embaudio You will need some files written specifically for the AUD cours: the examples/teensy/libraries/mydsp directory available in the github cloned . Copy the mydsp directory in the directory $ARDUINOPATH/libraries","title":"Installing Arduino/Teensyduino on Your Computer"},{"location":"lectures/intro/#getting-started-on-tc-machines","text":"Arduino is installed in directory /opt/arduino-1.8.19 . Launching arduino is done simply by typing the command arduino in a command line shell. However, the mydsp library must be made available to arduino. For that, you have to select a directory for additionnal arduino library, for instance /home/mylogin/Arduino and indicate it to arduino par writing the directory path in file->preference->sketchbook location . Then copy the mydsp directory in the /home/mylogin/Arduino directory.","title":"Getting Started on TC Machines"},{"location":"lectures/intro/#flashing-the-led-and-using-serial-terminal","text":"For programming the teensy: Connect the teensy on your computer or on the TC machine Launch arduino: > arduino Make sure that the correct board is selected: select Tools -> Boards -> Teensyduino -> Teensy 4.0 Make sure that the serial port is configured correctly: select Tools -> USB Type -> Serial Select the flashing led example: select File -> Examples -> 01.Basic -> Blink . A new arduino editor with the 'Blink' application code. Compile and download the code by clickin in 'Upload' button in ardiuno (big arrow). This should launch the teensy_loader, and upload the code, then the led should be blinking. Select the serial communication example: select File -> Examples -> Teensy -> Serial -> EchoBoth . A new arduino editor with the 'EchoBoth' application code. Compile and download the code. Launch the serial monitor window from arduino (magnifying glass on the upper right). This should launch a new window showing serial communcations. Type some characters in the windown and send them (i.e. type 'return'), what is happening? Try to understand the code of the EchoBoth application.","title":"Flashing the LED and Using Serial Terminal"},{"location":"lectures/intro/#audio-applications-on-teensy","text":"The application prepared for the AUD course are available in the embaudio git repository ( examples/teensy/projects directory). Open ( File -> open... in arduino) the crazy-sine/crazy-sine.ino project. Download it to teensy and ear the crazy sine.","title":"Audio applications on Teensy"},{"location":"lectures/os/","text":"Embedded Operating Systems This course will present the important notions of embedded operating systems and show how a primitive operating system (protothreads) can be deployed on teensy. Slides Slides are available here A teensy program with protothreads Download the protothreads_blink.tar archive and uncompress it. Have a look at the protothread_blink.ino program. This programs defines two protothreads: thread_red_led that switch the LED state every seconds, and thread_sine that changes the sine tune every 0.25 seconds. Both these thread are unlock by a dedicated timer. Can you understand how it works? This example, of course, is a bit trivial because the changes in the LED state and the sine tune could have been done in timer callbacks. But, first, we try to avoid writing to much code in the call back and, second, one has to imagine other conditions for unlocking the threads such as internal variable states or input on UART... What happens if one PT_WAIT_UNTIL is commented out? For instance in thread_led_red : //PT_WAIT_UNTIL(pt, TIMER_LED_RED_ON == 1); Note finaly, note that many variables have been set as global variables as local protothread variables are not remanent. Add a UART printing copy this protothreads_blink directory to a new protothreads_count and modify the program to print on UART serial port, every second, the number of blinking of the LED since the beginning of the program. This imply the following tasks (see UART teensy Documentation ): Initialize serial communication at 9600 bauds using Serial.begin(9600); Declare a global variable blink_count Declare a new timer counting each second (up to four interval timers can be used in teensy 4.0) Increment blink_count at each blink Print (using Serial.print ) the value of blink_count in new timer callback. Use the command minicom -D /dev/ttyACM0 to visualize what is send on serial port by the teensy. For the moment we do not really need a new protothread, as we can assume that two calls to Serial.printf are fast enough not to perturbate audio. Imagine that we want to control our audio device using the keyboard. Receiving a caracter from the keyboard and performing the action necessary might be too long for an interrupt handle function. Hence we will add a protothread to do that. Control LED and sound with keyboard copy this protothreads_count directory to a new protothreads_control declare a new protothread `static PT_THREAD(thread_receive(struct pt *pt)) that will receive characters on UART serial port for the host computer. the condition in the PT_WAIT_UNTIL will be Serial.available()>0 (indicating that the receiving buffer contains some characters, see UART teensy Documentation ). As soon as a character is typed, echo its ascii code. Use that mecanism to control the sound and LED: Typing 's' will switch off/on the sound (i.e. setting `myDsp.setFreq(0) ) Typing 'o' will switch off/on the LED blinking. Hint: you can switch the value of a variable var between 0 and 1 using the Xor operator: var = var ^ 1; ** Solution **","title":" Embedded OS, FreeRTOS, Embedded Linux Devices "},{"location":"lectures/os/#embedded-operating-systems","text":"This course will present the important notions of embedded operating systems and show how a primitive operating system (protothreads) can be deployed on teensy.","title":"Embedded Operating Systems"},{"location":"lectures/os/#slides","text":"Slides are available here","title":"Slides"},{"location":"lectures/os/#a-teensy-program-with-protothreads","text":"Download the protothreads_blink.tar archive and uncompress it. Have a look at the protothread_blink.ino program. This programs defines two protothreads: thread_red_led that switch the LED state every seconds, and thread_sine that changes the sine tune every 0.25 seconds. Both these thread are unlock by a dedicated timer. Can you understand how it works? This example, of course, is a bit trivial because the changes in the LED state and the sine tune could have been done in timer callbacks. But, first, we try to avoid writing to much code in the call back and, second, one has to imagine other conditions for unlocking the threads such as internal variable states or input on UART... What happens if one PT_WAIT_UNTIL is commented out? For instance in thread_led_red : //PT_WAIT_UNTIL(pt, TIMER_LED_RED_ON == 1); Note finaly, note that many variables have been set as global variables as local protothread variables are not remanent.","title":"A teensy program with protothreads"},{"location":"lectures/os/#add-a-uart-printing","text":"copy this protothreads_blink directory to a new protothreads_count and modify the program to print on UART serial port, every second, the number of blinking of the LED since the beginning of the program. This imply the following tasks (see UART teensy Documentation ): Initialize serial communication at 9600 bauds using Serial.begin(9600); Declare a global variable blink_count Declare a new timer counting each second (up to four interval timers can be used in teensy 4.0) Increment blink_count at each blink Print (using Serial.print ) the value of blink_count in new timer callback. Use the command minicom -D /dev/ttyACM0 to visualize what is send on serial port by the teensy. For the moment we do not really need a new protothread, as we can assume that two calls to Serial.printf are fast enough not to perturbate audio. Imagine that we want to control our audio device using the keyboard. Receiving a caracter from the keyboard and performing the action necessary might be too long for an interrupt handle function. Hence we will add a protothread to do that.","title":"Add a UART printing"},{"location":"lectures/os/#control-led-and-sound-with-keyboard","text":"copy this protothreads_count directory to a new protothreads_control declare a new protothread `static PT_THREAD(thread_receive(struct pt *pt)) that will receive characters on UART serial port for the host computer. the condition in the PT_WAIT_UNTIL will be Serial.available()>0 (indicating that the receiving buffer contains some characters, see UART teensy Documentation ). As soon as a character is typed, echo its ascii code. Use that mecanism to control the sound and LED: Typing 's' will switch off/on the sound (i.e. setting `myDsp.setFreq(0) ) Typing 'o' will switch off/on the LED blinking. Hint: you can switch the value of a variable var between 0 and 1 using the Xor operator: var = var ^ 1; ** Solution **","title":"Control LED and sound with keyboard"},{"location":"lectures/project/","text":"Personnal Project: Your Own LyraT Program The remaining course (approximately 3 TD of 2h) will be dedicated to a personal project that you will implement on LyraT, the idea it to explore further one aspect of the LyraT programming and to present it to the other students in a small d\u00e9mo during the last course. Possible Projects You are encouraged to proposed your own project, here are some idea of topics that can be studied. Note that the project will be done in a very short time , make sure to be able to present something, even if it is juste a study for a future implementation Explore Digital Synthesis or effect on the web (e.g., https://en.wikipedia.org/wiki/Category:Sound_synthesis_types , https://www.dafx.de/ , https://en.wikipedia.org/wiki/Sound_effect#techniques , https://ccrma.stanford.edu/~jos/ , etc.), and implement a particular algorithm Provide an OSC control via udp connection with Wifi Provide a BlueTooth/midi controler such as the one implemented here ( https://github.com/midibox/esp32-idf-blemidi/tree/master/components/blemidi ) Provide a USB/midi controler with an external midi/uSB device Think of a funny application using wireless communications between several LyraT Provide an HTTP server that can be used via wifi to control the LyraT program Explore (as above) digital synthesis of effect but in Faust to LyraT Rules: One or two students per project project should be presented to the class \"as it is\" on last course in less than 10mn (e.g. three slides and a demo)","title":"Personnal Project: Your Own LyraT Program"},{"location":"lectures/project/#personnal-project-your-own-lyrat-program","text":"The remaining course (approximately 3 TD of 2h) will be dedicated to a personal project that you will implement on LyraT, the idea it to explore further one aspect of the LyraT programming and to present it to the other students in a small d\u00e9mo during the last course.","title":"Personnal Project: Your Own LyraT Program"},{"location":"lectures/project/#possible-projects","text":"You are encouraged to proposed your own project, here are some idea of topics that can be studied. Note that the project will be done in a very short time , make sure to be able to present something, even if it is juste a study for a future implementation Explore Digital Synthesis or effect on the web (e.g., https://en.wikipedia.org/wiki/Category:Sound_synthesis_types , https://www.dafx.de/ , https://en.wikipedia.org/wiki/Sound_effect#techniques , https://ccrma.stanford.edu/~jos/ , etc.), and implement a particular algorithm Provide an OSC control via udp connection with Wifi Provide a BlueTooth/midi controler such as the one implemented here ( https://github.com/midibox/esp32-idf-blemidi/tree/master/components/blemidi ) Provide a USB/midi controler with an external midi/uSB device Think of a funny application using wireless communications between several LyraT Provide an HTTP server that can be used via wifi to control the LyraT program Explore (as above) digital synthesis of effect but in Faust to LyraT","title":"Possible Projects"},{"location":"lectures/project/#rules","text":"One or two students per project project should be presented to the class \"as it is\" on last course in less than 10mn (e.g. three slides and a demo)","title":"Rules:"},{"location":"lectures/teensy-faust/","text":"Faust on the Teensy and Advanced Control Generating and Using a Faust C++ Object In order to run the examples in this lecture, you should install the Faust distribution on your system from the Faust Git Repository . At the most fundamental level, the Faust compiler is a command line tool translating a Faust DSP object into C++ code. For example, assuming that Faust is properly installed on your system, given the following simple Faust program implementing a filtered sawtooth wave oscillator ( FaustSynth.dsp ): import(\"stdfaust.lib\"); freq = nentry(\"freq\",200,50,1000,0.01); gain = nentry(\"gain\",0.5,0,1,0.01) : si.smoo; gate = button(\"gate\") : si.smoo; cutoff = nentry(\"cutoff\",10000,50,10000,0.01) : si.smoo; process = os.sawtooth(freq)*gain*gate : fi.lowpass(3,cutoff) <: _,_; running: faust FaustSynth.dsp will output the C++ code corresponding to this file in the terminal. Faust comes with a system of C++ wrapper (called architectures in the Faust ecosystem) which can be used to customize the generated C++ code. faustMininal.h is a minimal architecture file including some C++ objects that can be used to facilitate interactions with the generated DSP: #include #include #include \"faust/gui/MapUI.h\" #include \"faust/gui/meta.h\" #include \"faust/dsp/dsp.h\" // BEGIN-FAUSTDSP <> <> // END-FAUSTDSP For instance, MapUI allows us to access the parameters of a Faust DSP object using the setParamValue method, etc. To generate a C++ file using this architecture, you can run: faust -i -a faustMinial.h FaustSynth.dsp -o FaustSynth.h which will produce a FaustSynth.h file (feel free to click on it). The -i inlines all the included C++ .h files in the generated file. The faust-synth Teensy example project demonstrates how FaustSynth.h can be used. First, it is included in MyDsp.cpp and the following elements are declared in the corresponding header file : private: MapUI* fUI; dsp* fDSP; float **outputs; dsp is the actual Faust DSP, MapUI will be used to interact with it, and outputs is the multidimensional output buffer. These objects are then allocated in the constructor of MyDsp.cpp : fDSP = new mydsp(); fDSP->init(AUDIO_SAMPLE_RATE_EXACT); fUI = new MapUI(); fDSP->buildUserInterface(fUI); outputs = new float*[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; ++channel){ outputs[channel] = new float[AUDIO_BLOCK_SAMPLES]; } buildUserInterface is used to connect fUI to fDSP and then memory is allocated for the output buffer. Note that memory should be de-allocated in the destructor after this. In the update method, we just call the compute method of fDSP and then reformat the generated samples to transmit them via i2s: void MyDsp::update(void) { fDSP->compute(AUDIO_BLOCK_SAMPLES,NULL,outputs); audio_block_t* outBlock[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; channel++) { outBlock[channel] = allocate(); if (outBlock[channel]) { for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { int16_t val = outputs[channel][i]*MULT_16; outBlock[channel]->data[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } Note that outputs is used as an intermediate here and the first dimension of the array is the channel number and the second dimension the samples themselves. The various parameters of the Faust object can then be changed just by calling the setParamValue method. The first argument of the method corresponds to the name of the parameter as specified in the Faust program: void MyDsp::setFreq(float freq){ fUI->setParamValue(\"freq\",freq); } void MyDsp::setCutoff(float freq){ fUI->setParamValue(\"cutoff\",freq); } void MyDsp::setGate(int gate){ fUI->setParamValue(\"gate\",gate); } Better Control on the Teensy In addition to controlling DSP parameters on the Teensy using external sensors connected to the board's GPIOs, other techniques can potentially be used. We briefly summarize this section. MIDI MIDI is THE standard in the world of music to control digital devices. It has been around since 1983 and even though it is very \"low tech,\" it is still heavily used. While MIDI was traditionally transmitted over MIDI ports, USB is used nowadays to send MIDI. USB MIDI is natively supported on the Teensy through the Teensy USB MIDI library: https://www.pjrc.com/teensy/td_midi.html . Interfacing this library with your DSP programs should be very straightforward. Please, also note that Teensys can be used to send MIDI messages over USB which means that implementing your own midi controller using a Teensy is fairly straightforward as well. If you're curious about this, you can check this page: https://ccrma.stanford.edu/courses/250a-winter-2018/labs/2/ . OSC Open Sound Control (OSC) is a more modern communication standard used in the field of music technology. It is based on UDP which means that information can be transmitted via Ethernet or Wi-Fi. OSC uses a system of address/values to access the different parameters of a system. An OSC message can therefore look like: /synth/freq 440 On the Teensy, dealing with OSC is a bit more tricky than MIDI because the Teensy 4.0 provided as part of your class kit don't have a built-in Ethernet port. Hence, the only way to get an Ethernet connection to the Teensy is to buy an external Ethernet adapter (that will likely connect to the Teensy through i2c, etc.). Another option is to buy a Teensy 4.1 which hosts an Ethernet chip (an Ethernet connector can just be soldered to the board). Exercises Faust Triangle Oscillator The Faust libraries host a triangle wave oscillator: os.triangle Try to replace the sawtooth wave oscillator from the previous example by a triangle wave oscillator in Faust and run it on the Teensy. Flanger The Faust libraries host a flanger function : pf.flanger_mono Turn your Teensy into a flanger effect processor!","title":" Faust on the Teensy and Advanced Control "},{"location":"lectures/teensy-faust/#faust-on-the-teensy-and-advanced-control","text":"","title":"Faust on the Teensy and Advanced Control"},{"location":"lectures/teensy-faust/#generating-and-using-a-faust-c-object","text":"In order to run the examples in this lecture, you should install the Faust distribution on your system from the Faust Git Repository . At the most fundamental level, the Faust compiler is a command line tool translating a Faust DSP object into C++ code. For example, assuming that Faust is properly installed on your system, given the following simple Faust program implementing a filtered sawtooth wave oscillator ( FaustSynth.dsp ): import(\"stdfaust.lib\"); freq = nentry(\"freq\",200,50,1000,0.01); gain = nentry(\"gain\",0.5,0,1,0.01) : si.smoo; gate = button(\"gate\") : si.smoo; cutoff = nentry(\"cutoff\",10000,50,10000,0.01) : si.smoo; process = os.sawtooth(freq)*gain*gate : fi.lowpass(3,cutoff) <: _,_; running: faust FaustSynth.dsp will output the C++ code corresponding to this file in the terminal. Faust comes with a system of C++ wrapper (called architectures in the Faust ecosystem) which can be used to customize the generated C++ code. faustMininal.h is a minimal architecture file including some C++ objects that can be used to facilitate interactions with the generated DSP: #include #include #include \"faust/gui/MapUI.h\" #include \"faust/gui/meta.h\" #include \"faust/dsp/dsp.h\" // BEGIN-FAUSTDSP <> <> // END-FAUSTDSP For instance, MapUI allows us to access the parameters of a Faust DSP object using the setParamValue method, etc. To generate a C++ file using this architecture, you can run: faust -i -a faustMinial.h FaustSynth.dsp -o FaustSynth.h which will produce a FaustSynth.h file (feel free to click on it). The -i inlines all the included C++ .h files in the generated file. The faust-synth Teensy example project demonstrates how FaustSynth.h can be used. First, it is included in MyDsp.cpp and the following elements are declared in the corresponding header file : private: MapUI* fUI; dsp* fDSP; float **outputs; dsp is the actual Faust DSP, MapUI will be used to interact with it, and outputs is the multidimensional output buffer. These objects are then allocated in the constructor of MyDsp.cpp : fDSP = new mydsp(); fDSP->init(AUDIO_SAMPLE_RATE_EXACT); fUI = new MapUI(); fDSP->buildUserInterface(fUI); outputs = new float*[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; ++channel){ outputs[channel] = new float[AUDIO_BLOCK_SAMPLES]; } buildUserInterface is used to connect fUI to fDSP and then memory is allocated for the output buffer. Note that memory should be de-allocated in the destructor after this. In the update method, we just call the compute method of fDSP and then reformat the generated samples to transmit them via i2s: void MyDsp::update(void) { fDSP->compute(AUDIO_BLOCK_SAMPLES,NULL,outputs); audio_block_t* outBlock[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; channel++) { outBlock[channel] = allocate(); if (outBlock[channel]) { for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { int16_t val = outputs[channel][i]*MULT_16; outBlock[channel]->data[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } Note that outputs is used as an intermediate here and the first dimension of the array is the channel number and the second dimension the samples themselves. The various parameters of the Faust object can then be changed just by calling the setParamValue method. The first argument of the method corresponds to the name of the parameter as specified in the Faust program: void MyDsp::setFreq(float freq){ fUI->setParamValue(\"freq\",freq); } void MyDsp::setCutoff(float freq){ fUI->setParamValue(\"cutoff\",freq); } void MyDsp::setGate(int gate){ fUI->setParamValue(\"gate\",gate); }","title":"Generating and Using a Faust C++ Object"},{"location":"lectures/teensy-faust/#better-control-on-the-teensy","text":"In addition to controlling DSP parameters on the Teensy using external sensors connected to the board's GPIOs, other techniques can potentially be used. We briefly summarize this section.","title":"Better Control on the Teensy"},{"location":"lectures/teensy-faust/#midi","text":"MIDI is THE standard in the world of music to control digital devices. It has been around since 1983 and even though it is very \"low tech,\" it is still heavily used. While MIDI was traditionally transmitted over MIDI ports, USB is used nowadays to send MIDI. USB MIDI is natively supported on the Teensy through the Teensy USB MIDI library: https://www.pjrc.com/teensy/td_midi.html . Interfacing this library with your DSP programs should be very straightforward. Please, also note that Teensys can be used to send MIDI messages over USB which means that implementing your own midi controller using a Teensy is fairly straightforward as well. If you're curious about this, you can check this page: https://ccrma.stanford.edu/courses/250a-winter-2018/labs/2/ .","title":"MIDI"},{"location":"lectures/teensy-faust/#osc","text":"Open Sound Control (OSC) is a more modern communication standard used in the field of music technology. It is based on UDP which means that information can be transmitted via Ethernet or Wi-Fi. OSC uses a system of address/values to access the different parameters of a system. An OSC message can therefore look like: /synth/freq 440 On the Teensy, dealing with OSC is a bit more tricky than MIDI because the Teensy 4.0 provided as part of your class kit don't have a built-in Ethernet port. Hence, the only way to get an Ethernet connection to the Teensy is to buy an external Ethernet adapter (that will likely connect to the Teensy through i2c, etc.). Another option is to buy a Teensy 4.1 which hosts an Ethernet chip (an Ethernet connector can just be soldered to the board).","title":"OSC"},{"location":"lectures/teensy-faust/#exercises","text":"","title":"Exercises"},{"location":"lectures/teensy-faust/#faust-triangle-oscillator","text":"The Faust libraries host a triangle wave oscillator: os.triangle Try to replace the sawtooth wave oscillator from the previous example by a triangle wave oscillator in Faust and run it on the Teensy.","title":"Faust Triangle Oscillator"},{"location":"lectures/teensy-faust/#flanger","text":"The Faust libraries host a flanger function : pf.flanger_mono Turn your Teensy into a flanger effect processor!","title":"Flanger"}]} \ No newline at end of file +{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Embedded Audio Signal Processing This course is a collaboration between Insa-Lyon (TC-Dept, Citi Lab ), INRIA , and GRAME-CNCM . The objective is to foster the development of emerging embedded audio devices and to take advantage of the resources of the CITI/INRIA Emeraude Team (Embedded Programmable Audio Systems) in this domain. In this course, students will learn about: Low-level embedded systems for real-time audio signal processing Digital audio system architecture Audio codec configuration IC communication protocols Audio signal processing Audio sound synthesis and effects design The Faust programming language Course git repository: https://github.com/grame-cncm/embaudio Instructors Romain Michon (INRIA) Tanguy Risset (INSA Lyon) Yann Orlarey (GRAME-CNCM) Organization and ECTS The course will consists of 32 hours (2 ECTS) divided into 16h TD (or CM, this is equivalent) and 16h TP (two instructors): 2x2h CM 6x2h CM/TD 8x2h TP Evaluation on TPs Course Overview 19/09/2023 -- 14h00-16h00: Course Introduction and Programming Environment Setup 26/09/2023 -- 14h00-16h00: Audio Signal Processing Fundamentals 03/10/2023 -- 14h00-16h00: Digital Audio Systems Architectures and Audio Callback 09/10/2023 -- 14h00-16h00: Hardware Control and Audio Codec Configuration 10/10/2023 -- 14h00-16h00: Audio Processing Basics I 06/11/2023 -- 08h00-10h00: Embedded System Peripherals 06/11/2023 -- 10h00-12h00: Embedded OS, FreeRTOS, Embedded Linux Devices 07/11/2023 -- 14h00-16h00: Audio Processing Basics II 08/11/2023 -- 10h00-12h00: Faust Tutotial 14/11/2023 -- 14h00-16h00: Embedded systems at Rtone 22/11/2023 -- 10h00-12h00: Faust on the Teensy and Advanced Control Sessions 12-16: Mini-project 29/11/2023 10h00-12h00 06/12/2023 10h00-12h00 12/12/2023 08h00-10h00 12/12/2023 10h00-12h00 19/12/2023 14h00-16h00","title":"Syllabus"},{"location":"#embedded-audio-signal-processing","text":"This course is a collaboration between Insa-Lyon (TC-Dept, Citi Lab ), INRIA , and GRAME-CNCM . The objective is to foster the development of emerging embedded audio devices and to take advantage of the resources of the CITI/INRIA Emeraude Team (Embedded Programmable Audio Systems) in this domain. In this course, students will learn about: Low-level embedded systems for real-time audio signal processing Digital audio system architecture Audio codec configuration IC communication protocols Audio signal processing Audio sound synthesis and effects design The Faust programming language Course git repository: https://github.com/grame-cncm/embaudio","title":"Embedded Audio Signal Processing"},{"location":"#instructors","text":"Romain Michon (INRIA) Tanguy Risset (INSA Lyon) Yann Orlarey (GRAME-CNCM)","title":"Instructors"},{"location":"#organization-and-ects","text":"The course will consists of 32 hours (2 ECTS) divided into 16h TD (or CM, this is equivalent) and 16h TP (two instructors): 2x2h CM 6x2h CM/TD 8x2h TP Evaluation on TPs","title":"Organization and ECTS"},{"location":"#course-overview","text":"19/09/2023 -- 14h00-16h00: Course Introduction and Programming Environment Setup 26/09/2023 -- 14h00-16h00: Audio Signal Processing Fundamentals 03/10/2023 -- 14h00-16h00: Digital Audio Systems Architectures and Audio Callback 09/10/2023 -- 14h00-16h00: Hardware Control and Audio Codec Configuration 10/10/2023 -- 14h00-16h00: Audio Processing Basics I 06/11/2023 -- 08h00-10h00: Embedded System Peripherals 06/11/2023 -- 10h00-12h00: Embedded OS, FreeRTOS, Embedded Linux Devices 07/11/2023 -- 14h00-16h00: Audio Processing Basics II 08/11/2023 -- 10h00-12h00: Faust Tutotial 14/11/2023 -- 14h00-16h00: Embedded systems at Rtone 22/11/2023 -- 10h00-12h00: Faust on the Teensy and Advanced Control Sessions 12-16: Mini-project 29/11/2023 10h00-12h00 06/12/2023 10h00-12h00 12/12/2023 08h00-10h00 12/12/2023 10h00-12h00 19/12/2023 14h00-16h00","title":"Course Overview"},{"location":"lectures/architecture/","text":"Digital Audio Systems Architectures and Audio Callback By the end of this lecture, you should be able to produce sound with your Teensy and have a basic understanding of the software and hardware architecture of embedded audio systems. Basic Architecture of a Digital Audio System All digital audio systems have an architecture involving at least an ADC and/or a DAC. Audio samples are processed on a computer (i.e., CPU, microcontroller, DSP, etc.) typically in an audio callback and are transmitted to the DAC and/or received from the ADC: The format of audio samples depends on the hardware configuration of the system. Architecture of Embedded Audio Systems Such as the Teensy In embedded audio systems, the component implementing the audio ADC and DAC is called an \"Audio Codec.\" This name is slightly ambiguous because it is also used in the context of audio compression (e.g., mp3) to designate a totally different concept. In the case of the Teensy kits that are provided to you as part of this class, the audio codec we use is an SGTL5000. It is mounted on a shield/sister board that has the same form factor as the Teensy. Audio samples are sent and received between the Cortex M7 and the audio codec using the i2s protocol (additional information on how this kind of system works is provided in this lecture ). As a microcontroller, the Cortex M7 has its own analog inputs which can be used to retrieve sensor datas (e.g., potentiometers, etc.). These analog inputs cannot be used for audio because of their limited precision and sampling rate. We'll briefly show in this lecture how these analog inputs can be used to use sensors to control audio algorithms running on the Teensy. Teensy and Audio Shield Overview Concept of Audio Blocks (Buffers), Audio Rate, and Control Rate A large number of audio samples must be processed and transmitted every second. For example, if the sampling rate of the system is 48 kHz, 48000 samples will be processed in one second. Digital audio is extremely demanding and if one sample is missed, the result on the produced sound will be very audible. Most processors cannot process and transmit samples one by one which is why buffers need to be used. Hence, most digital audio systems will process audio as \"blocks.\" The smallest size of a block will be determined by the performance of the system. On a modern computer running an operating system such as Windows, MacOS or Linux, the standard block size is usually 256 samples. In that case, the audio callback will process and then transmit to the DAC 256 samples all at once. An audio callback function typically takes the following form: void audioCallback(float *inputs, float *outputs){ // control rate portion int gain = mainVolume; for(int i=0; idata[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } The update method is called every time a new audio buffer is needed by the system. A new audio buffer audioBlock containing AUDIO_OUTPUTS channels is first created. For every audio channel, memory is allocated and a full block of samples is computed. Individual samples resulting from computing a sine wave through an echo ( echo and sine are defined in the lib folder and implement an echo and a sine wave oscillator, respectively) are stored in currentSample . currentSample is a floating point number whose range is {-1;1}. This is a standard in the world of digital audio, hence, a signal actually ranging between {-1;1} will correspond to the \"loudest\" sound that can be played on a given system. max(-1,min(1,currentSample)); ensures that currentSample doesn't exceed this range. AUDIO_BLOCK_SAMPLES corresponds to the block size (256 samples by default on the Teensy, but this value can potentially be adjusted). The values contained in currentSample (between -1 and 1) must be converted to 16 bits signed integers (to ensure compatibility with the rest of the Teensy audio library). For that, we just have to multiply currentSample by 2^{16-1} (if we were looking at unsigned integers, which can happen on some system, we would multiply currentSample by 2^{16} ). Note that currentSample is multiplied by 0.5 to control the output gain of the system here (we'll see later in this class that echos tend to add energy to the system hence we must limit the gain of the output signal to prevent potential saturation). Once a full block has been computed, it is transmitted to the rest of the system using the transmit function. Once this is done, the memory that was allocated for the audio block is freed using the release function. The update method is called over and over until the Teensy is powered out. C++ Sine Wave Oscillator Sine wave are at the basis of many algorithms in the field of audio. The sound of a sine wave is what we call a \"pure tone\" since it only has a single harmonic. One of the consequences of this is that all sounds can be synthesized using a combination of sine waves ( Fourier transform ). From a mathematical standpoint, a sine oscillator can be implemented with the following differential equation: x(t) = Asin(\\omega t + \\phi) with: A : the peak amplitude \\omega = 2 \\pi f : the radian frequency (rad/sec) f : the frequency in Hz t : the time seconds \\phi : the initial phase (radians) x(t) could be translated to C++ by writing something like ( \\phi is ignored here): float currentSample = A*std::sin(2*PI*f*t); however sine oscillators are rarely implemented as such since calling the std::sin function at every sample can be quite computationally expensive. For that reason, it is better to pre-compute the sine wave and store it in a wave table before computation starts. That kind of algorithm is then called a \"wave table oscillator.\" Sine.cpp , which is used in crazy-sine is a good example of that. It uses SineTable.cpp which pre-computes a sine table: table = new float[size]; for(int i=0; i #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; delay(500); } Basic Additive Synthesis One of the most basic kind of sound synthesis is \"additive synthesis.\" In consists of adding multiple sine wave oscillators together to \"sculpt\" the timbre of a sound. Both the frequency and the gain of each individual oscillator can then be used to change the properties of the synthesized sound. A simple additive synthesizer could be implemented from the crazy-sine example by declaring multiple instances of sine . E.g.: float currentSample = echo.tick(sine0.tick()*gain0 + sine1.tick()*gain1); but the problem with that option is that memory will be allocated twice for the sineTable array which is a terrible idea in the context of our embedded audio system with very little memory. Instead, the additive synthesizer should reuse the same instance of sineTable for each oscillator. In the tick method of Sine.cpp , try to call the sineTable a second time after float currentSample = sineTable[index]*gain; to add a second oscillator to the generated sample. The value of its index could be something like index = (int) (index*1.5)%SINE_TABLE_SIZE; so that the frequency of the second oscillator is one fifth above the main frequency. In other words, the differential equation of the synth should be: x(t) = sin(2 \\pi f t) + sin(2 \\pi (1.5f) t) Hint: Beware of clipping! Adding two sine waves together even though they don't have the same frequency will likely produce a signal whose range exceeds {-1;1}: you should take that into account for your final product. Solution: In Sine.cpp : float Sine::tick(){ int index = phasor.tick()*SINE_TABLE_SIZE; int index2 = (int) (index*1.5)%SINE_TABLE_SIZE; return (sineTable.tick(index)+sineTable.tick(index2))*gain*0.5; } Stereo Echo Reusing the result of the previous exercise, create a second instance of echo (connected to the same instance of sine ) with different parameters from the first one that will be connected to the second channel of the output (i.e., the first instance should be connected to the left channel and the second one to the right channel). The final algorithm should look like this: float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; Hint: Beware of memory allocation again! Make sure that the maxim delay of your echo (on the 2 parameters of the class constructor) doesn't exceed 10000 for now for both instances of the echo. Solution: In MyDsp.h : Sine sine; Echo echo0, echo1; }; In MyDsp.cpp : MyDsp::MyDsp() : AudioStream(AUDIO_OUTPUTS, new audio_block_t*[AUDIO_OUTPUTS]), sine(AUDIO_SAMPLE_RATE_EXACT), echo0(AUDIO_SAMPLE_RATE_EXACT,10000), echo1(AUDIO_SAMPLE_RATE_EXACT,7000) { ... // setting up DSP objects echo0.setDel(10000); echo0.setFeedback(0.5); echo1.setDel(7000); echo1.setFeedback(0.4); ... // processing buffers for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; ... }","title":" Digital Audio Systems Architectures and Audio Callback "},{"location":"lectures/architecture/#digital-audio-systems-architectures-and-audio-callback","text":"By the end of this lecture, you should be able to produce sound with your Teensy and have a basic understanding of the software and hardware architecture of embedded audio systems.","title":"Digital Audio Systems Architectures and Audio Callback"},{"location":"lectures/architecture/#basic-architecture-of-a-digital-audio-system","text":"All digital audio systems have an architecture involving at least an ADC and/or a DAC. Audio samples are processed on a computer (i.e., CPU, microcontroller, DSP, etc.) typically in an audio callback and are transmitted to the DAC and/or received from the ADC: The format of audio samples depends on the hardware configuration of the system.","title":"Basic Architecture of a Digital Audio System"},{"location":"lectures/architecture/#architecture-of-embedded-audio-systems-such-as-the-teensy","text":"In embedded audio systems, the component implementing the audio ADC and DAC is called an \"Audio Codec.\" This name is slightly ambiguous because it is also used in the context of audio compression (e.g., mp3) to designate a totally different concept. In the case of the Teensy kits that are provided to you as part of this class, the audio codec we use is an SGTL5000. It is mounted on a shield/sister board that has the same form factor as the Teensy. Audio samples are sent and received between the Cortex M7 and the audio codec using the i2s protocol (additional information on how this kind of system works is provided in this lecture ). As a microcontroller, the Cortex M7 has its own analog inputs which can be used to retrieve sensor datas (e.g., potentiometers, etc.). These analog inputs cannot be used for audio because of their limited precision and sampling rate. We'll briefly show in this lecture how these analog inputs can be used to use sensors to control audio algorithms running on the Teensy. Teensy and Audio Shield Overview","title":"Architecture of Embedded Audio Systems Such as the Teensy"},{"location":"lectures/architecture/#concept-of-audio-blocks-buffers-audio-rate-and-control-rate","text":"A large number of audio samples must be processed and transmitted every second. For example, if the sampling rate of the system is 48 kHz, 48000 samples will be processed in one second. Digital audio is extremely demanding and if one sample is missed, the result on the produced sound will be very audible. Most processors cannot process and transmit samples one by one which is why buffers need to be used. Hence, most digital audio systems will process audio as \"blocks.\" The smallest size of a block will be determined by the performance of the system. On a modern computer running an operating system such as Windows, MacOS or Linux, the standard block size is usually 256 samples. In that case, the audio callback will process and then transmit to the DAC 256 samples all at once. An audio callback function typically takes the following form: void audioCallback(float *inputs, float *outputs){ // control rate portion int gain = mainVolume; for(int i=0; idata[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } The update method is called every time a new audio buffer is needed by the system. A new audio buffer audioBlock containing AUDIO_OUTPUTS channels is first created. For every audio channel, memory is allocated and a full block of samples is computed. Individual samples resulting from computing a sine wave through an echo ( echo and sine are defined in the lib folder and implement an echo and a sine wave oscillator, respectively) are stored in currentSample . currentSample is a floating point number whose range is {-1;1}. This is a standard in the world of digital audio, hence, a signal actually ranging between {-1;1} will correspond to the \"loudest\" sound that can be played on a given system. max(-1,min(1,currentSample)); ensures that currentSample doesn't exceed this range. AUDIO_BLOCK_SAMPLES corresponds to the block size (256 samples by default on the Teensy, but this value can potentially be adjusted). The values contained in currentSample (between -1 and 1) must be converted to 16 bits signed integers (to ensure compatibility with the rest of the Teensy audio library). For that, we just have to multiply currentSample by 2^{16-1} (if we were looking at unsigned integers, which can happen on some system, we would multiply currentSample by 2^{16} ). Note that currentSample is multiplied by 0.5 to control the output gain of the system here (we'll see later in this class that echos tend to add energy to the system hence we must limit the gain of the output signal to prevent potential saturation). Once a full block has been computed, it is transmitted to the rest of the system using the transmit function. Once this is done, the memory that was allocated for the audio block is freed using the release function. The update method is called over and over until the Teensy is powered out.","title":"First Audio Program on the Teensy: crazy-sine"},{"location":"lectures/architecture/#c-sine-wave-oscillator","text":"Sine wave are at the basis of many algorithms in the field of audio. The sound of a sine wave is what we call a \"pure tone\" since it only has a single harmonic. One of the consequences of this is that all sounds can be synthesized using a combination of sine waves ( Fourier transform ). From a mathematical standpoint, a sine oscillator can be implemented with the following differential equation: x(t) = Asin(\\omega t + \\phi) with: A : the peak amplitude \\omega = 2 \\pi f : the radian frequency (rad/sec) f : the frequency in Hz t : the time seconds \\phi : the initial phase (radians) x(t) could be translated to C++ by writing something like ( \\phi is ignored here): float currentSample = A*std::sin(2*PI*f*t); however sine oscillators are rarely implemented as such since calling the std::sin function at every sample can be quite computationally expensive. For that reason, it is better to pre-compute the sine wave and store it in a wave table before computation starts. That kind of algorithm is then called a \"wave table oscillator.\" Sine.cpp , which is used in crazy-sine is a good example of that. It uses SineTable.cpp which pre-computes a sine table: table = new float[size]; for(int i=0; i #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; delay(500); }","title":"Looping Through a Small Tune: Making a Music Box"},{"location":"lectures/architecture/#basic-additive-synthesis","text":"One of the most basic kind of sound synthesis is \"additive synthesis.\" In consists of adding multiple sine wave oscillators together to \"sculpt\" the timbre of a sound. Both the frequency and the gain of each individual oscillator can then be used to change the properties of the synthesized sound. A simple additive synthesizer could be implemented from the crazy-sine example by declaring multiple instances of sine . E.g.: float currentSample = echo.tick(sine0.tick()*gain0 + sine1.tick()*gain1); but the problem with that option is that memory will be allocated twice for the sineTable array which is a terrible idea in the context of our embedded audio system with very little memory. Instead, the additive synthesizer should reuse the same instance of sineTable for each oscillator. In the tick method of Sine.cpp , try to call the sineTable a second time after float currentSample = sineTable[index]*gain; to add a second oscillator to the generated sample. The value of its index could be something like index = (int) (index*1.5)%SINE_TABLE_SIZE; so that the frequency of the second oscillator is one fifth above the main frequency. In other words, the differential equation of the synth should be: x(t) = sin(2 \\pi f t) + sin(2 \\pi (1.5f) t) Hint: Beware of clipping! Adding two sine waves together even though they don't have the same frequency will likely produce a signal whose range exceeds {-1;1}: you should take that into account for your final product. Solution: In Sine.cpp : float Sine::tick(){ int index = phasor.tick()*SINE_TABLE_SIZE; int index2 = (int) (index*1.5)%SINE_TABLE_SIZE; return (sineTable.tick(index)+sineTable.tick(index2))*gain*0.5; }","title":"Basic Additive Synthesis"},{"location":"lectures/architecture/#stereo-echo","text":"Reusing the result of the previous exercise, create a second instance of echo (connected to the same instance of sine ) with different parameters from the first one that will be connected to the second channel of the output (i.e., the first instance should be connected to the left channel and the second one to the right channel). The final algorithm should look like this: float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; Hint: Beware of memory allocation again! Make sure that the maxim delay of your echo (on the 2 parameters of the class constructor) doesn't exceed 10000 for now for both instances of the echo. Solution: In MyDsp.h : Sine sine; Echo echo0, echo1; }; In MyDsp.cpp : MyDsp::MyDsp() : AudioStream(AUDIO_OUTPUTS, new audio_block_t*[AUDIO_OUTPUTS]), sine(AUDIO_SAMPLE_RATE_EXACT), echo0(AUDIO_SAMPLE_RATE_EXACT,10000), echo1(AUDIO_SAMPLE_RATE_EXACT,7000) { ... // setting up DSP objects echo0.setDel(10000); echo0.setFeedback(0.5); echo1.setDel(7000); echo1.setFeedback(0.4); ... // processing buffers for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP float sineSample = sine.tick(); float currentSampleL = echo0.tick(sineSample)*0.5; float currentSampleR = echo1.tick(sineSample)*0.5; ... }","title":"Stereo Echo"},{"location":"lectures/control/","text":"Hardware Control and Audio Codec Configuration The two main goals of this lecture are: to show you how to control DSP algorithms running on your Teensy using hardware controllers (i.e., potentiometers and buttons); to give you a basic understanding of how audio codecs work and how they can be configured using the i2c protocol. Hardware Control Electronic Basics While the goal of this class is not to teach electronics nor to make projects involving complicated circuitry, some basic circuits do need to be implemented in order to control the various parameters of the DSP algorithms studied in class using hardware controllers such as buttons and potentiometers. Hence, in case you feel like your electronics skills are a bit rusty, feel free to review the following page: https://ccrma.stanford.edu/wiki/Introduction_to_Electronics_(condensed) . Teensy 4.0 Pinout The Teensy pins map can be seen in the following figure (directly taken from the PJRC website ). Most pins can be used as digital I/Os. Some pins noted \"A(N)\" can be used as analog inputs. Please, also note that 3.3v power can be retrieved from the top right corner pin and the ground from the top left pin. Make sure to never connect the 5.5v Vin pin to any other pin of the Teensy: that would probably fry it (the Cortex M7 inside the Teensy operates at 3.3v)! Teensy pinout. The Teensy audio shield uses a bunch of pins on the Teensy for i2c and i2s communication: Teensy audio shield pins. This means that these pins (besides GND and 3.3v, of course) cannot be used for something else (i.e., connecting external sensors). Bringing Power to Your Breadboard The first step in making your first circuit with the Teensy is to bring power to the breadboard included in your kit using jumper wires: Teensy connected to the breadboard. Basically connect the 3.3v pin to the red strip and the GND pin to the black strip of the breadboard. WARNING: Do not connect the 5.5v pin to the breadboard! Adding a Rotary Potentiometer to the Circuit Your kit should come with a couple of rotary potentiometers: Rotary potentiometer mounted on the breadboard. Place it on the breadboard, and connect its leftmost pin to power and its rightmost pin to the ground. Finally, connect its center pin to the A0 pin of the Teensy using a jumper wire. Please, note that we're using this pin since it is not used by the audio shield (see previous section). Testing the Potentiometer In the current configuration, the potentiometer will deliver a 3.3v current at its center pin if it is fully turned to the right side, and 0v if it is fully turned to the left side. The following Teensy program: void setup() { Serial.begin(9600); } void loop() { int sensorValue = analogRead(A0); Serial.println(sensorValue); delay(100); } displays the values measured at the A0 pin on the Teensy in the serial debugger. Values should be between 0 and 1023 (10 bits values). Make sure that the values you're getting are consistent with the position of the potentiometer. Controlling DSP Parameters With the Potentiometer Now that you know how to retrieve potentiometer values in the Teensy, plugging it to your audio DSP should be pretty straightforward. Hence, we can reuse the crazy-sine example and do: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } Note that sensorValue needs to be turned into a frequency in hertz so we just add 100 to it to get a frequency between 100 and 1123 hertz. Now take some time to have fun ;)! Using a Button Using a button with the Teensy is slightly more involving since the use of a pulldown resistor is required (alternatively, a pullup resistor could be used, of course). This is due to the fact that buttons are \"just\" circuit breakers: they don't have a dedicated output pin like potentiometers. The pulldown resistor is used to suck potential floating currents out of the output pin of the button in order to get a stable signal to be measured on the Teensy. Hence, the following circuit must be implemented: Circuit to connect a button to the Teensy. There's no need to use an analog pin on the Teensy to measure the voltage at the output of the button since we're looking at discrete values here (0 or 1). Hence, the button shall be connected to a digital pin (number 0, for example). Expanding on the previous example, we could write: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { pinMode(0, INPUT); // configuring digital pin 0 as an input AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { if (digitalRead(0)) { // button is pressed myDsp.setGain(1); } else { myDsp.setGain(0); } int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } (Assuming that a setGain method has been implemented, which is not the case in the previous example. It shouldn't be too hard though ;) ) Exercise: Looping Between Notes by Pressing a Button Expand the \"note looper\" that you implemented as part of this lecture so that new notes are triggered when a button is pressed (as opposed to be triggered automatically). Every time the button is pressed, a new note is produced. This means that you'll have to turn your push button into a switch using software techniques... Finally, make sure that gain is controllable using a rotary potentiometer. Solution: In crazy-sine.ino : #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; bool change = true; bool on = false; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); pinMode(0, INPUT); } void loop() { if (digitalRead(0)) { // button is pressed if(!on) change = true; on = true; } else { if(on) change = true; on = false; } if(change){ // if the status of the button changed if(on){ // if the button is pressed myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; } change = false; // status changed } } Audio Codec Configuration Audio Codec An audio codec is a hardware component providing an ADC and a DAC for audio purposes. Hence, it typically has analog audio inputs and outputs (stereo, in general) and digital audio inputs and outputs. Most audio codecs support standard audio sampling rate (i.e., 44.1, 48kHz, etc.) and bit depth (i.e., 16, 24 bits, etc.). Some high-end audio codecs also support higher sampling rates (e.g., 96, 192 kHz, etc.) and bit depth (32 bits, mostly) as well as more than a stereo interface (e.g., 4x4, 8x8, etc.). The price range of audio codecs can vary significantly impacting the quality of the components, i.e., audio is usually extremely sensitive to the quality of the hardware components of a system. Audio codecs usually use two different communication channels to connect to the processor unit (whether it's a CPU, a microcontroller, etc.): an i2c bus is used to configure the codec, an i2s bus is used to transmit digital audio data. i2c (pronounced I-squared-C) is a serial communication protocol heavily used in the field of microelectronics. Most digital elements in an electronic circuit communicate using i2c. i2s (pronounced I-squared-S) is also a serial communication protocol but targeting specifically audio applications. We'll see that they're very close to each other in practice later in this class. The Teensy Audio Shield hosts an audio codec (SGTL5000) which is connected to the Teensy using the following model: Interfacing of a microcontroller with an audio codec. You can check the Teensy audio shield connection map in the Teensy Pinout section for more details on how the audio shield is actually connected to the Teensy. Before audio data can be streamed to the audio codec through i2s, it needs to be configured with an audio driver which basically just sends a set of instructions from the microcontroller to the codec using i2c. The goal of this lecture is to get a basic understanding of how audio drivers work in the context of embedded systems. Quick Tour of the SGTL5000 The SGTL5000 is a low-power 24-bit, 8 kHz to 96 kHz audio codec. Its data sheet can be found on the course repository (feel free to download it now because you'll extensively need it later in this lecture). On the first page of the data sheet, you will find a block diagram indicating the different components of the codec: Block diagram of the SGTL5000 As you can see, the SGTL5000 hosts a stereo ADC and DAC. The codec has a stereo line input and a mono mic input (both accessible through solderable pins on the audio shield). The difference between the two is that the line input goes straight to the ADC while the mic input goes through a preamp first. The gain of the preamp can be adjusted independently from that of the line input. A similar pattern is used for the outputs and the DAC which are available as line outputs (through solderable pins on the audio shield) or amplified outputs (through the headphone jack on the audio shield). Finally, the codec also has a digital interface which is used for i2c (depicted at the bottom of the block diagram) and i2s (depicted on the left side of the block diagram) communication. Configuring an Audio Codec All audio codecs work the same way and are configured through their i2c bus. A system of register/value is used for that. A register corresponds to a set of parameters and a 16 bits value can be provided to configure them. A list of all available registers of the SGTL5000 can be seen on page 31-59 of the data sheet . Have a quick look at it! For example, register 0x0010 (which is documented on p. 36) allows us to configure the DAC left and right channel volume in dB. Hence setting that register to the 3C3C (0011110000111100 binary value) will set the volume to 0dB on both channels. If you're hex/binary is a bit rusty, you can use this tool: https://www.rapidtables.com/convert/number/hex-to-binary.html to carry out the conversion. Audio Codec Driver Writing an audio codec driver consists of sending the right sequence of register/value through i2c to the codec. This will set the signal routing, the i2s format, the sampling rate, the bit depth, etc. control_sgtl5000.cpp of the Teensy Audio Library implements a driver for the SGTL5000 codec. The write method can be used to set a register and its corresponding 16-bit value. Note that the Arduino Wire library is used for that. The enable method sets a bunch of register values to provide a basic working configuration to the codec in the context of the Teensy. A bunch of methods are implemented to set high-level parameters of the codec, such as volume , micGain , lineInLevel , etc. Note that a bunch of macros are defined at the beginning for various registers (which constitutes a potential alternative to the codec datasheet, etc.). In most cases, calling: AudioControlSGTL5000 audioShield; audioShield.enable(); in the .ino file will be sufficient to get the audio codec going and send it i2s data. Since the write method of AudioControlSGTL5000 is protected, it cannot be called outside of the class. Hence, to write custom methods to configure the codec and which are not currently available in AudioControlSGTL5000 , one would have to write a custom version of control_sgtl5000.cpp .","title":" Hardware Control and Audio Codec Configuration "},{"location":"lectures/control/#hardware-control-and-audio-codec-configuration","text":"The two main goals of this lecture are: to show you how to control DSP algorithms running on your Teensy using hardware controllers (i.e., potentiometers and buttons); to give you a basic understanding of how audio codecs work and how they can be configured using the i2c protocol.","title":"Hardware Control and Audio Codec Configuration"},{"location":"lectures/control/#hardware-control","text":"","title":"Hardware Control"},{"location":"lectures/control/#electronic-basics","text":"While the goal of this class is not to teach electronics nor to make projects involving complicated circuitry, some basic circuits do need to be implemented in order to control the various parameters of the DSP algorithms studied in class using hardware controllers such as buttons and potentiometers. Hence, in case you feel like your electronics skills are a bit rusty, feel free to review the following page: https://ccrma.stanford.edu/wiki/Introduction_to_Electronics_(condensed) .","title":"Electronic Basics"},{"location":"lectures/control/#teensy-40-pinout","text":"The Teensy pins map can be seen in the following figure (directly taken from the PJRC website ). Most pins can be used as digital I/Os. Some pins noted \"A(N)\" can be used as analog inputs. Please, also note that 3.3v power can be retrieved from the top right corner pin and the ground from the top left pin. Make sure to never connect the 5.5v Vin pin to any other pin of the Teensy: that would probably fry it (the Cortex M7 inside the Teensy operates at 3.3v)! Teensy pinout. The Teensy audio shield uses a bunch of pins on the Teensy for i2c and i2s communication: Teensy audio shield pins. This means that these pins (besides GND and 3.3v, of course) cannot be used for something else (i.e., connecting external sensors).","title":"Teensy 4.0 Pinout"},{"location":"lectures/control/#bringing-power-to-your-breadboard","text":"The first step in making your first circuit with the Teensy is to bring power to the breadboard included in your kit using jumper wires: Teensy connected to the breadboard. Basically connect the 3.3v pin to the red strip and the GND pin to the black strip of the breadboard. WARNING: Do not connect the 5.5v pin to the breadboard!","title":"Bringing Power to Your Breadboard"},{"location":"lectures/control/#adding-a-rotary-potentiometer-to-the-circuit","text":"Your kit should come with a couple of rotary potentiometers: Rotary potentiometer mounted on the breadboard. Place it on the breadboard, and connect its leftmost pin to power and its rightmost pin to the ground. Finally, connect its center pin to the A0 pin of the Teensy using a jumper wire. Please, note that we're using this pin since it is not used by the audio shield (see previous section).","title":"Adding a Rotary Potentiometer to the Circuit"},{"location":"lectures/control/#testing-the-potentiometer","text":"In the current configuration, the potentiometer will deliver a 3.3v current at its center pin if it is fully turned to the right side, and 0v if it is fully turned to the left side. The following Teensy program: void setup() { Serial.begin(9600); } void loop() { int sensorValue = analogRead(A0); Serial.println(sensorValue); delay(100); } displays the values measured at the A0 pin on the Teensy in the serial debugger. Values should be between 0 and 1023 (10 bits values). Make sure that the values you're getting are consistent with the position of the potentiometer.","title":"Testing the Potentiometer"},{"location":"lectures/control/#controlling-dsp-parameters-with-the-potentiometer","text":"Now that you know how to retrieve potentiometer values in the Teensy, plugging it to your audio DSP should be pretty straightforward. Hence, we can reuse the crazy-sine example and do: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } Note that sensorValue needs to be turned into a frequency in hertz so we just add 100 to it to get a frequency between 100 and 1123 hertz. Now take some time to have fun ;)!","title":"Controlling DSP Parameters With the Potentiometer"},{"location":"lectures/control/#using-a-button","text":"Using a button with the Teensy is slightly more involving since the use of a pulldown resistor is required (alternatively, a pullup resistor could be used, of course). This is due to the fact that buttons are \"just\" circuit breakers: they don't have a dedicated output pin like potentiometers. The pulldown resistor is used to suck potential floating currents out of the output pin of the button in order to get a stable signal to be measured on the Teensy. Hence, the following circuit must be implemented: Circuit to connect a button to the Teensy. There's no need to use an analog pin on the Teensy to measure the voltage at the output of the button since we're looking at discrete values here (0 or 1). Hence, the button shall be connected to a digital pin (number 0, for example). Expanding on the previous example, we could write: #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); void setup() { pinMode(0, INPUT); // configuring digital pin 0 as an input AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); } void loop() { if (digitalRead(0)) { // button is pressed myDsp.setGain(1); } else { myDsp.setGain(0); } int sensorValue = analogRead(A0); float freq = sensorValue + 100; myDsp.setFreq(freq); } (Assuming that a setGain method has been implemented, which is not the case in the previous example. It shouldn't be too hard though ;) )","title":"Using a Button"},{"location":"lectures/control/#exercise-looping-between-notes-by-pressing-a-button","text":"Expand the \"note looper\" that you implemented as part of this lecture so that new notes are triggered when a button is pressed (as opposed to be triggered automatically). Every time the button is pressed, a new note is produced. This means that you'll have to turn your push button into a switch using software techniques... Finally, make sure that gain is controllable using a rotary potentiometer. Solution: In crazy-sine.ino : #include #include \"MyDsp.h\" MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); float mtof(float note){ return pow(2.0,(note-69.0)/12.0)*440.0; } int tune[] = {62,78,65,67,69}; int cnt = 0; bool change = true; bool on = false; void setup() { AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); pinMode(0, INPUT); } void loop() { if (digitalRead(0)) { // button is pressed if(!on) change = true; on = true; } else { if(on) change = true; on = false; } if(change){ // if the status of the button changed if(on){ // if the button is pressed myDsp.setFreq(mtof(tune[cnt])); cnt = (cnt+1)%5; } change = false; // status changed } }","title":"Exercise: Looping Between Notes by Pressing a Button"},{"location":"lectures/control/#audio-codec-configuration","text":"","title":"Audio Codec Configuration"},{"location":"lectures/control/#audio-codec","text":"An audio codec is a hardware component providing an ADC and a DAC for audio purposes. Hence, it typically has analog audio inputs and outputs (stereo, in general) and digital audio inputs and outputs. Most audio codecs support standard audio sampling rate (i.e., 44.1, 48kHz, etc.) and bit depth (i.e., 16, 24 bits, etc.). Some high-end audio codecs also support higher sampling rates (e.g., 96, 192 kHz, etc.) and bit depth (32 bits, mostly) as well as more than a stereo interface (e.g., 4x4, 8x8, etc.). The price range of audio codecs can vary significantly impacting the quality of the components, i.e., audio is usually extremely sensitive to the quality of the hardware components of a system. Audio codecs usually use two different communication channels to connect to the processor unit (whether it's a CPU, a microcontroller, etc.): an i2c bus is used to configure the codec, an i2s bus is used to transmit digital audio data. i2c (pronounced I-squared-C) is a serial communication protocol heavily used in the field of microelectronics. Most digital elements in an electronic circuit communicate using i2c. i2s (pronounced I-squared-S) is also a serial communication protocol but targeting specifically audio applications. We'll see that they're very close to each other in practice later in this class. The Teensy Audio Shield hosts an audio codec (SGTL5000) which is connected to the Teensy using the following model: Interfacing of a microcontroller with an audio codec. You can check the Teensy audio shield connection map in the Teensy Pinout section for more details on how the audio shield is actually connected to the Teensy. Before audio data can be streamed to the audio codec through i2s, it needs to be configured with an audio driver which basically just sends a set of instructions from the microcontroller to the codec using i2c. The goal of this lecture is to get a basic understanding of how audio drivers work in the context of embedded systems.","title":"Audio Codec"},{"location":"lectures/control/#quick-tour-of-the-sgtl5000","text":"The SGTL5000 is a low-power 24-bit, 8 kHz to 96 kHz audio codec. Its data sheet can be found on the course repository (feel free to download it now because you'll extensively need it later in this lecture). On the first page of the data sheet, you will find a block diagram indicating the different components of the codec: Block diagram of the SGTL5000 As you can see, the SGTL5000 hosts a stereo ADC and DAC. The codec has a stereo line input and a mono mic input (both accessible through solderable pins on the audio shield). The difference between the two is that the line input goes straight to the ADC while the mic input goes through a preamp first. The gain of the preamp can be adjusted independently from that of the line input. A similar pattern is used for the outputs and the DAC which are available as line outputs (through solderable pins on the audio shield) or amplified outputs (through the headphone jack on the audio shield). Finally, the codec also has a digital interface which is used for i2c (depicted at the bottom of the block diagram) and i2s (depicted on the left side of the block diagram) communication.","title":"Quick Tour of the SGTL5000"},{"location":"lectures/control/#configuring-an-audio-codec","text":"All audio codecs work the same way and are configured through their i2c bus. A system of register/value is used for that. A register corresponds to a set of parameters and a 16 bits value can be provided to configure them. A list of all available registers of the SGTL5000 can be seen on page 31-59 of the data sheet . Have a quick look at it! For example, register 0x0010 (which is documented on p. 36) allows us to configure the DAC left and right channel volume in dB. Hence setting that register to the 3C3C (0011110000111100 binary value) will set the volume to 0dB on both channels. If you're hex/binary is a bit rusty, you can use this tool: https://www.rapidtables.com/convert/number/hex-to-binary.html to carry out the conversion.","title":"Configuring an Audio Codec"},{"location":"lectures/control/#audio-codec-driver","text":"Writing an audio codec driver consists of sending the right sequence of register/value through i2c to the codec. This will set the signal routing, the i2s format, the sampling rate, the bit depth, etc. control_sgtl5000.cpp of the Teensy Audio Library implements a driver for the SGTL5000 codec. The write method can be used to set a register and its corresponding 16-bit value. Note that the Arduino Wire library is used for that. The enable method sets a bunch of register values to provide a basic working configuration to the codec in the context of the Teensy. A bunch of methods are implemented to set high-level parameters of the codec, such as volume , micGain , lineInLevel , etc. Note that a bunch of macros are defined at the beginning for various registers (which constitutes a potential alternative to the codec datasheet, etc.). In most cases, calling: AudioControlSGTL5000 audioShield; audioShield.enable(); in the .ino file will be sufficient to get the audio codec going and send it i2s data. Since the write method of AudioControlSGTL5000 is protected, it cannot be called outside of the class. Hence, to write custom methods to configure the codec and which are not currently available in AudioControlSGTL5000 , one would have to write a custom version of control_sgtl5000.cpp .","title":"Audio Codec Driver"},{"location":"lectures/digital-audio/","text":"Audio Signal Processing Fundamentals The goal of this lecture is to provide an overview of the basics of digital audio. Analog Audio Signals Before the advent of digital audio, most audio systems/technologies were analog. An analog audio signal can take different forms: it can be electric (e.g., transmitted through an electric wire and stored on a magnetic tape) or mechanical (e.g., transmitted through the air as standing waves and stored on a vinyl disc). Acoustical mechanical waves can be converted into an electric signal using a microphone. Conversely, an electric audio signal can be converted into mechanical acoustical waves using a speaker. In nature, sounds almost always originate from a mechanical source. However, in the 20th century, many musicians, composers and engineers experimented with the production of sound from an electrical source. One of the pioneer in this field was Karlheinz Stockhausen . This lead to analog and modular synthesizers which are very popular among Croix-Roussian hipsters these days. A modular analog synthesizer The Discovery of Digital Audio Sampling theory dates back from the beginning of the 20th century with initial work by Harry Nyquist and was theorized in the 1930s by Claude Shannon to become the Nyquist-Shannon sampling theorem. Carrying sampling in the field of audio is relatively simple: voltage measurements are carried out at regular intervals of time on an analog electrical signal. Each individual acquired value is called a \"sample\" and can be stored on a computer. Hence, while an analog electric audio signal is a variation of tension in time in an electric cable, a digital audio signal is just series of samples (values) in time as well. Signal sampling representation. The continuous signal is represented with a green colored line while the discrete samples are indicated by the blue vertical lines. (source: Wikipedia ) ADC and DAC In the field of audio, an ADC (Analog to Digital Converter) is a hardware component that can be used to discretize (sample) an electrical analog audio signal. The reverse operation is carried out using a DAC (Digital to Analog Converter). In most systems, the ADC and the DAC are hosted in the same piece of hardware (e.g., audio codec, audio interface, etc.). Human Hearing Range and Sampling Rate One of the main factor to consider when sampling an audio signal is the human hearing range. In theory, humans can hear any sound between 20 and 20000 Hz. In practice, our ability to perceive high frequencies decays over time and is affected by environmental factors (e.g., if we're exposed to sound with high volume, if we contract some diseases such as hear infections, etc.). By the age of 30, most adults can't hear frequencies over 17 kHz. When sampling an audio signal, the number of samples per second also known as the sampling rate (noted fs ) will determine the highest frequency than can be sampled by the system. The rule is very simple: the highest frequency that can be sampled is half the sampling rate. Hence, in order to sample a frequency of 20 kHz, the sampling rate of the system must be at least 40 kHz which corresponds to 40000 values (samples) per second. The highest frequency that can be sampled is also known as the \" Nyquist Frequency \" ( fn ): fn=\\frac{fs}{2} The standard for modern audio systems is to use a sampling rate of 48 kHz. fs is 44.1 kHz on compact discs (CDs) and many home and recording studios use a sampling rate of 96 or 192 kHz. Sampling Theorem Let x(t) denote any continuous-time signal having a continuous Fourier transform : X(j\\omega) \\triangleq \\int_{-\\infty}^{\\infty}x(t)e^{-j \\omega t}dt Let x_d(n) \\triangleq x(nT), \\quad n=\\dots,-2,-1,0,1,2,\\dots, denote the samples of x(t) at uniform intervals of T seconds. Then x(t) can be exactly reconstructed from its samples x_d(n) if X(j\\omega)=0 for all \\vert\\omega\\vert\\geq\\pi/T . In other words, any frequency (harmonics) between 0 Hz and the Nyquist frequency can be exactly reconstructed without loosing any information. That also means that if the Nyquist frequency is above the upper threshold of the human hearing range (e.g., 20 kHz), a digitized signal should sound exactly the same as its analog counterpart from a perceptual standpoint. Additional proofs about the sampling theorem can be found on Julius Smith's website here . Aliasing Aliasing is a well known phenomenon in the field of video: In audio, aliasing happens when a digital signal contains frequencies above the Nyquist frequency. In that case, they are not sampled at the right frequency and they are wrapped. Hence, for all frequency fo above fn , the sampled frequency f will be: f = fn - (fo-fn) with fn = \\frac{fs}{2} Aliasing is typically prevented by filtering an analog signal before it is discretized by removing all frequency above fn . Aliasing can also be obtained when synthesizing a broadband signal on a computer (e.g., a sawtooth wave). It is the software engineer's role to prevent this from happening. Bit Depth, Dynamic Range and Signal-to-Noise Ratio Beside sampling rate, the other parameter of sampling is the bit depth of audio samples. Audio is typically recorded at 8, 16 (the standard for CDs), or 24 bits (and 32 bits in some rarer cases). A higher bit depth means a more accurate precision for a given audio sample. This impacts directly the dynamic range and the signal-to-noise (SNR) ratio of a digital signal. In other words, a smaller bit depth will mean more noise in the signal, etc. Additional information about this topic can be found here . Range of Audio Samples Audio samples can be coded in many different ways depending on the context. Some low-level systems use fixed-point numbers (i.e., integers) for efficiency. In that case, the range of the signal will be determined by the data type. For example, if audio samples are coded on 16 bits unsigned integers, the range of the signal will be 0 to 2^{16} - 1 (or 65535). At the hardware level (e.g., ADC/DAC), audio samples are almost exclusively coded on integers. On the other hand, fixed points are relatively hard to deal with at the software level when it comes to implementing DSP algorithms. In that case, it is much more convenient to use decimal numbers (i.e., floating points). The established standard in audio is that audio signals coded on decimal numbers always have the following range: {-1;1}. While this range can be exceeded within an algorithm without any consequences, the inputs and outputs of a DSP block must always be constrained between -1 and 1. Most systems will clip audio signals to this range to prevent warping and will hence result in clipping if exceeded. First Synthesized Sound on a Digital Computer While Shanon and Nyquist theorized sampling in the 1930s, it's only in 1958 that a sound was synthesized for the first time on a computer by Max Mathews at Bell Labs, giving birth a few years later to the first song synthesized (and sung) by a computer: This was by the way reused by Stanley Kubrick in one of his famous movie as HAL the computer is slowly dying as it's being unplugged: These technologies were then extensively exploited until today both for musical applications and in the industry at large.","title":" Audio Signal Processing Fundamentals "},{"location":"lectures/digital-audio/#audio-signal-processing-fundamentals","text":"The goal of this lecture is to provide an overview of the basics of digital audio.","title":"Audio Signal Processing Fundamentals"},{"location":"lectures/digital-audio/#analog-audio-signals","text":"Before the advent of digital audio, most audio systems/technologies were analog. An analog audio signal can take different forms: it can be electric (e.g., transmitted through an electric wire and stored on a magnetic tape) or mechanical (e.g., transmitted through the air as standing waves and stored on a vinyl disc). Acoustical mechanical waves can be converted into an electric signal using a microphone. Conversely, an electric audio signal can be converted into mechanical acoustical waves using a speaker. In nature, sounds almost always originate from a mechanical source. However, in the 20th century, many musicians, composers and engineers experimented with the production of sound from an electrical source. One of the pioneer in this field was Karlheinz Stockhausen . This lead to analog and modular synthesizers which are very popular among Croix-Roussian hipsters these days. A modular analog synthesizer","title":"Analog Audio Signals"},{"location":"lectures/digital-audio/#the-discovery-of-digital-audio","text":"Sampling theory dates back from the beginning of the 20th century with initial work by Harry Nyquist and was theorized in the 1930s by Claude Shannon to become the Nyquist-Shannon sampling theorem. Carrying sampling in the field of audio is relatively simple: voltage measurements are carried out at regular intervals of time on an analog electrical signal. Each individual acquired value is called a \"sample\" and can be stored on a computer. Hence, while an analog electric audio signal is a variation of tension in time in an electric cable, a digital audio signal is just series of samples (values) in time as well. Signal sampling representation. The continuous signal is represented with a green colored line while the discrete samples are indicated by the blue vertical lines. (source: Wikipedia )","title":"The Discovery of Digital Audio"},{"location":"lectures/digital-audio/#adc-and-dac","text":"In the field of audio, an ADC (Analog to Digital Converter) is a hardware component that can be used to discretize (sample) an electrical analog audio signal. The reverse operation is carried out using a DAC (Digital to Analog Converter). In most systems, the ADC and the DAC are hosted in the same piece of hardware (e.g., audio codec, audio interface, etc.).","title":"ADC and DAC"},{"location":"lectures/digital-audio/#human-hearing-range-and-sampling-rate","text":"One of the main factor to consider when sampling an audio signal is the human hearing range. In theory, humans can hear any sound between 20 and 20000 Hz. In practice, our ability to perceive high frequencies decays over time and is affected by environmental factors (e.g., if we're exposed to sound with high volume, if we contract some diseases such as hear infections, etc.). By the age of 30, most adults can't hear frequencies over 17 kHz. When sampling an audio signal, the number of samples per second also known as the sampling rate (noted fs ) will determine the highest frequency than can be sampled by the system. The rule is very simple: the highest frequency that can be sampled is half the sampling rate. Hence, in order to sample a frequency of 20 kHz, the sampling rate of the system must be at least 40 kHz which corresponds to 40000 values (samples) per second. The highest frequency that can be sampled is also known as the \" Nyquist Frequency \" ( fn ): fn=\\frac{fs}{2} The standard for modern audio systems is to use a sampling rate of 48 kHz. fs is 44.1 kHz on compact discs (CDs) and many home and recording studios use a sampling rate of 96 or 192 kHz.","title":"Human Hearing Range and Sampling Rate"},{"location":"lectures/digital-audio/#sampling-theorem","text":"Let x(t) denote any continuous-time signal having a continuous Fourier transform : X(j\\omega) \\triangleq \\int_{-\\infty}^{\\infty}x(t)e^{-j \\omega t}dt Let x_d(n) \\triangleq x(nT), \\quad n=\\dots,-2,-1,0,1,2,\\dots, denote the samples of x(t) at uniform intervals of T seconds. Then x(t) can be exactly reconstructed from its samples x_d(n) if X(j\\omega)=0 for all \\vert\\omega\\vert\\geq\\pi/T . In other words, any frequency (harmonics) between 0 Hz and the Nyquist frequency can be exactly reconstructed without loosing any information. That also means that if the Nyquist frequency is above the upper threshold of the human hearing range (e.g., 20 kHz), a digitized signal should sound exactly the same as its analog counterpart from a perceptual standpoint. Additional proofs about the sampling theorem can be found on Julius Smith's website here .","title":"Sampling Theorem"},{"location":"lectures/digital-audio/#aliasing","text":"Aliasing is a well known phenomenon in the field of video: In audio, aliasing happens when a digital signal contains frequencies above the Nyquist frequency. In that case, they are not sampled at the right frequency and they are wrapped. Hence, for all frequency fo above fn , the sampled frequency f will be: f = fn - (fo-fn) with fn = \\frac{fs}{2} Aliasing is typically prevented by filtering an analog signal before it is discretized by removing all frequency above fn . Aliasing can also be obtained when synthesizing a broadband signal on a computer (e.g., a sawtooth wave). It is the software engineer's role to prevent this from happening.","title":"Aliasing"},{"location":"lectures/digital-audio/#bit-depth-dynamic-range-and-signal-to-noise-ratio","text":"Beside sampling rate, the other parameter of sampling is the bit depth of audio samples. Audio is typically recorded at 8, 16 (the standard for CDs), or 24 bits (and 32 bits in some rarer cases). A higher bit depth means a more accurate precision for a given audio sample. This impacts directly the dynamic range and the signal-to-noise (SNR) ratio of a digital signal. In other words, a smaller bit depth will mean more noise in the signal, etc. Additional information about this topic can be found here .","title":"Bit Depth, Dynamic Range and Signal-to-Noise Ratio"},{"location":"lectures/digital-audio/#range-of-audio-samples","text":"Audio samples can be coded in many different ways depending on the context. Some low-level systems use fixed-point numbers (i.e., integers) for efficiency. In that case, the range of the signal will be determined by the data type. For example, if audio samples are coded on 16 bits unsigned integers, the range of the signal will be 0 to 2^{16} - 1 (or 65535). At the hardware level (e.g., ADC/DAC), audio samples are almost exclusively coded on integers. On the other hand, fixed points are relatively hard to deal with at the software level when it comes to implementing DSP algorithms. In that case, it is much more convenient to use decimal numbers (i.e., floating points). The established standard in audio is that audio signals coded on decimal numbers always have the following range: {-1;1}. While this range can be exceeded within an algorithm without any consequences, the inputs and outputs of a DSP block must always be constrained between -1 and 1. Most systems will clip audio signals to this range to prevent warping and will hence result in clipping if exceeded.","title":"Range of Audio Samples"},{"location":"lectures/digital-audio/#first-synthesized-sound-on-a-digital-computer","text":"While Shanon and Nyquist theorized sampling in the 1930s, it's only in 1958 that a sound was synthesized for the first time on a computer by Max Mathews at Bell Labs, giving birth a few years later to the first song synthesized (and sung) by a computer: This was by the way reused by Stanley Kubrick in one of his famous movie as HAL the computer is slowly dying as it's being unplugged: These technologies were then extensively exploited until today both for musical applications and in the industry at large.","title":"First Synthesized Sound on a Digital Computer"},{"location":"lectures/dsp1/","text":"Audio Processing Basics I This lecture and this one (Audio Processing Basics II) present a selection of audio processing and synthesis algorithms. It is in no way comprehensive: the goal is just to give you a sense of what's out there. All these algorithms have been extensively used during the second half of the twentieth century by musicians and artists, especially within the computer music community. White Noise White noise is a specific kind of signal in which there's an infinite number of harmonics all having the same level. In other words, the spectrum of white noise looks completely flat. White noise is produced by generating random numbers between -1 and 1. Noise.cpp demonstrates how this can be done in C++ using the rand() function: Noise::Noise() : randDiv(1.0/RAND_MAX){} float Noise::tick(){ return rand()*randDiv*2 - 1; } The Simple Filter: One Zero section presents a use example of white noise. Wave Shape Synthesis Wave Shape synthesis is one of the most basic sound synthesis technique. It consists of using oscillators producing waveforms of different shapes to generate sound. The most standard wave shapes are: sine wave , square wave , triangle wave , sawtooth wave . The crazy-sine example can be considered as \"wave shape synthesis\" in that regard. The crazy-saw example is very similar to crazy-sine , but it's based on a sawtooth wave instead. The sawtooth wave is created by using a phasor object. Just as a reminder, a phasor produces a signals tamping from 0 to 1 at a given frequency, it can therefore be seen as a sawtooth wave. Since the range of oscillators must be bounded between -1 and 1, we adjusts the output of the phasor such that: float currentSample = sawtooth.tick()*2 - 1; Feel free to try the crazy-saw example at this point. Amplitude Modulation (AM) Synthesis Amplitude modulation synthesis consists of modulating the amplitude of a signal with another one. Sine waves are typically used for that: Amplitude Modulation (Source: Wikipedia ) When the frequency of the modulator is low (bellow 20Hz), our ear is able to distinguish each independent \"beat,\" creating a tremolo effect. However, above 20Hz two side bands (if sine waves are used) start appearing following this rule: Amplitude Modulation Spectrum (Source: Wikipedia ) The mathematical proof of this can be found on Julius Smith's website . Am.cpp implements a sinusoidal amplitude modulation synthesizer: float Am::tick(){ int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float posMod = sineTable.tick(mIndex)*0.5 + 0.5; return sineTable.tick(cIndex)*(1 - posMod*modIndex)*gain; } Note that phasors are used instead of \"complete\" sine wave oscillators to save the memory of an extra sine wave table. The range of the modulating oscillator is adjusted to be {0,1} instead of {-1,1}. The amplitude parameter of the modulating oscillator is called the index of modulation and its frequency, the frequency of modulation . In practice, the same result could be achieved using additive synthesis and three sine wave oscillators but AM allows us to save one oscillator. Also, AM is usually used an audio effect and modulation is applied to an input signal in that case instead of a sine wave. Sidebands will then be produced for each harmonic of the processed sound. The am example demonstrates a use case of an AM synthesizer. Use the Rec and Mode button to cycle through the parameters of the synth and change their value. Frequency Modulation (FM) Synthesis Frequency modulation synthesis consists of modulating the frequency of an oscillator with another one: Frequency Modulation (Source: Wikipedia ) which mathematically can be expressed as: x(t) = A_c\\sin[\\omega_ct + \\phi_c + A_m\\sin(\\omega_mt + \\phi_m)] where c denotes the carrier and m , the modulator. As for AM, the frequency of the modulating oscillator is called the frequency of modulation and the amplitude of the modulating oscillator, the index of modulation . Unlike AM, the value of the index of modulation can exceed 1 which will increase the number of sidebands. FM is not limited to two sidebands and can have an infinite number of sidebands depending on the value of the index. The mathematical rational behind this can be found on Julius Smith's website . fm.cpp provides a simple example of how an FM synthesizer can be implemented: float Fm::tick(){ int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float modulator = sineTable.tick(mIndex); cPhasor.setFrequency(cFreq + modulator*modIndex); int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; return sineTable.tick(cIndex)*gain; } Note that as for the AM example, we're saving an extra sine wave table by using the same one for both oscillators. The examples folder of the course repository hosts a simple Teensy program illustrating the use of FM. Use the Rec and Mode button to cycle through the parameters of the synth and change their value. FM synthesis was discovered in the late 1960s by John Chowning at Stanford University in California. He's now considered as one of the funding fathers of music technology and computer music. FM completely revolutionized the world of music in the 1980s by allowing Yamaha to produce the first commercial digital synthesizers: the DX7 which met a huge success. FM synthesis is the second most profitable patent that Stanford ever had. Simple Filter: One Zero Filters are heavily used in the field of audio processing. In fact, designing filters is a whole field by itself. They are at the basis of many audio effects such as Wah guitar pedals, etc. From an algorithmic standpoint, the most basic filter is what we call a \"one zero\" filter which means that its transfer function only has numerators and no denominators. The differential equation of a one zero filter can be expressed as: y(n) = b_0x(n) + b_1x(n-1) where b_1 is \"the zero\" of the filter (also called feed forward coefficient ), b_0 can be discarded as it is equal to 1 in most cases. One zero filters can either be used as a lowpass if the value of b_1 is positive or as a highpass if b_1 is negative. The frequency response of the filter which is be obtained with H(e^{j \\omega T}) = b_0 + b_1e^{-j \\omega T} can be visualized on Julius Smith's website . Note that the gain of the signal is amplified on the second half of the spectrum which needs to be taken into account if this filter is used to process audio (once again, the output signal must be bounded within {-1,1}). OneZero.cpp implements a one zero filter: float OneZero::tick(float input){ float output = input + del*b1; del = input; return output*0.5; } Note that we multiply the output by 0.5 to normalize the output gain. The filtered-noise example program for the Teensy demonstrates the use of OneZero.cpp by feeding white noise in it. The value of b_1 can be changed by pressing the \"Mode\" button on the board, give it a try! Exercises LFO: Low Frequency Oscillator An LFO is an oscillator whose frequency is below the human hearing range (20 Hz). LFOs are typically used to create vibrato. In that case, the frequency of the LFO is usually set to 6 Hz. Modify the crazy-saw example so that notes are played slower (1 per second) and that some vibrato is added to the generated sound. Solution: Shall be posted here after class... In MyDsp.h : #include \"Sine.h\" ... private: float freq; Phasor sawtooth; Echo echo; Sine LFO; }; In MyDsp.cpp : MyDsp::MyDsp() : ... freq(440), sawtooth(AUDIO_SAMPLE_RATE_EXACT), echo(AUDIO_SAMPLE_RATE_EXACT,10000), LFO(AUDIO_SAMPLE_RATE_EXACT) { ... // setting up DSP objects echo.setDel(10000); echo.setFeedback(0.5); LFO.setFrequency(6); } ... // set sine wave frequency void MyDsp::setFreq(float f){ freq = f; } ... for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP sawtooth.setFrequency(freq*(1 + LFO.tick()*0.1)); float currentSample = echo.tick(sawtooth.tick()*2 - 1)*0.5; Towards the DX7 The DX7 carried out frequency modulation over a total of six oscillators that could be patched in different ways . So FM is not limited to two oscillators... Try to implement an FM synthesizer involving 3 oscillators instead of one. They should be connected in series: 3 -> 2 -> 1. Solution: Shall be posted after class... (non-exhaustive) In Fm.cpp : float Fm::tick(){ int m0Index = m0Phasor.tick()*SINE_TABLE_SIZE; float modulator0 = sineTable.tick(m0Index); modulator1.setFrequency(m1Freq + modulator0*mod0Index); int m1Index = m1Phasor.tick()*SINE_TABLE_SIZE; float modulator1 = sineTable.tick(m1Index); cPhasor.setFrequency(cFreq + modulator1*mod1Index); int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; return sineTable.tick(cIndex)*gain; }","title":" Audio Processing Basics I "},{"location":"lectures/dsp1/#audio-processing-basics-i","text":"This lecture and this one (Audio Processing Basics II) present a selection of audio processing and synthesis algorithms. It is in no way comprehensive: the goal is just to give you a sense of what's out there. All these algorithms have been extensively used during the second half of the twentieth century by musicians and artists, especially within the computer music community.","title":"Audio Processing Basics I"},{"location":"lectures/dsp1/#white-noise","text":"White noise is a specific kind of signal in which there's an infinite number of harmonics all having the same level. In other words, the spectrum of white noise looks completely flat. White noise is produced by generating random numbers between -1 and 1. Noise.cpp demonstrates how this can be done in C++ using the rand() function: Noise::Noise() : randDiv(1.0/RAND_MAX){} float Noise::tick(){ return rand()*randDiv*2 - 1; } The Simple Filter: One Zero section presents a use example of white noise.","title":"White Noise"},{"location":"lectures/dsp1/#wave-shape-synthesis","text":"Wave Shape synthesis is one of the most basic sound synthesis technique. It consists of using oscillators producing waveforms of different shapes to generate sound. The most standard wave shapes are: sine wave , square wave , triangle wave , sawtooth wave . The crazy-sine example can be considered as \"wave shape synthesis\" in that regard. The crazy-saw example is very similar to crazy-sine , but it's based on a sawtooth wave instead. The sawtooth wave is created by using a phasor object. Just as a reminder, a phasor produces a signals tamping from 0 to 1 at a given frequency, it can therefore be seen as a sawtooth wave. Since the range of oscillators must be bounded between -1 and 1, we adjusts the output of the phasor such that: float currentSample = sawtooth.tick()*2 - 1; Feel free to try the crazy-saw example at this point.","title":"Wave Shape Synthesis"},{"location":"lectures/dsp1/#amplitude-modulation-am-synthesis","text":"Amplitude modulation synthesis consists of modulating the amplitude of a signal with another one. Sine waves are typically used for that: Amplitude Modulation (Source: Wikipedia ) When the frequency of the modulator is low (bellow 20Hz), our ear is able to distinguish each independent \"beat,\" creating a tremolo effect. However, above 20Hz two side bands (if sine waves are used) start appearing following this rule: Amplitude Modulation Spectrum (Source: Wikipedia ) The mathematical proof of this can be found on Julius Smith's website . Am.cpp implements a sinusoidal amplitude modulation synthesizer: float Am::tick(){ int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float posMod = sineTable.tick(mIndex)*0.5 + 0.5; return sineTable.tick(cIndex)*(1 - posMod*modIndex)*gain; } Note that phasors are used instead of \"complete\" sine wave oscillators to save the memory of an extra sine wave table. The range of the modulating oscillator is adjusted to be {0,1} instead of {-1,1}. The amplitude parameter of the modulating oscillator is called the index of modulation and its frequency, the frequency of modulation . In practice, the same result could be achieved using additive synthesis and three sine wave oscillators but AM allows us to save one oscillator. Also, AM is usually used an audio effect and modulation is applied to an input signal in that case instead of a sine wave. Sidebands will then be produced for each harmonic of the processed sound. The am example demonstrates a use case of an AM synthesizer. Use the Rec and Mode button to cycle through the parameters of the synth and change their value.","title":"Amplitude Modulation (AM) Synthesis"},{"location":"lectures/dsp1/#frequency-modulation-fm-synthesis","text":"Frequency modulation synthesis consists of modulating the frequency of an oscillator with another one: Frequency Modulation (Source: Wikipedia ) which mathematically can be expressed as: x(t) = A_c\\sin[\\omega_ct + \\phi_c + A_m\\sin(\\omega_mt + \\phi_m)] where c denotes the carrier and m , the modulator. As for AM, the frequency of the modulating oscillator is called the frequency of modulation and the amplitude of the modulating oscillator, the index of modulation . Unlike AM, the value of the index of modulation can exceed 1 which will increase the number of sidebands. FM is not limited to two sidebands and can have an infinite number of sidebands depending on the value of the index. The mathematical rational behind this can be found on Julius Smith's website . fm.cpp provides a simple example of how an FM synthesizer can be implemented: float Fm::tick(){ int mIndex = mPhasor.tick()*SINE_TABLE_SIZE; float modulator = sineTable.tick(mIndex); cPhasor.setFrequency(cFreq + modulator*modIndex); int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; return sineTable.tick(cIndex)*gain; } Note that as for the AM example, we're saving an extra sine wave table by using the same one for both oscillators. The examples folder of the course repository hosts a simple Teensy program illustrating the use of FM. Use the Rec and Mode button to cycle through the parameters of the synth and change their value. FM synthesis was discovered in the late 1960s by John Chowning at Stanford University in California. He's now considered as one of the funding fathers of music technology and computer music. FM completely revolutionized the world of music in the 1980s by allowing Yamaha to produce the first commercial digital synthesizers: the DX7 which met a huge success. FM synthesis is the second most profitable patent that Stanford ever had.","title":"Frequency Modulation (FM) Synthesis"},{"location":"lectures/dsp1/#simple-filter-one-zero","text":"Filters are heavily used in the field of audio processing. In fact, designing filters is a whole field by itself. They are at the basis of many audio effects such as Wah guitar pedals, etc. From an algorithmic standpoint, the most basic filter is what we call a \"one zero\" filter which means that its transfer function only has numerators and no denominators. The differential equation of a one zero filter can be expressed as: y(n) = b_0x(n) + b_1x(n-1) where b_1 is \"the zero\" of the filter (also called feed forward coefficient ), b_0 can be discarded as it is equal to 1 in most cases. One zero filters can either be used as a lowpass if the value of b_1 is positive or as a highpass if b_1 is negative. The frequency response of the filter which is be obtained with H(e^{j \\omega T}) = b_0 + b_1e^{-j \\omega T} can be visualized on Julius Smith's website . Note that the gain of the signal is amplified on the second half of the spectrum which needs to be taken into account if this filter is used to process audio (once again, the output signal must be bounded within {-1,1}). OneZero.cpp implements a one zero filter: float OneZero::tick(float input){ float output = input + del*b1; del = input; return output*0.5; } Note that we multiply the output by 0.5 to normalize the output gain. The filtered-noise example program for the Teensy demonstrates the use of OneZero.cpp by feeding white noise in it. The value of b_1 can be changed by pressing the \"Mode\" button on the board, give it a try!","title":"Simple Filter: One Zero"},{"location":"lectures/dsp1/#exercises","text":"","title":"Exercises"},{"location":"lectures/dsp1/#lfo-low-frequency-oscillator","text":"An LFO is an oscillator whose frequency is below the human hearing range (20 Hz). LFOs are typically used to create vibrato. In that case, the frequency of the LFO is usually set to 6 Hz. Modify the crazy-saw example so that notes are played slower (1 per second) and that some vibrato is added to the generated sound. Solution: Shall be posted here after class... In MyDsp.h : #include \"Sine.h\" ... private: float freq; Phasor sawtooth; Echo echo; Sine LFO; }; In MyDsp.cpp : MyDsp::MyDsp() : ... freq(440), sawtooth(AUDIO_SAMPLE_RATE_EXACT), echo(AUDIO_SAMPLE_RATE_EXACT,10000), LFO(AUDIO_SAMPLE_RATE_EXACT) { ... // setting up DSP objects echo.setDel(10000); echo.setFeedback(0.5); LFO.setFrequency(6); } ... // set sine wave frequency void MyDsp::setFreq(float f){ freq = f; } ... for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { // DSP sawtooth.setFrequency(freq*(1 + LFO.tick()*0.1)); float currentSample = echo.tick(sawtooth.tick()*2 - 1)*0.5;","title":"LFO: Low Frequency Oscillator"},{"location":"lectures/dsp1/#towards-the-dx7","text":"The DX7 carried out frequency modulation over a total of six oscillators that could be patched in different ways . So FM is not limited to two oscillators... Try to implement an FM synthesizer involving 3 oscillators instead of one. They should be connected in series: 3 -> 2 -> 1. Solution: Shall be posted after class... (non-exhaustive) In Fm.cpp : float Fm::tick(){ int m0Index = m0Phasor.tick()*SINE_TABLE_SIZE; float modulator0 = sineTable.tick(m0Index); modulator1.setFrequency(m1Freq + modulator0*mod0Index); int m1Index = m1Phasor.tick()*SINE_TABLE_SIZE; float modulator1 = sineTable.tick(m1Index); cPhasor.setFrequency(cFreq + modulator1*mod1Index); int cIndex = cPhasor.tick()*SINE_TABLE_SIZE; return sineTable.tick(cIndex)*gain; }","title":"Towards the DX7"},{"location":"lectures/dsp2/","text":"Audio Processing Basics II Harmonic Distortion: Rock On! Distortion is one of the most common electric guitar effect. It consists of over driving a signal by increasing its gain to \"square\" the extremities of its waveform. This results in the creation of lots of harmonics, producing very \"rich\" sounds. Overdrive is easily achievable with an analog electronic circuit and \"sharp edges\" in the waveform are rounded thanks to the tolerance of the electronic components. In the digital world, things are slightly more complicated since clipping will happen resulting in a very dirty sound with potentially lots of aliasing. One way to solve this problem is to use a \"cubic function\" which will round the edges of the signal above a certain amplitude: f(x) = \\begin{cases} \\frac{-2}{3}, \\; \\; x \\leq -1\\\\ x - \\frac{x^3}{3}, \\; \\; -1 < x < 1\\\\ \\frac{2}{3}, \\; \\; x \\geq -1 \\end{cases} Distortion.cpp implements a cubic distortion as: float Distortion::cubic(float x){ return x - x*x*x/3; } float Distortion::tick(float input){ float output = input*pow(10.0,2*drive) + offset; output = fmax(-1,fmin(1,output)); output = cubic(output); return output*gain; } The range of drive is {0;1} which means that the value of input can be multiplied by a number as great as 100 here. offset is a common parameter which just adds a positive or negative DC offset to the signal. If this parameter is used, it is recommended to add a DC blocking filter after the distortion. Distortion is created here by clipping the signal using the fmin and fmax functions. Finally, the cubic polynomial is used to round the edges of the waveform of the signal as explained above. The distortion example program for the Teensy demonstrates the use of Distortion.cpp . Distortion is a very trendy field of research in audio technology these days especially using \"virtual analog\" algorithms which consists of modeling the electronic circuit of distortion on a computer. Echo An echo is a very common audio effect which is used a lot to add some density and depth to a sound. It is based on a feedback loop and a delay and can be expressed as: y(n) = x(n) + g.y(n - M) where g is the feedback between 0 and 1 and M the delay as a number of samples. It can be seen as a simple physical model of what happens in the real world when echo is produced: the delay represents the time it takes for an acoustical wave to go from point A to point B at the speed of sound and g can control the amount of absorption created by the air and the reflecting material. Echo.cpp implements an echo as: float Echo::tick(float input){ float output = input + delBuffer[readIndex]*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } Here, delBuffer is used as a \"ring buffer\": incoming samples are stored and the read and write indices loop around to buffer to write incoming samples and read previous ones. Note that memory is allocated in the constructor of the class for delBuffer based on the value of maxDel , the maximum size of the delay. The echo example program for the Teensy demonstrates the use of Echo.cpp . Comb A comb filter is a filter whose frequency response looks like a \"comb.\" Comb filters can be implemented with feed-forward filters (Finite Impulse Response -- FIR) or feedback filters (Infinite Impulse Response -- IIR). In fact, the Echo algorithm can be used as a comb filter if the delay is very short: y(n) = x(n)-g.y(n-M) where M is the length of the delay and g feedback coefficient. Julius Smith's website presents the frequency response of such filter and the mathematical rationals behind it. From an acoustical standpoint, a feedback comb filter will introduce resonances at specific point in the spectrum of the sound. The position and the spacing of these resonances is determined by the value of M . g , on the other hand, will determine the amplitude and sharpness of these resonances. The comb example program for the Teensy demonstrates the use of Echo.cpp as a comb filter. The \"Mode\" button can be used to change the value of the delay. Physical Modeling: the Simple Case of the Karplus Strong Physical modeling is one of the most advanced sound synthesis technique and a very active field of research. It consists of using physics/mathematical models of musical instruments or vibrating structures to synthesize sound. Various physical modeling techniques are used in the field of audio synthesis: Mass/Interaction (MI), Finite Difference Scheme (FDS), Signal models (e.g., waveguides, modal systems, etc.). While MI and FDS model the vibrational behavior of a system (i.e., using partial differential equation in the case of FDS), signal models model an object as a combination of signal processors. In this section, we will only look at this type of model the other ones being out of the scope of this class. An extremely primitive string model can be implemented using a delay line and a loop. The delay line models the time it takes for vibration in the string to go from one extremity to the other, and the loop models the reflections at the boundaries of the string. In other words, we can literally just reuse the echo algorithm for this. This primitive string model is called the \"Karplus-Strong\" algorithm: Karplus-Strong Algorithm (Source: Wikipedia ) The Karplus-Strong algorithm is typically implemented as: y(n) = x(n) + \\alpha\\frac{y(n-L) + y(n-L-1)}{2} where: x(n) is the input signal (typically an dirac or a noise burst), \\alpha is the feedback coefficient (or dispersion coefficient, in that case), L is the length of the delay and hence, the length of the string. \\frac{y(n-L) + y(n-L-1)}{2} can be seen as a one zero filter implementing a lowpass. It models the fact that high frequencies are absorbed faster than low frequencies at the extremities of a string. The length of the delay L can be controlled as a frequency using the following formula: L = fs/f where f is the desired frequency. At the very least, the system must be excited by a dirac (i.e., a simple impulse going from 1 to 0). The quality of the generated sound can be significantly improved if a noise impulse is used though. KS.cpp implements a basic Karplus-Strong algorithm: float KS::tick(){ float excitation; if(trig){ excitation = 1.0; trig = false; } else{ excitation = 0.0; } float output = excitation + oneZero(delBuffer[readIndex])*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } with: float KS::oneZero(float x){ float output = (x + zeroDel)*0.5; zeroDel = output; return output; } The examples folder of the course repository hosts a simple Teensy program illustrating the use of KS.cpp . Note that this algorithm could be improved in many ways. In particular, the fact that the delay length is currently expressed as an integer can result in frequency mismatches at high frequencies. In other words, our current string is out of tune. This could be fixed using fractional delay . In practice, the Karplus-Strong algorithm is not a physical model per se and is just a simplification of the ideal string wave equation . More advanced signal models can be implemented using waveguides. Waveguide physical modeling has been extensively used in modern synthesizers to synthesize the sound of acoustic instruments. Julius O. Smith (Stanford professor) is the father of waveguide physical modeling. Exercises Smoothing In most cases, DSP parameters are executed at control rate. Moreover, the resolution of the value used to configure parameters is much lower than that of audio samples since it might come from a Graphical User Interface (GUI), a low resolution sensor ADC (e.g., arduino), etc. For all these reasons, changing the value of a DSP parameter will often result in a \"click\"/discontinuity. A common way to prevent this from happening is to interpolate between the values of the parameter using a \"leaky integrator.\" In signal processing, this can be easily implemented using a normalized one pole lowpass filter: y(n) = (1-s)x(n) + sy(n-1) where s is the value of the pole and is typically set to 0.999 for optimal results. Modify the crazy-saw example by \"smoothing\" the value of the frequency parameter by implementing the filter above with s=0.999 . Then slow down the rate at which frequency is being changed so that only two new values are generated per second. The result should sound quite funny :). Solution: Shall be posted after class... Smoothing Potentiometer Values Try to use the smoothing function that you implemented in the previous step to smooth sensor values coming from a potential potentiometer controlling some parameter of one of the Teensy examples. The main idea is to get rid of sound artifacts when making abrupt changes in potentiometers.","title":" Audio Processing Basics II "},{"location":"lectures/dsp2/#audio-processing-basics-ii","text":"","title":"Audio Processing Basics II"},{"location":"lectures/dsp2/#harmonic-distortion-rock-on","text":"Distortion is one of the most common electric guitar effect. It consists of over driving a signal by increasing its gain to \"square\" the extremities of its waveform. This results in the creation of lots of harmonics, producing very \"rich\" sounds. Overdrive is easily achievable with an analog electronic circuit and \"sharp edges\" in the waveform are rounded thanks to the tolerance of the electronic components. In the digital world, things are slightly more complicated since clipping will happen resulting in a very dirty sound with potentially lots of aliasing. One way to solve this problem is to use a \"cubic function\" which will round the edges of the signal above a certain amplitude: f(x) = \\begin{cases} \\frac{-2}{3}, \\; \\; x \\leq -1\\\\ x - \\frac{x^3}{3}, \\; \\; -1 < x < 1\\\\ \\frac{2}{3}, \\; \\; x \\geq -1 \\end{cases} Distortion.cpp implements a cubic distortion as: float Distortion::cubic(float x){ return x - x*x*x/3; } float Distortion::tick(float input){ float output = input*pow(10.0,2*drive) + offset; output = fmax(-1,fmin(1,output)); output = cubic(output); return output*gain; } The range of drive is {0;1} which means that the value of input can be multiplied by a number as great as 100 here. offset is a common parameter which just adds a positive or negative DC offset to the signal. If this parameter is used, it is recommended to add a DC blocking filter after the distortion. Distortion is created here by clipping the signal using the fmin and fmax functions. Finally, the cubic polynomial is used to round the edges of the waveform of the signal as explained above. The distortion example program for the Teensy demonstrates the use of Distortion.cpp . Distortion is a very trendy field of research in audio technology these days especially using \"virtual analog\" algorithms which consists of modeling the electronic circuit of distortion on a computer.","title":"Harmonic Distortion: Rock On!"},{"location":"lectures/dsp2/#echo","text":"An echo is a very common audio effect which is used a lot to add some density and depth to a sound. It is based on a feedback loop and a delay and can be expressed as: y(n) = x(n) + g.y(n - M) where g is the feedback between 0 and 1 and M the delay as a number of samples. It can be seen as a simple physical model of what happens in the real world when echo is produced: the delay represents the time it takes for an acoustical wave to go from point A to point B at the speed of sound and g can control the amount of absorption created by the air and the reflecting material. Echo.cpp implements an echo as: float Echo::tick(float input){ float output = input + delBuffer[readIndex]*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } Here, delBuffer is used as a \"ring buffer\": incoming samples are stored and the read and write indices loop around to buffer to write incoming samples and read previous ones. Note that memory is allocated in the constructor of the class for delBuffer based on the value of maxDel , the maximum size of the delay. The echo example program for the Teensy demonstrates the use of Echo.cpp .","title":"Echo"},{"location":"lectures/dsp2/#comb","text":"A comb filter is a filter whose frequency response looks like a \"comb.\" Comb filters can be implemented with feed-forward filters (Finite Impulse Response -- FIR) or feedback filters (Infinite Impulse Response -- IIR). In fact, the Echo algorithm can be used as a comb filter if the delay is very short: y(n) = x(n)-g.y(n-M) where M is the length of the delay and g feedback coefficient. Julius Smith's website presents the frequency response of such filter and the mathematical rationals behind it. From an acoustical standpoint, a feedback comb filter will introduce resonances at specific point in the spectrum of the sound. The position and the spacing of these resonances is determined by the value of M . g , on the other hand, will determine the amplitude and sharpness of these resonances. The comb example program for the Teensy demonstrates the use of Echo.cpp as a comb filter. The \"Mode\" button can be used to change the value of the delay.","title":"Comb"},{"location":"lectures/dsp2/#physical-modeling-the-simple-case-of-the-karplus-strong","text":"Physical modeling is one of the most advanced sound synthesis technique and a very active field of research. It consists of using physics/mathematical models of musical instruments or vibrating structures to synthesize sound. Various physical modeling techniques are used in the field of audio synthesis: Mass/Interaction (MI), Finite Difference Scheme (FDS), Signal models (e.g., waveguides, modal systems, etc.). While MI and FDS model the vibrational behavior of a system (i.e., using partial differential equation in the case of FDS), signal models model an object as a combination of signal processors. In this section, we will only look at this type of model the other ones being out of the scope of this class. An extremely primitive string model can be implemented using a delay line and a loop. The delay line models the time it takes for vibration in the string to go from one extremity to the other, and the loop models the reflections at the boundaries of the string. In other words, we can literally just reuse the echo algorithm for this. This primitive string model is called the \"Karplus-Strong\" algorithm: Karplus-Strong Algorithm (Source: Wikipedia ) The Karplus-Strong algorithm is typically implemented as: y(n) = x(n) + \\alpha\\frac{y(n-L) + y(n-L-1)}{2} where: x(n) is the input signal (typically an dirac or a noise burst), \\alpha is the feedback coefficient (or dispersion coefficient, in that case), L is the length of the delay and hence, the length of the string. \\frac{y(n-L) + y(n-L-1)}{2} can be seen as a one zero filter implementing a lowpass. It models the fact that high frequencies are absorbed faster than low frequencies at the extremities of a string. The length of the delay L can be controlled as a frequency using the following formula: L = fs/f where f is the desired frequency. At the very least, the system must be excited by a dirac (i.e., a simple impulse going from 1 to 0). The quality of the generated sound can be significantly improved if a noise impulse is used though. KS.cpp implements a basic Karplus-Strong algorithm: float KS::tick(){ float excitation; if(trig){ excitation = 1.0; trig = false; } else{ excitation = 0.0; } float output = excitation + oneZero(delBuffer[readIndex])*feedback; delBuffer[writeIndex] = output; readIndex = (readIndex+1)%del; writeIndex = (writeIndex+1)%del; return output; } with: float KS::oneZero(float x){ float output = (x + zeroDel)*0.5; zeroDel = output; return output; } The examples folder of the course repository hosts a simple Teensy program illustrating the use of KS.cpp . Note that this algorithm could be improved in many ways. In particular, the fact that the delay length is currently expressed as an integer can result in frequency mismatches at high frequencies. In other words, our current string is out of tune. This could be fixed using fractional delay . In practice, the Karplus-Strong algorithm is not a physical model per se and is just a simplification of the ideal string wave equation . More advanced signal models can be implemented using waveguides. Waveguide physical modeling has been extensively used in modern synthesizers to synthesize the sound of acoustic instruments. Julius O. Smith (Stanford professor) is the father of waveguide physical modeling.","title":"Physical Modeling: the Simple Case of the Karplus Strong"},{"location":"lectures/dsp2/#exercises","text":"","title":"Exercises"},{"location":"lectures/dsp2/#smoothing","text":"In most cases, DSP parameters are executed at control rate. Moreover, the resolution of the value used to configure parameters is much lower than that of audio samples since it might come from a Graphical User Interface (GUI), a low resolution sensor ADC (e.g., arduino), etc. For all these reasons, changing the value of a DSP parameter will often result in a \"click\"/discontinuity. A common way to prevent this from happening is to interpolate between the values of the parameter using a \"leaky integrator.\" In signal processing, this can be easily implemented using a normalized one pole lowpass filter: y(n) = (1-s)x(n) + sy(n-1) where s is the value of the pole and is typically set to 0.999 for optimal results. Modify the crazy-saw example by \"smoothing\" the value of the frequency parameter by implementing the filter above with s=0.999 . Then slow down the rate at which frequency is being changed so that only two new values are generated per second. The result should sound quite funny :). Solution: Shall be posted after class...","title":"Smoothing"},{"location":"lectures/dsp2/#smoothing-potentiometer-values","text":"Try to use the smoothing function that you implemented in the previous step to smooth sensor values coming from a potential potentiometer controlling some parameter of one of the Teensy examples. The main idea is to get rid of sound artifacts when making abrupt changes in potentiometers.","title":"Smoothing Potentiometer Values"},{"location":"lectures/embedded/","text":"Embedded System Basics That course will explain in more details the principles of embedded programming, peripheral programming, and interrupt handling. At the end of the course is explained how to use simple makefiles to program the teensy, not the arduino IDE. Slides Slides courses are available here INTERRUPTS The principle of interrupt is fundamental in computers and it is the same on every machine. The processor running its program can receive interrupts (i.e. hardware interrupts no to mislead with software interrupts that are implemented by operating systems) at any time. An interrupt can be sent by a peripheral of the micro-controller (timer, radio chip, serial port, etc...), or received from the outside (via GPIOs) like the reset for example. There is another way to communicate with peripherals which is called polling : regularly look a the state of the peripheral to see if something has changed. It is simpler than using interrupts but much more compute-intensive for the CPU. It is the programmer who configures the peripherals (for example the timer) to send an interrupt during certain events. For instance the user can ask the timer to send an interruption every 1.5 second. We will see how to do that with the Teensy. The interrupt \"triggers\" an interrupt handler (or interrupt routine service : ISR), special function called for this particular interrupt. Each interrupt has its own ISR. The correspondence between the interrupt and the ISR is done with the interrupt vector table . This means that each processor knows exactly which interrupt can occur. Each interrupt has a flag bit, which is set by hardware when the interrupt trigger condition occurs. For instance the list of interrupt flags of the Teensy (i.e. of CorteX-M7) is here . The flag's purpose is to remember the interrupt condition has occurred until it has been handled by software. An interrupt is said to be \"pending\" if the trigger condition has set the flag but the interrupt service routine has not been called yet, which can happen if the main program has disabled interrupts or another interrupt service routine is running (usually, interrupts are disable during the execution of an ISR). Usually, interrupt flags are automatically reset when the interrupt service routine is called. Some flags must be reset by the software inside the interrupt service routine. Some flags are completely controlled by the peripheral to reflect internal state (such as UART receive) and can only be changed indirectly by manipulating the peripheral. All interrupts are used in roughly the same way. Configure The Peripheral Reset Interrupt Flag Set Interrupt Mask Enable Global Interrupt, with sei() When the interrupt condition occurs, the interrupt flag is set. The interrupt service routine will be called at the first opportunity. In \"traditionnal\" embedded microcontroler programming, this is done \"by hand\", i.e. all these steps are performed by the programmer. However with the advent of complex micro-controllers such as the ARM CorteX-M, these tasks are simplified with higher level API for peripherals and with the use of interrupt callback. Interrupt Callback Interrupt Callback principle (Image Source: Reusable Firmware Development book] In modern micro-controllers, interrupt handlers are usually in the kernel driver library, i.e. not written by the developper. However, the programmer has to indicate how to react to the interrupt. A way to have a flexible interrupt handler is to use a callback. An interrupt callback is a function, dynamically assigned, that will be called from the ISR . Using function pointer, the programmer can assign any function to the callback. Usually the kernel driver library provides a function that perform this assignement. For instance, the Teensy library propose an class IntervalTimer which uses the timers of the ARM CPU to send regular (i.e. at regular interval) an interrupt. The user can indicate the callback function foo() to be called at each timer interruption occuring, say, every 1.5 second: IntervalTimer myTimer; myTimer.begin(foo, 150000); For that, foo() function must have type: void foo() Exercice: LED blinking Create a new teensy project teensy_led which blinks the LED every 100ms using the delay(int numMilliSeconds) function: You have to know on which GPIO the LED has been soldered. You can look at the schematic of the Teensy 4.0 or a look at file $ARDUINOPATH/hardware/teensy/avr/cos/teensy4/pins_arduino.h and guess which macro can be used for the LED pin (let's call it ledPin ). Then you just have to know that this pin has to be configured in output mode: pinMode(ledPin, OUTPUT) and that switching on and off the led can be done using digitalWrite(ledPin, HIGH); and digitalWrite(ledPin, LOW); Using the built in function delay(int numMilliSeconds) write a program that blink the LED. Solution: Posted after class... Exercice: LED and timer Copy the teensy_led directory to a teensy_timer directory. write a function void toggle_LED() that uses a global variable LEDstate which correspond to the current status of the LED. (In general, it is very common in embedded system to have a local copy of the state of peripheral, just to know in which state we are). Instantiate an IntervalTimer and, as shown above, use toggle_LED() function as timer callback. Have the LED blinking every 0.15s What is the advantage of this approach (i.e. using timers instead of delays) Solution: Posted after class... Exercice: LED, timer and UART Create another project teensy_serial that prints, at each second, on the serial port the number of LED switch occured from the beguinning. Note that you will have to use a global variable shared by the ISR (in function toogle_LED() ) and the main code. It is recommended to disable interrupt when modifyng this variable in the main code, using noInterrupts() and Interrupts() functions. Solution: Posted after class... Exercice: LED, timer and Audio Download the teensy_audio project.. This project plays the crazy-sine sound while blinking the LED. Check that the sound is correct. Add a 10ms delai in the blinkLED callback. What do you notice. It is very important to spend a very short time in ISR, other wise your system can be blocked, miss interrupts or not respect real time constraints. Solution: Posted after class... (Optionnal) Compiling teensy project with Makefile Sometimes it may be interesting to get rid of the arduino environment and provide Makefile that can be easier to tune to various sytems. Here is an example of a teensy program using a Makefile rather than the arduino environment for compilation: - Download the teensy_makefile project here - Untar the archive tar xvf teensy_makefile.tar somewhere (this will create teensy_makefile directory) - Go into the teensy_makefile directory and edit the Makefile to fill up the definition of $ARDUINOPATH (your arduino installation) and $MYDSPPATH (location of the AUD MyDsp library on your computer). - Type make check that the compilation is going well. The object files are compiled in the build directory. The only file that are interesting for you are in the current directory: main.cpp and MyDsp.cpp . The files MyDsp.h and MyDsp.cpp are the same as in the the AUD examples projects. The main.cpp is this one: #include #include \"MyDsp.h\" int main(void) { MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); while (1) { myDsp.setFreq(random(50,1000)); delay(100); } } As one can see, we do not have a setup() and loop() function, but just a main() function with an initialization (which corresponds to the setup() function) and an infinite loop (which correponds to the loop() function). This is allways the case for embedded programming: initialization and infinite loop. Here, we know exactly what executes on the ARM CPU, and it is very explicit that the ARM processor is 100% running the loop doing nothing (i.e. no operating system is present on the ARM), hence sound processing relies on interrupts. A very common programming model for embedded system is to rely only on interrupts. The generic Makefile that you can use for your project is here .","title":" Embedded System Peripherals "},{"location":"lectures/embedded/#embedded-system-basics","text":"That course will explain in more details the principles of embedded programming, peripheral programming, and interrupt handling. At the end of the course is explained how to use simple makefiles to program the teensy, not the arduino IDE.","title":"Embedded System Basics"},{"location":"lectures/embedded/#slides","text":"Slides courses are available here","title":"Slides"},{"location":"lectures/embedded/#interrupts","text":"The principle of interrupt is fundamental in computers and it is the same on every machine. The processor running its program can receive interrupts (i.e. hardware interrupts no to mislead with software interrupts that are implemented by operating systems) at any time. An interrupt can be sent by a peripheral of the micro-controller (timer, radio chip, serial port, etc...), or received from the outside (via GPIOs) like the reset for example. There is another way to communicate with peripherals which is called polling : regularly look a the state of the peripheral to see if something has changed. It is simpler than using interrupts but much more compute-intensive for the CPU. It is the programmer who configures the peripherals (for example the timer) to send an interrupt during certain events. For instance the user can ask the timer to send an interruption every 1.5 second. We will see how to do that with the Teensy. The interrupt \"triggers\" an interrupt handler (or interrupt routine service : ISR), special function called for this particular interrupt. Each interrupt has its own ISR. The correspondence between the interrupt and the ISR is done with the interrupt vector table . This means that each processor knows exactly which interrupt can occur. Each interrupt has a flag bit, which is set by hardware when the interrupt trigger condition occurs. For instance the list of interrupt flags of the Teensy (i.e. of CorteX-M7) is here . The flag's purpose is to remember the interrupt condition has occurred until it has been handled by software. An interrupt is said to be \"pending\" if the trigger condition has set the flag but the interrupt service routine has not been called yet, which can happen if the main program has disabled interrupts or another interrupt service routine is running (usually, interrupts are disable during the execution of an ISR). Usually, interrupt flags are automatically reset when the interrupt service routine is called. Some flags must be reset by the software inside the interrupt service routine. Some flags are completely controlled by the peripheral to reflect internal state (such as UART receive) and can only be changed indirectly by manipulating the peripheral. All interrupts are used in roughly the same way. Configure The Peripheral Reset Interrupt Flag Set Interrupt Mask Enable Global Interrupt, with sei() When the interrupt condition occurs, the interrupt flag is set. The interrupt service routine will be called at the first opportunity. In \"traditionnal\" embedded microcontroler programming, this is done \"by hand\", i.e. all these steps are performed by the programmer. However with the advent of complex micro-controllers such as the ARM CorteX-M, these tasks are simplified with higher level API for peripherals and with the use of interrupt callback.","title":"INTERRUPTS"},{"location":"lectures/embedded/#interrupt-callback","text":"Interrupt Callback principle (Image Source: Reusable Firmware Development book] In modern micro-controllers, interrupt handlers are usually in the kernel driver library, i.e. not written by the developper. However, the programmer has to indicate how to react to the interrupt. A way to have a flexible interrupt handler is to use a callback. An interrupt callback is a function, dynamically assigned, that will be called from the ISR . Using function pointer, the programmer can assign any function to the callback. Usually the kernel driver library provides a function that perform this assignement. For instance, the Teensy library propose an class IntervalTimer which uses the timers of the ARM CPU to send regular (i.e. at regular interval) an interrupt. The user can indicate the callback function foo() to be called at each timer interruption occuring, say, every 1.5 second: IntervalTimer myTimer; myTimer.begin(foo, 150000); For that, foo() function must have type: void foo()","title":"Interrupt Callback"},{"location":"lectures/embedded/#exercice-led-blinking","text":"Create a new teensy project teensy_led which blinks the LED every 100ms using the delay(int numMilliSeconds) function: You have to know on which GPIO the LED has been soldered. You can look at the schematic of the Teensy 4.0 or a look at file $ARDUINOPATH/hardware/teensy/avr/cos/teensy4/pins_arduino.h and guess which macro can be used for the LED pin (let's call it ledPin ). Then you just have to know that this pin has to be configured in output mode: pinMode(ledPin, OUTPUT) and that switching on and off the led can be done using digitalWrite(ledPin, HIGH); and digitalWrite(ledPin, LOW); Using the built in function delay(int numMilliSeconds) write a program that blink the LED. Solution: Posted after class...","title":"Exercice: LED blinking"},{"location":"lectures/embedded/#exercice-led-and-timer","text":"Copy the teensy_led directory to a teensy_timer directory. write a function void toggle_LED() that uses a global variable LEDstate which correspond to the current status of the LED. (In general, it is very common in embedded system to have a local copy of the state of peripheral, just to know in which state we are). Instantiate an IntervalTimer and, as shown above, use toggle_LED() function as timer callback. Have the LED blinking every 0.15s What is the advantage of this approach (i.e. using timers instead of delays) Solution: Posted after class...","title":"Exercice: LED and timer"},{"location":"lectures/embedded/#exercice-led-timer-and-uart","text":"Create another project teensy_serial that prints, at each second, on the serial port the number of LED switch occured from the beguinning. Note that you will have to use a global variable shared by the ISR (in function toogle_LED() ) and the main code. It is recommended to disable interrupt when modifyng this variable in the main code, using noInterrupts() and Interrupts() functions. Solution: Posted after class...","title":"Exercice: LED, timer and UART"},{"location":"lectures/embedded/#exercice-led-timer-and-audio","text":"Download the teensy_audio project.. This project plays the crazy-sine sound while blinking the LED. Check that the sound is correct. Add a 10ms delai in the blinkLED callback. What do you notice. It is very important to spend a very short time in ISR, other wise your system can be blocked, miss interrupts or not respect real time constraints. Solution: Posted after class...","title":"Exercice: LED, timer and Audio"},{"location":"lectures/embedded/#optionnal-compiling-teensy-project-with-makefile","text":"Sometimes it may be interesting to get rid of the arduino environment and provide Makefile that can be easier to tune to various sytems. Here is an example of a teensy program using a Makefile rather than the arduino environment for compilation: - Download the teensy_makefile project here - Untar the archive tar xvf teensy_makefile.tar somewhere (this will create teensy_makefile directory) - Go into the teensy_makefile directory and edit the Makefile to fill up the definition of $ARDUINOPATH (your arduino installation) and $MYDSPPATH (location of the AUD MyDsp library on your computer). - Type make check that the compilation is going well. The object files are compiled in the build directory. The only file that are interesting for you are in the current directory: main.cpp and MyDsp.cpp . The files MyDsp.h and MyDsp.cpp are the same as in the the AUD examples projects. The main.cpp is this one: #include #include \"MyDsp.h\" int main(void) { MyDsp myDsp; AudioOutputI2S out; AudioControlSGTL5000 audioShield; AudioConnection patchCord0(myDsp,0,out,0); AudioConnection patchCord1(myDsp,0,out,1); AudioMemory(2); audioShield.enable(); audioShield.volume(0.5); while (1) { myDsp.setFreq(random(50,1000)); delay(100); } } As one can see, we do not have a setup() and loop() function, but just a main() function with an initialization (which corresponds to the setup() function) and an infinite loop (which correponds to the loop() function). This is allways the case for embedded programming: initialization and infinite loop. Here, we know exactly what executes on the ARM CPU, and it is very explicit that the ARM processor is 100% running the loop doing nothing (i.e. no operating system is present on the ARM), hence sound processing relies on interrupts. A very common programming model for embedded system is to rely only on interrupts. The generic Makefile that you can use for your project is here .","title":"(Optionnal) Compiling teensy project with Makefile"},{"location":"lectures/faust/","text":"Lecture 7: Faust Tutorial (TBD) Faust Web site","title":"Lecture 7: Faust Tutorial (TBD)"},{"location":"lectures/faust/#lecture-7-faust-tutorial-tbd","text":"Faust Web site","title":"Lecture 7: Faust Tutorial (TBD)"},{"location":"lectures/intro/","text":"Course Introduction and Programming Environment Setup This lecture is devoted to installing the software suite used in this course so that everybody can follow the other lectures from Insa or from his home if it needs to be done in distant work. The slides for the first courses can be found here Introduction to AUD2020 and Teensy Teensy 4.0, and the associated audio adaptor board Most of the document on this course come from PJRC . Actually most of the documentation on Teensy is from PJRC . The development in AUD are performed on teensy which is developped by PJRC. It is a microcontroller that offers many I/O pins and a USB interface. It is programmed using a custom/modified version of the arduino programming environment ( teensyduino ). Teensy is a brand of microcontroller development boards designed by the co-owner of PJRC, Paul Stoffregen . The first Teensy 2.0, Teensy++ 2.0 (and discontinued predecessors) use an 8-bit AVR microcontrollers. Teensy 3.0 (and up) have instead Freescale microcontrollers, running ARM Cortex-M CPUs. The technical characteritics of all Teensy can be compared here . In AUD, we use the Teensy 4.0 which contains an ARM Cortex-M7 at 600 MHz with a Floating point unit, hence it can handle non trivial audio treatment. Teensy 4.0 and Audio Shield (from PJRC Website) Teensy 4.0 uses many powerful CPU features useful for true real-time microcontroller platform. The CPU is an ARM Cortex-M7 dual-issue superscaler clocked at at 600 MHz. CPU performance is many times faster than typical 32 bit microcontrollers. The Floating Point Unit performs 32 bit float and 64 bit double precision math in hardware. DSP extension instructions accelerate signal processing, filters and Fourier transform. The Audio library automatically makes uses of these DSP instructions. Teensy performance from Core Benchmarks This pinout reference card comes with the Teensy 4.0 ( do not loose it! ). The pins are not 5V tolerant. Do not drive any digital pin higher than 3.3V. Teensy 4.0 pin map (from PJRC webite) The Teensy 4.0 has a total of 40 input/output signal pins. 24 are easily accessible when used with a solderless breadboard. The available pins include general purpose IO (GPIO, digital or analog, i.e. ADC), as well as integrated serial protocols (I2C, I2S, CAN, SPI and UART protocols) that are used to connect to other devices. In AUD, we use the audio adaptor board provided by PJRC that integrates a low power stereo codex (NXP Semiconductors SGTL5000 codec) and a SD card reader. Teensy audio adaptor (rev. D) from [PRJC](https://www.pjrc.com/store/teensy40.html), The audio codec connects to Teensy using 7 signals (Yellow signal in pin map above) which are used by two protocol: I2C and I2S. This is a traditional configuration for audio codec: the I2C (or I\u00b2C: Inter-Integrated Circuit) protocol is used to configure the codec (sample rate, input and output pins etc.) and the I2S (or I\u00b2S: Inter-IC Sound) is used to transfer samples bit by bit in both direction (i.e. from and to the teensy). The I2C pins SDA and SCL are used to control the chip and adjust parameters. Audio data uses I2S signals, TX (to headphones and/or line out) and RX (from line in or mic), and 3 clocks, LRCLK (44.1 kHz), BCLK (1.41 MHz) and MCLK (11.29 MHz). All 3 clocks are created by Teensy which means that the SGTL5000 operates in \"slave mode\". The schematics of the audio shield board, rev. D, can bee seen here and the schematic of the Teensy 4.0 can be seen at the end of the page here . Of course, as they are both made by PJRC, they are designed to be compatible. We (the teachers!) have soldered the connectors so that the audio shiel can be easily connected to the tennsy. The USB connector of the Teensy can support many serial communication from the host computer to the Teensy: (JTAG for flashing/programming, Serial UART, midi, mouse etc. see Tools -> USB Type menu in arduino IDE). In AUD, the USB connector is used to program the device (i.e., download binary code into flash memory) and ascii communication between the host and the Teensy (i.e. using UART/Serial communicatino protocol). In linux machines, when the teensy USB cable is connected, the serial port will appear as /dev/ttyACM0 Teensy 4.0 Processor: NXP i.MX RT1062 The Teensy uses the i.MX RT1062 processor chip from NXP (a model of the serie i.MX RT1060). The main components of the chip can be seen on the image extracted from the i.MX RT1060 datasheet . The processor used in the chip is an ARM Cortex-M7 ( technical reference manuel of Cortex-M7 here ). The ARM Cortex-M7 is the latest architecture that uses the ISA ARMv7 ( ARM v7 reference manuel ) The i.MX RT1060 used in Teensy 4.0 and the associated perifpherals Teensy 4.0 has 2 Mbyte of flash memory intended for storing your code. 1Mbyte of memory is available for execution (i.e., for variables and data storing during execution). Half of this memory (RAM1) is accessed as tightly coupled memory for maximum performance. The other half (RAM2) is optimized for access by DMA. Normally large arrays & data buffers are placed in RAM2, to save the ultra-fast RAM1 for normal variables. The mapping of variables to memories is indicated at the variables declaration by compiler directive (such as DMAMEM for variable in RAM2 or FASTRUN for variable in RAM1, see here . The memory map is the following: Teensy 4.0 pin map (from PJRC webite) Teensy Development Framework: teensyduino The Teensy can be programmed in many ways: Arduino's IDE software with the Teensyduino add-on is the primary programming environment for Teensy. Visual Micro , PlatformIO Makefiles: type make in directory $(arduino)/hardware/teensy/avr/cores/teensy4/ . in AUD we will use most popular Arduino's IDE with Teensyduino . In general, programming the Teensy amounts to compile an application to an executable ( main.elf usually) and then download the application on the teensy which is connected through its USB interface to your PC. The teensyduino software add the source file to arduino in order to compile code for the teensy and call the teensy_loader that flashes the main.elf in the connected Teensy. Cloning AUD github repository You will need to clone AUD github repository which is at https://github.com/grame-cncm/embaudio . For that use the following command on Linux: git clone https://github.com/grame-cncm/embaudio.git AUD The file will now be available in the AUD directory on your computer, you will need in particulat to install the mydsp library in arduino. This library will be found in AUD/examples/teensy/libraries/mydsp/ Installing Arduino/Teensyduino on Your Computer The toolchain can be installed on Macintosh, Linux or Windows platform, we recommend that you install it on your own machine, however it is also installed on TC machines. The Arduino version to install shoudl be 1.8.19, more recent version may provide probles (on linux at least) The installation procedure is the following: Install arduino from here: https://www.arduino.cc/en/software . Warning, make sure to install one of the supported version : 1.8.5, 1.8.9, 1.8.13, 1.8.15, 1.8.16 or (recommended:) 1.8.19. Download the installer corresponding to your OS (click \"just download\" if you do not want to donate), This installer is just an archive containing all the arduino files. Hence place it in an appropriate directory on your drive, Then unzip or untar it.The directory created ( /home/mylogin/arduino-1.8.19 for instance) will become your $ARDUINOPATH variable. Install teensyduino following these instructions: https://www.pjrc.com/teensy/td_download.html If you are on Linux, make sure to install the udev rules, the udev rules file is available here: https://www.pjrc.com/teensy/00-teensy.rules . execute the command: sudo cp 00-teensy.rules /etc/udev/rules.d Clone embaudio github repository (if not done already): https://github.com/grame-cncm/embaudio You will need some files written specifically for the AUD cours: the examples/teensy/libraries/mydsp directory available in the github cloned . Copy the mydsp directory in the directory $ARDUINOPATH/libraries Getting Started on TC Machines Arduino is installed in directory /opt/arduino-1.8.19 . Launching arduino is done simply by typing the command arduino in a command line shell. However, the mydsp library must be made available to arduino. For that, you have to select a directory for additionnal arduino library, for instance /home/mylogin/Arduino and indicate it to arduino par writing the directory path in file->preference->sketchbook location . Then copy the mydsp directory in the /home/mylogin/Arduino directory. Flashing the LED and Using Serial Terminal For programming the teensy: Connect the teensy on your computer or on the TC machine Launch arduino: > arduino Make sure that the correct board is selected: select Tools -> Boards -> Teensyduino -> Teensy 4.0 Make sure that the serial port is configured correctly: select Tools -> USB Type -> Serial Select the flashing led example: select File -> Examples -> 01.Basic -> Blink . A new arduino editor with the 'Blink' application code. Compile and download the code by clickin in 'Upload' button in ardiuno (big arrow). This should launch the teensy_loader, and upload the code, then the led should be blinking. Select the serial communication example: select File -> Examples -> Teensy -> Serial -> EchoBoth . A new arduino editor with the 'EchoBoth' application code. Compile and download the code. Launch the serial monitor window from arduino (magnifying glass on the upper right). This should launch a new window showing serial communcations. Type some characters in the windown and send them (i.e. type 'return'), what is happening? Try to understand the code of the EchoBoth application. Audio applications on Teensy The application prepared for the AUD course are available in the embaudio git repository ( examples/teensy/projects directory). Open ( File -> open... in arduino) the crazy-sine/crazy-sine.ino project. Download it to teensy and ear the crazy sine.","title":" Course Introduction and Programming Environment Setup "},{"location":"lectures/intro/#course-introduction-and-programming-environment-setup","text":"This lecture is devoted to installing the software suite used in this course so that everybody can follow the other lectures from Insa or from his home if it needs to be done in distant work. The slides for the first courses can be found here","title":"Course Introduction and Programming Environment Setup"},{"location":"lectures/intro/#introduction-to-aud2020-and-teensy","text":"Teensy 4.0, and the associated audio adaptor board Most of the document on this course come from PJRC . Actually most of the documentation on Teensy is from PJRC . The development in AUD are performed on teensy which is developped by PJRC. It is a microcontroller that offers many I/O pins and a USB interface. It is programmed using a custom/modified version of the arduino programming environment ( teensyduino ). Teensy is a brand of microcontroller development boards designed by the co-owner of PJRC, Paul Stoffregen . The first Teensy 2.0, Teensy++ 2.0 (and discontinued predecessors) use an 8-bit AVR microcontrollers. Teensy 3.0 (and up) have instead Freescale microcontrollers, running ARM Cortex-M CPUs. The technical characteritics of all Teensy can be compared here . In AUD, we use the Teensy 4.0 which contains an ARM Cortex-M7 at 600 MHz with a Floating point unit, hence it can handle non trivial audio treatment.","title":"Introduction to AUD2020 and Teensy"},{"location":"lectures/intro/#teensy-40-and-audio-shield-from-pjrc-website","text":"Teensy 4.0 uses many powerful CPU features useful for true real-time microcontroller platform. The CPU is an ARM Cortex-M7 dual-issue superscaler clocked at at 600 MHz. CPU performance is many times faster than typical 32 bit microcontrollers. The Floating Point Unit performs 32 bit float and 64 bit double precision math in hardware. DSP extension instructions accelerate signal processing, filters and Fourier transform. The Audio library automatically makes uses of these DSP instructions. Teensy performance from Core Benchmarks This pinout reference card comes with the Teensy 4.0 ( do not loose it! ). The pins are not 5V tolerant. Do not drive any digital pin higher than 3.3V. Teensy 4.0 pin map (from PJRC webite) The Teensy 4.0 has a total of 40 input/output signal pins. 24 are easily accessible when used with a solderless breadboard. The available pins include general purpose IO (GPIO, digital or analog, i.e. ADC), as well as integrated serial protocols (I2C, I2S, CAN, SPI and UART protocols) that are used to connect to other devices. In AUD, we use the audio adaptor board provided by PJRC that integrates a low power stereo codex (NXP Semiconductors SGTL5000 codec) and a SD card reader. Teensy audio adaptor (rev. D) from [PRJC](https://www.pjrc.com/store/teensy40.html), The audio codec connects to Teensy using 7 signals (Yellow signal in pin map above) which are used by two protocol: I2C and I2S. This is a traditional configuration for audio codec: the I2C (or I\u00b2C: Inter-Integrated Circuit) protocol is used to configure the codec (sample rate, input and output pins etc.) and the I2S (or I\u00b2S: Inter-IC Sound) is used to transfer samples bit by bit in both direction (i.e. from and to the teensy). The I2C pins SDA and SCL are used to control the chip and adjust parameters. Audio data uses I2S signals, TX (to headphones and/or line out) and RX (from line in or mic), and 3 clocks, LRCLK (44.1 kHz), BCLK (1.41 MHz) and MCLK (11.29 MHz). All 3 clocks are created by Teensy which means that the SGTL5000 operates in \"slave mode\". The schematics of the audio shield board, rev. D, can bee seen here and the schematic of the Teensy 4.0 can be seen at the end of the page here . Of course, as they are both made by PJRC, they are designed to be compatible. We (the teachers!) have soldered the connectors so that the audio shiel can be easily connected to the tennsy. The USB connector of the Teensy can support many serial communication from the host computer to the Teensy: (JTAG for flashing/programming, Serial UART, midi, mouse etc. see Tools -> USB Type menu in arduino IDE). In AUD, the USB connector is used to program the device (i.e., download binary code into flash memory) and ascii communication between the host and the Teensy (i.e. using UART/Serial communicatino protocol). In linux machines, when the teensy USB cable is connected, the serial port will appear as /dev/ttyACM0","title":"Teensy 4.0 and Audio Shield (from PJRC Website)"},{"location":"lectures/intro/#teensy-40-processor-nxp-imx-rt1062","text":"The Teensy uses the i.MX RT1062 processor chip from NXP (a model of the serie i.MX RT1060). The main components of the chip can be seen on the image extracted from the i.MX RT1060 datasheet . The processor used in the chip is an ARM Cortex-M7 ( technical reference manuel of Cortex-M7 here ). The ARM Cortex-M7 is the latest architecture that uses the ISA ARMv7 ( ARM v7 reference manuel ) The i.MX RT1060 used in Teensy 4.0 and the associated perifpherals Teensy 4.0 has 2 Mbyte of flash memory intended for storing your code. 1Mbyte of memory is available for execution (i.e., for variables and data storing during execution). Half of this memory (RAM1) is accessed as tightly coupled memory for maximum performance. The other half (RAM2) is optimized for access by DMA. Normally large arrays & data buffers are placed in RAM2, to save the ultra-fast RAM1 for normal variables. The mapping of variables to memories is indicated at the variables declaration by compiler directive (such as DMAMEM for variable in RAM2 or FASTRUN for variable in RAM1, see here . The memory map is the following: Teensy 4.0 pin map (from PJRC webite)","title":"Teensy 4.0 Processor: NXP i.MX RT1062"},{"location":"lectures/intro/#teensy-development-framework-teensyduino","text":"The Teensy can be programmed in many ways: Arduino's IDE software with the Teensyduino add-on is the primary programming environment for Teensy. Visual Micro , PlatformIO Makefiles: type make in directory $(arduino)/hardware/teensy/avr/cores/teensy4/ . in AUD we will use most popular Arduino's IDE with Teensyduino . In general, programming the Teensy amounts to compile an application to an executable ( main.elf usually) and then download the application on the teensy which is connected through its USB interface to your PC. The teensyduino software add the source file to arduino in order to compile code for the teensy and call the teensy_loader that flashes the main.elf in the connected Teensy.","title":"Teensy Development Framework: teensyduino"},{"location":"lectures/intro/#cloning-aud-github-repository","text":"You will need to clone AUD github repository which is at https://github.com/grame-cncm/embaudio . For that use the following command on Linux: git clone https://github.com/grame-cncm/embaudio.git AUD The file will now be available in the AUD directory on your computer, you will need in particulat to install the mydsp library in arduino. This library will be found in AUD/examples/teensy/libraries/mydsp/","title":"Cloning AUD github repository"},{"location":"lectures/intro/#installing-arduinoteensyduino-on-your-computer","text":"The toolchain can be installed on Macintosh, Linux or Windows platform, we recommend that you install it on your own machine, however it is also installed on TC machines. The Arduino version to install shoudl be 1.8.19, more recent version may provide probles (on linux at least) The installation procedure is the following: Install arduino from here: https://www.arduino.cc/en/software . Warning, make sure to install one of the supported version : 1.8.5, 1.8.9, 1.8.13, 1.8.15, 1.8.16 or (recommended:) 1.8.19. Download the installer corresponding to your OS (click \"just download\" if you do not want to donate), This installer is just an archive containing all the arduino files. Hence place it in an appropriate directory on your drive, Then unzip or untar it.The directory created ( /home/mylogin/arduino-1.8.19 for instance) will become your $ARDUINOPATH variable. Install teensyduino following these instructions: https://www.pjrc.com/teensy/td_download.html If you are on Linux, make sure to install the udev rules, the udev rules file is available here: https://www.pjrc.com/teensy/00-teensy.rules . execute the command: sudo cp 00-teensy.rules /etc/udev/rules.d Clone embaudio github repository (if not done already): https://github.com/grame-cncm/embaudio You will need some files written specifically for the AUD cours: the examples/teensy/libraries/mydsp directory available in the github cloned . Copy the mydsp directory in the directory $ARDUINOPATH/libraries","title":"Installing Arduino/Teensyduino on Your Computer"},{"location":"lectures/intro/#getting-started-on-tc-machines","text":"Arduino is installed in directory /opt/arduino-1.8.19 . Launching arduino is done simply by typing the command arduino in a command line shell. However, the mydsp library must be made available to arduino. For that, you have to select a directory for additionnal arduino library, for instance /home/mylogin/Arduino and indicate it to arduino par writing the directory path in file->preference->sketchbook location . Then copy the mydsp directory in the /home/mylogin/Arduino directory.","title":"Getting Started on TC Machines"},{"location":"lectures/intro/#flashing-the-led-and-using-serial-terminal","text":"For programming the teensy: Connect the teensy on your computer or on the TC machine Launch arduino: > arduino Make sure that the correct board is selected: select Tools -> Boards -> Teensyduino -> Teensy 4.0 Make sure that the serial port is configured correctly: select Tools -> USB Type -> Serial Select the flashing led example: select File -> Examples -> 01.Basic -> Blink . A new arduino editor with the 'Blink' application code. Compile and download the code by clickin in 'Upload' button in ardiuno (big arrow). This should launch the teensy_loader, and upload the code, then the led should be blinking. Select the serial communication example: select File -> Examples -> Teensy -> Serial -> EchoBoth . A new arduino editor with the 'EchoBoth' application code. Compile and download the code. Launch the serial monitor window from arduino (magnifying glass on the upper right). This should launch a new window showing serial communcations. Type some characters in the windown and send them (i.e. type 'return'), what is happening? Try to understand the code of the EchoBoth application.","title":"Flashing the LED and Using Serial Terminal"},{"location":"lectures/intro/#audio-applications-on-teensy","text":"The application prepared for the AUD course are available in the embaudio git repository ( examples/teensy/projects directory). Open ( File -> open... in arduino) the crazy-sine/crazy-sine.ino project. Download it to teensy and ear the crazy sine.","title":"Audio applications on Teensy"},{"location":"lectures/os/","text":"Embedded Operating Systems This course will present the important notions of embedded operating systems and show how a primitive operating system (protothreads) can be deployed on teensy. Slides Slides are available here A teensy program with protothreads Download the protothreads_blink.tar archive and uncompress it. Have a look at the protothread_blink.ino program. This programs defines two protothreads: thread_red_led that switch the LED state every seconds, and thread_sine that changes the sine tune every 0.25 seconds. Both these thread are unlock by a dedicated timer. Can you understand how it works? This example, of course, is a bit trivial because the changes in the LED state and the sine tune could have been done in timer callbacks. But, first, we try to avoid writing to much code in the call back and, second, one has to imagine other conditions for unlocking the threads such as internal variable states or input on UART... What happens if one PT_WAIT_UNTIL is commented out? For instance in thread_led_red : //PT_WAIT_UNTIL(pt, TIMER_LED_RED_ON == 1); Note finaly, note that many variables have been set as global variables as local protothread variables are not remanent. Add a UART printing copy this protothreads_blink directory to a new protothreads_count and modify the program to print on UART serial port, every second, the number of blinking of the LED since the beginning of the program. This imply the following tasks (see UART teensy Documentation ): Initialize serial communication at 9600 bauds using Serial.begin(9600); Declare a global variable blink_count Declare a new timer counting each second (up to four interval timers can be used in teensy 4.0) Increment blink_count at each blink Print (using Serial.print ) the value of blink_count in new timer callback. Use the command minicom -D /dev/ttyACM0 to visualize what is send on serial port by the teensy. For the moment we do not really need a new protothread, as we can assume that two calls to Serial.printf are fast enough not to perturbate audio. Imagine that we want to control our audio device using the keyboard. Receiving a caracter from the keyboard and performing the action necessary might be too long for an interrupt handle function. Hence we will add a protothread to do that. Control LED and sound with keyboard copy this protothreads_count directory to a new protothreads_control declare a new protothread `static PT_THREAD(thread_receive(struct pt *pt)) that will receive characters on UART serial port for the host computer. the condition in the PT_WAIT_UNTIL will be Serial.available()>0 (indicating that the receiving buffer contains some characters, see UART teensy Documentation ). As soon as a character is typed, echo its ascii code. Use that mecanism to control the sound and LED: Typing 's' will switch off/on the sound (i.e. setting `myDsp.setFreq(0) ) Typing 'o' will switch off/on the LED blinking. Hint: you can switch the value of a variable var between 0 and 1 using the Xor operator: var = var ^ 1; ** Solution **","title":" Embedded OS, FreeRTOS, Embedded Linux Devices "},{"location":"lectures/os/#embedded-operating-systems","text":"This course will present the important notions of embedded operating systems and show how a primitive operating system (protothreads) can be deployed on teensy.","title":"Embedded Operating Systems"},{"location":"lectures/os/#slides","text":"Slides are available here","title":"Slides"},{"location":"lectures/os/#a-teensy-program-with-protothreads","text":"Download the protothreads_blink.tar archive and uncompress it. Have a look at the protothread_blink.ino program. This programs defines two protothreads: thread_red_led that switch the LED state every seconds, and thread_sine that changes the sine tune every 0.25 seconds. Both these thread are unlock by a dedicated timer. Can you understand how it works? This example, of course, is a bit trivial because the changes in the LED state and the sine tune could have been done in timer callbacks. But, first, we try to avoid writing to much code in the call back and, second, one has to imagine other conditions for unlocking the threads such as internal variable states or input on UART... What happens if one PT_WAIT_UNTIL is commented out? For instance in thread_led_red : //PT_WAIT_UNTIL(pt, TIMER_LED_RED_ON == 1); Note finaly, note that many variables have been set as global variables as local protothread variables are not remanent.","title":"A teensy program with protothreads"},{"location":"lectures/os/#add-a-uart-printing","text":"copy this protothreads_blink directory to a new protothreads_count and modify the program to print on UART serial port, every second, the number of blinking of the LED since the beginning of the program. This imply the following tasks (see UART teensy Documentation ): Initialize serial communication at 9600 bauds using Serial.begin(9600); Declare a global variable blink_count Declare a new timer counting each second (up to four interval timers can be used in teensy 4.0) Increment blink_count at each blink Print (using Serial.print ) the value of blink_count in new timer callback. Use the command minicom -D /dev/ttyACM0 to visualize what is send on serial port by the teensy. For the moment we do not really need a new protothread, as we can assume that two calls to Serial.printf are fast enough not to perturbate audio. Imagine that we want to control our audio device using the keyboard. Receiving a caracter from the keyboard and performing the action necessary might be too long for an interrupt handle function. Hence we will add a protothread to do that.","title":"Add a UART printing"},{"location":"lectures/os/#control-led-and-sound-with-keyboard","text":"copy this protothreads_count directory to a new protothreads_control declare a new protothread `static PT_THREAD(thread_receive(struct pt *pt)) that will receive characters on UART serial port for the host computer. the condition in the PT_WAIT_UNTIL will be Serial.available()>0 (indicating that the receiving buffer contains some characters, see UART teensy Documentation ). As soon as a character is typed, echo its ascii code. Use that mecanism to control the sound and LED: Typing 's' will switch off/on the sound (i.e. setting `myDsp.setFreq(0) ) Typing 'o' will switch off/on the LED blinking. Hint: you can switch the value of a variable var between 0 and 1 using the Xor operator: var = var ^ 1; ** Solution **","title":"Control LED and sound with keyboard"},{"location":"lectures/project/","text":"Personnal Project: Your Own LyraT Program The remaining course (approximately 3 TD of 2h) will be dedicated to a personal project that you will implement on LyraT, the idea it to explore further one aspect of the LyraT programming and to present it to the other students in a small d\u00e9mo during the last course. Possible Projects You are encouraged to proposed your own project, here are some idea of topics that can be studied. Note that the project will be done in a very short time , make sure to be able to present something, even if it is juste a study for a future implementation Explore Digital Synthesis or effect on the web (e.g., https://en.wikipedia.org/wiki/Category:Sound_synthesis_types , https://www.dafx.de/ , https://en.wikipedia.org/wiki/Sound_effect#techniques , https://ccrma.stanford.edu/~jos/ , etc.), and implement a particular algorithm Provide an OSC control via udp connection with Wifi Provide a BlueTooth/midi controler such as the one implemented here ( https://github.com/midibox/esp32-idf-blemidi/tree/master/components/blemidi ) Provide a USB/midi controler with an external midi/uSB device Think of a funny application using wireless communications between several LyraT Provide an HTTP server that can be used via wifi to control the LyraT program Explore (as above) digital synthesis of effect but in Faust to LyraT Rules: One or two students per project project should be presented to the class \"as it is\" on last course in less than 10mn (e.g. three slides and a demo)","title":"Personnal Project: Your Own LyraT Program"},{"location":"lectures/project/#personnal-project-your-own-lyrat-program","text":"The remaining course (approximately 3 TD of 2h) will be dedicated to a personal project that you will implement on LyraT, the idea it to explore further one aspect of the LyraT programming and to present it to the other students in a small d\u00e9mo during the last course.","title":"Personnal Project: Your Own LyraT Program"},{"location":"lectures/project/#possible-projects","text":"You are encouraged to proposed your own project, here are some idea of topics that can be studied. Note that the project will be done in a very short time , make sure to be able to present something, even if it is juste a study for a future implementation Explore Digital Synthesis or effect on the web (e.g., https://en.wikipedia.org/wiki/Category:Sound_synthesis_types , https://www.dafx.de/ , https://en.wikipedia.org/wiki/Sound_effect#techniques , https://ccrma.stanford.edu/~jos/ , etc.), and implement a particular algorithm Provide an OSC control via udp connection with Wifi Provide a BlueTooth/midi controler such as the one implemented here ( https://github.com/midibox/esp32-idf-blemidi/tree/master/components/blemidi ) Provide a USB/midi controler with an external midi/uSB device Think of a funny application using wireless communications between several LyraT Provide an HTTP server that can be used via wifi to control the LyraT program Explore (as above) digital synthesis of effect but in Faust to LyraT","title":"Possible Projects"},{"location":"lectures/project/#rules","text":"One or two students per project project should be presented to the class \"as it is\" on last course in less than 10mn (e.g. three slides and a demo)","title":"Rules:"},{"location":"lectures/teensy-faust/","text":"Faust on the Teensy and Advanced Control Generating and Using a Faust C++ Object In order to run the examples in this lecture, you should install the Faust distribution on your system from the Faust Git Repository . At the most fundamental level, the Faust compiler is a command line tool translating a Faust DSP object into C++ code. For example, assuming that Faust is properly installed on your system, given the following simple Faust program implementing a filtered sawtooth wave oscillator ( FaustSynth.dsp ): import(\"stdfaust.lib\"); freq = nentry(\"freq\",200,50,1000,0.01); gain = nentry(\"gain\",0.5,0,1,0.01) : si.smoo; gate = button(\"gate\") : si.smoo; cutoff = nentry(\"cutoff\",10000,50,10000,0.01) : si.smoo; process = os.sawtooth(freq)*gain*gate : fi.lowpass(3,cutoff) <: _,_; running: faust FaustSynth.dsp will output the C++ code corresponding to this file in the terminal. Faust comes with a system of C++ wrapper (called architectures in the Faust ecosystem) which can be used to customize the generated C++ code. faustMininal.h is a minimal architecture file including some C++ objects that can be used to facilitate interactions with the generated DSP: #include #include #include \"faust/gui/MapUI.h\" #include \"faust/gui/meta.h\" #include \"faust/dsp/dsp.h\" // BEGIN-FAUSTDSP <> <> // END-FAUSTDSP For instance, MapUI allows us to access the parameters of a Faust DSP object using the setParamValue method, etc. To generate a C++ file using this architecture, you can run: faust -i -a faustMinial.h FaustSynth.dsp -o FaustSynth.h which will produce a FaustSynth.h file (feel free to click on it). The -i inlines all the included C++ .h files in the generated file. The faust-synth Teensy example project demonstrates how FaustSynth.h can be used. First, it is included in MyDsp.cpp and the following elements are declared in the corresponding header file : private: MapUI* fUI; dsp* fDSP; float **outputs; dsp is the actual Faust DSP, MapUI will be used to interact with it, and outputs is the multidimensional output buffer. These objects are then allocated in the constructor of MyDsp.cpp : fDSP = new mydsp(); fDSP->init(AUDIO_SAMPLE_RATE_EXACT); fUI = new MapUI(); fDSP->buildUserInterface(fUI); outputs = new float*[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; ++channel){ outputs[channel] = new float[AUDIO_BLOCK_SAMPLES]; } buildUserInterface is used to connect fUI to fDSP and then memory is allocated for the output buffer. Note that memory should be de-allocated in the destructor after this. In the update method, we just call the compute method of fDSP and then reformat the generated samples to transmit them via i2s: void MyDsp::update(void) { fDSP->compute(AUDIO_BLOCK_SAMPLES,NULL,outputs); audio_block_t* outBlock[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; channel++) { outBlock[channel] = allocate(); if (outBlock[channel]) { for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { int16_t val = outputs[channel][i]*MULT_16; outBlock[channel]->data[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } Note that outputs is used as an intermediate here and the first dimension of the array is the channel number and the second dimension the samples themselves. The various parameters of the Faust object can then be changed just by calling the setParamValue method. The first argument of the method corresponds to the name of the parameter as specified in the Faust program: void MyDsp::setFreq(float freq){ fUI->setParamValue(\"freq\",freq); } void MyDsp::setCutoff(float freq){ fUI->setParamValue(\"cutoff\",freq); } void MyDsp::setGate(int gate){ fUI->setParamValue(\"gate\",gate); } Better Control on the Teensy In addition to controlling DSP parameters on the Teensy using external sensors connected to the board's GPIOs, other techniques can potentially be used. We briefly summarize this section. MIDI MIDI is THE standard in the world of music to control digital devices. It has been around since 1983 and even though it is very \"low tech,\" it is still heavily used. While MIDI was traditionally transmitted over MIDI ports, USB is used nowadays to send MIDI. USB MIDI is natively supported on the Teensy through the Teensy USB MIDI library: https://www.pjrc.com/teensy/td_midi.html . Interfacing this library with your DSP programs should be very straightforward. Please, also note that Teensys can be used to send MIDI messages over USB which means that implementing your own midi controller using a Teensy is fairly straightforward as well. If you're curious about this, you can check this page: https://ccrma.stanford.edu/courses/250a-winter-2018/labs/2/ . OSC Open Sound Control (OSC) is a more modern communication standard used in the field of music technology. It is based on UDP which means that information can be transmitted via Ethernet or Wi-Fi. OSC uses a system of address/values to access the different parameters of a system. An OSC message can therefore look like: /synth/freq 440 On the Teensy, dealing with OSC is a bit more tricky than MIDI because the Teensy 4.0 provided as part of your class kit don't have a built-in Ethernet port. Hence, the only way to get an Ethernet connection to the Teensy is to buy an external Ethernet adapter (that will likely connect to the Teensy through i2c, etc.). Another option is to buy a Teensy 4.1 which hosts an Ethernet chip (an Ethernet connector can just be soldered to the board). Exercises Faust Triangle Oscillator The Faust libraries host a triangle wave oscillator: os.triangle Try to replace the sawtooth wave oscillator from the previous example by a triangle wave oscillator in Faust and run it on the Teensy. Flanger The Faust libraries host a flanger function : pf.flanger_mono Turn your Teensy into a flanger effect processor!","title":" Faust on the Teensy and Advanced Control "},{"location":"lectures/teensy-faust/#faust-on-the-teensy-and-advanced-control","text":"","title":"Faust on the Teensy and Advanced Control"},{"location":"lectures/teensy-faust/#generating-and-using-a-faust-c-object","text":"In order to run the examples in this lecture, you should install the Faust distribution on your system from the Faust Git Repository . At the most fundamental level, the Faust compiler is a command line tool translating a Faust DSP object into C++ code. For example, assuming that Faust is properly installed on your system, given the following simple Faust program implementing a filtered sawtooth wave oscillator ( FaustSynth.dsp ): import(\"stdfaust.lib\"); freq = nentry(\"freq\",200,50,1000,0.01); gain = nentry(\"gain\",0.5,0,1,0.01) : si.smoo; gate = button(\"gate\") : si.smoo; cutoff = nentry(\"cutoff\",10000,50,10000,0.01) : si.smoo; process = os.sawtooth(freq)*gain*gate : fi.lowpass(3,cutoff) <: _,_; running: faust FaustSynth.dsp will output the C++ code corresponding to this file in the terminal. Faust comes with a system of C++ wrapper (called architectures in the Faust ecosystem) which can be used to customize the generated C++ code. faustMininal.h is a minimal architecture file including some C++ objects that can be used to facilitate interactions with the generated DSP: #include #include #include \"faust/gui/MapUI.h\" #include \"faust/gui/meta.h\" #include \"faust/dsp/dsp.h\" // BEGIN-FAUSTDSP <> <> // END-FAUSTDSP For instance, MapUI allows us to access the parameters of a Faust DSP object using the setParamValue method, etc. To generate a C++ file using this architecture, you can run: faust -i -a faustMinial.h FaustSynth.dsp -o FaustSynth.h which will produce a FaustSynth.h file (feel free to click on it). The -i inlines all the included C++ .h files in the generated file. The faust-synth Teensy example project demonstrates how FaustSynth.h can be used. First, it is included in MyDsp.cpp and the following elements are declared in the corresponding header file : private: MapUI* fUI; dsp* fDSP; float **outputs; dsp is the actual Faust DSP, MapUI will be used to interact with it, and outputs is the multidimensional output buffer. These objects are then allocated in the constructor of MyDsp.cpp : fDSP = new mydsp(); fDSP->init(AUDIO_SAMPLE_RATE_EXACT); fUI = new MapUI(); fDSP->buildUserInterface(fUI); outputs = new float*[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; ++channel){ outputs[channel] = new float[AUDIO_BLOCK_SAMPLES]; } buildUserInterface is used to connect fUI to fDSP and then memory is allocated for the output buffer. Note that memory should be de-allocated in the destructor after this. In the update method, we just call the compute method of fDSP and then reformat the generated samples to transmit them via i2s: void MyDsp::update(void) { fDSP->compute(AUDIO_BLOCK_SAMPLES,NULL,outputs); audio_block_t* outBlock[AUDIO_OUTPUTS]; for (int channel = 0; channel < AUDIO_OUTPUTS; channel++) { outBlock[channel] = allocate(); if (outBlock[channel]) { for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { int16_t val = outputs[channel][i]*MULT_16; outBlock[channel]->data[i] = val; } transmit(outBlock[channel], channel); release(outBlock[channel]); } } } Note that outputs is used as an intermediate here and the first dimension of the array is the channel number and the second dimension the samples themselves. The various parameters of the Faust object can then be changed just by calling the setParamValue method. The first argument of the method corresponds to the name of the parameter as specified in the Faust program: void MyDsp::setFreq(float freq){ fUI->setParamValue(\"freq\",freq); } void MyDsp::setCutoff(float freq){ fUI->setParamValue(\"cutoff\",freq); } void MyDsp::setGate(int gate){ fUI->setParamValue(\"gate\",gate); }","title":"Generating and Using a Faust C++ Object"},{"location":"lectures/teensy-faust/#better-control-on-the-teensy","text":"In addition to controlling DSP parameters on the Teensy using external sensors connected to the board's GPIOs, other techniques can potentially be used. We briefly summarize this section.","title":"Better Control on the Teensy"},{"location":"lectures/teensy-faust/#midi","text":"MIDI is THE standard in the world of music to control digital devices. It has been around since 1983 and even though it is very \"low tech,\" it is still heavily used. While MIDI was traditionally transmitted over MIDI ports, USB is used nowadays to send MIDI. USB MIDI is natively supported on the Teensy through the Teensy USB MIDI library: https://www.pjrc.com/teensy/td_midi.html . Interfacing this library with your DSP programs should be very straightforward. Please, also note that Teensys can be used to send MIDI messages over USB which means that implementing your own midi controller using a Teensy is fairly straightforward as well. If you're curious about this, you can check this page: https://ccrma.stanford.edu/courses/250a-winter-2018/labs/2/ .","title":"MIDI"},{"location":"lectures/teensy-faust/#osc","text":"Open Sound Control (OSC) is a more modern communication standard used in the field of music technology. It is based on UDP which means that information can be transmitted via Ethernet or Wi-Fi. OSC uses a system of address/values to access the different parameters of a system. An OSC message can therefore look like: /synth/freq 440 On the Teensy, dealing with OSC is a bit more tricky than MIDI because the Teensy 4.0 provided as part of your class kit don't have a built-in Ethernet port. Hence, the only way to get an Ethernet connection to the Teensy is to buy an external Ethernet adapter (that will likely connect to the Teensy through i2c, etc.). Another option is to buy a Teensy 4.1 which hosts an Ethernet chip (an Ethernet connector can just be soldered to the board).","title":"OSC"},{"location":"lectures/teensy-faust/#exercises","text":"","title":"Exercises"},{"location":"lectures/teensy-faust/#faust-triangle-oscillator","text":"The Faust libraries host a triangle wave oscillator: os.triangle Try to replace the sawtooth wave oscillator from the previous example by a triangle wave oscillator in Faust and run it on the Teensy.","title":"Faust Triangle Oscillator"},{"location":"lectures/teensy-faust/#flanger","text":"The Faust libraries host a flanger function : pf.flanger_mono Turn your Teensy into a flanger effect processor!","title":"Flanger"}]} \ No newline at end of file diff --git a/docs/sitemap.xml b/docs/sitemap.xml index d21b6f4..af9d9c8 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -2,62 +2,62 @@ https://embaudio.grame.fr/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/architecture/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/control/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/digital-audio/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/dsp1/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/dsp2/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/embedded/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/faust/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/intro/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/os/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/project/ - 2023-11-03 + 2023-11-06 daily https://embaudio.grame.fr/lectures/teensy-faust/ - 2023-11-03 + 2023-11-06 daily \ No newline at end of file diff --git a/docs/sitemap.xml.gz b/docs/sitemap.xml.gz index 50bf595..29f0f96 100644 Binary files a/docs/sitemap.xml.gz and b/docs/sitemap.xml.gz differ diff --git a/web/docs/lectures/dsp1.md b/web/docs/lectures/dsp1.md index e3acbb8..2be1ec9 100644 --- a/web/docs/lectures/dsp1.md +++ b/web/docs/lectures/dsp1.md @@ -205,7 +205,6 @@ for (int i = 0; i < AUDIO_BLOCK_SAMPLES; i++) { The DX7 carried out frequency modulation over a total of six oscillators that could be patched in [different ways](https://static.righto.com/images/dx7-alg/algorithms-w800.jpg). So FM is not limited to two oscillators... Try to implement an FM synthesizer involving 3 oscillators instead of one. They should be connected in series: 3 -> 2 -> 1. -