User Manual (former)

February 1, 2016

 

Soundcool: User manual.

 

 

 

U0636222.png

 

 

Soundcool: User manual

 

 

LOGO.png

 

 

 

 

 

April, 2016


 

 

Contents

Introduction. 4

Requirements. 5

Installation Instructions. 5

File Compatibility. 7

Links to free audio files and VST instruments. 7

Basic operation. 9

Modules. 9

Relocating a module. 9

Connecting the two modules. 10

Producing the first sounds. 10

Disconnecting modules. 10

Multiple connections between modules. 11

Minimizing modules. 12

Operation of the audio modules. 14

MSpeakers. 14

Save. 14

Load. 14

MPlayer. 15

MDelay. 15

MDirectInput. 16

MKeyboard. 16

MVST. 18

MMixer. 19

MPan. 19

MPitch. 20

MRecord. 20

MFilter. 21

MRouting. 22

MSamplePlayer. 23

MScope. 23

MSpectroscope. 24

MSignalGen. 24

MSequencer. 26

Selection of an MSignalGen module. 26

Parameters for each of the eight boxes. 26

Launching the sequence. 27

MTransposer. 28

MEnvelope. 28

Operation of the video modules. 30

MVDirectInput. 30

MVPlayer. 30

Connecting Smartphones and Tablets using TouchOSC. 32

Configure the OSC menu. 32

Load the Soundcool layout. 32

Add from Editor. 32

Add from File. 33

Configure the module. 34

Example of a typical connection process. 34

Kinect body control of modules. 37

Installation of Kinect drivers for Synapse. 38

Installation process on a MAC. 38

Installation process on a PC. 38

Using Kinect with Soundcool modules through Synapse. 39

Movements for controlling the parameters. 40

Credits. 41

 


 

Introduction

                Soundcool is a collaborative system for sound and music creation using mobile phones, tablets, Kinect and MAX / MSP / Jitter. Soundcool has been introduced into educational centers at primary and secondary levels. The core of the Soundcool system is available for free. Soundcool is being developed at the “Universitat Politècnica de València, UPV” (Spain). The key projects relating Soundcool are:

  • Daniel and Nina Carasso Foundation grant 16-AC-2016: VideoArt software developmets 2016-2019.

14158-logo-es-afilada-sanstexte-hd-03

  • Soundcool has been adopted in the European project Erasmus+ KA201 “Technology to support learning and creativity: building European networks through collaborative music creation“ (“Tecnología al servicio del aprendizaje y la creatividad: tejiendo redes europeas a través de la creación musical colaborativa”, 2015-2017), led by Dr. Elizabeth Carrascosa, and with the following reference 2015-1-ES01-KA201-016139. As a result, Soundcool has been introduced into several European educational centers.

  • Generalitat Valenciana (Spain) grant AICO/2015/120: “New Technologies for Music Education and Sound Creation” (2015-2016), led by Dr. Jorge Sastre.  This project aims to continue developing and improving Soundcool as a modular software system based on low-cost interfaces such as tablets, smartphones and Kinect.

  • Universitat Politècnica de València (Spain) project PAID-05-12-SP20120470: “New Audiovisual Technologies and Interfaces for Education in Music and Sound Creation” (2013), directed by Dr. Jorge Sastre. This project led to the birth of Soundcool.

The following are two key publications:

          “New Technologies for Music Education”, Sastre, J. Cerdà, W. García, C.A. Hernández, N. Lloret, A. Murillo, D. Picó, J. E. Serrano, S. Scarani, R.B. Dannenberg. Presented on the 2nd International Conference on E-Learning and E-Technologies in Education (ICEEE2013), and considered one of the most relevant papers of the conference. Available at:

https://www.researchgate.net/publication/259900796_New_Technologies_for_Music_Education

 

          “Soundcool: New Technologies for Music Education”, J. Sastre, A. Murillo, E. Carrascosa, R. García, R.B. Dannenberg, N. Lloret, R. Morant, S. Scarani, A. Muñoz. Presented on the International Conference of Education, Research and Innovation ICERI2015. Available at:

https://www.researchgate.net/publication/284183213_SOUNDCOOL:_NEW_TECHNOLOGIES_FOR_MUSIC_EDUCATION

Back to Top

Requirements

                For the Soundcool modules to operate properly, the following components are needed:

      Max Runtime 6. This program opens the Soundcool modules. It is free and can be found here (you must use Max Runtime 6.1.10, not Max 7):

 https://cycling74.com/downloads/runtime

       QuickTime. This program is necessary because Max Runtime uses it to playback audio and video files. If it hasn’t been installed, the modules may not be able to play these files. The program is free and can be downloaded here:

http:/www.quicktime.com

    TouchOSC. Mobile application. It enables wireless control of modules, sending messages and receiving feedback from remote devices. The app costs 4.99€ ($4.99) in your app store (currently available only for Android in Google Play and iOS in App Store). The use of TouchOSC is a temporary solution that will be substituted in the future by a free app that we are currently developing.

         TouchOSC Editor. Program to quickly load the Soundcool layout onto mobile devices.  The program is free and can be found here:

http://hexler.net/software/touchosc#downloads

      Kinect. Microsoft’s sensor that controls some Soundcool modules by using physical movement, as though the whole body is an instrument. The device model compatible with Soundcool is Kinect for Xbox 360, Model 1414.

      Synapse. Program to process movement detected by the Kinect sensor. Free download here:

http://synapsekinect.tumblr.com/post/6305020721/download

 

Installation Instructions

1 – Install External Apps

Mac – OS X:

1. Install Max Runtime 6.1.10 (not Max 7) for Mac.

2. Install TouchOSC Editor for Mac.

 

Windows: (check what program versions can run in your computer: 64 bit or 32 bit versions)

1. Install Quicktime for Windows.

2. Install Max Runtime 6.1.10 (not Max 7) for Windows.

3. Install TouchOSC Editor for Windows.

At this point you can use Soundcool on your computer, even without using mobile devices or Kinect. For an example, follow the tutorial described on the “Basic Operation” section below. If you have Kinect for Xbox 360, Model 1414 available, you can install Synapse for using it with Soundcool.

 

2 – Install TouchOSC in your mobile device

[Only Android and iOS devices are supported]

1. Visit http://soundcool.org/downloads from your mobile device and click “Download TouchOSC for Android” or “Download TouchOSC for iOS” or search for “TouchOSC” in your app store.

The app costs 4.99€ ($4.99). As mentioned above, the use of TouchOSC is a temporary solution that will be substituted in the future by a free app that we are currently developing.

 

3 – Install Soundcool system on your computer

1. Drag the folder named Soundcool to your desktop. This folder is in the .ZIP file you downloaded from the Soundcool website.

2. Drag Soundcool_Layout_v1.0.touchosc to your Desktop. This file is in the .ZIP file you downloaded from the Soundcool website.

 

4 – Install Soundcool Layout in your mobile device

1. Be sure your computer and your mobile device(s) are connected to the same wireless/Wi-Fi network.

2. Open TouchOSC Editor, click on “Open” and load the file Soundcool_Layout_v1.0.touchosc from your desktop. Click the Sync button.

3. Open TouchOSC in your mobile device. Go to Layout and click on “Add”. Your Host computer should appear in the list. If not, tap Edit and enter your computer’s internal IP address.

How to get your internal IP adress:

                Windows: https://www.youtube.com/watch?v=HsCD7hUlSjk

                Mac OS X: https://www.youtube.com/watch?v=1mjryJqYSQo

4. Click on the computer’s name and the layout will automatically download.

5. Go back and select the layout Soundcool_Layout_v1.0. Press Done.

Back to Top

File Compatibility

               Soundcool-compatible files are identical to QuickTime-compatible files:
 

Supported audio files

Supported video files

iTunes Audio (.m4a, .m4b, .m4p)

QuickTime Movie (.mov)

MP3

MPEG-4 (.mp4, .m4v)

Core Audio (.caf)

MPEG-2 (OS X Lion or later)

AIFF

MPEG-1

AU

3GPP

SD2

3GPP2

WAV

AVCHD (OS X Mountain Lion or later)

SND

AVI (only Motion JPEG)

AMR

DV

               

                If you want to add more types of compatible files, such as .wma, .wmv, or divx or if Quicktime can’t open a file, then you may need to install additional codecs to QuickTime, by using the following link: https://support.apple.com/en-us/HT201290 or searching “If QuickTime Player can‘t open an audio or video file” in Apple Support https://www.apple.com/support/.

Back to Top

Links to free audio files and VST instruments

Links to free VST instruments:

          HELM VST – http://tytel.org/helm/

          CRYSTAL VST – http://www.greenoak.com/crystal/about.html

          ALTER EGO VST – https://www.plogue.com/products/alterego/

          DSK Asian DreamZ – http://www.dskmusic.com/dsk-asian-dreamz/

 

Links to free samples and sound effects:

          ROYALTY-FREE SOUNDS EFFECTS – https://www.audioblocks.com/royalty-free-audio/sound-effects

          2.000+ FREE SOUND EFFECTS – http://www.audiomicro.com/free-sound-effects

          FREE SOUND FX – http://www.freesfx.co.uk

          Freesound: collaborative database of creative-commons licensed sound for musicians and sound lovers – http://www.freesound.org

 

Back to Top


 

Basic operation

In this section you will be guided to produce your first sounds.

Modules

                The Soundcool system consists of different modules that can be interconnected together to produce different effects and sounds. Modules are typically opened from Max Runtime software. All Soundcool module names start by letter M. Now, you will open two modules, MPlayer and MSpeakers, by following these steps:

          Open Max Runtime (remember you installed it on step 1 of Installation Instructions).

          Go to File menu and select Open

          Browse to the Soundcool System folder (remember you installed it on step 3 of Installation Instructions).

          Select the file named MPlayer

Similarly, open the file named MSpeakers.

Relocating a module

                The modules can be placed on the screen wherever you want them. To change the location, hold the mouse cursor over the name of the module until the hand icon appears mano_open.png. By clicking and dragging, the module can then be moved to wherever desired. While doing this, the cursor will change to the image of a hand holding an object mano_drag.png.

You can arrange the two modules as shown in the following figure:

Figure 1: Two modules opened

Note that all modules generate an identification number, so that if two or more instances of the same module open up, each will be identified with a different number. If all instances of the same module close, the count is reset.

Connecting the two modules

                Soundcool modules can have inputs and/or outputs. To connect the two modules, first click on the output button of MPlayer, labeled out, to activate it. After that, the originally square button becomes round outredondo.png . At the same time, the in button on the modules which are available to be connected with it will become round inredondo.png . At this point, if you click on any of these in buttons, the connection between the two modules will be established. In our example, we only have the in  button of MSpeakers.

                Once connected, the output button will change to having a black background. In addition, the new input button will display the initial and identification number (here P1) and the background color of the module it originated from. The result can be seen in the following figure:

Figure 2: Connecting two modules

Producing the first sounds

To make this arrangement sound, follow these steps:

          Click the Open button of MPlayer .

          Select an audio file that is stored in your computer (Remember that you need to install QuickTime to playback audio files, and see QuickTime-compatible files above).

          Press the Play button MyPlay.png.

Disconnecting modules

                To disconnect two modules, simply hover over the in button. At that moment, an “X” will appear on the button. If you then click, the associated modules will be disconnected.

Figure 3: Disconnecting two modules

                Also, if any module that is part of the connection is closed, the connection is automatically broken.

Multiple connections between modules

                Now, we are going to make another arrangement using four modules. We will connect the MPlayer to two modules at the same time. This is because the output of an out button can be connected to one or more modules. In contrast, the in buttons only accept one connection. If you hold the mouse over the in button of certain module, the module which is connected to it will light up, and if you hover over the out button all the modules which are connected to if will light up.

In addition to the MPlayer and the MSpeakers, please, open the following modules: MPitch and MMixer. Initially, please, disconnect all the modules as explained above. The arrangement is shown in the next figure:

Figure 4: Arrangement disconnected

MPitch is an effect module that changes the pitch of its input. Similarly, MMixer has inputs and one output. Whatever is input into these kind of modules is processed and then output from the same module. In turn, whatever is output from any of those modules (usually audio) may lead into one or more other modules.

Please, make the following connections:

          Mplayer’s output will have a multiple connection: one to the input of MPitch and the other to the input 1 of MMixer. Please, proceed as explained above for multiple connections.

          MPitch’s output will be connected to the input 2 of MMixer.

          MMixer’s output will be connected to the input of MSpeakers.

The arrangement is shown in the following figure:

Figure 5: Arrangement with connections

Now, we have a typical arrangement where the original sound source and its processed version enter a mixer. By changing the levels of the mixer we can control the proportion of original and processed sound that we can listen through the speakers. That is, by increasing the volume of MMixer’s channel 1  you will hear the original sound louder, while by increasing the volume of MMixer’s channel 2  you will hear the effect, or processed sound, louder (move the Pitch 1 slider to change the pitch of the sound played by Player 1).

Please, note that we have not used the mobile devices, yet. That will be covered at the end of the manual.                  

            Minimizing modules

                All modules have a minimize button minimizar.png which compacts all modules to the same size, no matter what size they were when maximized. Once minimized, you can return to the previous size by clicking on the maximize button maximizar.png. The following figure shows how several modules are minimized to the same size:

Figure 6: Minimized modules

The next manual sections describe every Soundcool module in more detail, as a reference manual.

Back to Top


 

Operation of the audio modules

            MSpeakers

                This module corresponds to the audio output of a computer, so it will always be the final point in any configuration made with Soundcool. Audio output can be enabled or disabled instantly, with the speaker buttonaudio.png. There is also a control to select the language used in pop-up help messages (English/Spanish). In addition, there is a Close ALL button to close all the open modules simultaneously. The Audio Settings button opens a new window in which you can configure different parameters related to the computer’s audio card. Finally, you have the Save guarda.png and Load MyOpen_copia.png buttons, which are explained below.

                Save

                The MSpeakers module allows you to save any configuration of the modules that you wish to.  In this context, the Save function saves the names of open modules, the settings between the modules, the value of the parameters of each module, and finally, its location on the screen. To do this, click on the Save guarda.png icon. Then, the classic window for each operating system opens, in which you much choose the folder and the name. Note that, in addition, a popup window appears with the word Saving. This will automatically disappear when the setup is saved.

saving.png

Figure 7: The Saving popup appears during Save

                Load

                The MSpeakers module also allows you to load files that were previously saved. To do this, you must press the folder icon MyOpen_copia.png and select the name of the file. Saved modules and connections, the value of each module, and its location on the screen will be automatically loaded.

Mspeaker.png

Figure 8: The MSpeakers module

Back to Top

 


 

 

            MPlayer

                This module allows you to play audio files. Through the open button, different audio file formats can be loaded: .wav, .aiff, .mp3, etc. As mentioned previously, files created with Windows Media, .wma, are not compatible (see File Compatibility section for instructions on how to play this type of files). Once the audio file you loaded starts playing, you can pause and/or stop it by using the traditional playback control buttons, play MyPlay.png , pause MyPause.png and stopMyStop.png. To the right of these buttons is a third button which toggles between play forward MyForward.png or go backwardmyback.png. In addition, a check box labeled Loop enables looped playback; the vertical slider on the right controls volume; and the horizontal slider, labeled Speed, controls the playback speed. Finally, it also has a time bar that shows the temporal progress of the track being played. If you click anywhere on this bar, playback will move to that point in time. Note that there is not an input button in this module, since it gets its input by opening an audio file, and so, there is only an output button.

Figure 9: MPlayer Module

                The Kinect and OSC Port boxes that appear on this, as well as on most of the following modules, will be explained below.

Back to Top

            MDelay

                This module has two controls: delay and feedback.  With the Delay slider, you can control the delay of the signal that goes to the output. The Feedback slider indicates the proportion of the delayed signal that will be send again to the input to produce new delays. The number boxes above each slider allow you to manually modify the value of each parameter. Among each pair of number boxes the ones on the right side allow you to modify the maximum value of the slider range. To return to slider control, simply click again anywhere on the horizontal bar.

 

Figure 10: MDelay Module

Back to Top

            MDirectInput

                This module outputs the signal picked up directly from a computer’s input device, such as a microphone. It also has volume control. The default audio input of the computer can be changed in the Audio Settings window of the MSpeakers module.

Figure 11: MDirectInput Module

Back to Top

            MKeyboard

                The main element of this module is the piano keyboard. By clicking the X aspa_azul.png located above the keyboard, on the right side, the mode of operation can be changed:

          Check box activated aspa_azul.png: by pressing keys on the piano keyboard the module will send to the output information in the form of MIDI note and velocity, which can typically be sent to a VST module that will use a virtual instrument.

          Check box deactivatedaspa_azul.png: this will use the computer’s internal synthesizer to directly produce sound. There is no need to connect to another module. When working in this mode, you can select any MIDI instrument from the selection box located just above the keyboard.

                To control the volume, there is a vertical slider on the right side of the module. It also has an octave control control_octava.png at the bottom left of the module.

Figure 12: MKeyboard Module with the keyboard activated

Back to Top

 

           


 

            MVST

                With this module, you can use either instruments or effects, via the VST (Virtual Studio Technology) software interface:

          VST effects are used when there is audio connected to the MVST input.

          VST instruments will be used when there is a MIDI instrument connected to your computer or when MVST has a connection coming from the module MKeyboard.

                To load the effects/ VST instruments, press the Load button and locate the effect/ VST instrument on your computer. Once loaded, the Open button gives you access to its own graphical interface where make the desired changes. Under these buttons, there is a selection button Module Input/ MIDI Input.

          Select Module Input when working with a module connected to the MVST input. The input might be audio, or MIDI coming from the MKeyboard module.

          Select MIDI Input when working with a MIDI instrument connected to the computer. If you have multiple MIDI instrument connections, you can select which one you want to work with by using the selection box at the bottom of the module. The refresh button refresh.png will renew the list of connected instruments. If it appears that the MIDI instrument is not detected correctly, restart Soundcool after reconnecting the instrument.

                Select the Bypass checkbox when an audio signal is connected to the VST Host module input but you want the audio to pass directly to the output, without applying the VST effect.

Figure 13: MVST Module

Back to Top

           


 

            MMixer

                This module has eight audio inputs. Each has an individual volume control, and the sum of all the signals goes to the output. The output also has a volume control.  This module enables you to send more than one audio signal to the audio output of our computer (MSpeakers module).

Figure 14: MMixer Module

Back to Top

            MPan

                This module has a horizontal slider to control the left/right audio panning. If the slider control is on the left extreme, the audio will be entirely on the left channel with nothing on the right channel. As the slider moves to the right, the audio will move to the right channel until it is entirely on that side and the left channel disappears. Therefore, you are controlling the ratio between what is on the left and right channels.

Figure 15: MPan Module

Back to Top

 


 

            MPitch

                This module allows you to change the pitch of incoming audio. To do this, there is a horizontal slider that changes the pitch, ranging from – 400 cents (- 2 whole tones) to + 400 cents (+ 2 whole tones).  If the input of this module is connected to one MPlayer module, pitch control comes exclusively from MPitch, so that when the playback speed is changed in MPlayer, the tuning is not modified.

Figure 16: MPitch Module

Back to Top

            MRecord

                This module allows you to record audio from its input, or audio that comes directly from your computer’s input device, such as a microphone or other device. Typically, we are interested in recording the output of an MMixer module, where all the elements of our Soundcool circuit merge. To do this, there is a button at the bottom (Module Input / Input Device) to choose what you want to record, that can be either the module’s input in or the computer’s input devices (microphone…). After pressing the record button, a dialog box opens, where you can choose the file name and recording format (.wav or .aiff). Once chosen, it will start a count down from “3”, after which it will begin recording. When recording, the Record button changes to a Stop button, which can be pressed to stop the recording. The module provides a horizontal slider to control the final volume of the recorded file, and also a time indicator at the bottom of the module.

Figure 17: MRecord Module

Back to Top


 

            MFilter

                This module allows you to filter its input in different ways. In the dropdown menu, which defaults to Display, you can change the type of filter you want to use.

mfilterlowpass.pngmfilterlowpass1.png

                1) Nothing is selected.          2) Manipulating the cutoff frequency.  3) Manipulating the Q parameter.

 

Figure 18: MFilter Module, using the Low Pass Filter

                The available filters are as follows:

          Lowpass: This is a Low Pass Filter: it lets low frequencies pass through and reduces those that are above the cutoff frequency. When this filter is chosen, the graph changes and a vertical bar, which can slide sideways, appears allowing you to control the cutoff frequency and the quality factor or Q, which adjusts the filter slope. Note that to change the Q factor while on the computer, you need to place the cursor on one of the vertical blue lines on the chart (see Figure 18.2). The Q factor can be modified when the line changes from blue to brown (see Figure 18.3).  

          Highpass: This is a High Pass Filter: it passes high frequencies through and reduces those below the cutoff frequency. To modify the filter’s parameters, follow the same steps as for the Lowpass filter.

          Bandpass: This is a Bandpass Filter: it only lets those frequencies near the cutoff frequency pass through and reduces other frequencies. Parameter modification is the same as for the Lowpass Filter.

          Bandstop: This is a Band Rejection Filter: it reduces those frequencies around the cutoff frequency. By modifying the Q factor (see Lowpass filter), the width of the frequency spectrum to be canceled can be widened or narrowed.

          Peaknotch: This is a filter that allows you to reduce or amplify the gain on a certain frequency. This filter is at the top of the dropdown menu. Besides the previously described horizontal movements (see Lowpass filter) it also allows you to control the gain by means of vertical movements.

          Lowshelf: This filter allows you to modify the gain for all the frequency spectrum that are to the left of the cutoff frequency–lower frequencies. Vertical movements allow you to control the gain.

          Highshelf: This filter allows you to modify the gain for all the frequency spectrum that are on the right side of the cutoff frequency–higher frequencies. Vertical movements allow you to control the gain.

          Resonant: This is a special type of bandpass filter. The difference between the two lies in what happens when the Q parameter is modified: increasing or decreasing the Q in Resonant entails an increase or decrease of the gain at the cutoff frequency–see Lowpass filter to learn how to modify the Q.

          Allpass: This filter does not reduce any frequency. It is the default filter configuration. Notice that modification of the parameters does not produce any effect on this filter.

          Display: This option will lock the filter, so it cannot be manipulated.

                The following table shows the maximum and minimum values for each configurable parameter:

Parameter

Minimum

Maximum

Cutoff Frequency Hz

0*

22050

Gain dB

0

16

Q Factor

0.5

25

*Despite showing a zero on the graph, for mathematical reasons, that value will never be reached.

Back to Top

            MRouting

                This module has two audio inputs and two audio outputs. Each of the two inputs (Input 1 and Input 2) can be routed to either Output1, or Output 2, or can be left unconnected, in the Off position. Each of the two outputs is associated with a bar that shows the volume of whatever is connected at that moment to the output in question.

Routing.png

Figure 19: MRouting Module

Back to Top


 

            MSamplePlayer

                This module is designed to playback the different sounds that can be loaded in the 12 available positions. To load an audio track it is necessary to press the arrow flecha.png located under the Play button MyPlay.png for each position. Once you finish loading the files, you can start playing back any combination of sounds. The Play MyPlay.png button of each loaded file does not pause the ongoing playback but instead adds to it. Additionally, there is a vertical slider to control the volume and a horizontal slide to control the playback speed. Notice that the x0 speed position is equal to Pause MyPause.png.We can choose to play backwardsmyback.png or forwards MyForward.png using the button to the right of the horizontal slider. Finally, this module also allows you to choose random playback, by checking the RANDOM box on the bottom-left corner.

Sample_player.png

Figure 20: MSamplePlayer Module

Back to Top

            MScope

                This is a module that emulates an oscilloscope; which is a device that displays the variations of the audio signals over time. Since it is a display device, it only has the option of connecting just one input, which is the sound source for its functioning. Additionally, the size of the module can be enlarged with the Enlarge button zoom_grande.png which can be found on the bottom-right corner. The oscilloscope begins to work when the module or combination of modules connected to it (as one input) are working.

scope2.png

Figure 21: MScope Module displaying a 4 Hz pure tone

Back to Top

            MSpectroscope

                This is a module that works like a spectroscope or spectrometer. This device displays the frequency spectrum by analyzing and separating the audio input into different frequencies. The size of the module can be enlarged with the Enlarge button zoom_grande.png which can be found on the bottom-right corner. The spectroscope begins to work when the module or combination of modules connected to it (as one input) are working.                             

spectro1.png

Figure 22: MSpectroscope Module displaying a 200 Hz saw-tooth signal

Back to Top

            MSignalGen

                This module generates sound signals. Their frequency can be adjusted with the horizontal slider Frequency (Hz). The frequency value can be also entered numerically by using the box that is to the right of the words “Frequency (Hz)”, either by using the keyboard or by clicking on the box with the left mouse button and then scrolling the mouse up or down. The latter option allows for the frequency to be changed faster than with the slider.

The dropdown menu which is initially in the Silence position, allows you to select the type of signal:

          Silence: the signal generator is off.

          Sine wave: sinusoidal waveform.

          Triangle: triangular waveform.

          Square: square waveform.

          Sawtooth: sawtooth waveform.

          White noise: one type of colored noise.

          Pink noise: another type of colored noise.

                There is also a dropdown menu to choose the type of modulation, which is located to the right of the abovementioned dropdown menu for selecting the type of signal. You can choose between No Mod (without modulation), RM, FM and AM. Each modulation is explained in more detail as follows:

          No Mod: The module output will be the signal generated with the abovementioned menu for selecting the type of signal. Regardless of the module or combination of modules that are connected to the input in, no modulation is applied. The output is just the signal generated by MSignalGen.

 

 

          RM: Ring Modulation, which is based on the multiplication of two signals.


 

 

          AM: Amplitude Modulation. It consists in multiplying the input signal of the module by the MI factor (Modulation Index), adding the constant 1 and all that will multiply the signal generated by MSignalGen.

 

 

 

          FM: Frequency Modulation. In the FM technique, the input signal connected to the module “in” is multiplied by the FD parameter (which in turn is related to the frequency deviation of the FM technique). This controls the frequency of the signal generated by the module. When FM modulation is selected, the value of FD is controlled with the horizontal slider or the corresponding number box.


 

                FM Chowning: When FM is selected, the MSignalGen module will show two additional number boxes that allow you to use the methodology developed by John Chowning. Such parameters are modulation index I and harmonic rate HR. The two number boxes will function both as a display (their values change when other dependent parameters are changed) and as a control (their values can be adjusted).

signalgen.png signalGen.png

Figure 23: MSignalGen Module (left) and MSignalGen Module with FM modulation selected (right)

                When AM modulation is selected, the modulation index value can be controlled with the horizontal slider or the MI number box. Notice that the FD for FM and MI for AM are only available when FM and AM modulations are selected respectively from the menu. For any other case the slider will show Not applicable, and its movement will not affect the output of the module.

 

                Finally, there is a vertical slider for controlling the volume.

Back to Top

            MSequencer

                This module has the same purpose as a classic sequencer that controls a function generator. In Soundcool, the function generator is the MSignalGen module. Thus, the first thing to do is to select the ID number of the module that you want to control.     

                Selection of an MSignalGen module

                The MSequencer module has a number box which selects the MSignalGen you want to control.  Enter its ID number here. Then, MSequencer will send an automated sequence to it.

Figure 24: Selection of the MSignalGen Module that you want to control

                Parameters for each of the eight boxes

                To produce a sequence, you can use up to 8 independent boxes, whose parameters are described below. The order chosen for the description below is optimal to understand its functioning.

RecuadroSecuencer - English.png

Figure 25: Parameters for each of the MSequencer Module boxes.

1.       Box selection: To activate any box it is necessary to click theMySelect.pngswitch. A box that does not have this switch activated will not produce any effect on the MSignalGen module.

2.       Waveform selection: As in the explanation given for the MSignalGen module, it is necessary to select the type of waveform that you want to produce in the generator.

3.       Octave selection: Before deciding which note you want to produce in MSignalGen, it is necessary to choose an octave where the note selection knob (see below) can operate.

4.       Note selection knob: Once the octave has been selected, the note selection knob can be used to choose the desired note. The note selection knob has 24 positions, which are the number of quartertones in an octave.

5.       Note indicator: This is a display message that shows the symbol of the note that has been selected with the above knob (in the Anglo-Saxon music notation system). This message cannot be directly manipulated by the user. The quartertones between a natural note and a sharp note will be represented by a dot “.”. Thus, the note symbols that a knob can show are the following:

C · C# · D · D# · E · F · F# · G · G# · A · A# · B ·

6.       Note duration: Each of the eight boxes has a vertical slider that modifies the note duration for that particular box.

7.       Duration indicator: Here, the final value selected with the above slider is displayed. Additionally, this indicator can be modified manually to reach the desired value. 

8.       Box skip: As in the classic sequencers, the MySkip.pngswitch allows you to skip that particular box, resulting in a silence whose duration is equal to its chosen note-duration value. Notice that the Skip function is not equivalent to theMySelect.pngswitch being off. In order for the Skip switch to work, the Select switch must be activated. 

9.       Modulation selection: As in the MSignalGen module, the modulation will only be applied if another MSignalGen module is connected to the first MSignalGen. In this context, we could choose between RM, AM and FM.

10.   Modulation value knob: In the same way that the MSignalGen module has a horizontal slider to control the value of the modulation, this value is controlled here with a knob.

11.   Modulation value indicator: Under the duration indicator (7), there is a similar indicator for the value of the modulation. This indicator appears only when FM or AM are selected, which are the modulation types that allow the use of the above knob (10).

                Launching the sequence

                Once you have selected the MSignalGen module that you want to control, and all the desired parameters of the boxes have been configured, you can launch the sequence whenever you wish, using the buttons shown and described below (left to right):

secuencia.png

Figure 26: Buttons to launch the sequence of the MSequencer Module

·         Direct launchdirecto.png: The order of the sequence is as follows: the first of the eight boxes to be launched will be the one at the top-left corner, while the last box will be the one at the bottom-right corner. If not all eight boxes are used, the sequence can only be launched by using this button.

·         Inverse launchdirecto.png: The order of the sequence is the reverse of direct launch: the first box will be the one at the bottom-right corner and the last box will be the one at the top-left corner. This type of launch is only possible if all eight boxes are activated and configured.

·         Direct/inverse launch dirinv.png: This type of launch concatenates a direct launch, followed by a inverse launch. This type of launch is only possible if all eight boxes are activated and configured.

·         Loop selection MyLoop.png: This is not a standalone launch. It is a switch that makes the sequence repeat indefinitely. Notice that when it is active, you can press as many subsequent launches as you like; all of them will be heard simultaneously.

sequencer.png

Figure 27: MSequencer Module

Back to Top

            MTransposer

                This module allows you to raise or lower the pitch of the audio that is connected to its input. The functioning is as follows: the flat bemol.png and sharp sostenido.png buttons allow you to decrease or increase the pitch one semitone, respectively. Notice that the Cents box will change in steps of 100 cents (exactly the number of cents that a semitone has). Thus, those buttons represent a coarse tuning.  For fine tuning, a horizontal slider is available that allows you to make changes with step sizes as little as one cent. We can combine both controllers to achieve the desired pitch. Notice that the Cents box is display-only and cannot be directly manipulated. Finally, to return to the original pitch we can click on the Natural button becuadro.png.

 

Transposer.png

Figure 28: MTransposer Module

Back to Top

            MEnvelope

                This module allows you to create an amplitude envelope. To that end, a series of points can be created by clicking with the mouse. These points will be connected by a black-color line. This line will control the volume of the audio signal. The duration of the envelope can be established with the Dur number box. Additionally, there is a Loop switch causing an endless repetition of the envelope. To activate the envelope it is necessary to click the button to the right of the Dur number box.

                To illustrate all this, suppose that you want to increase the volume gradually, up to an end value. Then, you simply need to create two points, as in the following figure:

Envelope2.png

Figure 29: Example of a linear envelope

                Notice that the white-color background is like a plane with X-Y coordinates, where the horizontal coordinate X represents the time (within the duration established above) and the vertical coordinate Y represents the amplitude of the audio signal. Additionally, if an existing point is dragged to the bottom edge (the zero value of the Y axis), it will change from solid color to hollow and will represent the volume 0 or mute.

                The user is free to design any type of envelope. Additionally, any point can be deleted by using the combination “Shift + Click”. The Clear button will delete all the points at once.

                Finally, this module allows you to use two types of envelope:

          Curve: Besides allowing you the freedom to create points, it also allows you to modify the curvature of the line that joins those points. The curvature of each line segment can be modified with the number box on the top-left corner of the panel. You have to enter the value of the curvature (the allowed values are between -1 and 1) and press Enter on your computer keyboard. To change the line segment that you want to modify, you have to press the button next to the number box. You will identify the line segment that is ready to be modified because it automatically changes its curvature at the moment of being selected. The following figure shows a number of curves with positive and negative values: 

Envelope3.png

Figure 30: Example of curved envelope

          Line: This mode only allows you to create points but not to modify the curvature of the line, which will remain straight.

         Back to Top

Operation of the video modules

            MVDirectInput

                This module allows you to stream any video source that is connected to the computer, e.g., a WebCam. The checkbox on the top-left corner activates or deactivates this streaming.  When the streaming begins, the blue dropdown on the bottom-right side allows you to choose the desired video format. Additionally, the blue dropdown on the bottom-left side allows you to choose the desired video input. The white button on the bottom-left side allows you to watch the video in full screen. You can leave this mode by pressing ESC on your keyboard. The white button on the bottom-right side opens up the computer’s own menu, where you can configure any video parameters that the device allows you to.

vdirect.png

Figure 31: MVDirectInput Module

Back to Top

             MVPlayer

                As with the audio player (MPlayer) this module allows you to load any video file stored on your computer. As mentioned earlier, the player is not compatible with video files generated by Windows Media .wmv (see File Compatibility section for instructions on how to watch this type of video format). The Open button allows you to select the desired file. Next to this button, there is a dropdown displaying the history of files used, allowing you quick access to them. The player has these controls: a PlayPlayv.png/Pause pause.png toggle, Forward forward.png or Backward back.png and the ability to select the following video sequences:

          actualizar.png : The video will be repeated indefinitely.

          delantedetras.png : The video is played forwards and then backwards, indefinitely.

          delantesolo.png : The video is played only once.

 

                Finally, there is a button that allows you to watch the video in full screen. You can leave this mode by pressing ESC on your keyboard.

 

vplayer.png

Figure 32: MVPlayer Module

Back to Top


 

Connecting Smartphones and Tablets using TouchOSC

 

                Most Soundcool modules can be controlled from a mobile device (Tablet or Smartphone). To do that, it is necessary to:

 

          Connect the device to the same WiFi network as the computer on which the modules are open.

          Install the TouchOSC application on the mobile device (available both in the iTunes AppStore and Google Play).

 

                To function properly, these steps are needed:

            Configure the OSC menu

                Open the TouchOSC application on the mobile device. A menu called OSC will appear, which allows you to configure the parameters that are needed to establish communication with the computer.

          Host: Enter the IP address of the computer, typically X.X.X.X where the X’s are numbers between 0 and 255. The following website explains how to find out the computer’s IP address: http://www.howtofindmyipaddress.com/

          Port (outgoing): This is the port that the mobile device will use to communicate with the computer. Any port number can be chosen. However, it is recommended not to use the 7374 and 12345 ports because these are used by other devices (being default ports for the modules and Kinect, respectively).

          Port (incoming): The only requisite for this port is to choose any number that is not used by any other device.

          ZeroConf Name (only for Android devices): This text field will not be used for Soundcool.

 

Once you are done with these settings, you can exit the OSC menu.

            Load the Soundcool layout

                To control the Soundcool modules you need to load the layout that is included in the Soundcool program. This layout includes different control surfaces with buttons, faders, toggles, etc. To load the layout, open TouchOSC and select the option Layout. Then, on Android you can load the layout in two different ways, Add from Editor or Add from File (on iPhone only, you can use Add which is equivalent to Add from Editor):

                Add from Editor

                This option requires having the TouchOSC Editor installed (see “Requirements” section for more details). Once the program is open, select Open to find the layout called Soundcool_Layout_v1.0.touchosc and click the Sync button. Then, the screen will display a series of instructions to follow using the mobile device you want to load the layout to:

1.       Connect the device to the same WiFi network as the computer.

2.       Open the application on the mobile device and select Layout>Add from Editor.

3.       Select the name of the computer that appears on the Found Hosts list.

 

                After selecting the name of the computer, the layout will be automatically added to the mobile device.

               

                If the computer name does not appear automatically on your mobile devices Found Hosts list, you can directly add the IP address of the computer into the Host box (see “Configure the OSC menu” – “Host” above, for more instructions). After entering the IP address, select Download. The layout will be added to the mobile device.

               

                It is recommended that you don’t change the parameters related to either the type of mobile device or to the orientation of the layout in the TouchOSC Editor program because any change may alter the correct position of the objects.

layout.png

Figure 33: Soundcool layout loaded on the TouchOSC Editor

                Add from File

                Only available on Android devices. This option does not require any additional software. You only need to have the Soundcool layout stored in that mobile device. When you select the Add from File option, you need to find the folder that contains the layout and select it.

            Configure the module

                Once you have finished setting up your mobile device, you need to enter the number of its outgoing port in the OSC Port box of the module that you want to control wirelessly and press Enter. After this, you can press Done in the TouchOSC initial menu.

            Example of a typical connection process

 

To control the MPlayer module wirelessly by means of a mobile device.

 

Figure 34: Module that you want to control

                In this example, the computer’s local IP is 192.168.1.2 and the port you want to use to communicate is port 5000.

                Steps to follow for iPhone:

          Open the TouchOSC application on the iPhone and select OSC menu.

          Host: 192.168.1.2, Port (outgoing): 5000, Port (incoming): anything but 5000, 7374 and 12345.

          Close OSC menu.

          Layout->Add: to load the layout from the computer. You will load the layout called Soundcool_Layout_v1.0.touchosc.

          Close the Layout menu.

 

la foto.PNG

Figure 35: Configuration of the OSC menu on an iPhone

Skip the Android section and continue on with the Common Steps section.

                Steps to follow on Android:

          Open the TouchOSC application on the Android and select OSC menu.

          Host: 192.168.1.2, Port (outgoing): 5000, Port (incoming): anyone but 5000, 7374 and 12345; ZeroConf Name: this is not necessary to be modified.

          Close OSC menu.

 

Screenshot_2014-11-25-10-38-09.png

Figure 36: Configuration of OSC menu in Android

          Choose Layout->Add from Editor to load the layout from the computer. Choose Layout-> Add from File to load the layout from the mobile device itself. You have to load the layout named Soundcool_Layout_v1.0.touchosc. Close the Layout menu.

                 Common Steps:

          Enter the port number on the MPlayer module.

 

mvplayerhecho.png

Figure 37: MPlayer after entering the port number

          Select Done on the mobile device.

 

layout.png

Figure 38: The Soundcool layout is ready to use!

                At this point, the red slider controls the playback Speed, the yellow one controls the time position of playback, the blue one the volume, the brown buttons are the options for Play/Pause, Stop and Reverse, and the purple switch is the Loop option.

Back to Top

 


 

            Kinect body control of modules

                Soundcool modules allow manipulation of some of its parameters by means of body movements captured and processed in real time. This control is possible through the use of Kinect, a computer-vision camera developed by Microsoft. Kinect was originally included as a supplement to the Microsoft Xbox 360 video game console. However, the Kinect sensor can also be used without the Xbox console, so you can buy it separately and use it without restrictions.      

kinect.jpg

Figure 39: Kinect sensor

To connect Kinect to your computer (PC or MAC) you must use the USB cable that Microsoft provides for that specific purpose (power adapter included). This cable is included in Kinect if it is bought separately from the Microsoft Xbox 360 console. If Kinect and the Xbox are bought together as one unit, the cable needs to be purchased separately.      

powersupply.jpg

Figure 40: Kinect USB Cable with power adaptor

            Installation of Kinect drivers for Synapse

                Synapse is the program responsible for processing the information coming from the Kinect sensor. We recommend not connecting Kinect before doing the installation process described below. Synapse is free and can be found at:

http://synapsekinect.tumblr.com/post/6307790318/synapse-for-kinect

                Installation process on a MAC

                It is only needed to connect Kinect to a USB port and the adaptor to the power.

                Installation process on a PC

                It is recommended to strictly follow the sequence described in this section to install the necessary drivers. Do not connect the sensor until installation is complete. Otherwise, it is not guaranteed that the device will operate correctly. Furthermore, it is required the removal of any driver whose purpose is also the control of Kinect but not mentioned in this section. Otherwise, having multiple drivers installed for the same purpose can lead to errors.          

          Install the OpenNI library: http://hihigogo.com/OpenNI-Win32-1.1.0.41-Dev.msi

          Instalall NITE software, by PrimeSense: http://hihigogo.com/NITE-Win32-1.3.1.5-Dev.msi

          Instalall the SensorKinect driver: http://hihigogo.com/SensorKinect-Win-OpenSource32-5.0.1.msi     

          Reboot (the computer).

          Connect the Kinect sensor to power and to a USB port on your computer.

          Open Synapse.

                If, despite having followed the steps, the Synapse program does not work; you can find a list of common errors that are easy to fix, on their website:

http://synapsekinect.tumblr.com/post/6698860570/windows-install-instructions  

                Note:  Synapse does not work on Windows 8.

                Note: All the above links to software are provided by the Synapse website. However, these programs may have been subsequently improved. You can look for up-dates at the following link at which the driver downloads are packed in .zip files named RGBDemo… Win32:

http://sourceforge.net/projects/roboticslab/files/

           


 

            Using Kinect with Soundcool modules through Synapse

                All Soundcool modules that are compatible with body control incorporate a check box on the bottom left side that enables the Kinect sensor. To begin controlling any module using body gestures it is necessary to open the Synapse program and follow these steps:

          Once the Synapse is opened, the image captured by Kinect will be shown on the screen, as well as a colored dot on the top-left corner. It will initially be red meaning that it is not detecting any user in its vision field. As soon a user is detected, it will change to green.

 

Sin título.png

Figure 41: User detected using Kinect and Synapse

          When the user is detected, he or she must adopt a T position with their arms and hands up until Synapse is able to detect the joints of his/her body (only a few seconds are needed). Joints appear as black rectangles.

 

Sin título2.png

Figure 42: The user’s joints are detected

                These black rectangles will be captured as a reference for controlling the different parameters of the modules. At this moment you can activate the desired Kinect check box and immediately you will see that the movement of the body will produce changes in the parameters of the module.  

kinecton.png

Figure 43 : The Kinect check box is selected in the MDelay module

            Movements for controlling the parameters

                Sliders are controlled with the following movements:

          Horizontal Sliders can be controlled by moving your arm in the horizontal direction (left and right). Kinect captures the horizontal pathway of your hands as a reference for moving the horizontal sliders. Generally, one horizontal slider is associated with the left hand. If there are two sliders, one will be controlled with your right hand and the other with your left.

 

          Vertical Sliders can be controlled by arm movements in the vertical direction (up and down). Kinect captures the vertical pathway of your hands as a reference for moving the vertical sliders. Generally, one vertical slider is associated with the right hand. If there are two sliders, one will be controlled with your right hand and the other with your left.

 

Back to Top


 

Credits

The Soundcool project is led by:

                Dr. Jorge Sastre

                Director

                Electronic Music and Video Creation Courses (MEVIC)

                Universitat Politècnica de València

                Camino de Vera, s/n

                46022 Valencia (Spain)

                Email:  jorsasma@iteam.upv.es

Participants:

·         Faculty of the Master in Music and DCADHA UPV: Dr. Nuria Lloret, Dr. Adolfo Muñoz.

·         Pedagogic dimension of the project: Dr. Adolf Murillo (responsible for the educational aspects), Dr. Elizabeth Carrascosa, Dr. Remigi Morant.

·         Faculty of MEVIC UPV: Dr. Carlos Hernández, Dr. Ximo Cerdà.

·         Developers: Enrique Serrano, Rafael García, Han Deng, Giannino Clemente, Dr. Jaime Serquera.

Collaborators:

·         Faculty of MEVIC UPV: Stefano Scarani, Gregorio Jiménez, Joan Cerveró, Dr. David Picó.

·         Carnegie Mellon University (CMU): Professor Roger Dannenberg. CMU and U. of Pittsburgh:  Sara Masters.

Translation into English: Sara Masters, Jorge Sastre, Jaime Serquera.

                Soundcool is available for free at http://soundcool.org/. This software is provided “as is” without express or implied warranties, including, but not limited to warranties of commercialization and fitness for a particular purpose. In no event shall the creators or contributors be liable for any type of damage (direct, indirect, incidental, special, consequential, etc.) caused by Soundcool (including, but not limited to, procurement of substitute goods or services, loss of use, data or profits or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.

Back to Top