NICHOLAS BROWN

➡️ Try out the sketch described in this article via the online p5.js editor here

In this article, we’ll learn how to use the PolySynth function in p5.js to build a synthesizer designed to run in the web browser of mobile devices.

First of all let’s start by working out what we need to do.

We need to design a visual interface for our synthesizer in the browser window. We’ll base our design on a standard piano keyboard.

We’ll use some strong colours as the ‘keys’ and remove the distinction between sharps and flats so that playing the synth doesn’t require any keyboard training.

We’ll draw five vertical bars across the screen. Each bar will have a unique colour (out of a set of six) and will play one musical note.

We’ll add a volume slider to the top of the interface, leaving a top margin to separate the volume control from the keys.

When we code with p5.js we are able to work creatively with sound and image simultaneously, directly in the browser. This means that we can make creative connections between what we see and what we hear.

We can use information (data) from the domain of image and map it to the domain of sound (or vice versa). For more on this, check out my article on sound manipulation to see how to ‘map’ the horizontal position of a circle onscreen to the panning of a sound.

Back to our synth. How are we going to connect our five bars to the musical notes of the synth, assuming that each bar stands for one note?

Obviously, five bars suggests a five-note scale. In music theory, there is a five-note scale. It is called pentatonic. We will work with that.

Let’s choose F-minor pentatonic. This means that our pentatonic scale will begin on F and include the notes F, A-flat, B-flat, C and E-flat.

The choice of F-minor pentatonic is arbitrary. We could have chosen a different minor pentatonic scale, or a major pentatonic scale, or a different, five-note scale entirely.

The main thing to be aware of is that we’re making creative decisions across two forms of perception––seeing and hearing––in a way that connects them through digital code.

Another reason for limiting the synth to five coloured bars/five notes of a minor-pentatonic scale is that these two things––when combined-–make it easier to invent meaningful tunes that have musical coherence.

The reason why this is the case is beyond what I want to cover in this tutorial. But very often, in musical composition, the consequence of limiting the number of options is helps make our creative acts more coherent.

Once we have our visual interface in place and our synth set up correctly, we need to make sure that when we click a particular bar, we hear the relevant note from the pentatonic scale (e.g. bar two is an A flat etc.).

Here’s our task list, in pseudocode:

/* TASK  1. Create a visual interface by drawing five, vertical bars across the screen. 

TASK 2. Set up the sound element so that we can play an f-minor pentatonic scale.

TASK 3. Add interactivity so that musical notes are heard when we click (mouse/computer) or touch (mobile devices) the bars. 
*/

Draw the bars

OK. Let’s start coding. We’ll start by declaring some variables and arrays:

let numBars = 5; // variable for the number of bars

let bars = []; // array to hold each individual bar as an object

let xBar = []; // array to hold the x position of the LH side of each bar

let clr = ['#326CAD', '#9CAD3B', '#FA857A', '#61A9FA', '#DFFA48']; // array to hold the set of colours for the bars

Next, in setup(), we’ll create a canvas and give it a grey background:

createCanvas(windowWidth, windowHeight);
background(220);

The createCanvas() function uses the built-in variables in p5.js for the current width and height of the user’s browser window.

This means that our synthesizer will take up the whole of the browser, whatever its dimensions at the time of running the programme.

We’re also going to write a loop in setup() that calculates the x coordinate of the LH-side of each bar and stores that value in the xBar array. The reason we need to calculate this value is because we need it to draw the bars.

The p5.js function, rect(), takes an argument for the starting x position of the rectangle, as well as its width and height. (We already know that the starting y position will be 0.)

for (let i = 0; i < numBars; i++) {
 let w = windowWidth / numBars; // calculate the width of the bars
 x = w * i; // multiply the width by the current bar id
 xBar.push(x); // store the x position for the current bar in an array
}

We’re going to make each bar an instance of a class called ‘Bar’. This will be useful when we need to find out whether a user is touching a particular bar.

To make each bar a class, we need to write a constructor function. This is a kind of template that let’s you make specific instances of a general thing. The metaphor often used for a constructor is that it’s like a ‘cookie cutter’. 🍪

For our synth, we need to write a constructor that lets us make specific instances of a general thing called ‘bar’, each with a particular color.

To keep things organized, we will write the code for the constructor in a separate javascript file called bar.js.

We will need to remember to add a link to the bar.js file in the index.html file. Otherwise, the browser won’t be able to find the code for building the bars.

Here’s the code for the constructor:

function Bar(id) { // constructor for a general thing called 'Bar'

 this.display = function () { // function for displaying the bar

 noStroke(); // turn off the shape outline

 fill(clr[id]); // fill the bar with the right colour from the `clr` array

 rect(xBar[id], 50, windowWidth / numBars, windowHeight); // draw the bar
 } 
}

And here is the code we need to add to the <body> section of the index.html file. We’ll add it just below the reference to the sketch.js file.

<script src="bar.js"></script>

The constructor includes a parameter called id. This lets us pass an argument to the constructor to identify each new bar that we want build.

Having this ‘id’ parameter will be really useful when we want to ask questions like, ‘what colour should the second bar be?’.

To answer a question like this we will need to look up the relevant hexadecimal value in the clr array of colour values.

The ‘id’ parameter will also be useful when we want to know the x position of the left-side of a particular bar in the xBar array.

Fortunately, there is a straightforward way of setting this ‘id’ by using the i variable in the for() loop. This variable will be passed to the constructor and give each bar its own, unique id, i.e. 0, 1, 2 etc.

Now we just need to write a loop that actually draw the bars across the screen. We can do this using the keyword new to activate the constructor.

Using a for() loop is an efficient way of doing the same thing five times. It lets us create five separate bars, each with their own color and x position.

// draw the bars across the screen

for (let i = 0; i < numBars; i++) {
 // create a new bar and push to array of bars:
 bars.push(new Bar(i)); 
 // display bar by calling the .display() method:
 bars[i].display(); 
}

Each time we create a new bar, we store it in the array called bars[] using the .push() method. ‘Pushing’ every bar to an array lets us keep track of each one as it is created.

Having all the bars stored in one place (in an array) will make it easier when we need to check whether any of the bars are currently being played by the user.

Notice that the integer i has a value between 0 and 4. We use this value for incrementation in the for() loop. As mentioned, it also becomes the id for each new bar when we pass it to the constructor.

Remember: computers count from zero, so bar 1 is actually bar 0 and bar 5 is actually bar 4 etc.

We now have five bars, each of which is a unique instance of the class bar.

Set up the synth

Now that the visual interface is done, we need to get the sound working. We’ll use the built-in polyphonic synthesizer in p5.js called, polySynth();

First, some variables:

let polySynth; // for the synth itself

let notes = [349.23, 415.30, 466.16, 523.25, 622.25, 698.46];

The notes array stores the frequencies of the notes of the f-minor pentatonic scale. You can find pitch-frequency values on the web using charts such as this one.

We need to initialize the synthesis. This is something we’ll do in setup(), using the built-in p5.PolySynth() constructor to initialize our synthesizer instrument.

We use the new keyword to make a new instance of this class, much like we did with Bar(id), except here we use a ready-made constructor that is part of the p5.js library.

// set up the synthesis
polySynth = new p5.PolySynth();

Next, let’s set an envelope for our synth. You can find the parameters for the .setADSR method in the p5 reference here.

We’ll use an attack of 0.1 seconds, a decay of 0.4 seconds, a sustain ratio of 0.3 and a release of 0.01 seconds.

polySynth.setADSR(0.1, 0.4, 0.3, 0.05); // attackTime, decayTime, susRatio, releaseTime

Make it interactive

Let’s design our synth so that it is playable in a browser window on touchscreen/mobile devices.

We will need to connect the visual and sonic aspects of our sketch so that when we touch a bar, we play one of the notes of the pentatonic scale.

We do this is by writing a p5 function that is called whenever the screen is touched. This function will default to mouse clicks when a touchscreen isn’t present.

function touchStarted() {
  // some code
}

Inside this function we want to run a loop that looks through all the bars stored in the bars array and asks––for every bar––whether the user’s finger is currently touching that particular bar…

We’ll use a for() loop to cycle through the five instances of the Bar(id) class and call a method called played() for each bar.

function touchStarted() {
 for (let i = 0; i < numBars; i++) {
  bars[i].played();
 }
}

This played() method will include a conditional statement which will ask whether the finger is over the key. This conditional will go inside the constructor:

// put this inside the Bar(id) constructor'

this.played = function () {
 if (mouseY > 50 && mouseX > xBar[id] && mouseX < (xBar[id] + (windowWidth / numBars))) {
 polySynth.play(notes[id], 0.5, 0, 0.2 );
 }
}

The conditional here is a simple mouseover query. It takes into account the grey bar that contains the volume slider at the top of the screen by including a 50 pixel offset (mouseY > 50).

The reason for this offset is so that the user can change a volume slider (which we will add shortly) without actually playing any of the bars!

If the conditional is true then the play() method is called on the polySynth. The play() method accepts arguments to determine the note, velocity, seconds from now and sustain time. See the reference here.

We use the id parameter to access the correct note frequency for the relevant bar being played.

One other thing: it is best practice to give the user control over starting any audio inside a web browser. P5 has a function to let you do that called userStartAudio(). Here’s the reference.

We can add this userStartAudio() function to the top of the touchStarted() function in sketch.js:

function touchStarted() {
 userStartAudio(); 
 for (let i = 0; i < numBars; i++) {
  bars[i].played();
 }
}

The touchStarted() function responds to touch interaction on mobiles. But it also works according to mouse click, if the programme is alternatively run in a browser on a computer screen.

One final thing (also good practice for web audio)––we’re going to give the user the option to adjust the volume using a volume slider.

If you want to know more about adding sliders, see my previous article on sound manipulation in the browser with p5.js.

First the variable:

let volSlider; 

Then in setup():

volSlider = createSlider(0, 1, 0.5, 0);
volSlider.position(25, 25);

textSize(16);
fill(0);
text('volume', 25, 20);

Finally, we need to write some code that constantly checks for any changes to the slider. It should do this for as long as the programme is running and update the volume of the synth accordingly.

This line goes inside draw():

outputVolume(volSlider.value(), 0.025);

Here we put 25 ms (0.025s) of ramp time between each tick of the slider. This reduces any annoying audio glitches when adjusting the volume while the sound is still playing.

And that’s it. Now we have a basic synth inside the web browser.

Here’s the code in full:

/* Touch-Bar Synth
Nicholas Brown, 2023 */

// sketch.js

let numBars = 5; // variable for number of bars
let bars = []; // array to hold each individual bar as an object
let xBar = []; // array to hold the x coords of the LH side of each bar
let clr = ['#326CAD', '#9CAD3B', '#61A9FA', '#FA857A', '#DFFA48']; // array of colors for the bars
let notes = [349.23, 415.30, 466.16, 523.25, 622.25, 698.46]; // array to store the frequencies of each note of an f minor pentatonic scale
let volSlider;
let polySynth;

function setup() {
  createCanvas(windowWidth, windowHeight);
  background(220);

  // populate xCoord array values for the x-coordinate of the left side of each bar
  for (let i = 0; i < numBars; i++) {
    let w = windowWidth / numBars;
    x = w * i;
    xBar.push(x);
  }

  // draw the bars across the screen
  for (let i = 0; i < numBars; i++) {
    bars.push(new Bar(i)); // create a new bar and push to array of bars
    bars[i].display(); // call .display() function for that bar
  }

  // draw volume slider

  volSlider = createSlider(0, 1, 0.5, 0);
  volSlider.position(25, 25);

  textSize(16);
  fill(0);
  text('volume', 25, 20);

  // set up the synthesis

  polySynth = new p5.PolySynth();
  polySynth.setADSR(0.1, 0.4, 0.3, 0.05); // attackTime, decayTime, susRatio, releaseTime

}

function draw() {
  outputVolume(volSlider.value(), 0.025);
}

function touchStarted() {
  userStartAudio();
  for (let i = 0; i < numBars; i++) {
    bars[i].played();
  }
}

// bar.js

function Bar(id) {
  // bar constructor

  this.display = function () {
    // function for displaying the bars
    noStroke();
    fill(clr[id]);
    rect(xBar[id], 50, windowWidth / numBars, windowHeight);
  };

  this.played = function () {
    if (
      mouseY > 50 &&
      mouseX > xBar[id] &&
      mouseX < xBar[id] + windowWidth / numBars
    ) {
      polySynth.play(notes[id], 0.5, 0, 0.2);
    }
  };
}