This tutorial shows how to play and loop audio stored in an AudioSampleBuffer object. This is a useful basis for sampling applications that manipulate recorded audio data.
Intermediate
Windows, macOS, Linux
AudioBuffer, AudioFormatReader, AudioAppComponent
Getting started
- Note
- This tutorial assumes you have already completed Tutorial: Build a white noise generator and Tutorial: Build an audio player. If not, you should look at these first.
Download the demo project for this tutorial here: PIP | ZIP. Unzip the project and open the first header file in the Projucer.
If you need help with this step, see Tutorial: Projucer Part 1: Getting started with the Projucer.
The demo project
The demo project for this tutorial allows the user to open a sound file, read the whole file into an AudioSampleBuffer object, and play it in a loop. In Tutorial: Build an audio player we played sound files using an AudioFormatReaderSource object connected to an AudioTransportSource object to play the sound. Looping is possible using this method by enabling enabling the AudioFormatReaderSource object's looping flag — using the AudioFormatReaderSource::setLooping() function.
All of the code relevant to the discussion in this tutorial is in the MainContentComponent class of the demo project.
Loading sample data into memory
There are many cases where it is probably better to use the built-in classes for sound file playback. There may be occasions where you need to do this yourself and this tutorial gives an introduction to some of the techniques. Sampler applications commonly load the sound file data into memory like this, especially when the sounds are relatively short (see the SamplerSound class for an example). Synthesising sounds can also be achieved by storing a wavetable in an AudioSampleBuffer object and looping it at an appropriate rate to produce the required musical pitch. This is explored in Tutorial: Wavetable synthesis.
This tutorial also highlights some of the potential multi-threading issues you may encounter when combining access to files, and audio processing on the audio thread. Some of these problems seem simple on the surface but often require carefully applied techniques in order to avoid crashes and audio glitches. These techniques are explored further in Tutorial: Looping audio using the AudioSampleBuffer class (advanced).
Why the length limit?
The demo project limits the length of the sound file you can load to less than 2 seconds. This limit is rather arbitrary, but this is broadly for two reasons:
- If the whole file is very large then your computer might run out of physical memory. In a real application, of course, you would be able to use a much higher limit. A 2-second stereo audio file, at a sample rate of 44.1kHz, loaded into an AudioSampleBuffer object, will only occupy 705,600 bytes of memory. (See notes)
- Loading even quite short files doesn't take a trivial amount of time.
Regarding point 1: if we exceed the amount of physical memory the computer has, it may start to use virtual memory (that is, secondary storage such as a hard drive). This rather defeats the purpose of loading the data into memory in the first place! Of course the operation may just fail on some devices if it runs out of memory.
Regarding point 2: we keep the example simple by loading the audio data directly in, after the FileChooser::browseForFileToOpen() function has returned the file selected by the user. This means that the message thread will be blocked until all of the audio has been read in from disk into the AudioSampleBuffer object. Even with short sounds we should really do this on a background thread to keep the user interface as responsive as possible for the user. For long sounds the delay and unresponsiveness will be very noticeable. Adding another (background) thread would add to the complexity of this example. See Tutorial: Looping audio using the AudioSampleBuffer class (advanced) for example of how to load files on a background thread in this way.
- Exercise
- To keep it simple, demo project doesn't report an error if you try to load a longer file — it just fails. Adding error reporting like this is left for you as an additional exercise.
Reading the sound file
When the user clicks the Open... button they are presented with a file chooser. The whole file is then read into an AudioSampleBuffer member fileBuffer in our MainContentComponent class.
void openButtonClicked()
{
shutdownAudio();
chooser = std::make_unique<juce::FileChooser> ("Select a Wave file shorter than 2 seconds to play...",
juce::File{},
"*.wav");
auto chooserFlags = juce::FileBrowserComponent::openMode
| juce::FileBrowserComponent::canSelectFiles;
chooser->launchAsync (chooserFlags, [this] (const juce::FileChooser& fc)
{
auto file = fc.getResult();
if (file == juce::File{})
return;
std::unique_ptr<juce::AudioFormatReader> reader (formatManager.createReaderFor (file));
if (reader.get() != nullptr)
{
auto duration = (float) reader->lengthInSamples / reader->sampleRate;
if (duration < 2)
{
fileBuffer.setSize ((int) reader->numChannels, (int) reader->lengthInSamples);
reader->read (&fileBuffer,
0,
(int) reader->lengthInSamples,
0,
true,
true);
position = 0;
setAudioChannels (0, (int) reader->numChannels);
}
else
{
}
}
});
}
- : Notice that we shut down the audio system for the AudioAppComponent object each time we open a new file. This is to avoid some of the multithreading issues hinted at already. Once the audio system is shut down, there is no danger that our
getNextAudioBlock() function will be called on the audio thread while we are still within the call to the Button::onClick lambda function (which will have called this openButtonClicked() function from the message thread).
- : Here we create the AudioFormatReader object using the AudioFormatManager object. Notice that we store this in a std::unique_ptr object as we need to manage this object ourselves. (In Tutorial: Build an audio player we pass the AudioFormatReader object to the AudioFormatReaderSource object to manage for for us.) This operation may fail to create the reader object, therefore we have to check that the
reader pointer is not a nullptr value on the next line.
- : This is where we calculate the duration of the sound file by dividing the length of the file in samples by its sample rate. We check that this is less that 2 seconds on the next line.
- : Here we resize the AudioSampleBuffer object by calling the AudioSampleBuffer::setSize() function using the number of channels and length from the AudioFormatReader object.
- : This reads the audio data from the AudioFormatReader object into our AudioSampleBuffer
fileBuffer member using the AudioFormatReader::read() function. The arguments are:
- : The destination start sample in the AudioSampleBuffer object where the data will start to be written.
- : The number of samples to read.
- : The start samples in the AudioFormatReader object where reading will start.
- : For stereo (or other two-channel) files this flag indicates whether to read the left channel.
- : For stereo files this flag indicates whether to read the right channel.
- : We need to store the most recent read position within our buffer as we play it. This resets our
position member to zero.
- : This starts the audio system back up again. Here we have an opportunity to use the number of channels in the sound file to try and configure our audio device with the same number of channels.
Processing the audio
In the getNextAudioBlock() function the appropriate number of samples is read from our fileBuffer AudioSampleBuffer member and written out the the AudioSampleBuffer object in the AudioSourceChannelInfo struct.
While reading the audio data from the file we keep track of the current read position using the position member (being careful to update it after all the channels of the audio have been processed for the specified block of samples):
void getNextAudioBlock (const juce::AudioSourceChannelInfo& bufferToFill) override
{
auto numInputChannels = fileBuffer.getNumChannels();
auto numOutputChannels = bufferToFill.buffer->getNumChannels();
auto outputSamplesRemaining = bufferToFill.numSamples;
auto outputSamplesOffset = bufferToFill.startSample;
while (outputSamplesRemaining > 0)
{
auto bufferSamplesRemaining = fileBuffer.getNumSamples() - position;
auto samplesThisTime = juce::jmin (outputSamplesRemaining, bufferSamplesRemaining);
for (auto channel = 0; channel < numOutputChannels; ++channel)
{
bufferToFill.buffer->copyFrom (channel,
outputSamplesOffset,
fileBuffer,
channel % numInputChannels,
position,
samplesThisTime);
}
outputSamplesRemaining -= samplesThisTime;
outputSamplesOffset += samplesThisTime;
position += samplesThisTime;
if (position == fileBuffer.getNumSamples())
position = 0;
}
}
- : The
outputSamplesRemaining variable stores the total number of samples that the getNextAudioBlock() function needs to output taking a copy from the AudioSourceChannelInfo struct. We use this to check if we need to exit the while() loop that starts on the next line.
- : We also take a copy of the AudioSourceChannelInfo::startSample value to use as our offset within the destination buffer.
- : Here we calculate how many samples are left in the buffer from which we are reading.
- : For this pass of the
while() loop we need to output the smaller of the remaining samples for this call to the getNextAudioBlock() function and the remaining samples in the buffer — using the jmin() function. If this is less than the total number of samples for this call to the getNextAudioBlock() function, then there will be one more pass of the while() loop, before exiting.
- : For each output channel we use the AudioSampleBuffer::copyFrom() function to copy sections of data from one channel of one buffer to a channel of another buffer. Here we specify the destination channel index.
- : This is the sample offset within the destination buffer.
- : This is the source AudioSampleBuffer object from which to copy.
- : This is the channel index of the source buffer. In case the source buffer has fewer channels than our destination buffer we use this modulo calculation. For example, a mono source buffer will mean that this always results in zero, copying the same data to each of the output channels.
- : This is the position to start reading from in the source buffer.
- : The number of samples to read that we calculated earlier.
- : Now deduct the number of samples we just processed from the
outputSamplesRemaining variable.
- : Increment the
outputSamplesOffset by the same amount in case we have another pass of the while() loop.
- : Offset our
position member by the same amount too.
- : Finally, check if the
position member reached the end of the fileBuffer AudioSampleBuffer object and reset it to zero to form the loop if necessary.
- Exercise
- Add a level slider to control the audio playback level of the audio file (see Tutorial: Control audio levels). You can use the AudioSampleBuffer::applyGain() or AudioSampleBuffer::applyGainRamp() functions to apply the gain to the data in an AudioSampleBuffer object.
Multithreading issues
As discussed previously, this tutorial avoids multithreading issues by shutting down and restarting audio each time the user clicks the Open... button. But what if we didn't do this — what could happen? There many things that could go wrong, all of which have to do with the fact that both the getNextAudioBlock() and openButtonClicked() functions could be running at the same time in different threads. Here are some examples:
- Let's say that the application is already playing an audio file and the user clicks the Open... button and chooses a new file. Suppose the audio thread interrupts this function between and . The buffer has been resized but the data hasn't been written to the buffer. The buffer may still contain audio data from the previous file but it depends whether the memory for the buffer needed to be moved when it was resized. In any case we'll probably get a glitch.
- It's possible that the
getNextAudioBlock() function could be interrupted by code in the openButtonClicked() function. Suppose this happens just after and that the openButtonClicked() function has just reached . The buffer might be resized to be shorter than it was but we already calculated our starting point a few lines earlier. This could lead to a memory access error and the application could crash.
- The
getNextAudioBlock() function could be interrupted while calling the AudioSampleBuffer::copyFrom() function. Again depending in the implementation of this we could end up accessing memory that we shouldn't.
- Warning
- There are a number of other things that could go wrong. You may be familiar with using a critical section to synchronise memory access between threads. This is just one possible solution but care should be taken using a critical section in audio code as it can lead to priority inversion which could cause audio drop outs. We look at a solution that avoids critical sections in Tutorial: Looping audio using the AudioSampleBuffer class (advanced).
Notes
Two seconds of stereo audio at 44.1kHz would use 705,600 bytes in an AudioSampleBuffer object because there are:
- 2 channels
- 2 seconds
- 44,100 samples
- 4 bytes-per-sample (using the
float type)
Multiply these together and the result is: 2 x 2 x 44100 x 4 = 705600
Summary
In this tutorial we have introduced:
- How to read audio data directly from a sound file.
- How to copy the data into a buffer for playback.
- The basis for simple sampler applications and synthesisers using wavetable buffers.
- Some of the potential multithreading issues that exist in audio applications.
See also
