Tutorial: App analytics collection

Collect app usage data from users in JUCE applications. Send analytics events to Google Analytics using the analytics module.

Level: Intermediate

Platforms: Windows, macOS, Linux, iOS, Android

Classes: ThreadedAnalyticsDestination, ButtonTracker, WebInputStream, CriticalSection, CriticalSection::ScopedLockType

Warning
This project requires a Google Analytics account. If you need help with this, follow the instructions on the Google Analytics website to open an account.

Getting started

Download the demo project for this tutorial here: PIP | ZIP. Unzip the project and open the first header file in the Projucer.

If you need help with this step, see Tutorial: Projucer Part 1: Getting started with the Projucer.

Please make sure you have your Google Analytics API key written down and ready for this tutorial to fully work.

The demo project

The demo project shows a very simple UI with two buttons sending analytics events when pressed. Since the API key has not been set up yet, Google will not receive any events before implementation.

The demo project app window
Note
This project uses Google Analytics to track app analytics but you can apply this to any other service you wish to use.
The code presented here is broadly similar to the AnalyticsCollection from the JUCE Examples.

Anatomy of events

Events describe how the user has interacted with the content in applications and are sent to the analytics tracking system. To better categorise and filter the interactions, events are structured using the following keywords:

  • Category: Describes groups of events that are combined under the analytics reports.
  • Action: Specifies the action that was performed to trigger the event.
  • Label: Additional information explaining the specific object that interacted with the user.
  • Value: Optional integer to provide numerical data to the event in question.

All the events are sent with a unique user ID and a timestamp along with the keywords mentioned above. Additionally, users can be grouped into categories to better describe their capacity such as beta testers or developers.

API key set up

The first step for the project to work properly is to set up the Google Analytics API key. You can find the Tracking ID in your Google Analytics dashboard here:

Google Analytics Tracking ID

Copy this ID, and replace the apiKey placeholder variable in the GoogleAnalyticsDestination class:

apiKey = "UA-XXXXXXXXX-1"; // Change this part of the code to your personal API key.
Warning
Ideally, this API key should not be visible in your binary distribution as there could be all sorts of malicious uses if discovered and may pollute your analytics data with spam. One way to prevent this would be to retrieve the API key dynamically at runtime (such as from your own server).

Tracking app startup

Let's first start by tracking user-independent information such as app launches and define constant user information that will be used by the analytics system. In the constructor of the MainContentComponent class, we start by getting a reference to the Analytics singleton by calling Analytics::getInstance().

We can then set the user ID with setUserID() by choosing a unique identifier for this user [1]. Make sure not to include any sensitive personal information in this identifier. We can also set a user group on this user by calling setUserProperties() using a StringPairArray [2].

For the events to be received, we need to specify at least one destination to our Analytics instance. We can optionally add multiple destinations if we wish. In this case we add an instance of the GoogleAnalyticsDestination class to the singleton [3].

Since the MainContentComponent constructor gets called when the MainWindow is instantiated, we can log this event using the function logEvent() right when the component gets owned by the MainWindow [4].

MainContentComponent()
{
Analytics::getInstance()->setUserId ("AnonUser1234"); // [1]
StringPairArray userData;
userData.set ("group", "beta");
Analytics::getInstance()->setUserProperties (userData); // [2]
Analytics::getInstance()->addDestination (new GoogleAnalyticsDestination()); // [3]
Analytics::getInstance()->logEvent ("startup", {}, DemoAnalyticsEventTypes::event); // [4]
//...

Likewise, we can log the shutdown event in the MainContentComponent destructor right when the MainWindow gets deleted [5].

~MainContentComponent()
{
Analytics::getInstance()->logEvent ("shutdown", {}, DemoAnalyticsEventTypes::event); // [5]
//...

Tracking Button behaviour

In order to add tracking to specific user actions, we need to define which user interactions we want recorded and sent. Fortunately to record button behaviour, we can use a handy class included in the JUCE analytics module called ButtonTracker that will automatically handle this for us.

Let's first declare a ButtonTracker as a member variable in the MainContentComponent class [1].

private:
//==============================================================================
TextButton eventButton { "Press me!" }, crashButton { "Simulate crash!" };
std::unique_ptr<ButtonTracker> logEventButtonPress; // [1]
//...

Now in the MainContentComponent constructor, we can link the specific TextButton object we want to track by passing it as an argument to the ButtonTracker constructor. We also set the event category and action properties to send when the event is fired [2].

MainContentComponent()
{
//...
StringPairArray logButtonPressParameters;
logButtonPressParameters.set ("id", "a");
logEventButtonPress.reset (new ButtonTracker (eventButton, "button_press", logButtonPressParameters)); // [2]
}
Exercise
Create additional GUI components and implement tracking on them with different event parameters.

Sending events

The JUCE analytics module handles the logging of events on a dedicated thread and sends the analytics data in batches periodically. Therefore, we need to temporarily store the events on local storage until the data is sent. In the rest of this tutorial, we will be working in the GoogleAnalyticsDestination class.

We first need to specify a location to store our analytics event data in the application data directory. For this we use the special location File::userApplicationDataDirectory to find the correct location and navigate to the corresponding application folder for our app [1]. If the location does not exist we create the folder [2] and save the file path as an XML file name extension [3].

We can now start the thread by using the startAnalyticsThread() function and specifying the waiting time between batches of events in milliseconds [4].

GoogleAnalyticsDestination()
: ThreadedAnalyticsDestination ("GoogleAnalyticsThread")
{
{
.getChildFile (JUCEApplication::getInstance()->getApplicationName()); // [1]
if (! appDataDir.exists())
appDataDir.createDirectory(); // [2]
savedEventsFile = appDataDir.getChildFile ("analytics_events.xml"); // [3]
}
//...
startAnalyticsThread (initialPeriodMs); // [4]
}

In the class destructor, we have to ensure that the last batch of events can be sent without the application being killed by the operating system. To allow this, we provide one last batch period while sleeping the thread before stopping it forcibly after 1 second. This provides enough time for one last sending attempt without elongating too much the application shutdown time.

~GoogleAnalyticsDestination()
{
Thread::sleep (initialPeriodMs); // [5]
stopAnalyticsThread (1000); // [6]
}

We can supply the maximum number of events to send in batches by overriding the getMaximumBatchSize() function like so:

int getMaximumBatchSize() override { return 20; }

Formatting the HTTP request

Now we need to format the correct HTTP request to log these events to the analytics server. The URL we are trying to construct with its corresponding POST data in the case of a button press behaviour for example looks something like this:

POST /batch HTTP/1.1
Host: www.google-analytics.com
v=1 // Version Number
&aip=1 // Anonymise IP
&tid=UA-XXXXXXXXX-1 // Tracking ID
&t=event // Log Type
&ec=button_press // Event Category
&ea=a // Event Action
&cid=AnonUser1234 // User ID
  • [v]: The batch logging API version.
  • [aip]: The IP address of the sender is anonymised.
  • [tid]: The Tracking ID for the corresponding app.
  • [t]: The type of logging for the analytics system.
  • [ec]: The category identifier for the logged event.
  • [ea]: The action identifier for the logged event.
  • [cid]: The user ID for the corresponding user.

In a typical app lifecycle, the batched logger will first process the appStarted event when the application is fired up. Then when the user clicks on the button we log the button_press event and finally log the appStopped event when the application quits.

In order to account for these 3 logging scenarios, we need to construct different requests in the logBatchedEvents() function:

bool logBatchedEvents (const Array<AnalyticsEvent>& events) override
{
String appData ("v=1&aip=1&tid=" + apiKey); // [1]
StringArray postData;
for (auto& event : events) // [2]
{
switch (event.eventType)
{
case (DemoAnalyticsEventTypes::event):
{
data.set ("t", "event");
if (event.name == "startup")
{
data.set ("ec", "info");
data.set ("ea", "appStarted");
}
else if (event.name == "shutdown")
{
data.set ("ec", "info");
data.set ("ea", "appStopped");
}
else if (event.name == "button_press")
{
data.set ("ec", "button_press");
data.set ("ea", event.parameters["id"]);
}
else if (event.name == "crash")
{
data.set ("ec", "crash");
data.set ("ea", "crash");
}
else
{
continue;
}
break;
}
default:
{
break;
}
}
data.set ("cid", event.userID); // [3]
StringArray eventData;
for (auto& key : data.getAllKeys()) // [4]
eventData.add (key + "=" + URL::addEscapeChars (data[key], true));
postData.add (appData + "&" + eventData.joinIntoString ("&")); // [5]
}
auto url = URL ("https://www.google-analytics.com/batch")
.withPOSTData (postData.joinIntoString ("\n")); // [6]
//...
  • [1]: We start by adding the version number, anonymised IP and tracking ID to the appData string variable.
  • [2]: Then for each event in the batch, we determine the type of event in question to set its category and action properties. If the event is a startup or a shutdown, we set the event category to "info" and set the action property to "appStarted" or "appStopped" respectively. If the event is a button pressing, we set the event category to "button_press" and retrieve its action property from the id parameter of the ButtonTracker.
  • [3]: We also set the user ID for the event to log.
  • [4]: Now for all the individual StringPairArray entries, we concatenate keys with their corresponding values by inserting an equal sign in between and escaping any special characters from the URL.
  • [5]: Finally, we can join all the event parameters together with ampersand signs in between and by prepending the initial appData content to the front.
  • [6]: The URL is eventually constructed with its POST data appended line by line. This way we can send multiple events in a single HTTP request.
Exercise
Modify the code above to handle all event properties including label and value attributes.

Now that we have our URL ready we need to send the request to the server by creating a WebInputStream. We first have to lock the CriticalSection mutex declared as a member variable called webStreamCreation. Using a ScopedLock object allows us to automatically lock and unlock the mutex for the piece of code delimited by the curly brackets [1].

If the stopLoggingEvents() function was previously called due to the application terminating, we return immediately without attempting to initialise the WebInputStream [2]. Otherwise, we can create it in a std::unique_ptr by passing the previously constructed URL as an argument and using POST as the method [3].

We can then connect to the specified URL and perform the request using the connect() function on the WebInputStream [4]. If the response is successful, we just return positively from the function. Otherwise, we set an exponential decay on the batch period by multiplying the previous rate by 2 and return negatively from the function [5].

//...
{
const ScopedLock lock (webStreamCreation); // [1]
if (shouldExit) // [2]
return false;
webStream.reset (new WebInputStream (url, true)); // [3]
}
auto success = webStream->connect (nullptr); // [4]
if (success)
periodMs = initialPeriodMs;
else
periodMs *= 2;
setBatchPeriod (periodMs); // [5]
return success;
}

When the application shuts down, we need to cancel connections to the WebInputStream if there are any that are concurrently running. By first acquiring the lock from the same CriticalSection object using a ScopedLock, we ensure that the previously encountered critical section of the code in the logBatchedEvents() function will have terminated before [1]. Setting the shouldExit boolean to true prevents any new connections from being created subsequently [2]. Then we can finally cancel any WebInputStream connections using the cancel() function if there are any [3].

void stopLoggingEvents() override
{
const ScopedLock lock (webStreamCreation); // [1]
shouldExit = true; // [2]
if (webStream.get() != nullptr) // [3]
webStream->cancel();
}

This completes the part of the tutorial dealing with logging events. However, if the transmission of event data fails and the application terminates, we currently have no way of keeping track of unlogged events.

Save and restore unlogged events

This section will cover the use of XML files to store any unlogged events to disk in the case of a lost connection.

The XML document storing unlogged event information will look something like this for a single button press:

<?xml version="1.0"?>
<events> // Root XML element for the whole document.
<google_analytics_event name="button_press" type="event" timestamp="xxxx" user_id="AnonUser1234"> // Event node with name, type, timestamp and user ID.
<parameters id="a"></parameters> // Parameters related to the parent event.
<user_properties group="beta"></user_properties> // Properties for the user in the parent event.
</google_analytics_event>
//...
</events>

We will look at the saveUnloggedEvents() and restoreUnloggedEvents() functions that deal with saving and restoring events respectively. The saveUnloggedEvents() function will build an XML structure based on the format shown above and save the content in an XML file:

void saveUnloggedEvents (const std::deque<AnalyticsEvent>& eventsToSave) override
{
XmlDocument previouslySavedEvents (savedEventsFile);
std::unique_ptr<XmlElement> xml (previouslySavedEvents.getDocumentElement()); // [1]
if (xml.get() == nullptr || xml->getTagName() != "events") // [2]
xml.reset (new XmlElement ("events"));
for (auto& event : eventsToSave)
{
auto* xmlEvent = new XmlElement ("google_analytics_event"); // [3]
xmlEvent->setAttribute ("name", event.name);
xmlEvent->setAttribute ("type", event.eventType);
xmlEvent->setAttribute ("timestamp", (int) event.timestamp);
xmlEvent->setAttribute ("user_id", event.userID);
auto* parameters = new XmlElement ("parameters"); // [4]
for (auto& key : event.parameters.getAllKeys())
parameters->setAttribute (key, event.parameters[key]);
xmlEvent->addChildElement (parameters);
auto* userProperties = new XmlElement ("user_properties"); // [5]
for (auto& key : event.userProperties.getAllKeys())
userProperties->setAttribute (key, event.userProperties[key]);
xmlEvent->addChildElement (userProperties);
xml->addChildElement (xmlEvent); // [6]
}
xml->writeToFile (savedEventsFile, {}); // [7]
}
  • [1]: First we retrieve any previously saved events from the XML file stored at the previously defined file location and build an XmlElement based on it.
  • [2]: If the XmlElement does not exist or does not have the root "events" node, we create it.
  • [3]: For each unsaved event in the queue, we create a "google_analytics_event" node with the event name, type, timestamp and user ID as attributes.
  • [4]: We also create a "parameters" node as a child node to the previously created one with event parameters as attributes to it.
  • [5]: At the same hierarchy level, we create a "user_properties" node as a child node with user properties as attributes to it.
  • [6]: We can then add the individual event nodes as children to the root "events" node.
  • [7]: Finally, we write the XML structure to the XML file and store the events.

On the other hand, the restoreUnloggedEvents() function will in turn read an XML structure based on the same format shown previously and fill up the event queue:

void restoreUnloggedEvents (std::deque<AnalyticsEvent>& restoredEventQueue) override
{
XmlDocument savedEvents (savedEventsFile);
std::unique_ptr<XmlElement> xml (savedEvents.getDocumentElement()); // [1]
if (xml.get() == nullptr || xml->getTagName() != "events") // [2]
return;
auto numEvents = xml->getNumChildElements();
for (auto iEvent = 0; iEvent < numEvents; ++iEvent)
{
auto* xmlEvent = xml->getChildElement (iEvent); // [3]
StringPairArray parameters;
auto* xmlParameters = xmlEvent->getChildByName ("parameters"); // [4]
auto numParameters = xmlParameters->getNumAttributes();
for (auto iParam = 0; iParam < numParameters; ++iParam)
parameters.set (xmlParameters->getAttributeName (iParam),
xmlParameters->getAttributeValue (iParam));
StringPairArray userProperties;
auto* xmlUserProperties = xmlEvent->getChildByName ("user_properties"); // [5]
auto numUserProperties = xmlUserProperties->getNumAttributes();
for (auto iProp = 0; iProp < numUserProperties; ++iProp)
userProperties.set (xmlUserProperties->getAttributeName (iProp),
xmlUserProperties->getAttributeValue (iProp));
restoredEventQueue.push_back ({ // [6]
xmlEvent->getStringAttribute ("name"),
xmlEvent->getIntAttribute ("type"),
static_cast<uint32> (xmlEvent->getIntAttribute ("timestamp")),
parameters,
xmlEvent->getStringAttribute ("user_id"),
userProperties
});
}
savedEventsFile.deleteFile(); // [7]
}
  • [1]: Same as before, we retrieve any previously saved events from the XML file stored at the previously defined file location and build an XmlElement based on it.
  • [2]: If the XmlElement does not exist or does not have the root "events" node, we return from the function as there is nothing to do.
  • [3]: We first retrieve a single event child node to parse from the root parent.
  • [4]: For each attribute from the child "parameters" node, we set a key/value pair and add it to a StringPairArray.
  • [5]: For each attribute from the child "user_properties" node, we set a key/value pair and add it to a StringPairArray.
  • [6]: We can then push the individual events back into the event queue by setting the corresponding parameters from the StringPairArray objects.
  • [7]: Finally, we delete the XML file from disk when done.
Note
We used XML as a serialisation format but if we need to save large amounts of unsaved events, a binary format would be more efficient.
Exercise
Save and restore unlogged events in a different serialisation format such as JSON or in a binary format.

Summary

In this tutorial, we have learnt how to track usage data with Google Analytics and the JUCE analytics module. In particular, we have:

  • Sent analytics events to Google Analytics on a separate thread.
  • Stored unsent events locally in an XML document.
  • Restored saved events from the XML document to the event queue.

See also

WebInputStream
An InputStream which can be used to read from a given url.
Definition: juce_WebInputStream.h:36
TextButton
A button that uses the standard lozenge-shaped background with a line of text on it.
Definition: juce_TextButton.h:43
StringPairArray::set
void set(const String &key, const String &value)
Adds or amends a key/value pair.
String
The JUCE String class!
Definition: juce_String.h:42
File::getSpecialLocation
static File JUCE_CALLTYPE getSpecialLocation(const SpecialLocationType type)
Finds the location of a special type of file or directory, such as a home folder or documents folder.
GenericScopedLock
Automatically locks and unlocks a mutex object.
Definition: juce_ScopedLock.h:58
URL::addEscapeChars
static String addEscapeChars(const String &stringToAddEscapeCharsTo, bool isParameter, bool roundBracketsAreLegal=true)
Adds escape sequences to a string to encode any characters that aren't legal in a URL.
URL::withPOSTData
URL withPOSTData(const String &postData) const
Returns a copy of this URL, with a block of data to send as the POST data.
File::getChildFile
File getChildFile(StringRef relativeOrAbsolutePath) const
Returns a file that represents a relative (or absolute) sub-path of the current one.
StringPairArray::getAllKeys
const StringArray & getAllKeys() const noexcept
Returns a list of all keys in the array.
Definition: juce_StringPairArray.h:91
Array< AnalyticsEvent >
StringArray::add
void add(String stringToAdd)
Appends a string at the end of the array.
ThreadedAnalyticsDestination
A base class for dispatching analytics events on a dedicated thread.
Definition: juce_ThreadedAnalyticsDestination.h:57
File::createDirectory
Result createDirectory() const
Creates a new directory for this filename.
StringArray::joinIntoString
String joinIntoString(StringRef separatorString, int startIndex=0, int numberOfElements=-1) const
Joins the strings in the array together into one string.
URL
Represents a URL and has a bunch of useful functions to manipulate it.
Definition: juce_URL.h:41
ButtonTracker
A class that automatically sends analytics events to the Analytics singleton when a button is clicked...
Definition: juce_ButtonTracker.h:43
jassertfalse
#define jassertfalse
This will always cause an assertion failure.
Definition: juce_PlatformDefs.h:132
Thread::sleep
static void JUCE_CALLTYPE sleep(int milliseconds)
Suspends the execution of the current thread until the specified timeout period has elapsed (note tha...
File::userApplicationDataDirectory
The folder in which applications store their persistent user-specific settings.
Definition: juce_File.h:862
XmlElement
Used to build a tree of elements representing an XML document.
Definition: juce_XmlElement.h:141
JUCEApplication::getInstance
static JUCEApplication *JUCE_CALLTYPE getInstance() noexcept
Returns the global instance of the application object being run.
XmlDocument
Parses a text-based XML document and creates an XmlElement object from it.
Definition: juce_XmlDocument.h:67
StringPairArray
A container for holding a set of strings which are keyed by another string.
Definition: juce_StringPairArray.h:38
StringArray
A special array for holding a list of strings.
Definition: juce_StringArray.h:38