Brief
Whilst working on the design of an App-based platform, it became clear that little was known about the true usage of the system. Although usage data was being collected, the data was poorly organised, existing analyses were not particularly insightful and it was unclear whether the data was reliable.
I took responsibility for improving the analytics, with the aim of transforming understanding of usage and improving business and design decision making.
Approach
Tableau is a market-leading data visualisation tool that allows complex analytics to be created using a relatively easy-to-use desktop front end. These analyses can then be shred with a wider audience via Tableau Server.
A proof-of-concept integration of Tableau Server had already been created, connected to the usage data and I chose to continue with this approach. I learned to use Tableau and started work on analysing and cleaning up the data, working closely with the database development team.
Data Analysis
Usage data was collected via two separate routes – one dealing with installed applications and one dealing with web-based Apps. These two sources were collected into a single database, but events were still recorded in quite different ways, using separate sets of columns. This made it very difficult to perform any global analyses.
I examined the different structures, building a glossary of the fields and data values within them. As a result of the way the data was captured, three or four fields had to be examined just to determine what type of event was represented by a single record. Gaining a full understanding of each event required at least eleven fields. This meant that building analyses was extremely difficult, and there was always a high risk of making errors.
In order to make analysis simpler, what was required was that records were converted into a single data model, representing a ‘best interpretation’ of the meaning of each event.
To do this I developed a set of calculated fields in Tableau that would examine each record and return a set of seven clearly-defined fields, describing the key aspects of each record.
- User ID
- Event Type
- Event Source
- App Name
- Page Name
- Search Term
These ‘reference fields’ could then be used to easily and safely build global analyses.
Testing
To test the work, I used my own login to the product to run through ‘scripts’ of activity, opening pages and launching apps, and recording the time at which each action was taken. Once my usage trace appeared in the database, I could then link up the actions that I had taken with the events that had been recorded by the system. This allowed me to check my interpretation of the data and ensure that the ‘reference fields’ were giving a correct output.
SQL Implementation
In order to make the improved interpretation available to everyone and to reduce the calculation loads on Tableau, it was decided to implement the reference fields in the database. I worked closely with development during this process, explaining the purpose of each field and the calculation that I had used to create it. They implemented each calculation in SQL and I then tested the output, checking that it matched that from the Tableau calculations and working with them to correct any gaps.
Usage Analysis
Working with the new data interpretation, I developed analyses of key aspects of the system, showing how regularly users launched apps, what they searched for, and which apps were used the most. These helped to transform business understanding of how the system was being used and change the investment priorities going forward.
To assist sales and support staff, I created models which allowed them to select an individual client or end user and see details of how they were using the system.
Whilst the online reports were powerful, it became clear that many people did not have access, or would not regularly log in to see updates. To address this, automated PDF outputs were created which could reach a wider audience.
Martin Colebourne 