Whilst working in my current role as a Digital Analytics Manager, I oversee the analytics across various data sets for all of our food brands. While this might not be classified as ‘Big Data’, considering we are dealing with text file sizes of roughly 40mb or 50mb each, I have come to understand that data in itself is dumb.
The pitfalls of running an analytics division are many – over time, as end users become more educated, their analytics requests become advanced. This results in more time taken to do the analysis. However, the real analysis is still done by an analyst behind tableau dashboards and other SaaS tools. Therefore, while analyst resources in the team remain the same (man days available), the analytics becomes increasingly complicated over time.
This is a catch-22 situation for any analytics team. On one end, demand explodes and there is a need to do more and more complex pieces of analytics; however, the analyst resources available are not scalable. Whilst there are many things one can do to optimise the analytics delivery – sharper insights, prioritisation of work, etc. – these are stop-gap measures to delay the inevitable increase in team size. This ensures that the analysis of Big Data or any data is not a scalable process.
Up until now, the only algorithmic assistance analysts have had has been around alerts – when an event deviates above/below an average threshold, an automated email alert is sent to the recipient.
Thankfully, things are starting to look up. Over the last couple of months, I am starting to see a realisation dawn upon Analytics SaaS tool providers that understand the above catch-22 situation. One of the outcomes of this knowledge is the ability to offer Algorithmic Insight.
What is this you ask? Algorithmic Insight is the ability to derive qualitative/ quantitative insight from the various data sets without too much manual intervention. This allows for complex analytics capabilities in an automated fashion and frees up analyst time for more exploratory and innovative work.
So here is to 2016. Only time will tell where we end up but for now, the future does look promising,