According to Gartner, in 2022 data proliferation is one of the top trends to impact infrastructure and operations for many organisations.

The analyst firm says data will continue to multiply in variety, velocity and volume and, as a consequence, continued expansion of data collection and holding efforts will be key to guiding the policies surrounding processing, retention and legal requirements of the enterprise’s data.

In July 2019, a Harvard Business Review Analytics Services Survey reported that some of the top problems customers face result from data silos and the difficulty related to managing multiple new systems, outdated technology, and a greater variety of data formats. These problems ranged from making decisions based on opinions instead of facts because it is hard to make sense of the volume and variety of available data, to having a siloed view of customers across multiple data sources

Whilst many companies have vast amounts of data, very few utilise their data assets to drive real business value. In other words, they are not extracting meaningful insights from their data to make informed decisions, based on patterns, behaviours, trends, and preferences.

Before we get into specifics, it’s important organisations understand there are two parts to effective data analytics.

First, company data comes in a range of formats and sources, which has grown in variety and volume over the past decade. This has been compounded further by the pandemic, which has seen a rapid proliferation of remote work productivity and communications tools.

Second, many organisations still believe they cannot afford to implement data analytics because it will be too expensive, or that their data is too complex to analyse. That’s just not the case today as the cloud enables companies to reduce cost and eliminate complexity.

Looking at your data environment more holistically

Every single bit of content that a business creates is data. So, it’s essential for an organisation to know what data assets they have and where they live.

In today’s digital world, an organisation will have data sitting in the cloud, alongside more traditional on-premises databases or CRM systems. More and more, these applications connect to one another. For example, in a cloud environment you can bring in data from different locations and analyse against a whole range of other interesting content. This ability to look at your whole environment more holistically, instead of just looking at, say, a CRM system, has been a significant technological shift.

This ease of connectivity into your data is actually a real fundamental business driver and an opportunity. For example, Azure Purview is a unified data governance solution that enables you to manage and govern your on-premises, multi-cloud and software-as-a-service data. A solution like this pretty much harvests all that data for you.

These tools, and others, simplify the process of data analytics and reduce the costs. You also don’t need to be a data scientist to use them because the analytics and AI tools are designed to be straight forward and easy to use, analysing the data and generating reports, and removing the need to create the constructs yourself. And working with specialist consultants, like NovaWorks can make this process even more seamless.

Once you have centralised visibility of all your data, its format, where it resides and what information is in those sources, you can analyse it using already available AI and machine learning tools, automatically surfacing the data you need.  Another important layer of data analytics is building in compliance obligations. This may be conversations happening in Microsoft Teams and Zoom, which are recorded and become another dataset. This can include communications compliance around bullying and cyber harassment. For example, any content that contains profanity or contains a regulatory word or even a credit card can be automatically flagged.

Building a Business Case

 At NovaWorks we help our clients look at the holistic datasets and then work with them to identify what they want to change, or what data they want to be able to identify across their whole organisation. Essentially, what we are trying to do is bring data out of specific siloes, make it discoverable and searchable, so that data can be analysed from a central point. By surfacing all the necessary data, we can work out how a company wants to compartmentalise it across their organisation, because different teams or business functions will most likely have a different lens or filter on what’s going to be important to them.

For example, a senior management team is going to be more focused on business strategy and interrogating the data to perhaps predict what the next quarter is actually going to look like, or get some insights on trends or patterns in consumer behaviour. Whereas an IT manager may be more focused on data that tracks compliance with the IT policy or trends in data usage and how that impacts employee experience.

As such, data analytics is not necessarily just looking at a sales forecast. It can also be used to assess network traffic. Both are data, which is informed for a different purpose.

Whatever the reason, we work to create the data extract. We have extensive knowledge working with different tools and will determine the right approach and help manage the entire process, reducing the overall cost and simplifying implementation.

Each customer will have a unique business case. Once that business case is established, then there’s the case of executing on it. We help our clients with both. The business case boils down to actually understanding why this an important investment, in terms of what you want to know, understanding what you have and also potentially, how you may mitigate some of your risk in terms of liability or from a compliance or regulatory point of view.

Use Case – Australian Parliament customer

A Parliament in Australia was holding onto historically important records, stored on paper. These old written documents dated back to the 1800s and held important insights on the history of the Parliament and legislation. This was a goldmine of information about the historical decisions that shaped the laws of modern day, but in a format that could not be analysed. The challenge was not only that the data was on paper – it was also recorded in different and varying hand-writing styles, often using old-English terms that are no longer used in modern times.

Using Azure Cognitive Services, we developed custom handwritten recognition models to decode different handwriting styles and inspect old English terms and translate them to the language of today. This technology enabled the Parliament to make the data searchable – giving them the ability to understand matters of historical significance, research historical events, understand trends and make the data available to their citizens, offering complete transparency to the history of legislation.

Training your AI

 Once the data has been surfaced you can then use AI tools to auto classify it.

For example, within a data set you can ask yourself ‘is this particular data set sensitive?’. If it is, when you ingest data you can have a sensitive column which has to be obfuscated before it gets released to analysis. In this case, you protect yourself by not just looking at the security side, but what regulations you may need to have in place. Decisions then need to be made on whether to do that via humans or whether you let the machines work it out.

For classification, what we can actually end up doing is automate some of the metadata management in terms of when we actually import data. For example, if we look at this from a financial services point of view, let’s say a superannuation company, and extract a policy number, first name last name and a balance. The balance alone may be okay, but if it is linked to a policy number plus a first name, last name, tax file number, that may be deemed sensitive. So, what we do is use AI to actually understand what is sensitive data that should be masked by default.

With some of the parliamentary work we track sentiment. For example, from the way someone speaks you can tell if the person is passive, angry, happy or unhappy. Both facial recognition and voice analysis are data too and this type of analysis can also be extremely useful in market research scenarios.

Whilst there is a need for specialist data engineers and data scientists, much of the day to day analytics does not require such expertise. Many of the tools we have today are intuitive enough and remove complexity giving you the confidence to analyse a range of datasets and identify trends to inform decision-making.

Trusted Partner

NovaWorks is a trusted partner to customers with the most stringent levels of confidentiality and handles highly sensitive data, such as federal and state government, financial services and healthcare.

Through our engagements we have discovered that one of the biggest challenges is actually understanding what data an organisation has, and then demonstrating that you don’t have to spend millions and millions of dollars to actually understand it. We then determine the right value you can extract from it.

As with any technology decision, be clear on what it is you are setting out to achieve. For data analytics, once we’ve determined how you get all the data into a centralised point, the next task is establishing what lens you actually want to look at by business or role function. For example, what marketing people will want to see will be very different to the finance team.

If you looking to move from existing, piecemeal solutions to comprehensive analytics solutions that provide faster decision making, improved availability/uptime, scalability and generally avoiding technology performance related slowdowns, please get in touch.  We focus on combining data silos and work with you to:

  • Build a single source of truth, so you can leverage the data to make smarter business decisions in real time rather than retrospectively
  • Delivery of real-time insights and development of predictive capabilities
  • Powerful analysis and visualisation of large data sets across a range of business scenarios, turning data into real business insight and value
  • Leverage AI/ML capabilities to enable delivery at scale, advanced automation, reduce human error, enable faster decision-making, and 24/7 availability

Contact us to talk to one of our Data & Analytics experts.