Podcast: How (agenttic) AI can assist with unstructured data (19459000)
We talk to Boris Bialek in this podcast about how artificial intelligence can help with the discovery and management unstructured data.
Bialek explains how AI can be used to bring together different types of information an organisation may hold about its customers in order to make processes more efficient. He also discusses how multiple AI agents could work together to make the processes work.
In an agentic manner
Can AI help with the discovery and management unstructured data?
The recovery of unstructured information is one of the oldest IT tasks.
The first step was to scan papers and try to make pictures. Later, people began typing the information. Imagine that you receive a document written by hand about an accident and you have to try to make it readable. AI can now do this for you in a matter of seconds.
It can also understand and reason with the data. It can raise the intellectual level of “I have an image” to “I can extract sentences that consist of “accident”, “bicycle”, “street”, and “the mountain was steeper then I thought”.
This is where AI can really help. It can be images, text, or sound.
Classic database model, the
RDBMS is a great tool for structured data. This so-called structured information is mostly textual, and can also be numbers. But anything that has a structure can be put into a spreadsheet. Everything else is considered unstructured. This is unfair.
With AI, we are able to take this data to a new level and interpret it in an intelligent way.
Which approaches exist in the use AI to discover and maintain unstructured data for customers?
Ask any startup and they will tell that they are the only solution for this one.
When we take a more sophisticated view, there are essentially two ways. The first is to analyze the data and then build a solution based on it. The most important thing is to combine fresh data with existing information. I can use unstructured data like video, audio, and other things.
Boris, for example, has an insurance number and Boris also has a contract to work with Antony’s insurer. This first approach is to use mashups of operational data, metadata, and reference data with what we call signals.
The other option is to break it down into a horses for courses approach, the best horse on the best racetrack.
This is a problem that has solutions. EncoreCloudAI or PurpleFabricAI are two options.
These solutions allow me to convert the data to an intelligent form so I don’t have to start from scratch. I can then get my data and put it in an operational data store. I can then get my legacy data, and pull data from there. This could be documents or physical papers. These could be in legacy documents archives or document management system. This is, in my opinion the fastest way.
Having said that, there are many good reasons to build one yourself. If you have a specific need, for example, if there is video information you want to process in a specific format, you can build your own. You may want to ensure that someone pays the toll when they drive through a tollgate on a highway.
In some cases, writing your own code is a good idea. But it’s about combining the data from the existing data and new data, unstructured data.
This is what makes intelligence really work.
What are the main benefits of using these techniques on the data?
I can create a completely new picture of my environment. In a classical relational database such as an ERP [enterprise resource planning] that knows your sales numbers you know how much money you make.
Your CRM [customer relationship management] might tell you that Boris is a great customer and is currently on your website. What does Boris want? I could use the classic approach of a BI system [business intelligence] and say that Boris falls into the category white male, middle aged person, and perhaps he is looking to buy a new bike. Let’s give him a bike.”
However, that’s not what you might know about Boris. Boris might have bought a bike from you last week, and may be looking for a helmet now.
When you combine these things, you want to provide more intelligence to your customers in the retail space. In a positive sense, you want to be relevant and help them. You don’t wish for them to ask, “Why does he show me this stuff?” I’m not interested.”
Let’s also say we have an Insurance Case, someone bumped my bike, it was parked outside the house and now I have a Repair Case. I then go to my insurance. If they are able to understand the information I give them very quickly, then they can manage claims very quickly.
If they do that, I can be a satisfied client and not worry about my bicycle being damaged, who pays for that, etc. After an hour, I get a reply: “Yes, your bicycle is insured.” We will fix it, don’t be worried.
These are the reasoning parts that were not possible before. You couldn’t put so many data into context.
Second, there is the natural language processing. Boris can tell the insurance company, “My bicycle was damaged.” My bike was parked right in front of the house door. It was hit by a truck.”
By that time, the system could already interpret this as “bike, bike – he owns a bicycle and it’s insured. He’s probably referring to his house door”.
This is reasoning, so the system can assume a whole lot and say “Hey Boris. Is this the bicycle that you’re referring to? Was it parked outside your house in the village? You are insured. You are insured. I can always ask an agent to speak to me, but this is faster and there are no waiting times. I can solve my problem and move on.
This automation of routine tasks can be done by an AI system. Tagging things, entering things, all these things can be done with ease. It is also repeatable. You can do the same thing again with the same system.
There’s a lot to discuss about hallucinations. But today’s embedding model, such as VoyageAI are so good in terms of quality and reranking systems that allow answers to be categorized as good, bad, and ugly based upon my data.
Does agentic AI play a role in this? How would it work?
Agentic AI is like a soccer player, but you need 11 players to make a good team. Agents perform specific functions, just like different positions on a football team.
When we look at the insurance case, an agent will check out what contracts Boris holds, a system can determine Boris’s location and where this tractor might have come from. Is this a true description of the events?
As digital experts, different agents work together to create a framework. A soccer team of agents is created to help me as a customer, as well as the insurance company.
This system can provide very good answers to basic questions, bring them together and drive a solution. This is where agentic artificial intelligence really shines.
