We examine the areas of artificial intelligence risk. There are many potential exposures, including security and privacy issues as well as bias, accuracy, and complete fabrication of the results
.
The rapid development of artificial intelligence (AI), in particular generative AI (GenAI), and chatbots offers businesses a wealth opportunities to improve their customer service, drive efficiency, and speed up labor-intensive tasks. GenAI is not without its problems. These range from security issues and privacy concerns to bias, accuracy, and even hallucinations where the AI response was completely false. This has been brought to the attention by legislators and regulators. Customers’ internal compliance functions are now playing catch-up to a rapidly evolving and complex technology.
This article examines AI and the risks it poses to legal and regulatory environments. We also discuss why compliance teams should take a close look at GenAI in order to identify weaknesses and vulnerabilities and how reliable the source and output data are.
GenAI or large language models are the most common enterprise AI project. These can be used as chatbots to answer questions or make product recommendations. Another popular use is searching, summarizing or translating documents.
AI is also used in areas like fraud detection, surveillance and medical imaging and diagnostics; all areas with much higher stakes. This has led to questions regarding the use of AI. Organisations have found that AI systems can produce errors as well as inaccurate results.
Confidential Data
AI Tools have also leaked sensitive data, either directly through employees uploading confidential documents to an AI Tool or indirectly via employees.
And then there is bias. The latest AI algorithms are complex, especially those used in LLMs. It is difficult to understand how an AI system came to its conclusions. This makes it difficult for an enterprise to explain or justify what an AI tool such as a bot has done.
The result is a wide range of risks for businesses, particularly those in regulated industries or the public sector. On top of legislation like the European Union (EU) AI Act, regulators are updating existing compliance frameworks quickly to cover AI risks.
According to Forrester’s research, GenAI poses more than 20 new security threats. Some of these include the failure to use secure codes to build AI systems or malicious actors who tamper AI models. Other risks, such as data leakage and data tampering, or a lack in data integrity, can lead to regulatory failures, even if a model is secure.
The problem is exacerbated by the rise of “shadow AI”, in which employees use AI tools without official approval. James Bore warns that the most common deployments will be those enterprises aren’t aware of.
This can range from shadow IT within departments to individuals feeding corporate information to AI in order to simplify their role. Even those companies that have considered compliance with AI have only limited controls in place to prevent misuse.
AI source data issue
The enterprise’s first area to control is the way they use data in AI. This applies both to the model training phase and to the inference or production phase of AI.
Enterprises must ensure they have the right to use data in AI. Copyright is important, especially when it comes to third-party data. General Data Protection Regulations (GDPR) as well as industry regulations cover personal identifiable information used in AI. Organisations shouldn’t assume that existing data processing consent covers AI application.
There’s also the issue of data quality. If an organization uses poor-quality datasets to train a model the results are inaccurate or misleading.
This creates compliance risks – and the risks may not be eliminated, even if a company uses anonymised data.
Ralf Lindenlaub, chief solution officer at Sify Technologies and an IT and cloud service provider, warns that “source data remains one the most overlooked risks in enterprise AI.” “These practices are in violation of UK GDPR and EU Privacy laws,” he says. “Anonymisation can also give a false sense that you are protected. Many of these data can be reidentified or contain systemic bias.
“Public Data used in large language model from global tech companies often fails to meet European Privacy Standards.” To make AI truly reliable, organisations need to carefully curate and manage the datasets that they use, particularly when models can influence decisions that may affect individuals or regulated outcomes.
Another level of complexity comes from where AI models are used. The most common LLMs, despite the growing interest in on-premise AI, are cloud-based. Check that your cloud providers have given you permission to move your data.
AI models and compliance
There are also compliance and regulatory issues that apply to the outputs from AI models.
One of the most obvious risks is that confidential AI results are leaked or stolen. This risk increases as firms connect their AI systems with internal documents or data.
There are cases where AI users have accidentally or maliciously exposed confidential information through their prompts. One of the causes is using confidential data without proper safeguards to train models.
There’s also the risk that the AI model’s output will be incorrect.
“AI results can appear confident, but they could be completely false, biased or even violate privacy,” warns Sify’s Lindenlaub. “Enterprises underestimate the damage that a flawed outcome can cause, from discriminatory hiring through to incorrect legal and financial advice. These risks can become operational liabilities without rigorous validation and human oversight.
The risk is even greater with “agenttic” AI systems where multiple models work together to run an entire business process. If the output of one model is biased or incorrect, this error will be compounded by moving from agent to agent.
The regulatory consequences could be severe as one incorrect output may result in many customers being denied credit or a job interview.
James Bore says that the most obvious problem is that AI outputs generate language and not information. “Despite how they’re presented LLMs don’t analyse, they have no understanding, nor do they have weightings for truth versus fiction except those that are built into them when they are trained.
He adds, “They hallucinate in a very convincing way, because they are good with language.” “They cannot be trusted without thorough fact checking – and not by a second LLM.”
By Jerald Murph
