Public-generative AI chatbots – as exemplified by ChatGPT – have garnered significant attention and made headlines across both the legal and mainstream media. In-House Perspective delves into what they mean for in-house teams.

ChatGPT blazed onto the artificial intelligence (AI) scene when it was launched in November 2022. It experienced an unprecedented rate of uptake with 100 million users registered by January this year. The chatbot’s ability to produce respectable answers to challenging questions and to conduct human-like conversations has amazed users. Everyone from journalists to comedians have tested it out. Some specialists have even called for a halt in AI development until its processes and implications are better understood.

ChatGPT is a form of generative AI that uses a class of machine learning known as a large language model (LLM) to create human-like responses to prompts or questions put to it by humans. It’s trained on data found on the internet and can also use data given to it by users in their prompts to improve its algorithms. This particular AI model has excelled at tasks previous models struggled with, such as creativity and producing new content.

The technology ChatGPT uses is known as a ‘black box’ because it’s impossible to say exactly how a certain model reaches a particular conclusion in each set of circumstances. Fundamentally, its process is unknowable – already, then, we can see red flags for in-house legal.

Spotting the risks

Closer understanding of the way ChatGPT and other consumer-facing generative AI chatbots work begins to reveal some of the key risks for any business considering using them. To begin with, ChatGPT’s responses are generated using complex mathematical calculations and probabilities based on its training data sets. It doesn’t ‘think’ in the way that humans do, however convincing it may seem, and therefore it’s prone to making errors and presenting them as fact. This is referred to as hallucinations. It would be extremely risky, therefore, to rely solely on output generated by ChatGPT for any work that required factual accuracy, particularly for a business operating in a heavily regulated sector. According to Jonathan Emmanuel, a partner at Bird & Bird in London, ‘end users need to be aware that they’re using the AI system at their own risk. It’s unlikely they’re going to be able to bring a meaningful claim against the provider if there’s a mistake in the output the system has generated.’

If you ask ChatGPT a question it will give a different answer each time. This is a feature of the machine learning process it uses. John Buyers, a partner at Osborne Clarke who’s based in both the UK and the Middle East, says this feature is ‘problematic in a rules-based hierarchy where you’re going to be advising on the applicability of rules and laws. You don’t want a different answer every time you ask it the same question.’

The responses ChatGPT gives also reflect the content it’s reviewed, including any biases found there. Its output, therefore, may not be balanced or objective and could include harmful stereotypes or viewpoints. According to Jezah Khamisa, Knowledge and Innovation Manager at Slaughter and May in London, ‘much work is yet to be done in identifying and removing biases in training data sets and we are likely to see further investigation in this area’.

The opaque black box at the heart of ChatGPT also sets off intellectual property (IP) alarm bells. Alfred Meijboom, Co-Chair of the IBA Intellectual Property Law Committee and counsel at Kennedy Van der Laan in Amsterdam, highlights that because ChatGPT’s process is unknowable, it’s very hard to be certain whether a response it gives contains material that’s been copied verbatim from a third-party source and therefore infringes on that party’s copyright. Any output a business is considering using, therefore, should be thoroughly checked beforehand.

It’s also worth considering the risk of becoming the third party whose copyright is infringed by an AI system’s output. This is particularly pertinent for organisations with a business model based on monetising content, which therefore rely on protecting the IP associated with that content.

Meijboom distinguishes between learning and copying in relation to how generative AI works. Some systems may look at information on the internet to learn from it without copying it to their database. This method is less likely to be classed as copyright infringement than when a system copies material directly from the internet onto its database and then learns from that database.

In terms of asserting copyright over generative AI’s output, OpenAI, which developed and owns ChatGPT, assigns this to the user. However, Meijboom says the US Copyright Office (the 'Office') has rejected AI-generated works in the past as they don’t involve sufficient human authorship, a prerequisite for asserting copyright over a work. The Office argues there must be a significant degree of human involvement in the creative process for a copyright application to be successful. It views the human prompts given to generative AI such as ChatGPT as similar to the instructions given to an artist when commissioning a work and therefore not sufficient on their own to qualify as human authorship for the purposes of copyright registration. The Office also says it’ll be looking into ‘how the law should apply to the use of copyrighted works in AI training and the resulting treatment of outputs’.

The use of ChatGPT has also raised privacy and data protection concerns, particularly as some individuals and businesses are failing to exercise caution about the types of information they feed into the system. As part of OpenAI’s terms of use it states that, in relation to data inputted into ChatGPT, ‘we may use the data you provide us to improve our models’. Some argue the safest approach is to avoid putting personal, confidential or sensitive information into ChatGPT.

The Italian Data Protection Authority cited privacy concerns when it banned ChatGPT in the country at the beginning of April. It noted at the time that ‘there appears to be no legal basis underpinning the massive collection and processing of personal data in order to “train” the algorithms on which the platform relies’. ChatGPT was subsequently restored in Italy in late April after OpenAI addressed the regulator’s concerns. The regulator said that OpenAI had explained ‘that it had expanded the information to European users and non-users, that it had amended and clarified several mechanisms and deployed amenable solutions to enable users and non-users to exercise their rights. Based on these improvements, OpenAI reinstated access to ChatGPT for Italian users.’

Emmanuel says it’s important to understand how your supply chain uses generative AI and the risks that might pose. ‘Are [your vendors] sub-contracting part of their service to an AI system like ChatGPT to provide the service to you?’ he asks. ‘If so, what protections do you need in place?’ For example, the business would want to avoid a vendor sharing the organisation’s confidential information with ChatGPT. Equally, if the vendor supplies content to the business that’s been created using public generative AI, in-house legal would want to understand and protect against the risk of third-party IP infringement. It’s also worth considering whether the business is happy for a vendor’s AI system to reuse learnings derived from its data for the benefit of other customers.

Getting to the opportunities

Buyers argues that the most sensible way for a business to use generative AI is through an enterprise solution. ‘What you should be doing is taking the GPT AI model and licensing it through a SaaS [software as a service] provider’, he says. Crucially, in an enterprise solution, the GPT technology is trained predominately on data specific to the business. This gives the business much more control over what goes into the system and therefore what comes out of it. If there has been any pre-training of the system on the wider internet, this should be covered in the enterprise agreement.

The agreement should also include protections against some of the risks posed by using consumer-facing AI models such as ChatGPT – for example, an IP indemnity and IP warranties, data protection provisions and the use of correctly licensed content. It’s important to understand whether the solution plugs into OpenAI directly, which would make it as risky as using ChatGPT, or if it uses a segmented instance of the GPT model, which allows for more opportunities to mitigate risks.

There will be a cost associated with using a customised version of the GPT functionality. It’s an investment worth making, however, because it mitigates the significant risks of using any free versions, such as ChatGPT. Buyers believes there will always be a version of AI that’s more widely accessible to the general public, if not for free then perhaps through an open-source model.

Once the distinction has been made between public-facing generative AI such as ChatGPT and a much less risky private enterprise solution, it becomes possible to think about the opportunities this technology presents. It will be important for in-house counsel to educate the business on the difference between public and private generative AI models.

James Harper, Committee Liaison Officer for the IBA Corporate Counsel Forum and Director of Global Legal at LexisNexis, identifies an opportunity for businesses with databases of content to improve functions such as search and search analysis and to streamline browsing by running a bespoke LLM to generate conversational responses to user queries.

Harper also argues there are opportunities to improve productivity through using generative AI. The legal team could use it to summarise large amounts of information in more succinct forms and thereby optimise its processes. Generative AI could also be used to improve the legal team’s interactions with the business. For example, when providing contracts, an LLM chatbot could ask the user questions and generate a contract based on their responses. This could be a more appealing interaction for the business and therefore drive more people to the department.

Public-generative AI such as ChatGPT could be used by lawyers to spark the creative process, for example when drafting a contract. It could streamline the process and make it more efficient. It should, however, only be used as a starting point because of the risks associated with using the content generated by ChatGPT without reviewing it.

Lawmakers respond

In 2021, the European Commission proposed the first legal framework covering AI – the Artificial Intelligence Act (AIA). The proposals aim to address the risks of AI and give developers, deployers and users requirements and obligations in relation to specific uses of AI. The proposals set out areas in which the use of AI technology is considered ‘high risk’ and as such will be subject to strict obligations. They also include ‘limited’ and ‘minimal or no risk’ categories for AI systems, with chatbots cited as an example of a limited-risk AI system.

Buyers says the AIA is ‘going to be like the GDPR [the EU General Data Protection Regulation] all over again. We’re going to have to learn how to deal with artificial intelligence and systemise it in the way that we’ve dealt with personal data.’

Earlier in 2023, the UK launched a white paper outlining its proposed sector-based approach to regulating AI. The government will give existing regulators the power to develop ‘tailored, context-specific approaches’ to regulating AI depending on how it’s being used in their sector. The white paper sets out five principles for UK regulators to consider: safety, security and robustness, transparency and explainability, fairness, accountability and governance, and contestability and redress. UK regulators will issue guidance over the coming year.

In the US the proposed Algorithmic Accountability Act 2022 addresses the use of AI in critical decision-making processes. It would require companies to carry out a risk assessment for factors including bias and effectiveness when they create or implement systems capable of automating decision-making. In April, meanwhile, the US National Telecommunications and Information Administration put out an ‘AI Accountability Policy Request for Comment’, seeking feedback on the sorts of policies it could develop to ‘create earned trust in AI systems.’ Its request includes the role of regulators.

Richard Batstone, Senior Knowledge and Innovation Manager at Slaughter and May in London, says that when it comes to regulating AI, ‘businesses ultimately want a legal framework that builds trust in AI and does not stifle innovation’. They add that where multiple regulators are involved, businesses appreciate a consistent approach that avoids overlapping guidance.

Meijboom says lawmakers should decide whether training AI systems with existing protected materials should be allowed. If so, the conditions under which it’s allowed should be stipulated. For example, is a fee required and if so, how much should it be and who should it be paid to? ‘The users of AI systems should, within a certain limit, get some certainty that they can use the output of AI systems without infringing third parties’ rights’, he says.

Paulina Silva, Publications Officer of the IBA Technology Law Committee and a partner at Bitlaw in Santiago, argues that specific metrics on what companies should report or make public about their use of AI would be useful. These would underpin requirements for transparency and make it possible to hold organisations to account.

Key questions to ask now

Emmanuel says that implementing ‘a general policy on the use of AI within your firm is important so [employees] understand the risks [and] when they should be using AI and when they shouldn’t be’. Silva agrees and says companies should ‘assume their employees will be using these tools and regulate how they will be using them’. It might also be necessary to explain to employees how existing confidentiality requirements would be applied to the use of ChatGPT.  

In-house lawyers should begin researching AI to develop an understanding of how the technology works: its limits, what it does well and how its use can be optimised. Taking into account the aforementioned risks and the company’s policy, they could use test data to develop an understanding of systems such as ChatGPT. They should encourage others in the team to do the same. Their technology provider might offer an application that allows users to test generative AI capability in a safe environment.

In-house lawyers can also engage with the business to understand what sorts of AI it wants to use and for which applications. From there they can consider what data would be required and how it might need to be refined.

Harper says the challenge for in-house lawyers is that ‘the law doesn’t keep pace with developments in technology or society. So, lawyers are always in an inherent struggle to keep up.’ For him the role of in-house legal is to use their knowledge of the law to assess risks when the business presents them with a set of facts. Doing this in relation to rapidly evolving, cutting-edge technology when the law hasn’t caught up is particularly difficult.

Often in-house lawyers will be asked to make a call under pressure with commercial success at stake. Now that ChatGPT has brought generative AI using LLMs to the mainstream, there is increased urgency from businesses to be involved to avoid missing out. This pushes generative AI further up the agenda of in-house legal teams.

Harper says there are ways that in-house lawyers can craft their advice to a business to make it constructive and relatable. Understanding how the technology works can really help. For example, explaining that anything the business puts into a public-facing generative AI model could be made public and the company running the platform could take ownership of that content makes the risks very clear.

For Harper, AI highlights one of the challenges of the generalist in-house legal role – it can be difficult to know exactly what question you need to ask and of which outside counsel. For him it’s important that in-house lawyers are comfortable not knowing everything and are able to ask for an explanation when it’s needed. ‘One of the biggest failings sometimes of lawyers’, he says, ‘is refusing things because they don’t understand’. However, often being willing to show a lack of knowledge can put you in a powerful position. Harper says clients from the business usually want a view or a feeling from the legal team on an issue such as AI. This can be a first step that’s refined as knowledge is accumulated and outside counsel sought. ‘There’s a really nuanced approach to all of these things’, he says, ‘which is part of the craft and art of general counsel work’.