Ten principles shaping the role of GenAI in legal drafting according to Clarilis
We all have a front row seat to the evolution of GenAI in legal services. Initially, we saw a raft of new LegalTech emerging, often with a playground approach – encouraging law firms to experiment. Now that the market has explored GenAI and lived with the technology for a time, more mature use cases are emerging. One such use case, using GenAI for drafting, is widely acknowledged as a tougher nut to crack than, say, retrieval and extraction, but rapid progress is being made. At Clarilis we’re excited about the approach we have developed.
Over the last year we’ve been exploring the role large language models (‘LLMs’) might have in legal drafting, and how that relates to traditional drafting automation. Our challenge has been to work out how GenAI could augment the drafting experience in a thoughtful and meaningful way. How can we use it to address real problems for our customers? How can we use it to add genuine value? At our customer forum in September last year, we shared our approach to GenAI drafting, and what customers can expect from Clarilis. This included our core principles for GenAI development. These values guide our approach to integrating LLM-generated content into legal drafting and, as we come to release Clarilis AI Draft, we wanted to share them more widely as a reference point for law firms assessing their drafting technology choices.
North Star
Principle 1: Precedents are still the gold standard
There’s no doubt about it, templates (precedents) crafted by experienced knowledge lawyers are a far superior starting point when compared to content generated by an LLM. While an automated template can produce drafts of a consistently high quality, content generated from an LLM introduces risk of errors and ‘hallucinations’ – expertly written garbage. But this is not to say there is no place for LLMs in drafting. LLMs can provide a fantastic starting point when there’s no template available, or to help adapt a template to a novel scenario—as long as there is an understanding from lawyers that LLM generated content will need to be thoroughly reviewed.
Principle 2: GenAI is a tool, not a product
The potential of GenAI is enormously exciting, but translating that potential into useful solutions requires careful thought. Many users will already have access to LLM chat-style applications (e.g. through enterprise tools like Microsoft Copilot), so there’s little value in merely making that capability available through another product – simply passing user prompts to and receiving messages from an LLM. Instead, we need to challenge ourselves to use GenAI to add something more substantive. How can we save our users time? Can we use it to make it simpler for people to interact with our platform? Or augment it with trusted content and best practices?
Implementation
Principle 3: Be LLM agnostic (as far as you can)
Although GenAI has slipped comfortably into our everyday vocabulary, it continues to evolve rapidly. It is impossible to predict who is going to have the best language model in the future or what capabilities it will offer. OpenAI, Google, Anthropic, Meta—your guess is as good as ours. Rather than building a solution that works only with a single language model (or provider), an LLM agnostic approach is essential to maintain the flexibility to navigate and adapt to a constantly shifting landscape.
In reality, this is more easily said than done. Bringing GenAI capabilities to life (whether as a law firm or a LegalTech provider) forces you to tailor your approach, to an extent, towards certain model providers or LLMs: you cannot be entirely agnostic in a drafting context given the vastly different behaviours of each provider and model. An approach which works well with one model will not necessarily work well with another. But what we can do is set aside time to keep across the latest developments, be nimble and try to focus on content approaches that are more likely to be transferable, which takes us to Principle 4.
Principle 4: Focus on your task, not on prompting tricks
Most complex uses of GenAI require some level of ‘prompt engineering’: carefully tailoring instructions for an LLM to produce better outputs. Different models demand different approaches and, in the past, the technology community has highlighted many (sometimes surprising) techniques for getting the most out of LLMs. For example emotional prompting, telling the LLM that you will be fired if you do not get the correct answer, or that your life depends on it telling the truth. These quirks are interesting (and often entertaining), but it would be a mistake to rely on them for the success of a solution. That’s because these techniques are typically model specific (so do not promote agnosticism – see Principle 3) and, in any case, will likely be ironed out over time.
Instead, in the context of legal drafting, we think it is better to focus on the fundamentals of encoding legal knowledge and capturing best practices. For example, what are the characteristics of a good definition? What makes a clause unambiguous, or easily readable? Spending time clarifying these concepts (both for yourself and for prompting the LLM) is much more likely to produce an approach that translates between models.
Principle 5: More is often less
When introducing someone to GenAI for the first time, people often frame the tool as a ‘co-worker’. For example, when interacting with a chat-style LLM we might suggest giving it instructions in the way you would a new colleague: provide relevant background information and be clear about the output you would like (US Academic Ethan Mollick has an extended piece on this approach). We’ve found a similar approach is helpful in the context of legal drafting, particularly in relation to the volume of content that you provide the LLM. For example, imagine giving a new junior colleague an entire suite of documents and asking them to add a definition of a “Material Adverse Change”. Your colleague is likely to have questions: Which document should the definition appear in? Is there specific language it should take account of or be consistent with? What context do you need the definition to work in?
The same is true of GenAI drafting. Simply providing the LLM with all the information you have risks a variable and low-quality output. The quality of the generated output directly relates to the relevance and amount of context in the prompt. Providing focused, relevant information materially improves output quality, while irrelevant details materially degrade it. For tasks like this, it is generally better to have invested more time in understanding what information is likely to be relevant to the task and start from that small volume of content, rather than throwing everything in and hoping for the best (see further Principle 4).
Principle 6: Hallucinations are inherent, and mitigating them is not ‘one size fits all’
Text generated by an LLM might be false or misleading (‘hallucinated’), even if it appears very convincing and expertly written. Technology providers are working to address this, and introducing trusted reference information into the generation process (‘Retrieval Augmented Generation’) can improve the quality of outputs, but as yet there is no silver bullet. So, when we use GenAI, we need to consider: (1) whether the task is one in which hallucination matters; and (2) if it is, how will we enable users to verify the output.
In the context of legal drafting, accuracy and quality is crucial, but our approach to verification might vary. For example, suppose you use GenAI for the first draft of a deal-specific warranty that isn’t covered in your underlying precedent. When you review the generated drafting you would need to consider (at least):
- Internal consistency: should the drafting use existing defined terms? Are there other clauses that it should refer to?
- Intention: does the drafting accurately reflect your instructions? Does it capture the point you had in mind?
- Construction: is the drafting appropriate and effective in your wider legal and regulatory context?
For the first point, we can imagine ways that a product could start to automate verification, or otherwise make it straightforward for a user to check. But points two and three require much more expertise and judgement. We need to be clear with users about what steps have been taken by the product, and what is required of them.
Principle 7: Be clear where drafting has come from
A document generated by a traditional document automation engine will consistently reflect the drafting outcome set out by the automation logic. It delivers a predictable, accurate output, which has been reviewed and signed-off by the relevant law firm. Conversely, LLM-generated output is different every time. The language will not have been previously reviewed. This is fine where no appropriate precedent language exists but, as explored in Principle 6, it also means it always needs some form of review. This tension makes it crucial for documents that include GenAI drafting to be completely transparent about where the different content has come from. Lawyers shouldn’t be left in any doubt as to whether a particular clause or document was generated through automation, by an LLM or through some other process.
Responsible Development
Principle 8: Meet your users where they are
Different organisations – and individuals – are at different stages of AI adoption, and no two organisations have taken the same approach. Some will be eager to experiment and pioneer the use of the technology, whilst others will be more cautious. A firm’s approach will also reflect its client base, regulatory and jurisdictional context. This means that when building GenAI capability we should be clear which features leverage an LLM and, crucially, provide firms with the opportunity to opt-out.
Principle 9: Do not compromise security and data ownership
Building a GenAI capability should not negatively impact your existing information security standards or change how you and your customers approach data ownership. In the Clarilis context, our customers trust our hosted solution with their data and we apply the same standards to our GenAI development. We can leverage LLMs through a customer’s chosen provider, or through a customer’s own self-deployed model. The precedents which we automate belong to our customers, and we do not claim ownership in any LLM output: we also do not use customer inputs for any proprietary model training or fine-tuning.
Principle 10: Respect third-party content providers
Much of the content that law firms work with is provided by third party content providers. For example, industry bodies (like the Loan Market Association, the Model Commercial Lease committee and British Venture Capital Association) or knowledge publishers (like FromCounsel and Practical Law). Each content provider has their own rules in relation to the use of their content with LLMs – which need to be adhered to. For example, a few content providers permit their documents to be uploaded to an LLM based system, but most do not, and to upload their content would infringe their licence terms/membership byelaws. This requires conscientious governance and flexibility, whilst providing users with straightforward messaging about what they can and cannot do in a given context.
We hope the above principles are helpful as you explore your own interaction with LLMs and the LegalTech leveraging these exciting models.
What’s next?
We want to improve your drafting process through technology – whether that’s traditional document automation, by leveraging LLMs, or other techniques. Guided by our Principles, we’re continually developing Clarilis AI Draft to provide compelling value beyond traditional automation, and beyond a simple chat interface.
We’d love to explore how Clarilis AI Draft could work in your firm. If you’d like to learn more, click here or please feel free to reach out.
