Not all legal AI is created equal

Just as AI technology has quietly become part of our everyday personal lives, it has become part of our daily work lives in legal practice. Sometimes artificial intelligence drives subtle, intuitive features that make our work incrementally more efficient. Sometimes AI is the engine at the core of our most powerful software, making it possible to overcome previously impossible tasks with ease.

But it is always intended to support and empower human legal professionals, to help them be more efficient, and minimize tedious and time-consuming tasks so they can work on more strategic, higher value tasks.

AI technology can help legal professionals conduct research, analyze large data sets, monitor for risks and compliance obligations, and find answers in mountains of data. In short, it can help lawyers do the work they already do, just faster and more efficient.

Westlaw Edge is a prime example of AI’s gradual impact on the legal industry. Thomson Reuters has been integrating AI principles and technology into our legal software solutions and data systems over the past few decades. In fact, Westlaw implemented an AI technique – natural language processing – in the 1990s, which made it possible for users to search for content with normal search queries instead of complex Boolean search strings.

Now, Westlaw Edge includes many AI-enabled enhancements, such as:

Artificial intelligence, in some form, is now a necessity for legal professionals, due to:

Big Data is still growing.

When the term “Big Data” was coined, we were talking about gigabytes of data. Now, we’re talking about terabytes, petabytes, and even zettabytes of data.

The amount of data we generate in our ordinary personal and business processes is astounding. Researchers have estimated that we create 2.5 quintillion bytes of data every day,1 and that more data has been created in the past two years than in the previous entirety of the human history.2

The answer to meeting the demands of ever-increasing volumes of data is advanced technology that’s capable of processing the data to help lawyers find answers quickly. It’s not uncommon for legal professionals to be tasked with reviewing and processing hundreds of thousands of documents, provide analysis on high-volume data sets, or find connections and answers in disparate documents across various sources.

Just as online legal research have become standards in the industry, the AI technologies powering those solutions are here to stay.

Changing clients change attorney practices.

It’s in a lawyer’s nature to be cautious and risk-averse, to place high value on historical precedent, tradition, and order. So, it’s understandable when attorneys are slow to adopt the latest technology that’s pitched to them – especially when outsiders are doing the pitching.

Software startups, running on shoestring budgets and bravado, are not going to find lawyers eager to hear how they can completely revolutionize how a successful practice does business.

That’s not to say that lawyers can’t change to adopt technology, just that they will do it when it makes sense and will clearly benefit their clients. Economic changes, more competitive global and local markets, and several other factors have altered the market for legal work. Clients are demanding more for less, and getting it.

Artificial intelligence is inevitable as a part of long-term technology. The attorneys that leverage AI in their workflows to meet their clients’ high expectations will survive and prosper over the firms that cling to the mantra, “If it isn’t broke, don’t fix it.”

The ethical responsibility of an attorney.

Learning more about how AI technology may affect your legal workflow is something that you should do now, as these advancements are moving quickly in the legal technology space.

David Curle, the Director of Market Intelligence for Legal at Thomson Reuters, predicted that AI awareness and implementation may soon be an ethical responsibility. He noted the bar associations in 37 U.S. states have adopted a duty of technology competence. Specifically, the state bar associations adopted some form of the language in Comment 8 of ABA Model Rule 1.1 pertaining to technology:

Artificial intelligence has been around for years and is relevant to every practice area. Legal professionals who refuse to learn about or leverage this technology could face the same business and ethical risks as a lawyer who refuses to use email, word processing, or e-discovery software and overbills clients for doing things the “old-fashioned way.”

The three essential components of a successful AI solution are like three legs of a tripod:

Domain expertise refers to the specialized knowledge, experience, and input of people in a particular industry, subject matter, or domain. These domain experts know what data is important and why, and how people in this industry work.

Quality data includes vetted and relevant substantive data from trusted sources, metadata provided by experienced attorney-editors, and data on how users interact with the data. Anyone can provide a large quantity of data, but quality is what matters most.

AI expertise refers to the technical expertise and experience of working with AI and its many principles and techniques. This requires robust technology stacks, expertise in relevant and emerging cognitive technologies, and the practical abilities to apply the technology and AI techniques to quality data and domain expertise.

Why domain expertise matters

Specialization and accumulated experience is just as important to AI technology and software development as it is to practice area specialization in legal and medical fields. Using a general one-size-fits-all solution for specific, complex matters often results in generally bad outcomes.

Domain expertise ensures that an AI solution is built with the right end users in mind. In the legal world, this domain expertise comes from legal professionals imparting real-world subject matter experience and nuanced, peer-reviewed perspectives into the software, as well as human-created metadata, tags, editorial markup, and additional data that provides context and connections.

That’s why it’s important to consider who contributed to the creation of an AI solution. If you’re doing specialized work, you should expect the tools you use to do that work to be informed, made, and tested by your fellow specialists.

But legal experts are not data scientists, AI scientists, software developers, or designers. It takes broader teams of multiple disciplines collaborating to transform data, find connections, and deliver the answers end users need. That’s where AI expertise comes into play.

Collaboration creates well-rounded solutions

The people and organizations that play a part in educating the AI solution are critically important. One of the biggest advantages brings to its AI technology is the depth of knowledge and experience in critical disciplines that make it work.

The R&D team at Thomson Reuters has incorporated AI technology into several legal products:

• WestSearch
• KeyCite
• Folder Analysis
• Research Recommendations
• Monitor Suite
• Medical Litigator
• PeopleMap
• ResultsPlus
• NewsPlus

Attorney-editors and subject matter experts might know what data is most relevant to certain practice areas or legal issues, and what complete and thorough legal research looks like upon completion. But they might not know how different end users start their research or the different paths they’ll take to complete their research.

End users and customers can provide insight on the strategies they’re considering as they conduct their research, or the practical realities and personal preferences that affect their research. But they might not know of faster methods or more recent and relevant data.

Data scientists will be able to identify and manage the data that best suits the application and individual end users. But they might not know how best to present that information to users.

Designers and developers might know how to serve relevant data and ensure the user experience is streamlined and intuitive, but not know what data is most relevant.

Only by bringing all of the stakeholders to the table is it possible to truly understand and create a holistic research solution that works in the real world.

This sort of collaboration and expertise isn’t something we can get from a vendor – this innovation can only be created in-house. That’s why created our Center for Cognitive Computing and our Thomson Reuters Labs.

The Center for Cognitive Computing and Thomson Reuters Labs are where we start to answer the “what if?” questions of how AI can improve legal processes. We host collaborative teams of multiple stakeholders and disciplines to solve legal matters with insights on the complete picture. This approach has led to many of our greatest product enhancements, as well as new solutions.

About the Center for Cognitive Computing

Our team of scientists, engineers, and designers focuses on the development of smart applications by applying and extending state of the art in natural language processing, machine learning, deep learning, information retrieval, knowledge representation and reasoning, data mining, text analytics, and human-computer interactions.

About Thomson Reuters Labs

Thomson Reuters Labs are where we combine our data and human expertise to create innovative solutions. Our data scientists, research scientists, developers, and designers collaborate on the application of cutting-edge technology to create new ways to unlock answers within data. We have seven labs on four continents to bring a global perspective to global issues. Innovation is at the core of everything we do.

The principle of “garbage in, garbage out” dates back to the earliest days of computer science, referring to the fact that sloppy and incorrect data inputs will result in sloppy and incorrect data outputs. Despite our advances, this fact remains true. The most powerful AI-enabled computer will not provide great value if it is loaded with data that is incomplete, limited, irrelevant, inaccurate, or otherwise flawed.

If AI technology is loaded with bad data – incomplete, inaccurate, biased, outdated, etc. – and trained to find answers in bad data, the results can be much worse than not using AI technology.

When end users consider implementing AI technology, they are often pitched by startup companies and tech vendors on the promises of the technology, on the enormous volume of data that went into machine learning, the data points and variables, the simplicity and user-friendliness of the platform, and the cost. But they do not hear enough, nor will they ask enough questions, on the quality of the data and its sources.

Quality data comes from quality assurance and input.

Open source data can provide published cases, court decisions, enacted statutes, and regulations, but human intelligence and input is what defines quality.

Law firms should consider how additional data layers add to the value of the substantive data content. This includes metadata, analysis, tagging, organization, and quality assurance provided by legal professionals and data specialists who serve as stewards of the data. This data makes it possible for an AI solution to find more connections and specific answers. This blend of data quantity and quality is necessary to ensure thorough, efficient legal research.

The quality and breadth of the software’s user base should also be considered, as the users will provide continuous training to the AI solution by demonstrating how real-world users work, where they need support, and what they find valuable.

The legal content and data in Westlaw exemplifies how Thomson Reuters improves the quality of volumes of data. Our bar-admitted attorney-editors add rich editorial content and contextual metadata and ensure consistency and accuracy when it comes to how users’ research is organized and presented.

What makes Thomson Reuters legal data better?

Thomson Reuters has been a leading provider of trusted legal content for more than 100 years. Our attorney-editors review, organize, and enhance legal content with headnotes, Key Numbers, and highlights that contribute human perspectives and quality.

Attorney-editors go through extensive training to ensure consistency and accuracy when it comes to how users’ research is organized and presented in Westlaw Edge. Their legal knowledge and standards are similarly applied when they collaborate with product developers to create and refine AI enhancements. All of these elements are brought together to create a clean user interface that ensures maximum efficiency for legal researchers using Westlaw.

Our legal content and data is:

Curated, not harvested: Thomson Reuters has thousands of attorneys who review, annotate, organize, and provide stewardship of our legal content. Because of this careful oversight, users can feel confident that they are relying on the most-up-to-date, accurate, and relevant content.

Created and informed by subject matter experts: In addition to thousands of attorney-editors, our legal data is enhanced, informed, and supplemented by the renowned attorneys, judges, and professors who write our industry-leading law books, as well as a robust staff of reference attorneys. This ensures our data incorporates real-world perspective and practical advice for the complete spectrum of practice areas and subject matters.

Editorially enhanced: Our attorney-editors add human intelligence to our legal data by incorporating headnotes, notes of decisions, the Key Number System, and more to provide the contextual metadata needed to make machine learning more effective and connect to more relevant legal data.

Informed by actual users: Westlaw Edge users provide continuous improvement to the data, other users, and themselves by working within the system every day. By collecting documents in Westlaw Edge folders, the system learns to identify connections between the content. By selecting citations, the system is able to identify and suggest additional supporting content.

As you consider how AI can transform your organization’s legal work and help you meet the demands of working with larger data sets, it’s important to understand that:

• AI technology is only as good as the data and metadata content it has to work with
• Domain expertise and input from data scientists, legal experts, and users is the training the AI technology needs to make the data accessible and effective

What lawyers must consider when evaluating artificial intelligence as part of their legal practice

Artificial intelligence, like human intelligence, is created and influenced by multiple factors, each of which should be considered when evaluating AI solutions. Ask yourself and your potential AI technology provider the right questions to ensure AI technology is as intelligent as it should be.

Who created the solution and are they experienced in the field?

Consider if an AI technology solution is created and influenced by a software developer that might lack legal domain expertise. Are they retrofitting their technology to work for the legal industry, or was it made to excel for very specific legal purposes? An effective AI solution should be built by an organization with extensive history in the domain that has had time to evolve and refine their technology solutions.

What lawyers might not know is how often technology solutions are launched as MVPs, which stands for Minimally Viable Products. Lawyers provide their clients the best possible service, expertise, and value – not as little as they can get away with. They should expect the same robustness and quality from their technology solutions. You don’t want to serve your clients with half-baked solutions or help your technology providers learn as they go.

Who uses the AI-enabled technology and what value are they adding?

End users contribute continuously to the machine learning capabilities of an AI-enabled system every time they use it. The more users, the better. And the more qualified the users are, the better. Westlaw Edge for example, is the most preferred online legal research service. That means you benefit from the wisdom and practical habits of numerous experienced lawyers using the system.

Where does the data come from and how is it improved?

The data is the foundation of the AI solution. Consider how clean, robust, and relevant the data is, as well as, the specific practice areas and issues for which it will be used. Also, consider how the data is curated and enhanced by human intelligence before it gets to you. That includes the metadata that is added and the connections that are formed and identified between disparate data sets.

In summary, consider and evaluate artificial intelligence by the human intelligence that goes into it. Every interaction with AI technology should be considered as a collaboration with the teams of legal experts, AI experts, data scientists, developers, data stewards, and end users who contribute to the creation, training, and maintenance of the AI solution.

Thomson Reuters combines the human intelligence of our staff of thousands of attorneys and subject matter experts with industry-leading rich data content and the AI technology and expertise of the data scientists, AI experts, and developers working in our in-house innovation labs – The Center for Cognitive Computing and Thomson Reuters Labs – to provide innovative AI solutions that make attorneys more effective.

Siri, Apple, and iPhone are trademarks in the U.S. and other countries.

SOURCES:

• https://public.dhe.ibm.com/common/ssi/ecm/wr/en/wrl12345usen/watson-customer-engagement-watson-marketing-wr-other-papers-and-reports-wrl12345usen-20170719.pdf
• https://analyticsweek.com/content/big-data-facts/
• https://www2.deloitte.com/us/en/pages/deloitte-analytics/articles/cognitive-technology-adoption-survey.html
• https://www.seagate.com/files/www-content/our-story/trends/files/Seagate-WP-DataAge2025-March-2017.pdf

Disclaimer: The names of actual companies and products mentioned herein may be trademarks of their respective owners.