What it takes to write a better brief

Among the many breakthroughs, the biggest is how Quick Check analyzes the document to make recommendations. Our customers have already done good legal research. There may be gaps because they didn’t know about a case, or they didn’t have all the time in the world. Quick Check attempts to recommend highly relevant authorities to the issues in the user’s document and raise the confidence in their work.

This is a very complex task. We use natural language processing techniques to analyze the document and understand it. We use a plethora of content—primary sources (cases and statutes, appellate briefs and trial court documents); editorially enhanced content like headnotes, secondary sources, and metadata like key numbers; KeyCite information; citation graphs; and aggregated user data—to find potential authorities relevant to the issues at hand.

These sources consist of over 6 billion different data views. In short, we created an algorithm that looks in places humans cannot, and in ways that humans do not think to combine. We use machine learning models trained on metadata and linguistic features to find the candidates that are similar in both fact and law to the customer’s issues. We aim to be as precise as we can. And we made it fast.

Quick Check has two modes: analyze your own work or analyze your opponent’s work. Have you received any feedback from customers on how these have impacted their work?

CJ Lecthenberg: "For analyzing their own work, attorneys say that Quick Check helps streamline their research and gets them answers more quickly. For analyzing an opponent’s work, attorneys love that they can rapidly get a feel for where their opponent’s arguments are headed and what their options are for making their own arguments to the court. Attorneys want to see what their opponents are doing, but also what they are not citing to. Quick Check users say they can’t imagine doing legal research without it."

Quick Check uploads a brief that is not yet final or public. How do you ensure security?

Mernie Thomas: "We recognize that security and privacy of our customers’ work product is paramount. Attorney-client privilege is the most sacrosanct of rights in the adversarial process.

The uploaded file or plain text information is encrypted in transit via HTTPS. It stays encrypted as it gets to Quick Check. This cannot be accessed or viewed by any Thomson Reuters employee or contractors.

The uploaded file or plain text is deleted immediately after the Quick Check report is generated. It is not available in any of our systems in any form. The resulting Quick Check report is maintained temporarily subject to expiration for a maximum of 24 hours accessible only by the customer.

We remain committed to maintaining the security of our customers’ data."

Quick Check introduces some new ways to display information. Discuss some of these and the reaction from customers.

CJ Lechtenberg: "For Quick Check, we took a step back and thought about what attorneys do when they either write their own brief or when they examine the brief of an opponent. As Merine mentioned, the algorithms behind Quick Check are the most sophisticated we’ve ever created. Through machine learning and AI, Quick Check has the ability to look for content in places a researcher would not look, in ways they would not look, and in far less time than it would take them to look. Attorneys really appreciate that Quick Check harnesses the best legal research techniques, beyond what they can humanly do, and provides them with resources they may have missed doing their traditional legal research.

We’ve also added many features that aren’t found elsewhere to give the researcher as much context as possible while also making review of the report quick. First, we provide segment headings from the uploaded document, so the researcher knows to which issues the recommendations apply. This is far more beneficial to the researcher than just supplying a laundry list of cases with no indication of why they are relevant."

It’s important to note that these outcomes are not for the case as a whole, but rather for the individual issue for which the case is recommended.

Recommendation tags let the researcher know specific attributes about the case that might be helpful, such as whether they are frequently cited, from a high court, or decided within the last 2 years. (See Figure 1.)

Quick Check leverages Westlaw Edge features such as the “eyeglasses,” filtering by previous user interactions, folders, notes, and highlights. We also bring some KeyCite functionality to Quick Check so the user can get a holistic view of the table of authorities in their document and review any potential weakness in what they have already cited.

Speed is also an important feature, as Merine noted. One customer commented there is really no good reason not to use Quick Check given how fast it works.

KeyCite Overruling Risk, which Quick Check incorporates into its results, seems like an obvious feature. But only Westlaw Edge has it. What’s the story behind it and was it always on the Quick Check roadmap?

Mernie Thomas: "Yes, KeyCite Overruling Risk is only available in Westlaw Edge. We are constantly modernizing our platforms and developing new features and capabilities on these new platforms. We want our customers to benefit from that.

We have been thinking about the risk when an overruled case, also known as red flagged, extends beyond the overruled case itself—such as a case you found that seems to be good law. When the case you found cites the red-flagged case as an authority on the same issue, it could be at risk of being bad law. The challenge lies in identifying that your case cites the overruled case on the same issue, doing so reliably, and figuring out if there is a risk.

KeyCite Overruling Risk uses natural language processing and machine learning to find cases that have been overruled on the same issue. Our machine-learned models analyze the citation network, KeyCite features, and key number features, and then linguistically analyze the text of the cases themselves to identify if there’s a risk that a case has been implicitly overruled by another. (See Figure 2.)"

When we were developing Quick Check on Westlaw Edge, it felt natural to incorporate KeyCite Overruling Risk. It helps in presenting any potential weakness—both for Check Your Work and Analyze Opponent’s Work use cases—that the customer is not aware of because the risk is implicit. When making recommendations, we also do not want to recommend bad law. It is logical that a customer would not want to see bad law and waste their time on it.