Fine Tuning Justice: The Role of Pre-Trained LLMs in Enhancing Federal Investigations and Legal Procedures

Introduction

Large language models (LLMs) represent the next generation of artificial intelligence applications, attaining widespread attention and adoption. These models demand substantial energy and resources for training. Consequently, there has been a shift towards developing pre-trained models like bidirectional encoder representations from transformers (BERT), utilizing millions of parameters from training texts to create general use models. An evolution of this approach involves fine-tuning models with domain-specific data, enhancing their utility in fields such as medicine, law, and science.

One challenge with LLMs as it pertains to the federal government and law enforcement involves interacting with unsecured APIs, which communicate with external cloud-based servers. However, deploying pre-trained models fine-tuned on domain-specific data onto FedRAMP-compliant cloud solutions like AWS GovCloud, Google Cloud, or Microsoft Azure ensures security, eliminating vulnerabilities associated with third-party cloud solutions. Moreover, these models can undergo continuous training and improvement using newly generated legal, investigative, and enforcement texts.

According to the Pew Research Center, approximately 8% of criminal cases are dismissed, often due to insufficient evidence or procedural errors. A domain-specific assistive LLM could mitigate these issues by providing standard operating procedures and enforcement support to investigating officers. Federal law enforcement agencies stand to benefit from domain-specific LLMs, leveraging subject matter expertise from previous documentation to enhance investigations, safeguard enforcement personnel, and uphold citizens' rights.

Problem Statement

Many sensitive applications, including law enforcement, involve data that cannot be transmitted via API to third-party cloud-hosted LLMs due to concerns over personal identifying information (PII) and confidential investigation data. The Federal Risk and Authorization Management Program (FedRAMP), established in 2011 and codified in 2022, provides a standardized approach to security assessment and authorization for cloud computing products and services processing unclassified federal information.

FedRAMP facilitates the adoption of secure cloud services across the federal government, ensuring standardized security and risk assessment for cloud technologies used by federal agencies. FedRAMP approved cloud technologies, like AWS GovCloud, can host advanced analytics and AI tools like LLMs for federal departments without exposing information and data to third-party cloud hosted solutions or transmitting data over potentially compromised APIs.

Pre-Trained LLMs

Pre-trained LLMs come equipped with language understanding derived from extensive language data. These models, trained on millions of parameters, offer accuracy, energy efficiency, and the crucial ability to be secured within a FedRAMP-compliant environment without reliance on external cloud-hosted APIs. This allows fine-tuned models derived from pre-trained models to be securely and efficiently utilized by federal departments for domain-specific cases.

Multiple robust options exist for pre-trained LLMs, with HuggingFace BERT models being particularly noteworthy. Leveraging PyTorch or Keras, HuggingFace models offer top-tier pre-trained models that are easily fine-tuned and highly capable. The HuggingFace pre-trained BERT models are easily accessible and capable of being deployed on local hardware or, more appropriately, scalable secured FedRAMP-compliant cloud-hosted solutions.

Fine-tuning involves adapting a general-purpose model by training it on domain-specific data, such as investigative documents and standard operating procedure manuals. Pre-trained models can be effectively fine-tuned with as few as 1000 training documents. However, the best trained models will likely require 10’s of thousands or 100’s of thousands of documents to train. In short, the more text documents used, the better the outcome. A deployed model can be retrained and tuned as more documentation is produced, resulting in a continuous improvement model. However, there is a likely manpower cost in preparing the training documents. There are automated methods that can be used, but in some cases, a human is required.

Another cutting-edge option for leveraging the power of LLMs completely in a FedRamp compliant environment is AWS Bedrock. Searching large amounts of domain-specific documentation very quickly to answer questions is an excellent use case. Using standard operating procedure, training, and expert documentation would provide a fast, easy to use, and cost efficient tool for assisting law enforcement in their duties. Although this cannot take the place of a fully fine-tuned domain specific LLM, it is an excellent complementary tool to assist enforcement personnel.

Leveraging Domain-Specific LLMs

Domain-specific fine-tuned LLMs offer numerous benefits, serving as advisors or assistants to frontline personnel by suggesting best practices and standard operating procedures. These models standardize processes, reducing errors, increasing efficiency, and enhancing outcomes in complex procedures.

Deploying domain-specific LLMs on FedRAMP-compliant cloud services like AWS GovCloud, Google Cloud, or Microsoft Azure ensures a secure, continuously trainable, and scalable tool without external vulnerabilities. Models tailored for scientific, legal, and mental health data can be readily deployed for domain-specific purposes. Providing law enforcement with next-gen technology to assist them in their work is an area that is ripe for development.

Starting with a pre-trained general model like HuggingFace’s BERT, extensive domain-specific data can be used for fine-tuning within a FedRAMP-compliant environment. This approach combines the advantages of a well-trained model with the efficiency of domain-specific tailoring. The resulting model can be securely deployed with endpoints to various user interfaces, offering versatile applications such as documentation assistance, helpful chatbots, or documentation writers and editors.

Engaging a domain-specific model to edit and proofread charging documents, arrest reports, or warrant requests are some of the types of use-cases that would be beneficial to law enforcement. An accurate domain-specific LLM assistant would be able to provide advice and expertise to avoid cases being dismissed for lack of evidence by identifying patterns in previous enforcement activity and drawing on subject matter expertise and standard operating procedure that has been used as training material.

Empowering law enforcement personnel with domain-specific LLMs that integrate standard operating procedures and case documentation enhances citizens’ rights protection, ensures personnel safety, and expedites case closure. Next-generation solutions like domain-specific LLMs hold immense promise for law enforcement and citizens alike.

What’s going on right now

What I’m building right now:

🚧 Working with the speech_recognition package in Python
🚧 Building an “all in one” linux powered maching with 3d-printing, raspberry pi, and some coding

What I’m drinking right now:

🍺 New Realm variety

What I’m reading right now:

📚 Star Wars: The Princess and the Scoundrel

What I’m watching right now:

📺 X-Files

What I’m playing right now:

🎮 Fortnite
🎮 Palworld

What I’m Learning right now:

🎓 🎸 Scales
🎓 LLMs
🎓 SnowFlake