DEV Community

Maya Chen
Maya Chen

Posted on Originally published at courtrules.app

How we counted AI disclosure orders in federal and state courts

We wanted a number for a simple question. How often does a judge's own rules say something about AI in a filing? The answer sits in our database, but the counting took care. Here is the method, the fields we read, the review steps and the limits.

The search

The unit is a judge-linked source document. Each document belongs to a judge-court entry and links to a rule source. We read the production Supabase database on October 21, 2026. Every request was an HTTP GET that translates to a SELECT. No write or RPC ran.

The candidate set starts from 213,183 current-version rule rows. We narrowed the search to active sources and their newest active document version. Then we filtered on four predicates within source-ID batches:

  • content->'structured_data'->'required_elements'->>'ai_disclosure' = 'true'
  • content->'structured_data'->>'logic_type' = 'AIUsagePolicy'
  • a set of source-text terms such as ChatGPT, generative, large language and A.I.
  • a regex for standalone AI, GenAI, spelling variants and some OCR damage

The substring pass over-selects on purpose. LLM appears inside installment and in staff names. The union produced 568 candidate passages. Of those, 349 carry the true disclosure flag. No passage in the set used the legacy AI type.

The review

A flag alone is not evidence. We read every retrieved passage, including flagged passages with no literal AI term. We grouped identical passages and kept every rule ID. Then we coded each document with one or more labels:

  • disclose: tell the court that AI was used, name its role or tool, or declare use or non-use. AI-generated evidence notices qualify.
  • certify-human-review: an express certificate, sworn verification or filing attestation that a person reviewed the work.
  • verify-citations: an express check of legal authorities, quotations or their authenticity. Citation checks stay separate from disclosure.
  • prohibit: forbid AI use in a stated task. A limited ban counts. A rule against inaccurate work does not.
  • other: AI-specific accuracy duties, permissions or warnings that fit none of the above.

The headline uses the union of disclose and certify-human-review, deduplicated by judge ID for judge entries and court ID for court listings. A document with both labels counts once. That gives 35 of 516 federal judge-court entries and 94 of 1,741 state entries. At least one such document sits in 44 of the 474 covered court listings.

The limits

These are database entries. A judge-court entry is not always a unique person. A person who sits in several counties can appear more than once. An entry can also carry a predecessor's standing order or an adopted court policy reproduced on the current judge's page. The count describes stored evidence, not personal authorship.

The search reads extracted text, not every original PDF. Material omitted by extraction, unreadable OCR and product names we did not search can be missed. A missing match does not establish that a judge has no AI requirement. Treat the counts as lower bounds.

The CSV keeps the wording, source links, dates and overlapping labels. The full method and the per-court counts are on the data page: https://www.courtrules.app/rules/ai-orders?ref=devto

Top comments (0)