Okay the last few days have been a bit of a flurry. I’m still working on getting all the filings and the Defense responses uploaded, but I need to make sure we have a sane directory setup first because there’s quite a bit.
After a few weeks using these LLMs, my attitudes about them are starting to shift. Each of them seems to have their own “personality” above and beyond the artificial personality. The LLM I’ve been most amazed by so far is DeepSeek, which is funny because it was the LLM I was most skeptical of initially. Part of the issue with DeepSeek is that it doesn’t do as good of a job hand holding through the though process, or more appropriately, it’s probably the most “autistic” of the engines. The biggest standout for DeepSeek (especially compared to Gemini which is the most comparable) is that it’s SO GOOD with citations. It’s the only one that freely “admits” it’s plundering the WestLaw database meaning we have a TON of local Eastern District citations.
My citation strategy has been to use something from each level, Supreme Court, 9th Circuit, California (Fed), Eastern District. The reality is the magistrate doesn’t read them anyway, but DeepSeek ultimately gives me a pool of citations to choose from, something the other LLMs struggle with (or just don’t do no matter how you beat them).
Gemini (using 2.5 Pro/5-06) is now my primary writing starting point, even though I hate that it can’t export to .odf or even .docx formats. Because I have to reformat the documents significantly, it has the worst “canvas” style option. That’s kind of worth the annoyance though since Gemini is easily the most “thoughful” and well rounded of the LLMs I’ve used. Much like DeepSeek gives lots of citation options, Gemini gives the best pool of writing options, it’s ability to flexibly address issues in different ways is the best of the bunch so far.
Claude is still my pretty but shallow option, it comes up with the best/most compelling pure language, even if it’s depth of analysis falls short of the others. I wish I didn’t have to prompt it so explicitly to get decent results or it might be the primary canvas despite it’s painful context window. One thing I find it really helpful for is running my DeepSeek (which produces pretty.. “direct” language) through Claude to soften it up.
And finally there’s ChatGPT which up until the last few days was my primary for everything. It’s easy to get started with it, it’s encouraging, it produces compelling results, but it’s kind of a piece of shit. It’s constantly messing up context, screwing up citations, and arbitrarily rewriting things. It’s easily the most obstinately wrong of the LLMs, insisting that it’s doing something correctly no matter how many times you post the incorrect output back into it and it admits it messed up. It’s the only LLM out of the three that I’ve had to deal with “hallucinations”, and it explains it as “woops, I overwrite my own context a lot accidentally”. It’s absolutely maddening at times. I’m steadily weaning myself off of it and it’ll probably just be another voice in the room soon.
An interesting thing happened a bit ago, I had my first LLM disagreement! The Defense responded to a discovery request with bullshit and boilerplate, so I asked each of the LLMs what a good email response would be considering I’m already nearly done with a rule 37 motion to compel. Claude responded that it’s probably not a good idea to respond at all, just update the motion with the response as an exhibit, note the bullshit parts since they helped (a lot), and call it a day. DeepSeek was it’s typical “RIP THEIR THROATS OUT” and provided a detailed list of regulations and rules to go on the offensive instead of only using the motion route. Gemini pointed out that going the email route is more about reinforcing our meet and confer duty, we’re right and we are going to be integrating this in the motion, but the focus here should be on making the meet and confer rock solid (pretty good advice). ChatGPT recommended the email, but produced recommendations that didn’t focus on strategy at all, just a direct rebuttal to the discovery response.
Interestingly, opposing counsel asked for a discovery extension (after more than two years the suit has been active, they are suddenly finding SHRA bureaucracy hard to navigate), but refused to do a rolling production or produce electronic records (which should be pretty easy right?). It’s baffling to me that if I had access to these tools two years ago, this organization is such a mess that they’d have gotten crushed out of the gate. Instead they were able to delay and avoid discovery to the point that we now have to start talking about institutional spoliation because no one remembers anything.