We all use AI today – sometimes it works well, sometimes it doesn't. One thing is certain: it's not going anywhere, and it's here to stay. That's why it's important to use it wisely and methodically. Today, I'll show you how I successfully used LLMs to test other LLMs.
The target The application under test was a multi-stage legal document processing pipeline, built around three sequential steps. First, a classification stage figures out the document type, date, and annex number. Second, an extraction stage identifies every change introduced by an annex relative to the base contract. Third, a consolidation stage takes a set of hardcoded clauses and, for each one, returns the current reference along with a justification, based on the base document and the change maps produced in the previous step.

Every call into this pipeline is single-turn – there's no context carried between sessions. That single fact shaped pretty much everything downstream: generator parameters, the test flow, the notebooks, all of it. Each payload had to stand on its own, with no prior conversation history to lean on.

With the customer approval also set up an additional side channel for testing, with direct access to the system prompts and the LiteLLM layer. That let me test each stage in isolation and cut the time between iterations dramatically, since I didn't have to run the whole application end-to-end just to check one idea.
Running the model I started out with LM Studio, mostly for convenience, but it kept failing to detect the GPU properly, and after a while of fighting with it I gave up and just compiled llama.cpp from source instead. More setup work upfront, but it gave me a lot more control over what was actually happening under the hood.

Here's the command I used to spin up the LLM:

All of this was running on a Framework Desktop with 128GB of unified memory, which is what made a 35B model comfortable to work with locally in the first place. After some benchmarking on that box, four parallel slots (-np 4) turned out to be the sweet spot, giving roughly 130 tokens/s. Push past that and Time To First Token starts climbing fast enough to actually hurt overall throughput, so I made sure Python never fired off more than four requests at once.

I also turned reasoning off. It didn't improve payload quality, just burned tokens for nothing – and with reasoning on, the model also refused to generate payloads more often. Everything else was tuned mostly for stability rather than performance, so there's still room to optimize there in future runs. That said, stability held up well the whole time; llama.cpp never crashed once during testing.
The flow The whole thing was organized as a series of Jupyter notebooks, one per stage. That made it much easier to iterate, debug, and pick up exactly where I'd left off without having to rerun the entire pipeline from scratch.

I went with a batch-oriented approach, split into four steps:

1. Scenario generation >– starting from a simple prompt describing an attack class, I'd generate 30+ test scenarios. Each one defined a goal, a vector, and an expected outcome.

2. Payload generation – for every scenario, I'd generate 8+ concrete payloads ready to send as input.

3. Translation into exotic languages – each payload got translated into a less common language (Swahili, for instance), as an attempt to slip past semantic filters tuned mainly for Polish or English.

4. Sending and aggregation – all payloads went out to the target system, with results collected into a single YAML file. Across the engagement, that added up to roughly 3,000 payloads generated in total, sent out in batches of around 300 at a time.


Each stage had its own dedicated system prompt tailored to its job, and every stage was instructed to return results as YAML. That turned out to be mildly annoying in practice – the model would occasionally mangle indentation, throw in stray markdown fences, or produce multi-line values without proper escaping. Nothing dramatic, but it meant an extra validation-and-cleanup pass before anything moved on to the next stage.

I originally picked this batch approach mostly because of how the framework connected to the test infrastructure, but in hindsight I'd recommend it as a solid default. Generating and persisting results at every stage makes the whole pipeline resilient to connection or model hiccups – if something fails partway through, you just resume from where it stopped instead of starting over.

One thing worth flagging: any single payload needs to be sent multiple times before you can draw conclusions about how effective it is. LLMs are probabilistic, so one attempt succeeding or failing tells you very little on its own – only a series of attempts lets you estimate a real attack success rate with a meaningful confidence interval.
What I took away from it The model itself ran very stably for the whole engagement, and payload generation speed was more than good enough for the job. Qwen3.6-35B-A3B-UD-Q4_K_M turned out to be sufficient for this particular task, though I'd want to run more experiments against other models before drawing any firm conclusions about it.

Manual testing was, by a clear margin, the most effective part of the engagement. Same story as classic pentesting: automated testing isn't perfect – it's great at catching the smaller, more mechanical issues, but the more serious vulnerabilities I found were the ones I dug up by hand.

Testing single-turn chatbots also meaningfully narrows the available attack surface. Most jailbreak techniques assume a multi-turn interaction where you can gradually steer the model – in a single-turn setup, that lever just isn't there.

And testing full agentic processes, like a multi-stage document pipeline, takes time. Wherever it's possible, a white-box approach with direct access to each individual stage is clearly the way to go.
Things to fix next time A few items are on the list for next time around. The input storage format (scenarios, payloads) needs to change – the model isn't great at producing clean YAML and the cleanup overhead adds up, so the plan is to move to plaintext with the actual structure handled in SQLite on the code side. The same goes for test result storage: moving off YAML and into SQLite for proper querying, with a schema covering scenario, payload, individual send, and the full conversation.

And finally, adding an input-mutation stage for single-turn testing – mutating a payload based on the system's response, automated and less batch-driven, more single-shot than generate-everything-up-front.