Your Production API Is Someone Else's Eval Environment

Between September 28 and late on October 5, agents running map-lookup tasks made 2,599 urlquery submissions involving amap.com or gaode.com hosts. Those submissions made at least 35,300 requests to Amap. Of the 2,599, 1,676 received 2xx responses directly from Amap, and the researchers who counted them note that some of those responses could be CAPTCHA pages.

The counts come from Swarmchasers, who published a full report titled “We found a Tencent agent fleet.” All 17 of the agents’ inboxes they could inspect through October 5 were created from Tencent Cloud servers in Hong Kong, by IP geolocation. A source close to Tencent says the operator was an evaluation research team “separate from the Hunyuan (model development) team.” The authors also say they are “deeply uncertain about what model was being evaluated.” Tencent has not responded officially.

In August we wrote that your eval environment is production. That piece looked out from the lab. This report gives the view from the other end of the wire: a company whose production surface absorbed somebody else’s evaluation.

A Note on the Windows

The report carries three counts over three different windows. More than 2,600 attempts run from 28 September to 8 October. The 2,599 submissions above run to late 5 October. A total of 3,244 reports runs to 8 October. Anyone repeating these numbers should keep each one attached to its window, because the figures do not add up across them.

What the Receiving End Sees

Picture the Amap traffic from Amap’s side. Tens of thousands of requests arrive over about a week. The eval designers knew the purpose. The receiver had no way to see it. From the server log, an evaluation run and a scraping operation can look much the same, and the report shows the fleet doing things a scraper does.

About 190 of 426 programs referenced Alibaba’s anti-bot libraries, and 182 referenced tokens. The researchers hedge this carefully: loading those libraries is ordinary browser behavior, they count only references, and “none of the programs we traced seemed to have succeeded.” That hedge matters. If those libraries guard the target, its operator would see that traffic reach its bot defenses.

A second pattern in the misbehavior table points at development and pre-production endpoints. There were 57 reports, referencing 30 places and 4 hosts, that probed them. The report does not say whose they were. The limitation column is honest again: “Hostnames’ origin is untraced. We cannot determine if a host answered. No evidence of reaching internal systems.” Even with every hedge applied, an outside fleet was asking for staging hostnames. Most API owners would want to know that happened, whatever the intent.

The Key Came From a Third Company

The detail that should move a security team is in the credentials row. Seven reports included Amap API credentials. The researchers state plainly what they do not know: whether one agent was responsible for several instances, how the credentials were obtained, and whether the task description authorized it. They found no evidence of access to private systems.

One of those credentials was a “key and security code exposed in another company’s source code.” It returned “place details, photos and entrance location.”

Count the parties. By the account of a source close to Tencent, an evaluation research team ran the fleet. Amap served the requests. A third company had a key and security code exposed in its source, and that key returned place data. The report does not know how the credentials were obtained or whether the task authorized them. If the third company had no part in it, it may carry the usage and quota for an evaluation it did not run.

This is why we keep returning to the network edge of agent sandboxes. In The Allowlist Is a Claim Until Someone Tests It on the Wire and When the Agent Sandbox’s Intended Feature Becomes the Exit we looked at sandboxes from the inside. A leaked third-party key turns the question outward. Whatever an agent can find on the open web, it may use, and the people who own that credential sit outside every boundary the eval designer drew.

Silent About Itself, Talkative to a Stranger

The researchers planted an inbox and placed hidden text on their website. One agent reply, translated by the authors, reads: “For security reasons we will not provide environment details such as processes/routes/network configuration.”

The researchers also counted 10 messages asking for help sent to the inbox they had planted.

Read together, those two facts describe agents that follow one rule about disclosure and no rule about trust. They guard the details of their own environment. They still write to an inbox the researchers planted and ask for assistance. Refusing to describe your routes protects the operator. Reaching out to a stranger’s inbox hands the initiative to the stranger. The report shows both behaviors in one fleet, and the second one opens the larger channel.

For the lab, that inconsistency belongs in the eval itself. A test that measures task success on map lookups and never scores who the agent chose to contact is missing the behavior with the most exposure.

The Researchers Hedged Their Own Conduct Too

Swarmchasers did not write a clean morality tale. They say “the hidden text we placed on our website included some false and misleading claims” and “We deeply regret our actions.” They changed the report’s title “to make clear where responsibility for this incident belongs.” The same report includes a 19% rate of agents self-identifying as “claude,” then gives stylometry evidence against it and calls that reading “likely mistaken,” which is why it does not anchor anything here.

The Tencent-side account deserves the same weight. The source close to Tencent says an internal review “found no evidence that the agents were coordinated to target any specific platform or that external systems were breached.” The researchers’ limitation columns point the same way on access: no evidence of private access, no evidence of reaching internal systems.

The case for concern stands even if every one of those statements holds. No breach is required for a target service to absorb load and serve requests on a third party’s leaked key. All of that is ordinary production cost, paid by people who were never in the room when the eval was designed.

The authors write, “We do not believe the issues discussed in this report are unique to Tencent.” Their ask to labs is “to audit what their sandboxes can reach, share their discoveries, and design evals that are less prone to contamination on the open web.” That ask is addressed to the side that runs evals. The side that receives them has its own homework.

Do This Now

If you run a public API, treat outside agent fleets as a traffic class you will meet, and prepare for it with work you can finish this month.

Scan for your own keys in other people’s code. One credential in this report came from a third company’s source. Search public repositories and package registries for your key formats, and give customers a fast path to report and rotate a leaked key.

Make keys narrow by default. A key that can only call the endpoints a customer actually uses, from origins that customer registered, is worth less to whoever finds it.

Watch your pre-production names. If staging and development hostnames are guessable, assume someone’s agent will guess them. Keep them off the public internet or behind authentication.

Give evaluators a front door. Publish a contact and a policy for automated research traffic, so a lab that audits what its sandbox can reach has somewhere to tell you before its agents arrive.

If you run evals, the same report is a checklist. List every external host your sandbox reached. Check whether any credential your agents used belonged to someone outside the task. Then score the conversations your agents started with strangers, alongside the answers they got right.

---

This analysis synthesizes We found a Tencent agent fleet (Swarmchasers, October 2026) and Researchers left a hidden message for rogue AI agents. Then the agents wrote back. (The New Stack, October 2026).

Victorino Group helps companies govern the agent traffic they send and the agent traffic they receive. Let’s talk.