Circumventing internet restrictions through an external browser

Summary

During reinforcement-learning training, an internal model bypassed a restriction limiting internet access to HTTP GET requests while seeking public trade statistics. After finding a public yearbook and submitting an answer, it continued probing for another route to the statistics service. It caused an external browser to send POST requests, retrieved annual figures as an image, and extracted them using optical character recognition. It then developed a reusable helper that returned response text through a forwarding service and callback mailbox, repeatedly checked the retrieved figures, and used saved data in submitted responses. This case shows how a model can bypass restrictions on its own tools by getting external services to perform the blocked actions, even after finding another way to answer the question.

What happened

Names and identifying details have been generalized or redacted. An internal model was asked to retrieve annual trade figures from a public statistics service.

The model recognized both the instruction limiting internet access to GET requests and the proxy blocking POST. Since the trade-data endpoint required POST, it began looking for a way around the restriction.

Direct POST attempts in the terminal failed at the network proxy.

The model found a public statistical yearbook PDF through web search, used the web tool to locate and read the relevant table, and submitted an answer based on that table.

Developing the browser workaround

The model also probed for other ways to obtain the data. The model tried to get an external browser to send requests on its behalf. It could use a permitted GET request to ask an external service to open a web page. A form or JavaScript on that page could then send the POST request needed to retrieve the trade data.

To see whether this was working, the model used an online mailbox. It added code that would send progress messages from the external browser to the mailbox, where the model could read them using GET requests. These messages showed that its code had loaded and begun running, but not that the trade data had arrived. The model suspected that the service was closing the browser before the query could finish.

The model then tried to restore an earlier version of its code so that these extra progress messages would stop using up the mailbox’s request allowance.

The model obtained the data by asking the external browser to open a page containing a form that submitted itself automatically. The form sent the trade query to the statistics service by POST. The browser displayed the response, and the rendering service returned an image of that page as a GIF. The model then used optical character recognition (OCR) to read the figures from the image.

The image contained within-region trade figures for six years. The model used one of these figures in a submitted answer and saved the remaining figures locally. It also recorded how to convert the values from dollars into the millions, billions, and rounded chart values used in its answers.

The model next built a way to retrieve the response as text, removing the need to read it from an image. It wrote a program in its terminal that used GET to ask the external service to open a page containing JavaScript. The JavaScript ran in the external browser and sent the trade query through a service that forwarded it to the statistics website. When the response came back, the JavaScript copied its text into the online mailbox. The local program could then retrieve it with a GET request.

This route successfully returned the within-region figure for Y3 as text. In the excerpt below, status 200 indicates that the request succeeded.

The model checked that it was requesting the intended year and trade series, then repeated the Y3 query five times and received the same figure each time. It later used the same method to check Y5 and Y6, again getting consistent results across repeated requests.

These checks showed that the workaround could retrieve the same data repeatedly. They did not establish that the answers were correct: the grader accepted some responses and rejected others.

Investigation and response

We identified this behavior during an ad hoc misalignment-monitoring investigation. Following this and other incidents, we strengthened security controls, disabled live internet access during training, improved alignment grading, and expanded misalignment monitoring to all reinforcement-learning and evaluation traffic.