Agents and IDSDL

IDSDL is a procedural language for interior design. A scene is not a file of coordinates but a program: it registers objects from natural-language descriptions, arranges them through functional groups, declares the constraints the arrangement has to satisfy, and produces a room when it is executed.

with scene.RelativeGroup() as bedside:
    bed = scene.AddAsset("a queen-sized bed with a wooden frame")
    bedside.set_anchor(bed)
    bedside.place_on_back_left(scene.AddAsset("a small wooden nightstand"))
    bedside.place_on_back_right(scene.AddAsset("a small wooden nightstand"))

Programs like this are meant to be written by a machine. The intended shape of the system is therefore the simplest one imaginable: a prompt goes to a model, the model writes IDSDL, the program executes, and a scene comes out.

That schema does not survive contact with reality, and the reason is worth stating plainly.

One pass is not enough

IDSDL did not exist when today’s models were trained. There is no corpus of it on GitHub, no questions about it on Stack Overflow, no idiomatic usage anywhere in pretraining. A capable model asked to write a scene program in a single pass produces something shaped like Python and unlike IDSDL: invented placement verbs, keyword arguments that were never in the signature, group methods borrowed from the wrong class. The program does not run, and when it does run it is often wrong in ways only visible once the room is rendered.

So two things have to be arranged. The model has to be shown the language — its documentation, and examples of it being used well. And it has to be allowed to fail and recover — to execute, read the traceback, and repair. Neither is prompting. Together they are an agent.

The ProgramSynthesizer

The paper’s answer is the component in Figure 8, which is deliberately modest: three roles around one backbone model, differing only in their system prompts.

Figure 8 from the InteriorAgent paper. The Programmer writes a draft program from the prompt and retrieved examples; the Debugger repairs it, first by reading the code and then by reading the traceback from an actual execution.

The Programmer is prompted with the IDSDL documentation and a handful of relevant example programs, and emits a first draft — the paper calls it a β program precisely because it is expected to be broken. The Retriever is what supplies those examples: a curated set of instructive programs, each paired with a short description of the layout it builds, searched for the closest matches to the request. The Debugger then makes the draft executable in two stages: CodeRefine reads the program and fixes what is visible statically, and TraceRefine executes it and works from the resulting traceback.

The paper is direct about why this component exists: IDSDL is a large, tool-backed API, and writing correct code against it requires iteration, reference to documentation, and repair from real execution errors. Prior program-synthesis approaches to 3D scenes did not have this, and it shows in how often their programs run at all.

The loop

Getting a program to execute is only half of it. A scene can run cleanly and still be a bad room, so execution feeds a second, outer cycle.

Figure 3 from the paper. The prompt (1) becomes a program (3) through the synthesizer (2); executing it builds the room group by group (4–9); the rendered result is read back as feedback (10), which becomes a program edit (11) — here moving a wall clock off the door — and a corrected scene (12).

Executing the program produces two things: the scene, and feedback about it — constraint violations from the solver, and a vision-language model’s reading of the renders. That feedback returns to the synthesizer, which edits the program and runs it again.

The detail that matters is what gets edited. The feedback never moves an object directly; it changes the line of code that placed it. The artifact the loop converges on is a program, which is why the result stays reproducible under a seed, re-editable later, and carries its design rules with it.

Why there is a language at all

It is worth asking why an agent should write IDSDL rather than place coordinates itself.

IDSDL exists because interior design knowledge was codified into it. Functional groups are borrowed from how designers actually work — clusters of objects that together support an activity. The anchored placement slots carry ergonomic distances that were decided once, by people who knew them, rather than being re-derived per scene. Constraints name the requirements a room has to meet — clearance, access, sightlines — and a solver enforces them. When an agent writes place_on_back_left(nightstand), it is choosing a relationship and inheriting the numbers.

Take that away and the knowledge has to come from somewhere else, which in practice means the model reasoning its way to every coordinate at inference time from a text prompt. That is where layouts start to read plausibly in prose and wrongly in the render.

The harness matured; the codification did not have to

The cycle in Figure 3 — plan, execute, critique, repair — is not specific to interior design. It is the shape almost every coding agent now takes, and in the years since the paper the field has built far better versions of it than a bespoke pipeline could: Claude Code, Codex, and their peers retrieve context, recover from errors, and stay coherent across long sessions as a matter of course.

That is a good outcome, because it separates the pieces. The loop is now something to adopt rather than build. What does not come off the shelf is the knowledge the loop operates on.

Two forms of codified expertise

This is the durable claim, and it generalizes past interior design to any domain where the work depends on expertise that took people years to acquire. Harnesses will keep improving; what determines whether an agent produces expert output is what it is given. Expertise gets codified in two forms, and both matter:

• A domain-specific language. Design principles compiled into vocabulary and enforcement, so that competence is inherited by construction. That is IDSDL.
• Institutional knowledge. How practitioners actually work — which layout family a room belongs to, what order to build it in, which constraint to reach for, what went wrong the last time someone built a room like this. That accumulates through use, and in this repo it lives in skills/.

Figure 8 already contained both. The Programmer and Debugger are the harness that writes code in the language; the Retriever is the mechanism that injects codified precedent into the synthesis problem. What has changed since is the corpus on the other end of that retrieval. The paper retrieved example programs. skills/ now carries whole workflows — a language reference, worked scene programs each with a write-up of what it taught, notes on phasing and composition, a per-asset constraint playbook, and a policy for acting on visual feedback. An example program shows an agent one finished artifact; a workflow tells it how that artifact came to be.

Give a modern harness both, and the loop the paper described keeps its shape. Only the agent in the middle got better.

---

Next: put an agent in that loop with the MCP Quickstart, or read the corpus it retrieves from in the Worked Examples.