Aeneas Factory: an asset factory on a Mac that AI agents can use safely
Games need a lot of stuff: props, textures, little sounds, a voice line here and there. I wanted my AI agents to be able to make those for me, on my own machine, with open models, and without me having to check every file by hand. So I built Aeneas Factory.
It generates images, seamless textures, textured 3D models, sound effects and speech locally on an Apple Silicon Mac. Every model is open, pinned and checksummed, and nothing leaves the machine after the downloads. It’s open source, under MIT, on GitHub. This post describes version 0.2.1.
That chest is the gallery in one picture. Z-Image Turbo made it from a sentence. FLUX.2 [klein] repainted it from the instruction “paint the wooden planks a deep blue, keep the iron bands, the brass lock and the white background”. TRELLIS.2 turned the first picture into a textured, UV-unwrapped 3D model, rendered here in Blender. All three came out of one command each, with fixed seeds, so anyone can make them again.
Built with DINOv3. The 3D models in this post (the chest on the right above and the bucket further down) were made with TRELLIS.2, which uses Meta’s DINOv3 image features under the DINOv3 License.
A failure is never a success
The thing that made me build it properly was a blank image. The Metal VAE decode in the stable-diffusion.cpp build I use returns plain white pictures, and the first version of this repo happily reported them as successes. An agent would have taken that and moved on.
So now every output is checked before it counts. Images are checked for being blank or flat, tiles for visible seams, sounds for silence, meshes for being empty or flat. A failed check is an error, never a result.
On top of that, --candidates N makes several, drops the failures, ranks the rest by how well they match the prompt (SigLIP for pictures, CLAP for sound) and keeps the best. All the candidates stay on disk.
Every file says how it was made
Each file carries its own record, inside it and beside it: the PNG text chunk, the glTF extras, the WAV INFO block, and a .json next to it. The record lists the models with every file’s SHA-256, the versions of the factory and its tools, the prompt, the seed, the settings, the checks it passed and the licence policy that applied. Whenever I find an asset later, I can tell exactly where it came from.
Licences are handled per project. A small policy.toml says which licences and models a project allows. One of TRELLIS.2’s parts, the DINOv3 features, is under Meta’s own DINOv3 License, so it only downloads after you accept it, and a project has to allow it before it’s used. That licence allows commercial use, asks for “Built with DINOv3” to be shown wherever DINOv3 or its derivatives are distributed, and rules out military, weapons and espionage uses. Every 3D model the factory makes carries “Built with DINOv3” in its provenance and credits.
Model cards matter too. Image ranking used to run on OpenAI’s CLIP, but its model card puts any deployed use out of scope, so the factory now ranks with SigLIP (Apache-2.0). The records state licences as the model cards give them, which is not legal advice.
Finished assets, not raw generations
Most of the real work sits between the model and something a game can actually use. Recipes do that part: several steps in a row, every intermediate kept, one manifest at the end.
The prop recipe made that bucket. It picks the best of two concept images, turns it into 3D with TRELLIS.2, and then Blender makes it 0.4 m tall, puts its origin at the bottom and cuts its 149,556 faces down to LODs of 20,000, 10,000 and 5,000. That took 11.6 minutes. The sprite recipe cut the key out onto a transparent background with BiRefNet. The cobblestones are a seamless tile, shown here 2x2, and they come with height, normal, roughness and AO maps.
Sounds go through the same mill. This is the chest lid creaking open and thudding shut, from MOSS-SoundEffect:
The sfx recipe goes further: two takes ranked by CLAP, silence trimmed, loudness set to -18 LUFS. You can listen to the chest and the door on GitHub.
Built for agents
The point of all this is that an agent can use it without supervision. Claude Code or Codex connects through an MCP server with sixteen tools: generate an image, make a texture, turn a picture into 3D, make a sound, run a recipe, search the library, read a file’s provenance, and so on.
Every result comes back with a thumbnail the agent can actually look at: the image, a 2x2 tile, a render of the model, or a waveform. Long jobs like 3D return a job id straight away. And the agent can’t do anything dangerous with it: the server never runs code it is sent, never downloads models, and only writes under your home folder, outside the factory.
Underneath there is one factory service per machine. It owns the GPU with a single queue, so two agents can’t start two jobs at once and run out of memory. It keeps the sound model loaded between jobs and listens only on 127.0.0.1, with a token, refusing anything that looks like it came from a web page. The MCP server, the aen command line and ComfyUI all go through it, so the checks, provenance and policy apply the same way everywhere.
There is also an asset library that remembers everything generated or imported (a downloaded CC0 pack, say), searchable by words and, for images, by meaning. aen library credits writes a project’s credits page from it. Style packs keep a project’s look consistent across every asset.
Under the hood
The engines are stable-diffusion.cpp for Z-Image Turbo and FLUX.2 [klein] 4B, trellis.cpp for TRELLIS.2, and MLX for MOSS-SoundEffect and Kokoro speech, plus Real-ESRGAN, BiRefNet, Blender and ffmpeg. ComfyUI is an optional engine and a playground, reading the same model files in place. The four core models are 40.5 GB together; make install GROUPS=image gets you started with just the image ones.
On my M5 Pro a 1024 px image takes two to three minutes, three 512 px candidates ranked by prompt match take 75 seconds, and a 3D model takes about five minutes.
It’s Mac only for now. The MLX parts (sound and speech) are Apple-only. The image and 3D engines also build on Linux with NVIDIA, but I haven’t tested that. If you try it, I’d love to hear how it goes.
// comments