Neural Cellular Automata for Procedural Terrain Generation

Procedural terrain generation means algorithms produce landscapes, caves, dungeons, and open worlds instead of artists placing every rock and ridge by hand. Neural cellular automata, or NCA, are the newest entrant in that lineage, and they solve a problem classical methods never really cracked: how to make a generator that scales to any map size without being retrained or redesigned for it.

Classical cellular automata set the baseline for this kind of work. Early work showed that simple CA rulesets could produce playable, tunnel-based dungeon maps at very low computational cost, and that result has served as a useful reference point in the field. Noise functions like Perlin and simplex noise, along with grammar-based generation methods, round out the classical toolkit. All of them share the same ceiling, though. Their rules are hand-crafted by a designer, and changing the output means changing the rule by hand. Scaling a hand-authored rule to a bigger or more varied domain doesn't make the design effort grow in a straight line; it compounds.

What a neural cellular automaton is

A cellular automaton, at its simplest, is a grid of cells where each cell holds a state, and a fixed rule updates every cell by looking only at its immediate neighbors. No cell ever sees the whole board. A famous cellular-automaton example follows this same logic, but the same structure underlies terrain-generating CA: local rule, local view, global pattern that emerges after enough steps.

The neural version keeps that structure and swaps out the rule itself. Instead of a designer writing an if-then rule by hand, a convolutional neural network reads the neighborhood and outputs the next state, and the network's weights are learned from data rather than authored line by line. The foundational paper here is Mordvintsev, Randazzo, Niklasson, and Levin's "Growing Neural Cellular Automata," published in Distill in 2020 (DOI: 10.23915/distill.00023). It trains an end-to-end differentiable, self-organizing CA that can grow a target image pattern from a single seed cell and regenerate that pattern after damage. The paper leans on a biological analogy to explain why this matters: morphogenesis, the process by which a single fertilized egg cell and its descendants self-assemble into a complete, complex anatomy, with no cell holding a blueprint of the final organism, just a shared local rule every cell obeys. An NCA is that same idea encoded in a neural network: no cell knows what a mountain range looks like, but the shared rule causes one to emerge anyway.

How the update rule is learned

Every cell in an NCA runs the same network. It looks at its immediate neighbors' states, and it outputs its own next state, and because that network is shared across every cell and every position on the grid, the model stays compact no matter how large the grid gets at inference time.

Training happens through backpropagation, run through however many update steps it takes for the grid to settle. The loss function gets applied to the final grid state, and it can be defined against a target pattern, a functional metric like path connectivity, or a quality-diversity objective that rewards varied but valid outputs. What ends up encoded in the weights isn't a set of terrain features directly. There's no ridge template and no lake template sitting inside the network. What's encoded is a rule that, applied over and over locally, causes those features to emerge on their own.

One notable extension here is conditional NCA, or c-NCA. Sakour's work showed that a single set of local rules can encode multiple distinct structural outcomes, with the specific structure selected by a class vector spatially broadcast across every cell in the grid. That's a meaningful proof of concept for terrain generation specifically: one trained NCA, without retraining, can grow a desert map or a mountain map or a cave system, depending only on which condition vector it's given.

The scalability property that separates NCA from other learned generators

The single property that makes NCA worth taking seriously for terrain work is locality. Because the update rule only ever reads a cell's immediate neighbors, a generator trained on a small grid carries over directly to a much larger grid at test time. No retraining. No change to the network's architecture. The same weights, applied over more cells, for however many steps it takes.

Zhang et al. demonstrated this directly. They trained a class of learned environment generators on small warehouse layouts, 36 by 33 cells, and then used those same trained models to generate environments considerably larger in both dimensions and larger still, entirely at test time. The generated environments held onto consistent, regularized structure regardless of size, and when applied to multi-robot coordination, that consistency translated into throughput gains tested with up to 2,350 robots, a considerably larger scale than the roughly 350-robot ceiling that prior quality-diversity-only approaches had been tested against.

What "consistent patterns" means in practice deserves precision. It's not that the NCA tiles a small pattern repeatedly across a bigger canvas, the way a texture might repeat. The local spatial regularities the model learns at small scale, things like path connectivity, the density of open areas, how obstacles get distributed, replicate coherently as the grid expands. The pattern grows outward rather than getting stamped and copied.

Handling the hard problem: making NCA output functionally valid terrain

Locality is also where NCA runs into its hardest problem. Because no single cell ever sees the whole map, a purely local update rule can easily produce a map with a blocked path, a disconnected region, or some other violation of a hard constraint the designer actually cares about. The rule optimizes for what's visible nearby; it does not optimize for whether the finished layout works as a whole.

Zhang et al.'s answer to this was to pair the NCA with a solver. A Mixed Integer Linear Programming, or MILP, solver runs as a post-processing step on the NCA's raw output, repairing it to satisfy hard constraints like full connectivity and required storage capacity in the warehouse layouts. Just as important, the training objective itself was designed to push the NCA toward producing layouts that were already nearly valid before the solver ever touched them, which cuts down how much repair work the solver has to do. Learning and constraint satisfaction aren't competing steps here, they're working together, with the network doing the bulk of the work and the solver cleaning up what's left.

A similar pattern appears elsewhere in procedural content generation. A related hybrid approach in procedural content generation pairs a constraint-based pattern method with a reinforcement-learning generation approach, using local pattern learning to guide an agent that optimizes for global properties. It's a different mechanism than NCA plus MILP, but the underlying idea is the same: pair something that's good at local pattern coherence with something that's good at checking or enforcing global correctness.

Extending NCA beyond flat grids: mesh, particle, and 3D surface terrain

Most published NCA terrain work assumes a flat height-map grid, and that's a fine assumption for a lot of game terrain, but it breaks down fast for cliffs, overhangs, cave systems, or anything at planetary scale where the geometry isn't a simple height field.

MeshNCA generalizes the approach from a flat grid to an arbitrary 3D mesh. Cells live on mesh vertices instead of grid cells, and perception runs over the mesh's own connectivity rather than a fixed neighbor pattern. Trained only on an icosphere, MeshNCA can synthesize textures on meshes it never saw during training, and it supports real-time interactive editing. Related work builds on this, extending mesh-based NCA perception via message passing over mesh connectivity. Coordinate-aware perception appears to play a meaningful role in keeping the surface coherent.

Neural Particle Automata, from Kim, Pajouheshgar, Süsstrunk, Jakob, and Park, pushes the generalization further still. Instead of cells fixed to a lattice, this is a Lagrangian version of the idea: particles with continuous positions, neighborhoods that shift as the particles move, and differentiable smoothed-particle-hydrodynamics operators standing in for grid-based perception. It keeps the robustness and self-regeneration that made the original Growing NCA work notable, while allowing for uneven, heterogeneous dynamics and concentrating compute where the action actually is, rather than spending it uniformly across an entire grid.

Variants that add control, diversity, and designer intent to NCA terrain

Base NCA gives a generator that scales and self-repairs. It doesn't, by itself, give a designer much of a way to steer the output or ask for something specific. Several variants close that gap.

VNCA pairs an NCA with a variational autoencoder to learn a smooth latent space. A designer samples a latent vector, and a hypernetwork generates the actual NCA weights from that vector, which makes it possible to interpolate smoothly between distinct terrain shapes rather than jumping between discrete presets. The payoff is a continuous space a designer can actually navigate.

Attention-based NCA variants modify the perception step to let each cell weigh its neighbors selectively, which can enable sharper boundaries between regions and smoother transitions where they're wanted.

Goal-guided NCA incorporates a conditioning signal into the generation process, pointing toward terrain built explicitly around a purpose: shaped to satisfy some spatial objective a designer sets out ahead of time.

There's also the quality-diversity archive approach: rather than training one generator, train a whole population of them, each one different, and let a designer pick from that archive based on whatever properties matter for the task at hand. That maps cleanly onto what a terrain designer actually wants day to day, which is rarely a single best output, but a palette of options to choose from.

How NCA terrain compares to diffusion-based and hybrid alternatives

Diffusion models are the other active track in learned terrain generation, and they're solving a related but different problem. Barazandeh and Zachmann's "Earthbender" (ACM SIGGRAPH MIG 2025, DOI: 10.1145/3769047.3769053) uses guided diffusion for interactive, stylistic heightmap generation. Jain, Gain, and Cordonnier's "Pixels2Peaks" (ACM Transactions on Graphics, Volume 45, Issue 4, 2026, DOI: 10.1145/3811288) converts real terrain images into heightmaps. The terrain-diffusion approach takes a coarse map, whether procedurally generated or hand-drawn, and produces a high-resolution heightmap from it. A broader survey of the field compares search-based, machine-learning-based, noise-function-based, and LLM-based PCG methods across content types.

Where diffusion wins is fidelity: high-quality single-shot output, and strong support for image-conditioned authoring, where a designer feeds in a reference image and gets a matching heightmap back. Where NCA wins is different: scalability to arbitrary sizes without retraining, iterative self-repair built into the generation process itself, computation that's local and can run progressively rather than all at once, and a natural fit for settings where agents operate inside the generated environment, as in the multi-robot warehouse case.

The hybrid approach mentioned earlier, pairing a constraint-based pattern method with a reinforcement-learning generation approach, points at the same underlying convergence rather than competing with the other model. Local pattern constraint paired with global-property optimization appears across multiple corners of procedural generation research, whether the local piece is WFC, an NCA, or something else. That convergence, more than any single technique, is the actual trend to watch.

Building with NCA terrain in practice today

The most directly usable artifact for anyone wanting to build with NCA terrain generation right now is Zhang et al.'s NeurIPS 2023 source code, released publicly at github.com/lunjohnzhang/warehouse_env_gen_nca_public. It's the clearest working example in the current research of an NCA generator paired with a constraint solver, applied to a real coordination problem at meaningful scale.

The original Growing Neural Cellular Automata work from Distill remains the reference implementation for understanding the core mechanism, the differentiable, self-organizing update rule trained end to end, even though its published demonstrations are image patterns rather than terrain. Everything downstream, from c-NCA's conditional generation to MeshNCA's jump onto arbitrary 3D surfaces to Neural Particle Automata's move off the grid entirely, builds on that same foundational mechanism: a local rule, shared everywhere, that turns iteration into structure. For terrain generation specifically, the warehouse results already show the mechanism works at meaningful scale. The constraint-solving and conditioning machinery must grow before terrain generated by this kind of model can handle the full range of hard requirements, connectivity, navigability, biome logic, that a shipped game or simulation actually demands.