comfyui_pcg docs

Terrain to AI texture#

A complete procedural terrain -> diffusion model -> back onto the mesh chain. A noise height field is eroded and classified into five material layers; those layers become regional prompts and the height becomes a ControlNet depth image; SDXL paints a top-down texture that lines up pixel for pixel with the height field; the texture is put back into the terrain and turned into a coloured mesh.

Load it#

Drag example_workflows/pcg_terrain_to_ai_texture.json onto the ComfyUI canvas.

Requirements

WhatDefault in the workflowNotes
SDXL checkpointSDXL\RealVisXL_V4.0.safetensorsPick your own SDXL checkpoint in CheckpointLoaderSimple.
SDXL ControlNetSDXL\controlnet-union-sdxl-1_promax.safetensorsA union or a depth ControlNet for SDXL both work. SetUnionControlNetType is set to depth.
Custom nodesnone besides comfyui_pcgThe AI part uses ComfyUI core nodes only: CheckpointLoaderSimple, CLIPTextEncode, ControlNetLoader, SetUnionControlNetType, ControlNetApplyAdvanced, EmptyLatentImage, KSampler, VAEDecode, SaveImage, PreviewImage.

The erosion runs at 512 x 512; the terrain is then upsampled to 1024 x 1024 (with a fine relief on top), the size the image is generated at, so masks, control image and texture line up pixel for pixel.

Step by step#

1. Terrain (left two columns)

  1. COP Noise (PCG): 512 x 512 simplex noise, standard fractal, frequency 3, 6 octaves, amplitude 0.06, seed 7.
  2. COP HeightField Stream Power Erosion (PCG): stream-power erosion (60 iterations, rho8 routing) cuts river valleys.
  3. COP Erosion Filter (PCG): the erosion filter (amplitude 0.003, wavelength 40).
  4. COP HeightField Thermal Erosion (PCG): thermal erosion (talus angle 38, 12 iterations).
  5. COP HeightField Hydraulic Erosion (PCG): hydraulic erosion: gullies plus alluvium, accumulated into the debris layer.
  6. COP Flow Accumulation (PCG): flow accumulation (d8) for rivers and lakes.
  7. COP HeightField Splat Map (PCG): splits the terrain into grass / sediment / rock / snow / water; the five layers sum to 1 everywhere (rock angle 42, snow line 0.78).

2. Prompts and control image

  1. Regional Prompt from COP (PCG) turns the splat layers into regional conditioning: a base prompt plus one line per layer (layer: prompt); later lines take priority. Its preview output shows the regions.
  2. Control Image (PCG) in height mode outputs a top-down depth map (high = white) for the ControlNet.

3. Generation

  1. SDXL with the union ControlNet in depth mode (strength 0.85, ending at 85% of the steps), a negative prompt against perspective, sky, text and similar, and KSampler (seed 7, 28 steps, CFG 5, dpmpp_2m / karras) produce a top-down texture aligned with the height field. Save Image writes it as pcg/terrain_albedo.

4. Back to the mesh

  1. COP Import (PCG) reads the generated image back into a COPIMAGE (layer albedo).
  2. COP Composite (PCG) (mode replace) puts it into the terrain image's albedo layer.
  3. COP HeightField to Mesh (PCG) (step 2, attributes Cd=albedo) builds the mesh with the texture as point colour.
  4. Geometry Preview (PCG) in the Studio look shows the textured terrain, and Export Mesh (PCG) writes pcg/terrain_ai (glb) to the output folder.

Worth tweaking#

NodeParameterEffect
COP Noiserand_seed, frequency, amplitudeA different terrain.
Splat Maprock_angle, snow_lineHow much rock and snow the prompts get.
Regional Promptbase_prompt, regionsThe look of each material.
ControlNetApplyAdvancedstrength, end_percentHow strictly the image follows the height.
KSamplerseedA different texture for the same terrain.

See also#