Building a Local 3D AI Pipeline with Claude, ComfyUI, and Blender
This guide shows you how to connect Claude AI to Blender and ComfyUI to create a powerful local 3D AI pipeline. You'll learn how to generate images, 3D models, and even animate characters using open-source tools like Trellis 2, KimoDo, and the image — all controlled through natural language prompts.
What You'll Learn
Prerequisites
Before starting, ensure you have:
- Blender 5.1 or newer installed
- ComfyUI installed and running (see our ComfyUI Installation Guide)
- Claude Desktop installed
- Basic familiarity with ComfyUI workflows and Blender navigation
Hardware Considerations
Running this pipeline locally requires a capable GPU. If your hardware isn't powerful enough or you want to speed up generations, consider using RunPod's cloud GPU service.
Special Offer - $5 Credit Included!
When you sign up for RunPod using our affiliate link, you'll receive a $5 credit that can be used to generate up to 9,000 images and 300 videos. This gives you plenty of resources to explore ComfyUI and AI image/video generation without any upfront cost!
Step-by-Step Process
Step 1: Install the Blender MCP Server
The Blender MCP (Model Context Protocol) server provides the bridge between Claude and Blender. This is the foundation of the entire pipeline.
- Download the Blender MCP add-on from the GitHub repository (link in video description)
- Install the add-on by dragging and dropping the file into Blender, or download it directly from the repository page
- Verify the installation by going to Edit > Preferences > Add-ons and searching for MCP
- Confirm the MCP checkbox is checked — this means the add-on is active
- Ensure the MCP server is running in Blender before proceeding
Pro Tip
Keep Blender open and running throughout the entire process. Claude needs an active Blender instance to communicate with.
Step 2: Set Up the ComfyUI MCP Server
The ComfyUI MCP server allows Claude to read, execute, and manage ComfyUI workflows directly.
- Download the ComfyUI MCP from GitHub (link in video description)
- Open Claude Desktop and click the three dots in the top-right corner
- Navigate to File > Settings > Developer mode
- Click Edit Config to open your Claude configuration folder
- Locate the
claude_desktop_config.jsonfile in the roaming Claude folder - Right-click the file and open it with Notepad or Visual Studio Code
- Copy the MCP server code from the GitHub repository and paste it into the config file
- Save the file and restart Claude Desktop
Config File Location
If the Edit Config button doesn't work, navigate manually to your App Data location where Claude Desktop is installed and find the claude_desktop_config.json file there.
Step 3: Install the Blender Extension in Claude
Claude needs the Blender extension to properly interface with the Blender MCP server.
- Open Claude Desktop and go to File > Settings
- Navigate to Developer > Extensions
- Click Browse Extensions
- Search for "Blender" in the extensions marketplace
- Install the Blender extension
- Verify it appears in the developer extensions list
Pro Tip
Make sure both ComfyUI and Blender are running before testing the connection. Claude can't access them if they're not active.
Step 4: Test the Connection
Before building complex workflows, verify that Claude can communicate with Blender.
- Open a new chat in Claude Desktop
- Type a test command like: "Delete the cube in the scene and create a new one called test succeeded"
- Press Enter and watch Claude's response
- Check Blender — you should see the cube renamed to "test succeeded"
Connection Successful!
If Claude renames the cube, your pipeline is working. Claude will also notify you if it can't access the scene.
Step 5: Prepare ComfyUI Workflows for Claude
Claude needs API-ready workflows to generate images and 3D models. You'll prepare two workflows: one for image generation and one for 3D asset creation.
- Open ComfyUI and ensure it's running
- Load the image generation workflow — go to Templates and search for "the image Turbo text-to-image"
- Verify all models are downloaded and the workflow runs locally
- Enable API nodes by going to Settings and ensuring "API nodes" is enabled
- Export the workflow by going to File > Export API
- Name the file something like
image_turbo_api.json - Repeat for the Trellis 2 workflow — load it from Templates, verify it works, and export as API
Trellis 2 Optimization
For faster 3D asset generation, check out the Trellis 2 GT AF workflow mentioned in the video. It guarantees the fastest generation speeds, especially when generating multiple prompts.
Pro Tip
You can disable texturing in the Trellis 2 workflow if you only need the raw 3D models. This speeds up batch generation significantly.
Step 6: Feed Workflows to Claude
Claude needs to understand your workflows before it can use them.
- Open a new chat in Claude Desktop
- Drag and drop the
image_turbo_api.jsonfile into the chat - Type: "Read this ComfyUI workflow JSON and make it ready so you can use it in connection with the ComfyUI MCP"
- Press Enter and wait for Claude to process
- Repeat with the Trellis 2 API file
- Type: "Read these ComfyUI workflows JSON and make them ready so you can use them in connection with the ComfyUI MCP"
Allow File Access
When prompted, make sure to allow Claude Desktop to access the files. Without this permission, Claude can't read or use the workflows.
Step 7: Generate a Batch of 3D Assets
Now you can use Claude to generate multiple 3D assets at once and import them into Blender.
- Type a prompt like: "Build me a medieval kit consisting of 10 different props for a game level. Use the image turbo for the image generation and Trellis 2 GGF locally for the asset generation. Afterwards, import them all into Blender and arrange them into a game level."
- Press Enter and let Claude work
- Monitor progress — Claude will generate images first, then convert them to 3D models
- Watch as assets appear in Blender with textures applied
Customization Options
You can use LoRAs to customize the style of your generated assets. This helps achieve a consistent look across your game level or 3D scene.
Pro Tip
The generation process takes time. You can lean back and let Claude work — it will keep you posted on progress and fix minor issues automatically.
Step 8: Fix Low-Quality Textures with Trellis Splat
If your ground textures appear blurry, use Trellis Splat (focused on Gaussian Splat) to improve quality.
- Open ComfyUI and go to Templates
- Search for "Trellis Splat" — make sure you're on the latest ComfyUI version
- Load the workflow and go to File > Export as API
- Drag and drop the exported file into Claude
- Prompt Claude to generate the ground and import it into Blender
- Position the new ground in your scene
Step 9: Clean Up AI-Generated 3D Models
AI-generated models often have messy wireframes. Claude can automate the cleanup process.
- Import your 3D AI model into Blender
- Prompt Claude with instructions like: "Clean up the mesh I just imported. Merge the vertices, fix the scaling, make a low poly version using quad remesher, and bake the texture from high to low poly model"
- Install the Quad Remesher add-on for Blender (link in video description)
- Let Claude process — it will create retopology geometry and bake textures
Memory Advantage
Claude stores the cleanup steps in memory. You can load any model and Claude will remember the process, making future cleanups much faster.
Manual Fixes May Be Needed
The eye area or other complex geometry might need manual adjustment. Claude can get into an endless loop trying to fix certain issues, so manual fixes are often faster.
Step 10: Set Up KimoDo in Blender
KimoDo is a text-to-animation generator. The KimoDo Blender Bridge add-on connects it directly to Blender.
- Download the KimoDo Blender Bridge from GitHub
- Open Blender and go to Edit > Preferences > Add-ons
- Click the arrow and select Install from Disk
- Select the KimoDo Blender Bridge zip file and install
- Restart Blender
- Press N to open the sidebar — you should see a KimoDo tab
Two Installation Options
The KimoDo add-on offers two modes:
- Default mode: Easier but uses more VRAM
- Python path mode: For users who followed the 6GB VRAM setup from the previous KimoDo video
Step 11: Configure KimoDo for Low VRAM
If you're running KimoDo on limited VRAM (6GB), use the Python path configuration.
- Navigate to your KimoDo virtual environment folder
- Copy the path to the VM folder
- In Blender, go to the KimoDo tab in the sidebar
- Paste the path into the KimoDo Python path field
- Click Start KimoDo
- Wait for "Ready, KimoDo" and the version number to appear
Pro Tip
If you're on an older KimoDo version, delete the old folders and re-run the setup files. The Blender add-on requires the latest version to work properly.
Step 12: Generate a Character with Rodin
For hero characters, use Rodin Gen 2.5 — it creates high-quality models in under 4 seconds.
- Go to the Rodin website (link in video description with discount code)
- Upload your reference image
- Choose the thinking effort — Extreme High for best results, High for 40-second generation
- Wait for the model to generate
- Use the Smart Low Poly mode for retopology
- Generate the mesh and create materials
- Download as GLB with your preferred texture size
Rodin Features
Rodin offers:
- 12K texturing with no blind spots
- Auto bang to parts for splitting models
- Blender add-on for direct generation in Blender
- Batch generation for multiple assets
Step 13: Rig the Character
Use ComfyUI Skin Token for fast character rigging.
- Open ComfyUI and load the Skin Token workflow
- Launch the workflow and wait for it to finish
- Find the rigged model in your output folder
- Import the rigged character into Blender
Pro Tip
You can also feed this workflow to Claude and let it handle the rigging automatically. Claude will remember the process for future use.
Step 14: Generate and Retarget Animations
Now combine everything to create animated characters.
- Open a new Claude window
- Prompt Claude: "Use the KimoDo add-on in my running Blender and generate five different animations. Duplicate the character five times and retarget the KimoDo skeletons to the rigged characters using the Rokoko add-on"
- Let Claude work — it will inspect the scene, check add-ons, and start generating
- Watch as KimoDo generates skeletons based on prompts like "a person walks forward"
- Claude will retarget the animations to your character using Rokoko
Rokoko Add-on Setup
If you don't have the Rokoko add-on installed, follow the same installation process as the KimoDo Blender Bridge. It's straightforward and covered in the previous video.
Animation Issues
Some animations may have arm clipping issues if your character isn't in a perfect T-pose. You can fix this by:
- Having Claude fix the hands with a simple prompt
- Manually selecting bones and adjusting keyframes
Related Guides
- ComfyUI Installation Guide - Complete installation process for ComfyUI and custom nodes
- Running ComfyUI on RunPod - Cloud GPU setup for faster generations
- Deploy Your First RunPod - Basic RunPod account and pod management
Next Steps
Now that you've built your local 3D AI pipeline:
- Experiment with different prompts to discover what Claude can do for your specific workflow
- Build your own pipeline by feeding Claude more information — it remembers processes for future use
- Use Claude for debugging — it's excellent at troubleshooting technical issues and installation problems
- Explore Rodin for faster character generation with their Blender add-on integration
Pipeline Complete!
You now have a fully functional local 3D AI pipeline that connects Claude, ComfyUI, and Blender. This setup lets you generate images, create 3D models, clean up geometry, and animate characters — all through natural language commands.