> For the complete documentation index, see [llms.txt](https://metaverse-imagen.gitbook.io/ai-tools-research/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://metaverse-imagen.gitbook.io/ai-tools-research/ai-technology/generative-ai-architectures-and-models/controlnet.md).

# ControlNet

## ControlNet

ControlNet is a neural network structure that allows fine-grained control of diffusion models by adding extra conditions. It was introduced by Lvmin Zhang and Maneesh Agrawala in the paper "Adding Conditional Control to Text-to-Image Diffusion Models."

ControlNet works by making a copy of each block of the diffusion model into two variants: a trainable variant and a locked variant. The trainable variant is used to learn the control conditions, while the locked variant is used to preserve the original diffusion model.

The control conditions can be anything that can be represented as a neural network input, such as a depth map, a segmentation map, or a scribble. When the ControlNet is trained, it learns to use the control conditions to generate images that match the desired conditions.

ControlNet has been shown to be effective in controlling the generation of images in a variety of ways. For example, it can be used to generate images with specific objects or scenes, or to generate images with a particular style.

Here are some of the benefits of using ControlNet:

* It allows for fine-grained control of the image generation process.
* It can be used to control the generation of images in a variety of ways.
* It is relatively easy to train.

Here are some of the limitations of using ControlNet:

* It can be computationally expensive to train.
* It can be difficult to find the right control conditions for a particular image.

Overall, ControlNet is a powerful tool for controlling the generation of images. It is still under development, but it has the potential to revolutionize the way we create images.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://metaverse-imagen.gitbook.io/ai-tools-research/ai-technology/generative-ai-architectures-and-models/controlnet.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.