ComfyUI Layer Style Advance
中文说明点这里
The nodes detached from ComfyUI Layer Style are mainly those with complex requirements for dependency packages.
Example workflow
Some JSON workflow files in the workflow directory, That’s examples of how these nodes can be used in ComfyUI.
How to install
(Taking ComfyUI official portable package and Aki ComfyUI package as examples, please modify the dependency environment directory for other ComfyUI environments)
Install plugin
-
Recommended use ComfyUI Manager for installation.
-
Or open the cmd window in the plugin directory of ComfyUI, like ComfyUI\custom_nodes，type
git clone https://github.com/chflame163/ComfyUI_LayerStyle_Advance.git
-
Or download the zip file and extracted, copy the resulting folder to ComfyUI\custom_nodes
Install dependency packages
-
for ComfyUI official portable package, double-click the install_requirements.bat in the plugin directory, for Aki ComfyUI package double-click on the install_requirements_aki.bat in the plugin directory, and wait for the installation to complete.
-
Or install dependency packages, open the cmd window in the ComfyUI_LayerStyle plugin directory like
ComfyUI\custom_nodes\ComfyUI_LayerStyle_Advance and enter the following command,
for ComfyUI official portable package, type:
..\..\..\python_embeded\python.exe -s -m pip install .\whl\docopt-0.6.2-py2.py3-none-any.whl
..\..\..\python_embeded\python.exe -s -m pip install .\whl\hydra_core-1.3.2-py3-none-any.whl
..\..\..\python_embeded\python.exe -s -m pip install -r requirements.txt
.\repair_dependency.bat
for Aki ComfyUI package, type:
..\..\python\python.exe -s -m pip install .\whl\docopt-0.6.2-py2.py3-none-any.whl
..\..\python\python.exe -s -m pip install .\whl\hydra_core-1.3.2-py3-none-any.whl
..\..\python\python.exe -s -m pip install -r requirements.txt
.\repair_dependency.bat
Download Model Files
Chinese domestic users from BaiduNetdisk or QuarkNetdisk , other users from huggingface.co/chflame163/ComfyUI_LayerStyle
download all files and copy them to ComfyUI\models folder. This link provides all the model files required for this plugin.
Or download the model file according to the instructions of each node.
Some nodes named “Ultra” will use the vitmatte model, download the vitmatte model and copy to ComfyUI/models/vitmatte folder, it is also included in the download link above.
Common Issues
If the node cannot load properly or there are errors during use, please check the error message in the ComfyUI terminal window. The following are common errors and their solutions.
Warning: xxxx.ini not found, use default xxxx…
This warning message indicates that the ini file cannot be found and does not affect usage. If you do not want to see these warnings, please modify all *.ini.example files in the plugin directory to *.ini.
This error is that the psd_tools were not installed correctly.
Solution:
- Close ComfyUI and open the terminal window in the plugin directory and execute the following command:
../../../python_embeded/python.exe -s -m pip install psd_tools
If error occurs during the installation of psd_tool, such as ModuleNotFoundError: No module named 'docopt' , please download docopt’s whl and manual install it.
execute the following command in terminal window:
../../../python_embeded/python.exe -s -m pip install path/docopt-0.6.2-py2.py3-none-any.whl the path is path name of whl file.
Cannot import name ‘guidedFilter’ from ‘cv2.ximgproc’
This error is caused by incorrect version of the opencv-contrib-python package，or this package is overwriteen by other opencv packages.
NameError: name ‘guidedFilter’ is not defined
The reason for the problem is the same as above.
This error is caused by the low version of transformers package.
insightface Loading very slow
This error is caused by the low version of protobuf package.
For the issues with the above three dependency packages, please double click repair_dependency.bat (for Official ComfyUI Protable) or repair_dependency_aki.bat (for ComfyUI-aki-v1.x) in the plugin folder to automatically fix them.
onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasn’t able to be loaded. Please install the correct version of CUDA and cuDNN as mentioned in the GPU requirements page
Solution:
Reinstall the onnxruntime dependency package.
Error loading model xxx: We couldn’t connect to huggingface.co …
Check the network environment. If you cannot access huggingface.co normally in China, try modifying the huggingface_hub package to force the use hf_mirror.
-
Find constants.py in the directory of huggingface_hub package (usually Lib/site packages/huggingface_hub in the virtual environment path),
Add a line after import os
os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
ValueError: Trimap did not contain foreground values (xxxx…)
This error is caused by the mask area being too large or too small when using the PyMatting method to handle the mask edges.
Solution:
- Please adjust the parameters to change the effective area of the mask. Or use other methods to handle the edges.
Requests.exceptions.ProxyError: HTTPSConnectionPool(xxxx…)
When this error has occurred, please check the network environment.
UnboundLocalError: local variable ‘clip_processor’ referenced before assignment
UnboundLocalError: local variable ‘text_model’ referenced before assignment
If this error occurs when executing JoyCaption2 node and it has been confirmed that the model file has been placed in the correct directory,
please check the transformers dependency package version is at least 4.43.2 or higher.
If transformers version is higher than or equal to 4.45.0, and also have error message:
Error loading models: De️️scriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
......
Please try downgrading the protobuf dependency package to 3.20.3, or set environment variables: PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python.
Update
**If the dependency package error after updating, please double clicking repair_dependency.bat (for Official ComfyUI Protable) or repair_dependency_aki.bat (for ComfyUI-aki-v1.x) in the plugin folder to reinstall the dependency packages.
- Fix the issue where Florence2 run with higher versions of Transformers, this solution comes from kijai, Thanks to @flybirdxx for feedback.
After updating plugin, find modeling_florence2.py and configuration_florence2.py from the florence2_models folder, copy and overwrite them to the model folder in ComfyUI/models/florence2.
- Commit JimengImageToImageAPI node, edit images using the Instant Dreaming Image 3.0 API. Create an account on Volcano Engine and apply for API AccessKeyID and SecretAccessKey. Fill them into the
api_key.ini directory in the plugin directory.
- Commit SAM2UltraV2 and LoadSAM2Model nodes, Change the SAM model to an external input to save resources when using multiple nodes.
- Commit JoyCaptionBetaOne, LoadJoyCaptionBeta1Model, JoyCaptionBeta1ExtraOptions nodes, Generate prompt words using the JoyCaption Beta One model.
- Commit SaveImagePLusV2 node, add custom file names and setting up the dpi of image.
- Commit GeminiImageEdit node, support using gemini-2.0-flash-exp-image-generation API for image editing.
- Commit GeminiV2 and ObjectDetectorGeminiV2 nodes, used google-genai dependency package that supports the gemini-2.0-flash-exp and gemini-2.5-pro-exp-03-25 models.
- Add QuarkNetdisk model download link.
- Support numpy 2.x dependency package.
- Commit DeepseekAPI_V2 noee, supporting AliYun and VolcEngine API.
- Commit Collage node to collage images into one.
- Commit DeepSeekAPI node, Use DeepSeek API for text inference.
- Commit SegmentAnythingUltraV3 and LoadSegmentAnythingModels nodes, Avoid duplicating model loading when using multiple SAM nodes.
- Commit ZhipuGLM4 and ZhipuGLM4V nodes, Use the Zhipu API for textual and visual inference. Among the current Zhipu models, GLM-4-Flash and glm-4v-flash models are free.
Apply for an API key for free at https://bigmodel.cn/usercenter/proj-mgmt/apikeys, fill your API key in
zhipu_api_key=.
- Commit Gemini node, Use Gemini API for text or visual inference.
- Commit ObjectDetectorGemini node, Use Gemini API for object detection.
- Commit DrawBBOXMaskV2 node, can draw rounded rectangle masks.
- Commit SmolLM2, SmolVLM, LoadSmolLM2Model and LoadSmolVLMModel nodes, use SMOL model for text inference and image recognition.
download the model file from BaiduNetdisk or huggingface and copy to
ComfyUI/models/smol folder.
- Florence2 add support gokaygokay/Florence-2-Flux-Large and gokaygokay/Florence-2-Flux models,
download Florence-2-Flux-Large and Florence-2-Flux folder from BaiduNetdisk or huggingface and copy to ```ComfyUI\models\florence2`` folder.
- Discard the dependencies required for the ObjectDetector YOLOWorld node from the requirements. txt file. To use this node, please manually install the dependency package.
- Strip some nodes from ComfyUI Layer Style to this repository.
Description
Randomly collage the input images into one large image.
Node Options:
- Add multiple languages and increase support for 5 languages: Chinese, French, Japanese, Korean and Russian. This feature producted by ComfyUI-Globalization-Node-Translation, thank you to the original author.
- images: The input images.
- florence2_model: Optional input for object recognition and cropping.
- canvas_width: Output the width of the image.
- canvas_height: Output the height of the image.
- border_width: The border width.
- rounded_rect_radius: The border fillet radius.
- uniformity: The randomness of image stitching size. The value range is 0-1, and the larger the value, the greater the randomness of the size.
- background_color: The background color.
- seed: The seed of random number.
- control_after_generate: Seed change options. If this option is fixed, the generated random number will always be the same.
- object_prompt: When connecting to florence2_model, fill in the prompt words for object recognition here.
Inference the prompts based on the image. this node is repackage of the ComfyUI_VLM_nodes’s UForm-Gen2 Qwen Node, thanks to the original author.
Download model files from huggingface or Baidu Netdisk to ComfyUI/models/LLavacheckpoints/files_for_uform_gen2_qwen folder.
Node Options:
- question: Prompt of UForm-Gen-QWen model.
Use the Llama 3.2 vision model for local inference. Can be used to generate prompt words. part of the code for this node comes from ComfyUI-PixtralLlamaMolmoVision, thank you to the original author.
To use this node, the transformers need upgraded to 4.45.0 or higher.
Download models from BaiduNetdisk or huggingface/SeanScripts , and copy to ComfyUI/models/LLM.
Node Options:
- image: Image input.
- model: Currently, only the “Llama-3.2-11B-Vision-Instruct-nf4” is available.
- system_prompt: System prompt words for LLM model.
- user_prompt: User prompt words for LLM model.
- max_new_tokens: max_new_tokens for LLM model.
- do_sample: do_sample for LLM model.
- top-p: top_p for LLM model.
- top_k: top_k for LLM model.
- stop_strings: The stop strings.
- seed: The seed of random number.
- control_after_generate: Seed change options. If this option is fixed, the generated random number will always be the same.
- include_prompt_in_output: Does the output contain prompt words.
- cache_model: Whether to cache the model.
Use the JoyCaption-alpha-two model for local inference. Can be used to generate prompt words. this node is https://huggingface.co/John6666/joy-caption-alpha-two-cli-mod Implementation in ComfyUI, thank you to the original author.
Download models form BaiduNetdisk and BaiduNetdisk ,
or huggingface/Orenguteng and huggingface/unsloth , then copy to ComfyUI/models/LLM,
Download models from BaiduNetdisk or huggingface/google , and copy to ComfyUI/models/clip,
Donwload the cgrkzexw-599808 folder from BaiduNetdisk or huggingface/John6666 , and copy to ComfyUI/models/Joy_caption。
Node Options:
- image: Image input.
- extra_options: Input the extra_options.
- llm_model: There are two LLM models to choose, Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 and unsloth/Meta-Llama-3.1-8B-Instruct.
- device: Model loading device. Currently, only CUDA is supported.
- dtype: Model precision, nf4 and bf16.
- vlm_lora: Whether to load text_madel.
- caption_type: Caption type options, including: “Descriptive”, “Descriptive (Informal)”, “Training Prompt”, “MidJourney”, “Booru tag list”, “Booru-like tag list”, “Art Critic”, “Product Listing”, “Social Media Post”.
- caption_length: The length of caption.
- user_prompt: User prompt words for LLM model. If there is content here, it will overwrite all the settings for caption_type and extra_options.
- max_new_tokens: The max_new_token parameter of LLM.
- do_sample: The do_sample parameter of LLM.
- top-p: The top_p parameter of LLM.
- temperature: The temperature parameter of LLM.
- cache_model: Whether to cache the model.
The node of JoyCaption2 separate model loading and inference, and when multiple JoyCaption2 nodes are used, the model can be shared to improve efficiency.
Node Options:
- image: Image input.
- joy2_model: The JoyCaption model input.
- extra_options: Input the extra_options.
- caption_type: Caption type options, including: “Descriptive”, “Descriptive (Informal)”, “Training Prompt”, “MidJourney”, “Booru tag list”, “Booru-like tag list”, “Art Critic”, “Product Listing”, “Social Media Post”.
- caption_length: The length of caption.
- user_prompt: User prompt words for model. If there is content here, it will overwrite all the settings for caption_type and extra_options.
- max_new_tokens: The max_new_token parameter of model.
- do_sample: The do_sample parameter of model.
- top-p: The top_p parameter of model.
- temperature: The temperature parameter of model.
JoyCaption2’s model loading node, used in conjunction with JoyCaption2Split.
Node Options:
- llm_model: There are two LLM models to choose, Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 and unsloth/Meta-Llama-3.1-8B-Instruct.
- device: Model loading device. Currently, only CUDA is supported.
- dtype: Model precision, nf4 and bf16.
- vlm_lora: Whether to load text_madel.
The extra_options parameter node of JoyCaption2.
Node Options:
- refer_character_name: If there is a person/character in the image you must refer to them as {name}.
- exclude_people_info: Do NOT include information about people/characters that cannot be changed (like ethnicity, gender, etc), but do still include changeable attributes (like hair style).
- include_lighting: Include information about lighting.
- include_camera_angle: Include information about camera angle.
- include_watermark: Include information about whether there is a watermark or not.
- include_JPEG_artifacts: Include information about whether there are JPEG artifacts or not.
- include_exif: If it is a photo you MUST include information about what camera was likely used and details such as aperture, shutter speed, ISO, etc.
- exclude_sexual: Do NOT include anything sexual; keep it PG.
- exclude_image_resolution: Do NOT mention the image’s resolution.
- include_aesthetic_quality: You MUST include information about the subjective aesthetic quality of the image from low to very high.
- include_composition_style: Include information on the image’s composition style, such as leading lines, rule of thirds, or symmetry.
- exclude_text: Do NOT mention any text that is in the image.
- specify_depth_field: Specify the depth of field and whether the background is in focus or blurred.
- specify_lighting_sources: If applicable, mention the likely use of artificial or natural lighting sources.
- do_not_use_ambiguous_language: Do NOT use any ambiguous language.
- include_nsfw: Include whether the image is sfw, suggestive, or nsfw.
- only_describe_most_important_elements: ONLY describe the most important elements of the image.
- character_name: Person/Character Name, if choice
refer_character_name.
Generate prompt words using the JoyCaption Beta One model. This node is https://huggingface.co/fancyfeast/llama-joycaption-beta-one-hf-llava Implementation in ComfyUI.
The first time using the node, the model will be automatically downloaded to the ComfyUI/models/LLavacheckpoints/llama-joycaption-beta-one-hf-llava folder.
You can also download llama-joycaption-beta-one-hf-llava folder from BaiduNetdisk or Quark or huggingface/fancyfeast/llama-joycaption-beta-one-hf-llava and copy to ComfyUI/models/LLavacheckpoints
Node Options:
- image: Image input.
- joycaption_beta1_model: JoyCaption Beta One model input. The model is loaded from the
Load JoyCaption Beta One Model node.
- extra_options: Input the extra_options.
- caption_type: Caption type options, including: “Descriptive”, “Descriptive (Casual)”, “Straightforward”, “Stable Diffusion Prompt”, “MidJourney”, “Danbooru tag list”, “e621 tag list”, “Rule34 tag list”, “Booru-like tag list”, “Art Critic”, “Product Listing” and “Social Media Post”.
- caption_length: The length of caption.
- max_new_tokens: The max_new_token parameter of model.
- top-p: The top-p parameter of model.
- top-k: The top-k parameter of model.
- temperature: The temperature parameter of model.
- user_prompt: User prompt words for model. If there is content here, it will overwrite all the settings for caption_type and extra_options.
The model loading node of JoyCaption Beta One, used in conjunction with JoyCaption Beta One.
Node Options:
- model: Only the
fancyfeast/llama-joycaption-beta-one-hf-llava model is available for selection currently.
- quantization_mode: The model quantization mode has three options: nf4, int8, and bf16.
- device: The model loading device.
The extra_options parameter node of JoyCaption Beta One.
Node Options:
- refer_character_name: If there is a person/character in the image you must refer to them as {name}.
- exclude_people_info: Do NOT include information about people/characters that cannot be changed (like ethnicity, gender, etc), but do still include changeable attributes (like hair style).
- include_lighting: Include information about lighting.
- include_camera_angle: Include information about camera angle.
- include_watermark: Include information about whether there is a watermark or not.
- include_JPEG_artifacts: Include information about whether there are JPEG artifacts or not.
- include_exif: If it is a photo you MUST include information about what camera was likely used and details such as aperture, shutter speed, ISO, etc.
- exclude_sexual: Do NOT include anything sexual; keep it PG.
- exclude_image_resolution: Do NOT mention the image’s resolution.
- include_aesthetic_quality: You MUST include information about the subjective aesthetic quality of the image from low to very high.
- include_composition_style: Include information on the image’s composition style, such as leading lines, rule of thirds, or symmetry.
- exclude_text: Do NOT mention any text that is in the image.
- specify_depth_field: Specify the depth of field and whether the background is in focus or blurred.
- specify_lighting_sources: If applicable, mention the likely use of artificial or natural lighting sources.
- do_not_use_ambiguous_language: Do NOT use any ambiguous language.
- include_nsfw: Include whether the image is sfw, suggestive, or nsfw.
- only_describe_most_important_elements: ONLY describe the most important elements of the image.
- do_not_include_artist_name_or_title: If it is a work of art, do not include the artist’s name or the title of the work.
- identify_image_orientation: Identify the image orientation (portrait, landscape, or square) and aspect ratio if obvious.
- use_vulgar_slang_and_profanity: Use vulgar slang and profanity.
- do_not_use_polite_euphemisms: Do NOT use polite euphemisms—lean into blunt, casual phrasing.
- include_character_age: Include information about the ages of any people/characters when applicable.
- include_camera_shot_type: Mention whether the image depicts an extreme close-up, close-up, medium close-up, medium shot, cowboy shot, medium wide shot, wide shot, or extreme wide shot.
- exclude_mood_feeling: Do not mention the mood/feeling/etc of the image.
- include_camera_vantage_height: Explicitly specify the vantage height (eye-level, low-angle worm’s-eye, bird’s-eye, drone, rooftop, etc.).
- mention_watermark: If there is a watermark, you must mention it.
- avoid_meta_descriptive_phrases: Your response will be used by a text-to-image model, so avoid useless meta phrases like “This image shows…”, “You are looking at…”, etc.
- character_name: Person/Character Name, if choice
refer_character_name.
Use Microsoft Phi 3.5 text and visual models for local inference. Can be used to generate prompt words, process prompt words, or infer prompt words from images. Running this model requires at least 16GB of video memory.
Download model files from BaiduNetdisk or huggingface.co/microsoft/Phi-3.5-vision-instruct and huggingface.co/microsoft/Phi-3.5-mini-instruct and copy to ComfyUI\models\LLM folder.
Node Options:
- image: Optional input. The input image will serve as the input for Phi-3.5-vision-instruct.
- model: Selectable to load Phi-3.5-vision-instruct or Phi-3.5-mini-instruct model. The default value of auto will automatically load the corresponding model based on whether there is image input.
- device: Model loading device. Supports CPU and CUDA.
- dtype: The model loading accuracy has three options: fp16, bf16, and fp32.
- cache_model: Whether to cache the model.
- system_prompt: The system prompt of Phi-3.5-mini-instruct.
- user_prompt: User prompt words for LLM model.
- do_sample: The do_Sample parameter of LLM defaults to True.
- temperature: The temperature parameter of LLM defaults to 0.5.
- max_new_tokens: The max_new_token parameter of LLM defaults to 512.
Use Google Gemini API for text and visual models for local inference. Can be used to generate prompt words, process prompt words, or infer prompt words from images.
Apply for your API key on Google AI Studio, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after google_api_key= and save it.
Node options:
- image_1: Optional input. If there is an image input here, please explain the purpose of ‘image_1’ in user_dempt.
- image_2: Optional input. If there is an image input here, please explain the purpose of ‘image_2’ in user_dempt.
- model: Choose the Gemini model.
- max_output_tokens: The max_output_token parameter of Gemini defaults to 4096.
- temperature: The temperature parameter of Gemini defaults to 0.5.
- words_limit: The default word limit for replies is 200.
- response_language: The language of the reply.
- system_prompt: The system prompt.
- user_prompt: The user prompt.
On the basis of Gemini nodes, switch to using the new google-genai dependency package, which supports the latest gemini-2.0-flash, gemini-2.0-flash-lite, and gemini-2.5-pro-exp-03-25 models.
Add on the original node:
- seed: Seed value used when requesting Google API.
Implement multimodal image editing using the gemini-2.0-flash-exp-image-generation model.
Apply for your API key on Google AI Studio, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after google_api_key= and save it.
Node Options:
- image: The input image.
- image_2: Optional second image input.
- image_3: Optional third image input.
- model: Choose the Gemini model. Currently, only the gemini-2.0-flash-exp-image-generation model is supported.
- temperature: The temperature parameter of Gemini defaults to 0.5.
- seed: Seed value used when requesting Google API.
- control_after_generate: Set whether to change the seed every time.
- user_prompt: The user prompt.
Use the DeepSeek API for text inference, supporting multi node context concatenation.
Apply for an API key for free at https://platform.deepseek.com/api_keys, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after deepseek_api_key= and save it.
Node Options:
- history: History of DeepSeekAPI node, optional input. If there is input here, historical records will be used as context.
- model: Choose the DeepSeek model, currently there is only one option: “deepseek-chat”, which is the DeepSeek-V3 model.
- max_tokens: The max_token parameter of DeepSeek defaults to 4096.
- temperature: The temperature parameter of DeepSeek defaults to 1.
- top_p: The top_p parameter of DeepSeek defaults to 1.
- presence_penalty: The presence_penalty parameter of DeepSeek defaults to 0.
- frequency_penalty: The frequency_penalty parameter of DeepSeek defaults to 0.
- history_length: History record length. Records exceeding this length will be discarded.
- system_prompt: The system prompt.
- user_prompt: The user prompt.
Outputs:
- text: Output text of DeepSeek.
- history: History of DeepSeek conversations.
On the basis of the DeepSeekAPI node, DeepSeek API supporting AliYun and VolcEngine will be added, and these two Chinese cloud service providers will provide more stable API services.
-
On VolcEngine Applying for the VolcEngine API key, there is a free quota of 500 thousand tokens. If you fill in my invitation code 27RVS1QN when applying, you will receive an additional 3.75 million R1 model free tokens.
-
On AliYun Apply for AliYun API key.
-
Fill in the obtained API key into the fields volcengine_api_key and aliyun_api_key of api_key.ini. This file is located in the root directory of the plugin, with a default name of api_key.ini.example. Edit it and change the file extension to ‘.ini’.
Add Options:
- time_out: The timeout period is set to 300 seconds by default.
Use the Zhipu API for text inference, supporting multi node context concatenation.
Apply for an API key for free at https://bigmodel.cn/usercenter/proj-mgmt/apikeys, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after zhipu_api_key= and save it.
Node Options:
- history: History of GLM4 node, optional input. If there is input here, historical records will be used as context.
- model: Select GLM4 model. GLM-4-Flash is a free model.
- user_prompt: The user prompt.
- history_length: History record length. Records exceeding this length will be discarded.
Outputs:
- text: Output text of GLM4.
- history: History of GLM4 conversations.
Use the Zhipu API for visual inference.
Apply for an API key for free at https://bigmodel.cn/usercenter/proj-mgmt/apikeys, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after zhipu_api_key= and save it.
Node Options:
- image: The input image.
- model: Select the GLM4V model. glm-4v-flash is a free model.
- user_prompt: The user prompt.
Output:
- text: Output text of GLM4V.
Use the SmolLM2 model for local inference.
Download model files from BaiduNetdisk or huggingface,
find the SmolLM2-135M-Instruct, SmolLM2-360M-Instruct, SmolLM2-1.7B-Instruct folders, download at least one of them, copy to ComfyUI/models/smol folder.
Node Options:
- smolLM2_model: The input of SmolLM2 model is loaded from the LoadSmolLM2Model node.
- max_new_tokens: The maximum number of tokens is 512 by default.
- do_sample: The do_Sample parameter defaults to True.
- temperature: The temperature parameter defaults to 0.5.
- top-p: The top_p parameter defaults to 0.9.
- system_prompt: System prompt words.
- user_prompt: User prompt words.
Load SmolLM2 model.
Node Options:
- model: There are three options for selecting the SmolLM2 model: SmolLM2-135M-Instruct, SmolLM2-360M-Instruct and SmolLM2-1.7B-Instruct.
- dtype: The model accuracy has two options: bf16 and fp32.
- device: The model loading device has two options: cuda or cpu.
Using SmolVLM lightweight visual models for local inference.
Donwload the SmolVLM-Instruct folder from BaiduNetdisk or huggingface and copy to ComfyUI/models/smol folder.
Node Options:
- image: Image input, supports batch images.
- smolVLM_model: The input of the SmolVLM model is loaded from the LoadSmolVLMModel node.
- max_new_tokens: The maximum number of tokens is 512 by default.
- user_prompt: User prompt words.
Load SmolVLM model.
Node Options:
- model: The SmolVLM model selection currently only has the option of SmolVLM-Instruct.
- dtype: The model accuracy has two options: bf16 and fp32.
- device: The model loading device has two options: cuda or cpu.
Edit images using the Jimeng API.
Please create an account on Volcano Engine, apply for API AccessKeyID and SecretAccessKey, and fill them in api_key. ini. This file is located in the root directory of the plugin and its default name is api_key.ini.example. When using this file for the first time, you need to change the file extension to ‘.ini’. Open with text editing software and fill in the corresponding values after volcengine_SecretAccessKey= and volcengine_SecretAccessKey=.
Node Options:
- image: The input image.
- model: Choose the DreamMap and Life Map model. Currently, only the Jimeng_i2i-v30 model is supported.
- time_out: The maximum time limit for waiting for API return, in seconds. If this time is exceeded, the node will end running.
- scale: The scale parameter of jimeng_i2iuv30 is set to 0.5 by default.
- seed: The seed value.
- prompt: The prompt.
UserPrompt preset for generating SD text to image prompt words.
Node options:
- template: Prompt word template. Currently, only the ‘SD txt2img prompt’ is available.
- describe: Prompt word description. Enter a simple description here.
- limit_word: Maximum length limit for output prompt words. For example, 200 means that the output text will be limited to 200 words.
UserCompt preset for generating SD text to image prompt words based on input content.
Node options:
- reference_text: Reference text input. Usually it is a style description of the image.
- template: Prompt word template. Currently, only the ‘SD txt2img prompt’ is available.
- describe: Prompt word description. Enter a simple description here.
- limit_word: Maximum length limit for output prompt words. For example, 200 means that the output text will be limited to 200 words.
UserPrompt preset used to replace a keyword in text with different content. This is not only a simple replacement, but also a logical sorting of the text based on the context of the prompt words to achieve the rationality of the output content.
Node options:
- orig_prompt: Original prompt word input.
- template: Prompt word template. Currently, only ‘prompt replace word’ is available.
- exclude_word: Keywords that need to be excluded.
- replace_with_word: That word will replace the exclude_word.
Inference the prompts based on the image. it can replace key word for the prompt. This node currently uses Google Gemini API as the backend service. Please ensure that the network environment can use Gemini normally.
Apply for your API key on Google AI Studio, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after google_api_key= and save it.
Node options:
- api: The Api used. At present, there are two options “gemini-1. 5-flash” and “google-gemini”.
- token_limit: The maximum token limit for generating prompt words.
- exclude_word: Keywords that need to be excluded.
- replace_with_word: That word will replace the exclude_word.
Enter simple prompt words, output polished prompt words, and support inputting images as references, and support Chinese input. This node currently uses Google Gemini API as the backend service. Please ensure that the network environment can use Gemini normally.
Apply for your API key on Google AI Studio, And fill it in api_key.ini, this file is located in the root directory of the plug-in, and the default name is api_key.ini.example. to use this file for the first time, you need to change the file suffix to .ini. Open it using text editing software, fill in your API key after google_api_key= and save it.
Node options:
- image: Optional, input image as a reference for prompt words.
- api: The Api used. At present, there are two options “gemini-1. 5-flash” and “google-gemini”.
- token_limit: The maximum token limit for generating prompt words.
- discribe: Enter a simple description here. supports Chinese text input.
Use the Florence 2 model to infer prompt words. The code for this node section is fromyiwangsimple/florence_dw, thanks to the original author.
*When using it for the first time, the model will be automatically downloaded. You can also download the model file from BaiduNetdisk to ComfyUI/models/florence2 folder.
Node Options:
- florence2_model: Florence2 model input.
- image: Image input.
- task: Select the task for florence2.
- text_input: Text input for florence2.
- max_new_tokens: The maximum number of tokens for generating text.
- num_beams: The number of beam searches that generate text.
- do_sample: Whether to use text generated sampling.
- fill_mask: Whether to use text marker mask filling.
Obtain the main color or average color from the image and output RGB values.
Node options:
- mode： There are two modes to choose from, with the main color and average color.
Output type:
- RGB color in HEX: The RGB color described by hexadecimal RGB format, like ‘#FA3D86’.
- HSV color in list: The HSV color described by python’s list data format.
V2 upgrade of GetColorTone. You can specify the dominant or average color to get the body or background.
The following changes have been made on the basis of GetColorTong:
- color_of: Provides 4 options, mask, entire, background, and subject, to select the color of the mask area, entire picture, background, or subject, respectively.
- remove_background_method: There are two methods of background recognition: BiRefNet and RMBG V1.4.
- invert_mask: Whether to reverse the mask.
- mask_grow: Mask expansion. For subject, a larger value brings the obtained color closer to the color at the center of the body.
Output:
- image: Solid color picture output, the size is the same as the input picture.
- mask: Mask output.
Rating bulk pictures and outputting top-ranked pictures. it used [ImageReward] (https://github.com/THUDM/ImageReward) for image scoring, thanks to the original authors.
Node options:
- prompt: Optional input. Entering prompt here will be used as a basis to determine how well it matches the picture.
- output_nun: Number of pictures outputted. This value should be less than the picture batch.
Outputs：
- images: Bulk pictures output from high to low in order of rating.
- obsolete_images: Knockout pictures. Also output in order of rating from high to low.
Erase objects from the image based on the mask. this node is repackage of IOPaint, powered by state-of-the-art AI models, thanks to the original author.
It is have LaMa, LDM, ZITS,MAT, FcF, Manga models and the SPREAD method to erase. Please refer to the original link for the introduction of each model.
Please download the model files from lama models(BaiduNetdisk) or lama models(Google Drive) to ComfyUI/models/lama folder.
Node optons:
- lama_model: Choose a model or method.
- device: After correctly installing Torch and Nvidia CUDA drivers, using cuda will significantly improve running speed.
- invert_mask: Whether to reverse the mask.
- grow: Positive values expand outward, while negative values contract inward.
- blur: Blur the edge.
Automatically cutout and crop the image according to the mask. it can specify the background color, aspect ratio, and size for output image. this node is designed to generate the image materials for training models.
*Please refer to the model installation methods for SegmentAnythingUltra and RemBgUltra.
Node options:
- background_color4: The background color.
- aspect_ratio: Here are several common frame ratios provided. alternatively, you can choose “original” to keep original ratio or customize the ratio using “custom”.
- proportional_width: Proportional width. if the aspect ratio option is not “custom”, this setting will be ignored.
- proportional_height: Proportional height. if the aspect ratio option is not “custom”, this setting will be ignored.
- scale_by_longest_side: Allow scaling by long edge size.
- longest_side: When the scale_by_longest_side is set to True, this will be used this value to the long edge of the image. when the original_size have input, this setting will be ignored.
- detect: Detection method, min_bounding_rect is the minimum bounding rectangle, max_inscribed_rect is the maximum inscribed rectangle.
- border_reserve: Keep the border. expand the cutting range beyond the detected mask body area.
- ultra_detail_range: Mask edge ultra fine processing range, 0 is not processed, which can save generation time.
- matting_method: The method of generate masks. There are two methods available: Segment Anything and RMBG 1.4. RMBG 1.4 runs faster.
- sam_model: Select the SAM model used by Segment Anything here.
- grounding_dino_model: Select the Grounding_Dino model used by Segment Anything here.
- sam_threshold: The threshold for Segment Anything.
- sam_prompt: The prompt for Segment Anything.
Output:
cropped_image: Crop and replace the background image.
box_preview: Crop position preview.
cropped_mask: Cropped mask.
The V2 upgrad version of ImageAutoCrop, it has made the following changes based on the previous version:
- Add optional input for mask. when there is a mask input, use that input directly to skip the built-in mask generation.
- Add
fill_background. When set to False, the background will not be processed and any parts beyond the frame will not be included in the output range.
aspect_ratio adds the original option.- scale_by: Allow scaling by specified dimensions for longest, shortest, width, or height.
- scale_by_length: The value here is used as
scale_by to specify the length of the edge.
Automatically crop the image to the specified size. You can input a mask to preserve the specified area of the mask. This node is designed to generate image materials for training the model.
Node Options:
- image: The input image.
- mask: Optional input mask. The masking part will be preserved within the range of the cutting aspect ratio.
- aspect_ratio: The aspect ratio of the output. Here are common frame ratios provided, with “custom” being the custom ratio and “original” being the original frame ratio.
- proportional_width: Proportionally wide. If the aspect_ratio option is not ‘custom’, this setting will be ignored.
- proportional_height: High proportion. If the aspect_ratio option is not ‘custom’, this setting will be ignored.
- method: Scaling sampling methods include Lanczos, Bicubic, Hamming, Bilinear, Box, and Nearest.
- scale_to_side: Allow scaling to be specified by long side, short side, width, height, or total pixels.
- scale_to_length: The value here is used as the scale_to-side to specify the length of the edge or the total number of pixels (kilo pixels).
- round_to_multiple: Multiply to the nearest whole. For example, if set to 8, the width and height will be forcibly set to multiples of 8.
Outputs:
cropped_image: The cropped image.
box_preview: Preview of cutting position.
Enhanced save image node. You can customize the directory where the picture is saved, add a timestamp to the file name, select the save format, set the image compression rate, set whether to save the workflow, and optionally add invisible watermarks to the picture. (Add information in a way that is invisible to the naked eye, and use the ShowBlindWaterMark node to decode the watermark). Optionally output the json file of the workflow.
Node Options:
- image: The input image.
- custom_path*: User-defined directory, enter the directory name in the correct format. If empty, it is saved in the default output directory of ComfyUI.
- filename_prefix*: The prefix of file name.
- timestamp: Timestamp the file name, opting for date, time to seconds, and time to milliseconds.
- format: The format of image save. Currently available in
png and jpg. Note that only png format is supported for RGBA mode pictures.
- quality: Image quality, the value range 10-100, the higher the value, the better the picture quality, the volume of the file also correspondingly increases.
- meta_data: Whether to save metadata to png file, that is workflow information. Set this to false if you do not want the workflow to be leaked.
- blind_watermark: The text entered here (does not support multilingualism) will be converted into a QR code and saved as an invisible watermark. Use
ShowBlindWaterMark node can decode watermarks. Note that pictures with watermarks are recommended to be saved in png format, and lower-quality jpg format will cause watermark information to be lost.
- save_workflow_as_json: Whether the output workflow is a json file at the same time (the output json is in the same directory as the picture).
- preview: Preview switch.
* Enter%date for the current date (YY-mm-dd) and %time for the current time (HH-MM-SS). You can enter / for subdirectories. For example, %date/name_%tiem will output the image to the YY-mm-dd folder, with name_HH-MM-SS as the file name prefix.
Added custom file name and dpi options on the SaveImagePlus node.
NodeOptions:
- custom_filename*: User defined file name, if entered here, will be used as the file name. Please note that duplicate files will be overwritten. If this is empty, use the file name prefix and timestamp as the file name.
- dpi: Set the DPI value of the image file.
* Enter%date for the current date (YY-mm-dd) and %time for the current time (HH-MM-SS). You can enter / for subdirectories. For example, %date/name_%tiem will output the image to the YY-mm-dd folder, with name_HH-MM-SS as the file name prefix.
Add an invisible watermark to a picture. Add the watermark image in a way that is invisible to the naked eye, and use the ShowBlindWaterMark node to decode the watermark.
Node Options:
- iamge: The input image.
- watermark_image: Watermark image. The image entered here will automatically be converted to a square black and white image as a watermark. It is recommended to use a QR code as a watermark.
Decoding the invisible watermark added to the AddBlindWaterMark and SaveImagePlus nodes.
Generate a square QR code picture.
Node Options:
- size: The side length of image.
- border: The size of the border around the QR code, the larger the value, the wider the border.
- text: Enter the text content of the QR code here, and multi-language is not supported.
Decoding the QR code.
Node Options:
- image: The input QR code image.
- pre_blur: Pre-blurring, you can try to adjust this value for QR codes that are difficult to identify.
Load the PSD format file and export the layers.
Note that this node requires the installation of the psd_tools dependency package, If error occurs during the installation of psd_tool, such as ModuleNotFoundError: No module named 'docopt' , please download docopt’s whl and manual install it.
Node Options:
- image: Here is a list of *.psd files under
ComfyUI/input, where previously loaded psd images can be selected.
- file_path: The complete path and file name of the psd file.
- include_hidden_layer: whether include hidden layers.
- find_layer_by: The method for finding layers can be selected by layer key number or layer name. Layer groups are treated as one layer.
- layer_index: The layer key number, where 0 is the bottom layer, is incremented sequentially. If include_hiddenlayer is set to false, hidden layers are not counted. Set to -1 to output the top layer.
- layer_name: Layer name. Note that capitalization and punctuation must match exactly.
Outputs:
flat_image: PSD preview image.
layer_iamge: Find the layer output.
all_layers: Batch images containing all layers.
Encapsulate the four nodes of Negative Condition in SD3 into a separate node.
Node Options:
- zero_out_start: Set the ConditioningSetTimestepRange start value for Negative ConditioningZeroOut, which is the same as the ConditioningSetTimestepRange end value for Negative.
It is the implementation of PramaLLC/BEN project in ComfyUI. Thank you to the original author.
Download all files from huggingface or BaiduNetdisk and copy to ComfyUI/models/BEN folder.
Node Options：
- ben_model: Ben model input. There are two models to choose: BEN_Base and BEN2_base.
- image: Image input.
- detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards.
- detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair.
- detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair.
- black_point: Edge black sampling threshold.
- white_point: Edge white sampling threshold.
- process_detail: Set to false here will skip edge processing to save runtime.
- max_megapixels: Set the maximum size for VitMate operations.
Load the BEN model.
Node Options:
- model: Select the model. Currently, only the Ben_Sase model is available for selection.
Improvements to ComfyUI Segment Anything, thanks to the original author.
*Please refer to the installation of ComfyUI Segment Anything to install the model. If ComfyUI Segment Anything has been correctly installed, you can skip this step.
Node options:
- sam_model: Select the SAM model.
- ground_dino_model: Select the Grounding DINO model.
- threshold: The threshold of SAM.
- detail_range: Edge detail range.
- black_point: Edge black sampling threshold.
- white_point: Edge white sampling threshold.
- process_detail: Set to false here will skip edge processing to save runtime.
- prompt: Input for SAM’s prompt.
- cache_model: Set whether to cache the model.
The V2 upgraded version of SegmentAnythingUltra has added the VITMatte edge processing method.(Note: Images larger than 2K in size using this method will consume huge memory)
On the basis of SegmentAnythingUltra, the following changes have been made:
- detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards.
- detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair.
- detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair.
- device: Set whether the VitMatte to use cuda.
- max_megapixels: Set the maximum size for VitMate operations.
Separate model loading from inference nodes to avoid duplicate model loading when using multiple SAM nodes.
Node Options:
Same as SegmentAnythingUltra, removed sam_comodel and ground-dino_comodel, changed them to be obtained from node input.
Load SegmentAnything models.
This node is modified from kijai/ComfyUI-segment-anything-2. Thank to kijai for making significant contributions to the Comfyui community.
SAM2 Ultra node only support single image. If you need to process multiple images, please first convert the image batch to image list.
*Download models from BaiduNetdisk or huggingface.co/Kijai/sam2-safetensors and copy to ComfyUI/models/sam2 folder.
Node Options:
- image: The image to segment.
- bboxes: Input recognition box data.
- sam2_model: Select the SAM2 model.
- presicion: Model’s persicion. can be selected from fp16, bf16, and fp32.
- bbox_select: Select the input box data. There are three options: “all” to select all, “first” to select the box with the highest confidence, and “by_index” to specify the index of the box.
- select_index: This option is valid when bbox_delect is ‘by_index’. 0 is the first one. Multiple values can be entered, separated by any non numeric character, including but not limited to commas, periods, semicolons, spaces or letters, and even Chinese.
- cache_model: Whether to cache the model. After caching the model, it will save time for model loading.
- detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards.
- detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair.
- detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair.
- black_point: Edge black sampling threshold.
- white_point: Edge white sampling threshold.
- process_detail: Set to false here will skip edge processing to save runtime.
- device: Set whether the VitMatte to use cuda.
- max_megapixels: Set the maximum size for VitMate operations.
On the basis of SAM2 Ultra nodes, changing the SAM2 model to an external input saves resources when using multiple nodes.
Modified node options:
- sam2_model: SAM2 model input, the model is loaded by the
Load SAM2 Model node.
Load SAM2 model.
Node Options:
- sam2_model: Select the SAM2 model.
- presicion: Model’s persicion. can be selected from fp16, bf16, and fp32.
- device: Set whether to use cuda.
SAM2 Video Ultra node support processing multiple frames of images or video sequences. Please define the recognition box data in the first frame of the sequence to ensure correct recognition.
https://github.com/user-attachments/assets/4726b8bf-9b98-4630-8f54-cb7ed7a3d2c5
https://github.com/user-attachments/assets/b2a45c96-4be1-4470-8ceb-addaf301b0cb
Node Options:
- image: The image to segment.
- bboxes: Optional input of recognition bbox data.
bboxes and first_frame_mask must have least one input. If first_frame_mask inputed, bbboxes will be ignored.
- first_frame_mask: Optional input of the first frame mask. The mask will be used as the first frame recognition object.
bboxes and first_frame_mask must have least one input. If first_frame_mask inputed, bbboxes will be ignored.
- pre_mask: Optional input mask, which will serve as a propagation focus range limitation and help improve rec
README 内容过长，已截取部分内容。完整内容请前往 GitHub 仓库查看。
评论