Most successful MiniMax H3 reference-to-video prompt structure so far (3 img + 1 vid example)

The official open-weights MiniMax H3 (Ref2VA) model was trained on a rigid six-section schema (subject_definitions, summary, retention_analysis, detailed_description, overall_soundscape, non_diegetic_music) paired with exact angle-bracketed asset tags (<Picture 1>, <Picture 2>, <Picture 3>, <Video 1>, <Audio 1>) and internal entity aliases (<Subject 1>, <Subject 2>). When you feed the local ComfyUI node casual prose or mismatched tags, the Qwen3-VL text encoder fails to bind your reference slots to the diffusion layers, causing the swap to break or blend identities. For a three-image, one-video character replacement, the model expects <Picture 1>, <Picture 2>, and <Picture 3> to be unified under a single reusable character alias (<Subject 1>), while the person being replaced in <Video 1> is isolated as <Subject 2>. In the retention_analysis section, you must use the exact pre-trained trigger keywords fully_preserved, attribute_transfer, and partially_preserved so the transformer strips the original actor's appearance while locking onto their choreography and the scene geometry. Copy and paste the exact prompt structure below into your text node, replacing only the bracketed physical descriptions with concrete visual traits of your character, the original performer, and the action in the clip: subject_definitions: <Subject 1> is the [age/gender, e.g., 30-year-old woman] shown across <Picture 1>, <Picture 2>, and <Picture 3>, featuring [facial structure, skin tone, eye color, and hairstyle from Picture 1], wearing [exact outfit, colors, and footwear shown in Picture 2 and Picture 3]. Ignore the background, lighting, and static poses in <Picture 1>, <Picture 2>, and <Picture 3>. <Subject 2> is the original [man/woman/performer] in <Video 1> wearing [briefly describe original actor's hair and clothing so the model isolates who to remove]. <Video 1> is the source video for the target video edit, providing the exact environment, lighting, camera trajectory, framing, and action choreography. <Audio 1> is the synchronized audio track of <Video 1> and is reused in the target video. summary: [video editing + reference generation + audio reuse] The target video is an edited version of <Video 1>. <Subject 2> is completely replaced by <Subject 1>, who takes over their exact screen position, body mechanics, and facial performance while the background environment and camera work of <Video 1> remain unchanged. retention_analysis: <Subject 1> (appears in [Shot 1]): fully_preserved - facial identity, hairstyle, body proportions, and complete wardrobe from <Picture 1>, <Picture 2>, and <Picture 3> are retained throughout. <Subject 2> (appears in [Shot 1]): attribute_transfer - only their exact pose, movement, facial expressions, head rotations, and screen positioning are transferred to <Subject 1>; do not retain <Subject 2>'s face, hair, or clothing. <Video 1> (source of video edit): partially_preserved - the background environment, scene geometry, lighting, shadows, camera movement, and timing are retained unchanged, with only <Subject 2> replaced by <Subject 1>. <Audio 1>: fully_copy - the original audio signal is copied directly. detailed_description: [Shot 1] Live-action, cinematic. The target video keeps <Video 1>'s original camera angle, lens behavior, lighting, and background environment completely intact. <Subject 1> stands in the exact position of <Subject 2> and performs the complete observable movement from <Video 1>, [briefly describe the primary physical action, e.g., walking forward, turning toward the lens, and gesturing with their right hand], matching every gesture, head turn, and expression without identity drift or background warping. overall_soundscape: [Describe the ambient room tone or physical sounds present in the clip in one sentence, e.g., Soft indoor room tone with faint footsteps on wood, or write N/A if silent.] non_diegetic_music: N/A For the cleanest result when connecting your three images to MiniMaxH3ReferenceToVideo, wire your clearest close-up face/headshot into ref_image_1 (<Picture 1>), your full-body front view showing the outfit into ref_image_2 (<Picture 2>), and your side or three-quarter angle into ref_image_3 (<Picture 3>). Keep the sentence "The target video is an edited version of <Video 1>." verbatim right after the bracketed task prefix in the summary section—that exact phrase is a hardcoded trigger from MiniMax's training data that switches the model from free generation into frame-by-frame video editing mode. One last quirk of the local H3 weights: never cite <Picture 1>, <Picture 2>, or <Picture 3> inside the detailed_description section. If a tag appears near the end of the prompt, the model misinterprets it as a last-frame convergence command and will abruptly morph your video into the static reference photo during the final second. Always define the images once inside subject_definitions and retention_analysis, and use only <Subject 1> inside detailed_description. submitted by /u/photobombolo [link] [comments]

Source: r/StableDiffusion | 2026-10-18

Related