Skip to content

Improve conditioning to preserve original objects, backgrounds, materials, and identity #6324

Description

@duhsnest-ai

MediaPipe Solution (you are using)

MediaPipe Image Generator / Image Generation API

Programming language

Android Java (with MediaPipe Tasks / TensorFlow Lite backend)

Are you willing to contribute it

Yes

Describe the feature and the current behaviour/state

Current image generation conditioning can guide the generated result, but it does not always preserve important details from the reference image. The output may change original objects, background details, material appearance, textures, colors, and the identity of people or subjects. It would be valuable to have stronger image-to-image style conditioning that preserves the original scene while still allowing controlled generation changes. Similar capabilities in other diffusion systems use reference-image conditioning, appearance adapters, or feature-map injection to maintain details such as object identity, background structure, and material properties. A stronger conditioning mechanism or optional plugin/API support for high-fidelity reference image preservation would make MediaPipe Image Generation more useful for editing workflows.

Will this change the current API? How?

Ideally this could be added as an optional conditioning mode or additional plugin/adapter input rather than changing the existing API. For example, users could provide a reference image or appearance feature input when they need higher fidelity preservation. Existing generation workflows could remain unchanged.

Who will benefit with this feature?

Developers building mobile AI image editing applications, creative tools, AR experiences, and offline/on-device generative AI applications would benefit. Users would benefit from more realistic image editing where the generated result maintains the original subject and environment.

Please specify the use cases for this feature

Please specify the use cases for this feature Image-to-image editing where the original scene should remain recognizable Removing or changing parts of an image while keeping the rest unchanged Preserving people, pets, products, and objects during generation Maintaining materials, textures, lighting, and backgrounds Mobile photo editing applications requiring reliable reference-image fidelity Creative effects where users expect controlled transformations rather than a completely new image

Any Other info

Must be 16kb page size and implementation("com.google.mediapipe:tasks-vision-image-generator:x.xx.xx") { exclude(group = "com.google.protobuf", module = "protobuf-javalite") } implementation("com.google.protobuf:protobuf-java:4.32.0")

Metadata

Metadata

Labels

platform:androidIssues with Android as Platformtask:image generatorIssues related to Image generator: lightweight text-to-image generation tasktype:featureEnhancement in the New Functionality or Request for a New Solution

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions