[Community] DreamArtist : One-Shot Text-to-Image Generation For Diffusers

[adhikjoshi]

Is your feature request related to a problem? Please describe.
There is new, One-Shot Text-to-Image for Texual Inversion

Twitter Link

There is implementation for Automatic1111, can we have same for diffusers?

Github Code

Describe the solution you'd like

DreamArtist for Dreambooth Diffusers

[aandyw]

Very interested in this issue. I would like to take this on with some guidance if possible :) .

[isamu-isozaki]

@pie31415 Nice! I might implement it in my fork so you can check that if you want. The main changes for this work seems to be

1. Using negative prompts too to learn what the image is not at the same time as the regular tokens
2. Using a discriminator instead of the noise loss

[aandyw]

@isamu-isozaki Hey, sounds good. Thanks for the first 2 points. Also, do you mind if we work on the issue together? I've just started learning about diffusers so my knowledge is a little limited. Would you possibly be able to give me some guidance or resources to get started?

[aandyw]

Also, this maybe a dumb question, but I'm assuming we're implementing a new pipeline with the 2 previously mentioned changes instead of a new model?

[isamu-isozaki]

@pie31415 For sure! I guess for resources maybe the textual inversion paper but I was thinking that we can copy the code from the textual inversion, and just make slight changes to do this job.
We might do our custom pipeline but pipelines are mainly for inference and not training. So I'm thinking first we make the training script-> pipeline.

[isamu-isozaki]

I can try setting up some code by the end of the day

[aandyw]

@isamu-isozaki Thanks a lot for the help. Would you like to move this conversation to discord? I think it'll be more convenient for communication.

[isamu-isozaki]

@pie31415 yup sounds good. My name's Chad Kensington on discord. I'm in the Huggingface discord server.