Reprinted from | Machine Heart
Recently, the text-to-image model Stable Diffusion has become extremely popular in the AI community. It was implemented by researchers from the University of Munich and Runway based on the CVPR 2022 paper “High-Resolution Image Synthesis with Latent Diffusion Models,” and it can run on consumer-grade GPUs.
Since its release, researchers have conducted deeper studies on Stable Diffusion. Some researchers have combined it with a Web UI, creating a painting tool that allows those without systematic UI knowledge to operate it easily.
To help more people get started quickly, researchers have also compiled a guide for running Stable Diffusion on the Apple M1 chip.
However, the original model was implemented based on Torch, and now, Divam Gupta, a researcher from Meta, has announced that Stable Diffusion implemented with TensorFlow/Keras is here.
Project address: https://github.com/divamgupta/stable-diffusion-tensorflow
In summary, the project has three main features: converting pre-trained models; easy-to-understand code; and a small amount of code.
Divam Gupta stated that when running Stable Diffusion on an 8GB M1 MacBook Air, the speed is about 4 times faster compared to the Torch implementation.
Keras creator François Chollet stated: it works out of the box on the M1 MacBook Pro’s GPU and can also be used for multi-GPU inference out of the box.
Chollet also mentioned that besides the GPU, it can also be used for TPU inference out of the box: just get a TPU VM and add the TPU strategy scope to the code. This can significantly speed up inference and reduce costs when processing large batches.
Finally, Chollet expressed “great thanks to Divam Gupta for creating this port. It is a high-quality work that will benefit everyone involved in creative AI. I am always amazed at the speed of development in the open-source community.”
1
Project Introduction
How to run this project?
First, go to this address: https://colab.research.google.com/drive/1zVTa4mLeM_w44WaFwl7utTaa6JcaH1zK
Run the command line:
python text2image.py --prompt="An astronaut riding a horse"Use the Python interface:
pip install git+https://github.com/fchollet/stable-diffusion-tensorflowfrom stable_diffusion_tf.stable_diffusion import Text2Imagefrom PIL import Imagegenerator = Text2Image( img_height=512,img_width=512,jit_compile=False,)img = generator.generate("An astronaut riding a horse",num_steps=50,unconditional_guidance_scale=7.5,temperature=1,batch_size=1,)Image.fromarray(img[0]).save("output.png")It can be completed in just two simple steps, and the output result is as follows: a Rococo-style werewolf drinking coffee in a burning café.
Spider-Man in pink and white on a skyscraper:
The scene in heaven:
Those interested can give it a try.
Reference links:
https://twitter.com/divamgupta/status/1571234504320208897?s=20&t=kpyjZ8LLSmgJFnUKqyepAw
https://twitter.com/fchollet/status/1571874757582389250
Technical Group Invitation
Scan the QR code to add the assistant on WeChat
About Us
