🔲AI RESOURCES

Resources and Tools for Generative AI Enthusiasts

Tools and Resources for AI Art

If you are looking to get started with AI art, then a good place to start is one of the popular apps like DreamStudio , midjourney , Wombo , or NightCafe . You can get a quick sense of how you can use words and phrases to guide image generation. Read up on prompt engineering to improve your results. Then you may want to move on to using Google Colab notebooks linked below like Deforum. If you have a good nVidia GPU of your own then you can also use NMKD Stable Diffusion GUI or Visions of Chaos to run the most popular notebooks locally. If you want to train your own Ai models check out the Ai art model training page, for animations check Stable Diffusion animations .

You can follow on twitter @pharmapsychotic

Active Ai Art Competitions

• Ending June 8: Thomas & GPB art contest (tweet ) ( link )

• Ending June 22: Ai Art Weekly theme: censorship $50 (tweet)

Google Colab notebooks

Text to Image

There are a TON of shared Google Colab notebooks floating around for doing text to image with pre-trained GAN and diffusion models. I've been compiling the ones I come across and try out and find interesting. Please hit me up on twitter (@pharmapsychotic) if you know a cool notebook that I am missing! Stable Diffusion is most popular right now.

• Stable Diffusion WebUI by automatic1111 - run SD local with lots of features and extensions

• Deforum Stable Diffusion 0.7 - group effort for ultimate SD notebook (discord) (youtube tutorial) (guide)

• Disco Diffusion v5.6 by Somnai, gandamu, zippy721 (guide) (new guide) (youtube tutorial)

• Huemin Jax Diffusion 2.7 by nshepperd, huemin_art (guide) (stitching guide)

• pytti-tools v0.10 by DigThatData and sportsracer

• VQGAN+CLIP by remi_durant

• [2023/04/28] DeepFloyd IF (huggingface) (github)

• [2023/04/05] Kandinsky 2.1 Batching+Dynamic prompting Colab by @jrobocat

• [2023/04/03] Kandinsky 2.1 (huggingface) (site)

• [2023/03/23] Image-to-text-to-image Colab by @jrobocat - batch CLIP Interrogator + SD generations

• [2023/03/14] Unidiffuser - unified diffusion framework (github)

• [2023/02/20] Stable Diffusion Auto Stitching by @oleg_ai_art (guide)

• [2023/02/15] ControlNet - control Stable Diffusion with extra conditioning (youtube) (huggingface) (github) (models)

• [2023/02/14] Pix2Pix video with coherence by ​@johnowhitaker - stylize video inputs!

• [2023/01/30] Tune-a-Video - create short text2video sequences (github) (paper)

• [2023/01/21] KLMC2 Animation - @DigThatData's fork with lots of additions

• [2023/01/20] InstructPix2Pix - use text instructions to modify images (huggingface)

• [2023/01/19] Image Mixer by @Buntworthy - mix up to 5 images together with SD

• [2023/01/14] Latent Blending by @j_stelzer - smooth transition between SD latents (github)

• [2023/01/10] Custom Diffusion - fast SD finetune with multiple concepts (github)

• [2022/12/22] Karlo - unCLIP architecture like DALLE-2 (huggingface) (github)

• [2022/12/08] Stable Diffusion KLMC2 Animation by @RiversHaveWings

• [2022/11/30] BAOAB-limit sampler - new SD sampler that can also make anims hella fast (paper)

• [2022/11/25] Stable Diffusion 2.0 Web UI - by @anzorq (run SD 2.0 in colab using Diffusers)

• [2022/11/24] Stable Diffusion 2.0 w Diffusers - by @amrrs (youtube)

• [2022/11/08] Midjourney v4 Style - (dreambooth SD finetune on midjourney v4 outputs)

• [2022/11/03] All-in-one Private Diffusions Colab - fork and upgrades to WD notebook (website)

• [2022/10/25] Fast Dreambooth by TheLastBen (easy fast finetune of stable diffusion in colab)

• [2022/10/08] Stable Worlds by @NaxAlpha (create panoramas with SD!)

• [2022/09/29] MathRockDiffusion by ethansmith2000 (mods and improvements on Disco) ( guide )( cuts )

• [2022/09/29] robo_diffusion_v1 by @nousr (a DreamBooth fine tune of stable diffusion)

• [2022/09/27] Video Killed The Radio Star Diffusion by @DigThatData (transform music videos from YouTube)

• [2022/09/25] fast-stable-diffusion - automatic111 ui, hlky ui, github (+25% speed and low VRAM)

• [2022/09/18] Doohickey Diffusion by aicrumb (stable diffusion with CLIP guidance, perlin init, lots more)

• [2022/09/18] optimized colab by neonsecret (stable diffusion with nice gradio gui in colab)

• [2022/09/13] Stable Diffusion Batch by visoutre (includes tiled upscaling!) (tutorial)

• [2022/09/11] Easy Diffusion by WASasquatch and NOP (stable diffusion with lots of still image features)

• [2022/09/07] NMKD Stable Diffusion GUI (nice easy Windows GUI for stable by Noomkrad)

• [2022/08/30] Simple Stable Diffusion by @ai_curio (supports prompt weighting)

• [2022/08/29] Stable Diffusion WebUi by @altryne (fancy Gradio UI for stable diffusion)

• [2022/08/28] Prompt Parrot v2.0 by @KyrickYoung (train gpt2 on prompt list then generate with stable-diff)

• [2022/08/23] Stable Diffusion Interpolation by @ygantigravity (animate from own prompt to another!)

• [2022/08/23] Deforum Stable Diffusion (discord link)

• [2022/08/23] FunkyHorses Stable Diffusion by Coskaiy/Corran (has neat import from spreadsheet)

• [2022/08/23] NOP's Stable Diffusion Colab v0.19 by NOP#1337

• [2022/08/23] Stable Diffusion Lite by @future__art (prompt queueing and seed mining)

• [2022/08/23] Interactive notebook for Stable Diffusion

• [2022/08/22] Stable Diffusion HuggingFace space by stabilityai

• [2022/08/22] Stable Diffusion notebook by @pharmapsychotic (easy to use and batch to gdrive) (tutorial)

• [2022/08/22] Official Stable Diffusion notebook - requires hugging face account

• [2022/08/22] DiscoStream v1.1 by @WASasquatch

• [2022/08/20] Disco Diffusion v5.6 with Inpainting by @cut_pow

• [2022/08/18] DiscoArt [w/ Batch Prompts + GPT3 generator] by Skquark

• [2022/08/16] WAS's Disco Diffusion v5.6-9 Portrait Generator Playground by WASasquatch

• [2022/08/08] Paint Pour Diffusion by @EclecticBeams (diffusion trained on paint pour art)

• [2022/07/31] Huemin Jax Diffusion 2.7 August 2022 by @huemin_art

• [2022/07/30] CLIP Prior + VQGAN by @RiversHaveWings and @jd_pressman (a new VQGAN notebook)

• [2022/07/23] Textile Diffusion by @KaliYuga (diffusion trained on textiles)

• [2022/07/21] Floral Diffusion by @jags111 (fine tunes for floral)

• [2022/07/18] Liminal Diffusion v1 by @BrainArtLabs (diffusion trained on liminal photographs)

• [2022/07/18] DifNESfusion 1.35 by @LufiQ (fork or PixelArtDiffusion with NES dataset)

• [2022/07/18] Medieval Diffusion by @KaliYuga (diffusion trained on medieval art)

• [2022/07/17] FeiArt_Handpainted CG Diffusion by @FeiArt_AiArt

• [2022/07/17] Fantasy Diffusion by @LaVista (diffusion trained on fantasy art)

• [2022/07/15] Ukiyo-e Portrait Diffusion by @avantcontra

• [2022/07/15] Lithography Diffusion by @KaliYuga (diffusion trained on lithographic landscapes and portraits)

• [2022/07/06] Disco v5.2 Dynamic Prompting (dynamic prompt variations - tutorial video )

• [2022/07/06] Watercolor Diffusion by @KaliYuga (diffusion trained on watercolor paintings)

• [2022/07/05] EnzymeZoo edits to Huemin Jax Diffusion by @EnzymeZoo (brought over masking from Majesty)

• see older notebooks in the archive

StyleGAN

• [2022/08/23] Painting with StyleGAN by @jmoso13 (tutorial) - use VAE to navigate and animate!

• [2022/04/25] StyleGAN-Humans + CLIP modified by Diego Porres to use StyleGAN3

• StyleGAN2-ADA - train your own StyleGAN2 model from an image set you create

• StyleCLIP - Text-drive manipulation of StyleGAN imagery

• Structured Dreaming - Styledreams With helpers

• Structured Dreaming (CLIP+StyleGAN) by @ArYoMo (tweet)

• StyleGAN 2 pretrained models - can use these with Structured Dreaming

• StyleGAN 2 awesome pretrained models - BIG collection of models

• StyleGAN 3 training - train a StyleGAN and do interpolation video by @dvsch (currently busted)

• StyleGAN 3 music video generation - (tweet)

• StyleGAN 3 + CLIP by Annas

• StyleGAN3 + CLIP by @nshepperd1 and @RiversHaveWings

• StyleGANXL + CLIP by Eugenio Herrera and Rodrigo Mello

• Lucid Sonic Dreams - animate path through StyleGAN latent space with music (github)

Video

Text to video

• ModelScope (colab) (huggingface) - super fun but prominant shutterstock watermarks

• [2023/03/20] ModelScope text-to-video Colab by @camenduru (youtube) (github)

• [2023/03/18] ModelScope text-to-video huggingface space

• Text2Video-zero (colab) (github) (huggingface) (webui ext) - zero shot video from Stable Diffusion

Interpolation

• Video Enhance AI by Topaz Labs - commercial upscaling and frame interpolation <- excellent

• AnimationKit AI - video upscaling and interpolation tool <- great

• FILM colab - by @KyrickYoung has pause, loops, reverse <- my fave FILM

• 3D Ken Burns Effect from single image - animated video from 2D image

• 3D Photo Inpainting - cool 3D effects for 2D images

• Animating Pictures with Eulerian Motion Fields - code not out yet, looks like it'll be awesome

• DAIN colab - depth aware interpolation

• EbSynth - stylize video by giving it ai or hand painted key frames from video

• ESRGAN 4 Video - increase resolution of video with ESRGAN

• FILM: Frame Interpolation for Large Motion - (replicate link) smooth interpolation/morphing

• Flowframes - free Windows tool with patreon option, uses RIFE and other models

• PyTTI-Tools: FILM - @DigThatData 's version of FILM for video frames

• RIFE - smooth interpolation of video to increase frame rate

• Sequence Frame Interpolation - batch version of FILM

• Super Slomo - another way to increase frame rate of video

• Video Art and Styling Tools - by @Coskaiy (style transfer, interpolation, superres, and more)

Prompt Engineering

To get good results with CLIP guided diffusion and VQGAN+CLIP you need to find the right words and phrases that will direct the neural network to the content and style you are looking for.

Image to Text

• Antarctic-Captions by @dzryk

• BLIP image captioning HuggingFace space

• CLIP Interrogator by @pharmapsychotic - image to prompt! (huggingface) (lambda) (replicate)

• CLIP prefix captioning inference notebook (github)

• LLaVa: Large Language and Vision Assistant - ask vision model to describe image

• personality-clip by @dzryk

• PEZ: Prompts made EZ - prompt from image or long to short prompt (huggingface) (colab)

Other

• sdtools.org - cool wiki covering tools and methods related to Stable Diffusion

• JAX CLIP Guided Diffusion 2.7 Guide - Google doc from huemin

• Zippy's Disco Diffusion Cheatsheet - Google Doc guide to Disco and all the parameters

• EZ Charts - Google Doc Visual Reference Guides for CLIP-Guided Diffusion (see what all the parameters do!)

• Hitchhiker's Guide To The Latent Space - a guide that's been put together with lots of colab notebooks too

• Resources for GAN Artists - another big Google Doc with notebooks and resources for AI art

• Way of the TTI Artist - pytti guide

• Guide to install Disco Diffusion 5 on Windows with WSL - haven't tried this yet challenge is pytorch3d

• Great explanation of VQGAN+CLIP - https://ljvmiranda921.github.io/notebook/2021/08/08/clip-vqgan/

• Nice overview of lots of different optimization algorithms SGD, Adam, RMSProp etc and their differences (also covered in this lecture)

• Stanford's Convolutional Neural Networks class on YouTube - https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv

• ClipMatrix - text controlled 3D mesh deformation and stylization

• CLIP-Mesh - text to 3D mesh with texture and normal map (still pretty simple and mixed results)

• DreamFields - latest text to 3D (github)

• ImageSorter by @pharmapsychotic - sort images by similarity (nice for StyleGAN/FiLM animated loops)

• PIFuHD Colab - Human photo to 3D mesh of the human

• Point-E - OpenAI's text to 3d point clouds (github)

• text2mesh - Kaggle notebook for text to 3D mesh

• Watermark images - little notebook to add text watermark to images

• Zero-Shot Text-Guided Object Generation with Dream Fields - text to 3D render

AI Art Discord Servers

There are quite a few Discord servers dedicated now to AI artists or discussing text to image techniques.

• Ai NFT Discord - AI NFT Consortium. Has especially useful StyleGAN training resources

• Disco Diffusion Discord - chat and tech support for the Disco notebook

• EleutherAI Discord - researchers and good art room with more technical discussions

• Jukebox Community Discord - server for using OpenAI Jukebox for music generation

• LAION Discord - group working on replicating a full DALLE-E

• NeuralismAI Discord - AI art competitions and knowledge exchange

• Prompt Sharing Discord - community for sharing text to image prompts

• VQGAN+CLIP Discord - home of Instagram #vqganclipcommunitycolab

• Zoetrope Central Spoke Discord - support and discussion of the Looking Glass notebook

Online Galleries to Showcase Art

• OnCyber art galleries - https://oncyber.io - Cool 3D art gallery to showcase your art with links to NFT market

• Spatial - https://spatial.io

You can follow on twitter @pharmapsychotic