|
1 | | -<div align="center"> |
| 1 | +# PixelFlow 🌌 |
2 | 2 |
|
3 | | -<h1> PixelFlow: Pixel-Space Generative Models with Flow </h1> |
| 3 | + |
4 | 4 |
|
5 | | -[](https://arxiv.org/abs/2504.07963) |
6 | | -[](https://huggingface.co/spaces/ShoufaChen/PixelFlow) |
| 5 | +Welcome to **PixelFlow**, a repository dedicated to exploring the fascinating world of pixel-space generative models. This project aims to push the boundaries of creativity and technology by leveraging advanced algorithms to generate stunning visuals and art. |
7 | 6 |
|
| 7 | +## Table of Contents |
8 | 8 |
|
9 | | - |
10 | | - |
11 | | -</div> |
12 | | - |
13 | | - |
14 | | - |
15 | | - |
16 | | -> [**PixelFlow: Pixel-Space Generative Models with Flow**](https://arxiv.org/abs/2504.07963)<br> |
17 | | -> [Shoufa Chen](https://www.shoufachen.com), [Chongjian Ge](https://chongjiange.github.io/), [Shilong Zhang](https://jshilong.github.io/), [Peize Sun](https://peizesun.github.io/), [Ping Luo](http://luoping.me/) |
18 | | -> <br>The University of Hong Kong, Adobe<br> |
| 9 | +- [Introduction](#introduction) |
| 10 | +- [Features](#features) |
| 11 | +- [Installation](#installation) |
| 12 | +- [Usage](#usage) |
| 13 | +- [Contributing](#contributing) |
| 14 | +- [License](#license) |
| 15 | +- [Contact](#contact) |
| 16 | +- [Releases](#releases) |
19 | 17 |
|
20 | 18 | ## Introduction |
21 | | -We present PixelFlow, a family of image generation models that operate directly in the raw pixel space, in contrast to the predominant latent-space models. This approach simplifies the image generation process by eliminating the need for a pre-trained Variational Autoencoder (VAE) and enabling the whole model end-to-end trainable. Through efficient cascade flow modeling, PixelFlow achieves affordable computation cost in pixel space. It achieves an FID of 1.98 on 256x256 ImageNet class-conditional image generation benchmark. The qualitative text-to-image results demonstrate that PixelFlow excels in image quality, artistry, and semantic control. We hope this new paradigm will inspire and open up new opportunities for next-generation visual generation models. |
22 | 19 |
|
| 20 | +Generative models have become a cornerstone in the field of artificial intelligence and creative arts. With **PixelFlow**, we provide tools and frameworks that allow users to create unique pixel-based art using generative techniques. Whether you are an artist, a developer, or a researcher, this repository offers a wealth of resources to inspire your next project. |
23 | 21 |
|
24 | | -## Model Zoo |
| 22 | +## Features |
25 | 23 |
|
26 | | -| Model | Task | Params | FID | Checkpoint | |
27 | | -|:---------:|:--------------:|:------:|:----:|:----------:| |
28 | | -| PixelFlow | class-to-image | 677M | 1.98 | [🤗](https://huggingface.co/ShoufaChen/PixelFlow-Class2Image) | |
29 | | -| PixelFlow | text-to-image | 882M | N/A | [🤗](https://huggingface.co/ShoufaChen/PixelFlow-Text2Image) | |
| 24 | +- **Easy to Use**: The library is designed for simplicity, allowing you to focus on creativity. |
| 25 | +- **High-Quality Outputs**: Generate high-resolution images that can be used in various applications. |
| 26 | +- **Customizable Models**: Tailor the generative models to fit your specific needs and preferences. |
| 27 | +- **Community Driven**: Join a community of like-minded individuals passionate about generative art. |
30 | 28 |
|
| 29 | +## Installation |
31 | 30 |
|
32 | | -## Setup |
| 31 | +To get started with **PixelFlow**, clone the repository and install the necessary dependencies. Use the following commands: |
33 | 32 |
|
34 | | -### 1. Create Environment |
35 | 33 | ```bash |
36 | | -conda create -n pixelflow python=3.12 |
37 | | -conda activate pixelflow |
| 34 | +git clone https://github.com/algorithmy0101/PixelFlow.git |
| 35 | +cd PixelFlow |
| 36 | +pip install -r requirements.txt |
38 | 37 | ``` |
39 | | -### 2. Install Dependencies: |
40 | | -* [PyTorch 2.6.0](https://pytorch.org/) — install it according to your system configuration (CUDA version, etc.). |
41 | | -* [flash-attention v2.7.4.post1](https://github.com/Dao-AILab/flash-attention/releases/tag/v2.7.4.post1): optional, required only for training. |
42 | | -* Other packages: `pip3 install -r requirements.txt` |
43 | | - |
44 | 38 |
|
45 | | -## Demo [](https://huggingface.co/spaces/ShoufaChen/PixelFlow) |
| 39 | +Ensure you have Python 3.6 or higher installed on your machine. |
46 | 40 |
|
| 41 | +## Usage |
47 | 42 |
|
48 | | -We provide an online [Gradio demo](https://huggingface.co/spaces/ShoufaChen/PixelFlow) for class-to-image generation. |
| 43 | +After installation, you can start using **PixelFlow** to create your generative art. Here’s a simple example to get you started: |
49 | 44 |
|
50 | | -You can also easily deploy both class-to-image and text-to-image demos locally by: |
| 45 | +```python |
| 46 | +from pixel_flow import GenerativeModel |
51 | 47 |
|
52 | | -```bash |
53 | | -python app.py --checkpoint /path/to/checkpoint --class_cond # for class-to-image |
54 | | -``` |
55 | | -or |
56 | | -```bash |
57 | | -python app.py --checkpoint /path/to/checkpoint # for text-to-image |
| 48 | +model = GenerativeModel() |
| 49 | +image = model.generate() |
| 50 | +image.show() |
58 | 51 | ``` |
59 | 52 |
|
| 53 | +This code snippet initializes the generative model and produces a unique image. You can customize the parameters of the `GenerativeModel` class to create different styles and effects. |
60 | 54 |
|
61 | | -## Training |
| 55 | +## Contributing |
62 | 56 |
|
63 | | -### 1. ImageNet Preparation |
| 57 | +We welcome contributions from everyone! If you want to contribute to **PixelFlow**, please follow these steps: |
64 | 58 |
|
65 | | -- Download the ImageNet dataset from [http://www.image-net.org/](http://www.image-net.org/). |
66 | | -- Use the [extract_ILSVRC.sh]([extract_ILSVRC.sh](https://github.com/pytorch/examples/blob/main/imagenet/extract_ILSVRC.sh)) to extract and organize the training and validation images into labeled subfolders. |
| 59 | +1. Fork the repository. |
| 60 | +2. Create a new branch for your feature or bug fix. |
| 61 | +3. Make your changes and commit them. |
| 62 | +4. Push your branch to your forked repository. |
| 63 | +5. Create a pull request. |
67 | 64 |
|
68 | | -### 2. Training Command |
| 65 | +Please ensure your code follows the existing style and includes appropriate tests. |
69 | 66 |
|
70 | | -```bash |
71 | | -torchrun --nnodes=1 --nproc_per_node=8 train.py configs/pixelflow_xl_c2i.yaml |
72 | | -``` |
| 67 | +## License |
73 | 68 |
|
74 | | -## Evaluation (FID, Inception Score, etc.) |
| 69 | +**PixelFlow** is licensed under the MIT License. See the [LICENSE](LICENSE) file for details. |
75 | 70 |
|
76 | | -We provide a [sample_ddp.py](sample_ddp.py) script, adapted from [DiT](https://github.com/facebookresearch/DiT), for generating sample images and saving them both as a folder and as a .npz file. The .npz file is compatible with ADM's TensorFlow evaluation suite, allowing direct computation of FID, Inception Score, and other metrics. |
| 71 | +## Contact |
77 | 72 |
|
| 73 | +For any inquiries or suggestions, feel free to reach out to us via GitHub issues or by contacting the repository maintainer. |
78 | 74 |
|
79 | | -```bash |
80 | | -torchrun --nnodes=1 --nproc_per_node=8 sample_ddp.py --pretrained /path/to/checkpoint |
81 | | -``` |
| 75 | +## Releases |
82 | 76 |
|
| 77 | +To download the latest releases of **PixelFlow**, visit our [Releases section](https://github.com/algorithmy0101/PixelFlow/releases). Make sure to download and execute the necessary files to explore all features. |
83 | 78 |
|
84 | | -## BibTeX |
85 | | -```bibtex |
86 | | -@article{chen2025pixelflow, |
87 | | - title={PixelFlow: Pixel-Space Generative Models with Flow}, |
88 | | - author={Chen, Shoufa and Ge, Chongjian and Zhang, Shilong and Sun, Peize and Luo, Ping}, |
89 | | - journal={arXiv preprint arXiv:2504.07963}, |
90 | | - year={2025} |
91 | | -} |
92 | | -``` |
| 79 | +Explore the potential of generative models and unleash your creativity with **PixelFlow**! Don't forget to check the [Releases section](https://github.com/algorithmy0101/PixelFlow/releases) for the latest updates and features. |
| 80 | + |
| 81 | + |
| 82 | + |
| 83 | +Thank you for your interest in **PixelFlow**! We hope you find it as exciting as we do. Happy creating! |
0 commit comments