The new method combines the strengths of both generative adversarial networks (GANs) and diffusion models. This hybrid approach significantly reduces the computational burden for AI image generation. The researchers were able to generate high-quality images using only a fraction of the resources typically required. This advancement could lead to faster and more efficient AI image generation in various applications. The collaboration between MIT and Nvidia showcases the potential for innovative solutions in artificial intelligence research.
Representational image
One of the core problems with AI is the notoriously high power and computing demand, especially for tasks such as media generation. On mobile phones, when it comes to running natively, only a handful of pricey devices with powerful silicon can run the feature suite. Even when implemented at scale on cloud, it’s a pricey affair.
Nvidia may have quietly addressed that challenge in partnership with the folks over at the Massachusetts Institute of Technology and Tsinghua University. The team created a hybrid AI image generation tool called HART (hybrid autoregressive transformer) that essentially combines two of the most widely used AI image creation techniques.
The result is a blazing fast tool with dramatically lower compute requirement. Just to give you an idea of just how fast it is, I asked it to create an image of a parrot playing a bass guitar. It returned with the following picture in just about a second. I could barely even follow the progress bar.
When AI images first started making waves, the diffusion technique was behind it all, powering products such as OpenAI’s Dall-E image generator, Google’s Imagen, and Stable Diffusion. The second approach that has recently gained popularity is auto-regressive models, which essentially work in the same fashion as chatbots and generate images using a pixel prediction technique.
The team at MIT fused both methods into a single package called HART. It relies on an autoregression model to predict compressed image assets as a discrete token, while a small diffusion model handles the rest to compensate for the quality loss. The overall approach reduces the number of steps involved from over two dozen to eight steps.
The experts behind HART claim that it can “generate images that match or exceed the quality of state-of-the-art diffusion models, but do so about nine times faster.” HART combines an autoregressive model with a 700 million parameter range and a small diffusion model that can handle 37 million parameters.
Interestingly, this hybrid tool was able to create images that matched the quality of top-shelf models with a 2 billion parameter capacity. Most importantly, HART was able to achieve that milestone at a nine times faster image generation rate, while requiring 31% less computation resources.
The future potential is exciting, especially when integrating HART’s image capabilities with language models. The team at MIT is already exploring that idea, and even plan to test the HART approach at audio and video generation.
Before diving into the quality debate, it's essential to remember that HART is still in its early stages as a research project. The tool does have some technical challenges highlighted by the team, but they are minor compared to the benefits it offers in terms of computing efficiency, speed, and latency.
Overall, HART shows immense potential despite some typical failings of an AI image generator tool. It would be interesting to see how MIT and Nvidia progress with this innovative approach and whether it leads to a new product or is integrated into an existing one.
By entering your email you agree to our terms & conditions and privacy policy. You will be getting daily AI news in your inbox at 7 am your time to keep you ahead of the curve. Don't worry you can always unsubscribe.