Tiny language models and tiny image generation
19/07/25 10:01
Now my team is developing a tiny image-generating model, inspired by the low-bitrate videos of the Small File Media Festival. It’s important, because each inference of image-generation AI uses a crazy amount of electricity! We are learning from machine learning designers who work to achieve efficiencies, though the purpose is mostly to increase speed and accuracy, not to decrease electricity use. Small language models can run on a single GPU, using relatively little electricity in training and use (Schick and Schütze 2021; Thomas 2023, Wang & Wang 2024).
*However*, in the rebound effect, increased efficiency will lead to more electricity consumption, as users will be able to run AI models on a GPU-equipped device.
Some lower-electricity AI hacks:
· approximate or inexact computing: computing that solves to fewer decimal points
· “few-shot” algorithms and parameter counts that are degrees of magnitude smaller than those of large language models. For specific, not general purposes
· one-bit parameter representation of LLMs is pretty good for common-sense reasoning and world knowledge. (Xu 2024, “OneBit”)
· training on small hardware like Raspberry Pi, Arduino, SparkFun Edge, and others (Merenda 2020)
· and as always, keeping devices for as long as possible and avoiding activities that require new devices!
