As generative AI fashions develop extra highly effective, their power use is turning into a severe bottleneck. A brand new absolutely optical generative AI chip might assist by operating superior picture and video technology duties at speeds and efficiencies orders of magnitude past right this moment’s {hardware}.
Coaching generative AI fashions requires an infinite quantity of computing energy and power. However as demand explodes, the method of really operating the fashions to create photographs, textual content, or video—often known as inference—is rapidly turning into a fair larger drain on sources.
Video and picture technology fashions are significantly power intensive. Whereas the effectivity of those fashions is consistently enhancing, a 2023 research discovered that producing 1,000 photographs utilizing a number one mannequin produced carbon emissions equal to driving a gas-powered automotive greater than 4 miles.
One promising method for slashing power use is photonic computing, the place processors use gentle as a substitute of electrical energy. It’s a tactic a number of well-funded startups are pursuing in earnest. However most advances have been restricted to easier duties like picture classification or textual content technology.
Now, researchers from Shanghai Jiao Tong College and Tsinghua College in China have demonstrated an all-optical chip they name LightGen that’s greater than 100 occasions quicker and extra power environment friendly than a number one Nvidia GPU on duties like video and picture technology.
“LightGen gives a brand new solution to bridge the brand new chip architectures to every day sophisticated AI with out impairment of efficiency and with pace and effectivity which are orders of magnitude larger,” the researchers write in a latest paper on the chip in Science.
A key side of the brand new design is its density. Generative fashions sometimes require thousands and thousands of parameters to provide high-quality outputs, however earlier photonic chips have had, at most, a couple of thousand synthetic neurons. Utilizing 3D packaging, nonetheless, LightGen integrates greater than two million onto a tool measuring only a quarter of a sq. inch.
The ensuing processing increase permits the chip to work with photographs at resolutions as much as 512-by-512 pixels. Older photonic chips sometimes broke up high-resolution photographs into smaller patches to course of them. This not solely takes longer but in addition reduces a mannequin’s potential to attract statistical correlations between the completely different patches.
The researchers additionally innovated one thing known as an “optical latent house.” Generative AI fashions work, partially, by compressing high-dimensional information into easier representations. This forces them to take away much less necessary info and solely retain the bits which are integral to the enter.
These condensed representations are then saved in a multi-dimensional map of ideas known as a latent house. Fashions use these representations to generate new outputs when given a immediate.
LightGen’s builders replicated this course of totally optically. Of their chip, a full-resolution picture is transmitted by an optical encoder made up of a number of metasurfaces—ultra-thin constructions designed to control gentle—after which coupled into an array of optical fibers.
This course of naturally filters out higher-order information, successfully condensing the data into easier representations, that are then saved within the fiber array because the optical latent house. One other set of metasurfaces on the different finish of the machine, which could be switched relying on the duty, then take the output from this latent house and use it to generate high-resolution photographs.
The researchers additionally got here up with a novel coaching method. Right here, the chip learns probabilistic representations of coaching information, which makes it potential to sort out extra complicated duties, like creating novel outputs. It is a promising improvement. To this point, most photonic chips have targeted on inference not coaching.
The crew examined their chip on a number of demanding duties, together with the technology of high-resolution photographs of animals, changing photographs into completely different creative types, and even turning 2D photographs into 3D fashions. Notably, the chip achieved speeds and power efficiencies greater than two orders of magnitude higher than Nvidia’s A100 GPU, one of many firm’s strongest AI chips.
The brand new optical chip isn’t prepared to interrupt out of the lab simply but. It nonetheless depends on cumbersome lasers and spatial gentle modulators to generate enter alerts, and the metasurfaces central to its design are at the moment made with specialised processes quite these you may discover in normal chip factories.
Nonetheless, with additional improvement, the work suggests optical processors may very well be a quick, energy-efficient solution to energy the cutting-edge of an more and more power-hungry AI business.
