It’s the world’s largest chip manufactured to 7 nm standards.
“Father of all chips” is the top Intel Xe line GPU, a photo of which two weeks ago surpassed the pages of all thematic publications, perhaps the largest with some reservations. But among 7–nanometer chips “father” can certainly be called a chip Nvidia GA100. On the chip of this Ampere generation GPU there are 54 billion transistors (not counting the transistors in the HBM2E memory stacks).
According to Nvidia, the new GPU is 20 times faster than its predecessor, the Tesla V100, in both AI and single–precision computing (FP32). Nvidia plans to offer a new numerical format for AI computing called TF32 (32–bit floating point tensor). TF32 uses a 10–bit mantissa from FP16 and an 8–bit order from FP32. This format will significantly improve productivity.
Another feature of the GA100 is the “multi–instance” graphics processor, which allows several complex applications to run on a single physical graphics processor without sharing resources such as memory bandwidth. The user can divide the physical GA100 into several virtual (up to seven) applications with different characteristics. Not surprisingly, 54 billion transistors were required to implement such features.
It’s all good when a task is perfectly paralleled into thousands of similar computers. But these tens of millions of transistors will not help you at all: there will be a small speed gain due to faster memory and a little bit more in details there.
That’s why cunning marketers like Nvidia and others are pushing 8K standards, because video processing is in >90% of cases and there is a huge number of similar operations, which are paralleled by thousands of simple calculators ideally the better, the more points in the raster picture: convolution (blurring, increasing “sharpness“, etc.), brightness–contrast–color, compression of pictures and video streams, etc. There are not so many tasks in serious computational mathematics where these transistors could be sensibly applied. Almost completely empirical AI craft is not to be pulled up here: the efficiency of these “neurons” is about 5% like that of a steam locomotive, and so far this sad fact can only be hidden by the huge power of such calculators.
Academic mathematicians look with a smile at all these new “datascientologists“, who in the university matan is not always at 4 were able to pass, but stuffed his hand on the same type of packages of neurocomputers and thought of themselves the ignorance of a friendly choir of mercantile marketers, not hesitant to add oil to the fire of excitement allegedly real AI for its immediate benefit.