Generative AI vs genAI™

The market witnessess a flood of freshly baked AI experts which tell you how genAI is different from non-genAI and how latter is by far not as powerful as former. They will also tell you how genAI emerged in 2022 or actually in the lab in 2017 as described in the inception paper about attention mechanism, but actually become powerful enough in 2022 and prior to it there were no genAI stuff. At all. All very new, very fresh and development is exponential.

Well, we have to distinguish between generative artificial intelligence models (generative AI) as a class of AI models which have been available for decades, including first attempts at transformer like archtecture (see for exanple notes by J. Schmidhuber) and the commercial product, which aforementioned experts associate with modern AI. We call it genAI™.

In short, generative AI is based on models which learn a conditional probability distribution over outputs given inputs or a joint distribution over inputs and outputs. The latter is more powerful since it allows for all kinds of inference but is more likely to be computationally intractable. Transformer architecture is one particular example of a generative model. Bayesian networks are a notable another example.

Typically the story goes like, the non-genAI™ can not generate stuff. Well genAI™, genAI and discriminative AI are all just functions. Given input they compute output. While both former categories compute multiple output options explicitly and one typically has to pick one from them (via sampling, but the distribution itself may be of interest as well), the discriminative models (for example a spam classifier, which outputs spam or non-spam, given mail text) compute the options implicitly and in process compute a function over those options, the expected value. So it is just computationally faster special case. Models which can generate stuff and in particular generate distributions of stuff have been there for decades and are nothing new. Whats new, and this is truly what makes difference is the scale. The scale of available data, compute and financial resources required to make it happen is what made the characteristics seen in genAI™ possible. You may say I forgot model architecture. Well, given the resources it was arguably not hard to come up with something like attention mechanism, given all the previous work. Also see the variations on the theme.

Why does it matter: Well as usual to be able to assess the situation to your advantage as opposed to others advantage.