A Guide to AI for Gonzaga Faculty

Generative artificial intelligence is AI that is designed to create (i.e. generate) new data. This differs from AI models common before generative AI, which are designed to make predictions about existing data, for example to state how likely a certain image is to be of a cow or a dog, or how likely a borrower was to default on a loan.

Generative AI creates new information, usually in the form of text (Large Language Models such as ChatGPT) or images (image generation models such as DALL•E). This guide and resource repository will focus primarily on Large Language Models and the tools which use them.

The GPT in ChatGPT stands for Generative, Pre-Trained Transformer. This bit of technobabble actually describes how a Large Language Model works. 

Large Language Models are generative, that is, they create data. The data they create is human language. In other words, LLMs are human language simulators. They represent words as numbers, then do calculations on those numbers in order to predict which words are most probabilistically likely to occur next. 

Put simply, Large Language Models are very complex and very capable versions of the autocomplete ability that has been in our phones for years. 

What sets LLMs apart from the old autocomplete systems is their magical-seeming ability to work with large amounts of language and its context, which is made possible by a neural network architecture called a transformer. Transformers, first described in a Google paper in 2017, include a multi-step attention process by which an LLM is able to determine information about words in relation to their context, then pass that information forward to inform future calculations.

To use the example from the Google paper, in the sentence "I arrived at the bank after crossing the river," the proximity of the word "bank" to the word "river" means that it is likely a river bank, rather than a financial institution. A transformer step in an LLM will see the proximity of the word "bank" to the word "river" and create a new mathematical representation of the word "bank" which bakes in that relationship. It will then pass that new representation forward, ready for the next step in the prediction process. 

LLMs are able to do this because they have been already been trained (pre-trained) on a massive corpus of human language, allowing them learn the patterns in human language, i.e. the probabilistic likelihood of words being close to each other. They are then able to predictively duplicate or continue those patterns.

Learn more:

Large Language Models, Explained With a Minimum of Math and Jargon, Understanding AI