Understanding GPT2 and Transformers with spreadsheets

Spreadsheets-are-all-you-need

Understanding AI, GPT2, and Transformers with spreadsheets

What?

Spreadsheets-are-all-you-need implements the forward pass of GPT2 (an ancestor of ChatGPT) entirely in Excel using standard spreadsheet functions.

Show me!

Certainly, here's a 10 minute video about the project from the Seattle AI Tinkerers meetup:

By using a spreadsheet anyone (even non-developers) can explore and play directly with how a “real” transformer works under the hood with minimal abstractions to get in the way.

Tutorials

Lesson 1: Demystifying GPT with Excel

Lesson 2: Byte Pair Encoding & Tokenization

Extra: A walk through of the Excel sheet

More lessons to come! Get notified!

Future videos will walk through more details on the internals of modern AI. Subscribe below to get notified about new tutorials and updates.

Implementation

The implementation is just enough to run very small workloads:

• Full GPT2 small (124M parameters) model including byte pair encoding, embeddings, multi-headed attention, and multi-layer perceptron stages
• Inference/forward pass only (no training)
• Context is limited to 10 tokens in length
• 10 characters per word limit
• Zero temperature output only

This sheet is very big. Unfortunately, it is not unusual for Excel to lock up while using this spreadsheet. It is highly recommended to use the manual calculation mode in Excel.

Try it yourself

The sheet is available as an xlsb (Excel binary) file in the Releases section of the github repo. You should be able to download and run this file in Excel for Mac or PC.

Issues

Bugs are not out of the question. Please file issues on Github

Contact

@ianand on Twitter

ianand/spreadsheets-are-all-you-need on Github

What about Google Sheets?

This project actually started on Google Sheets but the full 124M model was too big and switched to Excel. I'm still exploring ways to make this work in Google Sheets but it is unlikely to fit into a single file as it can with Excel.

Why can't I chat with it like ChatGPT? It doesn't match the output of ChatGPT?

Aside from the minuscule context length, it also lacks the instruction tuning and reinforcement learning from human feedback (RLHF) that turn a large language model into a chatbot.

Why is it called Spreadsheets-are-all-you-need

The name is a play on the title of the famous Attention Is All You Need paper which first decribed the Transformer machine learning architecture that underlies ChatGPT, Claude, Bard, and many of the latest Generative AI tools.

Site layout: PicoCSS