How long would it take to think like human?
Preface
All we were excited about when OpenAI announced GPT-3 API with its use cases. Most of us may have thought like “If I train a model with the whole internet, I can create a model which knows anything that I ask it”. Let’s check out that what it is conceptually and how to use it.
Not thinking but acting.
What is GPT-3?
GPT means Generative Pre-Trained Transformer. It has built-in Natural Language Processor models so it can extract meaning from a text as we do. However, it has a difference from other NLP models, besides its NLP feature, it’s a generative model so it can generate linguistic data. It reads data, understands, learns, and uses it to answer your input.
GPT-3 can talk to and understand you.
How to use GPT-3?
GPT-3 is available as OpenAI’s private beta API by applying the waitlist form and is also a paid service. But there are also 2 open-source alternatives called GPT-J and GPT-Neo.
GPT-Neo has three different versions:
- 125 million parameters
- 1.3 billion parameters
- 2.7 billion parameters
There is only one version of GPT-J with 6 billion parameters, making it the most generative alternative to GPT-3.
System Requirements to use GPT-J
First we will talk about system requirements. There is no official information about that but according to this guys post,
- On CPU, 40GBs of RAM for loading model, after it loaded it requires 20GBs of RAM during runtime.
- On CPU, Standard text generation (around 50 words) requires 12 seconds for 12 CPUs
- On GPU, the model needs 40GBs of RAM to load, and then around 3GBs of RAM during runtime + 24GBs of GPU memory
- On GPU, Standard text generation (around 50 words) requires 1.5 seconds
So before you start doing some staffing with GPT models, you need to fix the huge amount of RAM and GPU issues.
How to use GPT-J?
You can follow this Colab Notebook to test it on the web (thanks to Google). I won’t show you the code but can show you what can you do with it, these and more can be found here:
- You can give some of a text about specific content and ask the model to write of the rest, it will try to write it with the narration style of the author.
- AI-generated coding. This guy is giving the name of function and purpose of function as a command and the model tries to write it, and when he calls the same command again, the model tries to do the same functionality with different code.
- You can generate images that never exist.
- You can compose music that has never been composed.
- You can create a chatbot that acts like a human.
You should check this philosophic conversation with GPT-3 by Merzmensch.
Epilogue
In my opinion, this approach is just the beginning. We can generate but not extract humanistic mean from data. I’m not sure if we need it, but until we do, we’ll think we talking to a machine. Maybe someday we can link these models with sensors to get real world’s dynamic data and when they mix up with knowledge talking with a computer can come more realistic.
