ChatGPT With Unity AI Assistant: A Mind-Blowing Innovation Thanks To OpenAI!
Hello everyone 👋!
After a few days of coding with ChatGPT API and Meta Voice SDK, I've created a new AI YouTube video showcasing a ChatGPT AI Assistant that I built with Unity. The video covers all the details of the assistant, highlighting its features and customizability. With this creation, I hope to inspire others to explore the possibilities of AI and push the boundaries of technology.
📚Unity ChatGPT AI Assistant Flow:
The first thing we need to do is to obtain an API Key and Organization Id by creating a paid account through OpenAI Developer Platform (keep in mind that OpenAI Chat requires a credit card, however the price per 1K tokens is $0.002 USD for gpt-3.5-turbo which makes this very affordable for anyone to test this mind-blowing AI technology)
Next, we need to concentrate in what I did on the video, I went ahead and created a C# ChatGPTClient singleton which allows me to interact with ChatGPT API via HTTP. I’m including an example below of a HTTP POST request going out to OpenAI via CURL with a model and message specified in the POST Body, but in the case of using Unity take a look at the C# ChatGPTClient from the video which provides additional implementation details.
curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Can you create a MonoBehaviour script instantiates a new cube and rotates it constantly?"}]
}'ChatGPTResponse C# object is populated with the source and explanations by using a simple parser.
Created a C# ChatGPTAssistant singleton to integrate ChatGPT responses with Meta Voice SDK. ChatGPTAssistant is responsible for communicating with OpenAI through the ChatGPTClient, once a response comes back from OpenAI, I hand out all the explanation to Meta Voice SDK for Text-To-Speech conversion.
ChatGPT explanations are put into a list containing a max of 280 characters per line since Meta Voice SDK TTS has that as a character limit 😩.
A Unity coroutine is used to send each TTS request in chunks = one request per explanation line.
AI Robot uses a built-in Meta Voice TTS Voice which makes use of gain, pitch, and speed to simulate a robot voice.
Repeated voice commands are cached such as “Welcome” and “Transcription in progress” messages, this helps alleviate the delays from Wit AI when processing which takes 3-5+ seconds per request to complete.
Here’s a C# example of ChatGPTAssistant SpeakInChunks method which processes each explanation by calling Meta Voice SDK TTS services.
public void ChatGPTAISpeak(string text) => StartCoroutine(SpeakInChunks(text));
private IEnumerator SpeakInChunks(string text)
{
List<string> captions = SplitIntoMultipleCaptions(text);
captions.Insert(0, transcribingInProgressMessage);
linesProcessed = 0;
while (true)
{
if (!speaker.IsSpeaking && !speaker.IsLoading)
{
Debug.Log($"Speak[{linesProcessed}] : {captions[linesProcessed]}");
speaker.Speak(captions[linesProcessed]);
}
yield return new WaitWhile(() => speaker.IsSpeaking);
if (linesProcessed >= captions.Count) break;
}
yield return null;
}Well, that concludes this new post on ChatGPT and Unity. I personally feel honored to live in an era where I have access to tools for working with AI. I would like to leave you with that thought and also take the opportunity to make the same tools available to you!
If you would like access to the entire source code, please let me know. Alternatively, you can sign up for my 'Full Source Code' tier on Patreon, where I can grant you access to the repository along with many other experiments that I consistently work on.
