ChatGPT 3.5 vs ChatGPT 4 for customer support

Now that the latest version of ChatGPT 4 has been released, users of SiteSpeakAI can use the newest model for their customer support automation. I've put ChatGPT 3.5 and ChatGPT 4 to the test with customer support questions to see how they compare.

Training

To train the chatbot, I used the standard text-embedding-ada-002 embedding model from OpenAI to create the embeddings and store them in Pinecone for querying. This chatbot will be the base for each of the comparisons.

The questions

To see how each of the models would respond, I asked it three questions:

1. Why should I use SiteSpeakAI?
2. How much does it cost?
3. Thanks. Does it work with WordPress?
4. Where can I find the tutorial?

GPT-3.5

First up, using the latest gpt-3.5-turbo-1106 model, I got the following results for the questions asked:

Question: "Why should I use SiteSpeakAI?"
Response: "You should use SiteSpeakAI because it can automatically answer visitor questions about your products and services, reducing the load on your customer support staff and enabling your website to speak to your visitors in real-time, 24/7."

Question: "How much does it cost?"
Response: "The pricing plans for SiteSpeakAI are as follows: Free: $0/month or $0/year Startup: $30/month or $330/year Pro: $60/month or $660/year Business: $500/month or $5500/year You can find more information about the pricing plans here."

Question: "Thanks. Does it work with WordPress?"
Response: "Yes, it works with WordPress. You can find tutorials on how to add ChatGPT to WordPress here."

Question: "Where can I find the tutorial?"
Response: "You can find the tutorial on how to add ChatGPT to WordPress here."

GPT-4

Now for the latest model from OpenAI, here are the results for the questions asked:

Question: "Why should I use SiteSpeakAI?"
Response: "We'd love to tell you why! SiteSpeakAI is great because it can automatically answer questions about your products and services. This means it can take a load off your customer support team and chat with your visitors any time of the day, all week long. Isn't that handy? It's like having a friendly helper on your site around the clock!"

Question: "How much does it cost?"
Response: "The pricing plans for SiteSpeakAI are: Free: $0/month or $0/year Startup: $30/month or $330/year Pro: $60/month or $660/year Business: $500/month or $5500/year You can find more information about the pricing plans here."

Question: "Thanks. Does it work with WordPress?"
Response: "Absolutely, it works with WordPress! You can add your custom trained ChatGPT chatbot to your WordPress site. We've got a tutorial to help you out with that. If you need it, just let us know, and we'll be more than happy to assist you further!"

Question: "Where can I find the tutorial?"
Response: "Sure thing! You can find the tutorial on how to add your custom trained ChatGPT chatbot to your WordPress site at this link: How to add ChatGPT to WordPress. Hope that helps you out!"

Results

Even though I used the same base and system prompts for these results, you can see that GPT-4 returned a more detailed response and seemed friendlier 😜.

GPT-3.5 and GPT-4 are excellent options for automating your customer support and responding to your visitors, but GPT-4 has an edge, albeit at a slightly increased cost.

If you'd like to try GPT-4 (or GPT-3.5) on your own site, SiteSpeakAI would love to help you create your own chatbot.