Custom model training and fine-tuning for GPT-3.5 Turbo

Herman Schutte

@hermanschutte

Custom model training and fine-tuning for GPT-3.5 Turbo

Today OpenAI announced that businesses and developers can now fine-tune GPT-3.5 Turbo using their own data. This means you can now take advantage of creating a custom trained and tuned GPT model for your business that can more accurately answer visitor questions and match your tone.

Use cases for fine-tuning

  • Allow the model to follow instructions better specifically when language choices are involved.
  • Output formatting is more reliable and can be tuned to consistantly output JSON in a format that can be consumed by your API.
  • A business with a recognizable brand voice can use fine-tuning for the model to be more consistent with their tone.

Besides boosting performance, fine-tuning lets companies trim down their prompts without losing efficiency. With GPT-3.5-Turbo, fine-tuning can even manage 4k tokens—twice what our earlier fine-tuned models could handle. Early users have managed to shrink prompt size by as much as 90% by embedding the fine-tuning instructions right into the model, making each API call faster and reducing expenses.

Fine-tuning your own custom GPT-3.5 Turbo model

Step 1: Prepare your data

You will need to prepare and provide a dataset to the OpenAI API that can be used to fine-tune your model. The minimum number of examples to provide is 10, but most improvements will come from providing more than 50.

Every sample in the dataset should follow the same structure as our Chat completions API, specifically being a list of messages where each one includes a role, content, and possibly a name. Some of the training examples should focus on instances where the prompted model isn't acting as expected. The assistant messages included in the data should represent the perfect responses you'd like the model to give.

Example dataset

{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "What is SiteSpeakAI?"}, {"role": "assistant", "content": "SiteSpeakAI is a tool that allows you to create a custom trained chatbot that uses ChatGPT to answer you visitor questions."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "How much does it cost?"}, {"role": "assistant", "content": "We have 3 pricing plans, $30/mo, $60/mo and $500/mo."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "Does it support Wordpress?"}, {"role": "assistant", "content": "Yes! You can easily install SiteSpeakAI on Wordpress and add a chatbot to your website."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "Does it support Carrd?"}, {"role": "assistant", "content": "Yes! You can easily install SiteSpeakAI on Carrd and add a chatbot to your website."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "How do I install it on my website?"}, {"role": "assistant", "content": "To install SiteSpeakAI, simply copy the code snippet and add it the `<head>` section of your website."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "How can I train my chatbot on my website content?"}, {"role": "assistant", "content": "To train your chatbot, selected the sources you would like to use and click the add selected button. Your chatbot will now be trained and ready to answer your visitor questions."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "What content can I use to train my chatbot?"}, {"role": "assistant", "content": "You can use your website content, knowledge base, support docs, PDF's, CSV's, API or SQL Database to train your chatbot."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "Do you support fine-tuning?"}, {"role": "assistant", "content": "Yes! You can add custom echo's to fine-tune your chatbot and we can also offer fine-tuning using GPT-3.5-Turbo to create a custom model for your business."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "What if a user asks my chatbot a question it can't answer?"}, {"role": "assistant", "content": "You can enable escalations for your chatbot that allows your visitors to leave their email addresses so you can follow up with them."}]}
{"messages": [{"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."}, {"role": "user", "content": "How do I get started?"}, {"role": "assistant", "content": "To get started, simply create an account and add the code snippet to your website. Your chatbot will be ready to answer your visitor questions in minutes."}]}

Step 2: Upload your training data

I'll provide a complete script at the end of this post that will allow you to upload and train your model, but if you just need to upload a file:

import os
import openai

openai.api_key = os.getenv("OPENAI_API_KEY")

# Upload file
print('Uploading file...')
file = openai.File.create(
    file=open("yourfile.jsonl", "rb"),
    purpose='fine-tune'
)

print('File uploaded: ', file)

Step 3: Create your fine-tuned model

After uploading your file and making sure the status has changed from uploaded to processed, you can create your fine-tune training job. Training jobs take a while to complete. For the short example provided the job took around 5 minutes to complete. Once the training job is done, the status of the job will change to succeeded. You will then be able to get the fine_tune_model from the response and use this model ID for inference.

import os
import openai

openai.api_key = os.getenv("OPENAI_API_KEY")

# Create fine-tuning job
print('Creating fine-tuning job...')
job = openai.FineTuningJob.create(
    training_file="file-123abc", model="gpt-3.5-turbo")

print('Job created.')
print(job)

Step 4: Use your fine-tuned model

You can now use your newly trained GPT-3.5 Turbo model for inference.

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")

completion = openai.ChatCompletion.create(
    model="your-fine-tune-model-id",
    messages=[
        {"role": "system", "content": "SiteSpeak AI Assistant is a customer support virtual assistant that is helpful and friendly."},
        {"role": "user", "content": "What is SiteSpeakAI?"}
    ]
)

print(completion.choices[0].message)

Here is a complete python script that uploads a file, waits for it to processed, creates a training job and outputs the model when it's ready:

import os
import openai
import time

openai.api_key = os.getenv("OPENAI_API_KEY")

# Upload file
print('Uploading file...')
file = openai.File.create(
    file=open("training.jsonl", "rb"),
    purpose='fine-tune'
)

print('File uploaded: ', file)

# Check if file is ready
print('Checking file status...')
while file.status != 'processed':
    time.sleep(10)
    file = openai.File.retrieve(file.id)
    print('File status:', file.status)

    # If file fails, print error and exit
    if file.status != 'processed':
        print('File failed:', file)
        exit(1)

# Create fine-tuning job
print('Creating fine-tuning job...')
job = openai.FineTuningJob.create(
    training_file=file.id, model="gpt-3.5-turbo")

print('Job created.')
print(job)

# Loop until status is succeeded. Print status every 60 seconds
while job.status != 'succeeded':
    time.sleep(10)
    job = openai.FineTuningJob.retrieve(job.id)
    print('Job status:', job.status)

    # If job fails, print error and exit
    if job.status != 'running' and job.status != 'succeeded':
        print('Job failed:', job)
        exit(1)

print('Job completed.')
print('Fine-tuned model:', job.fine_tuned_model)

Need help fine-tuning GPT-3.5 Turbo for your business?

If you don't want to go through the process of setting up your own training data and creating your fine-tuned models, SiteSpeakAI is a great alternative. Our chatbots can already be trained on your website content, knowledge base, API's and databases using embeddings and similarity search.

For business plan users we also offer the ability to create a custom fine-tuned GPT model trained on your business data that can exactly match your brand and provide AI powered "human-like" assitance to your visitors and customers.

More from the blog

Ready to automate your customer support with AI?

Join over 150+ businesses, websites and startups automating their customer support with a custom trained GPT chatbot.

Create Free Chatbot