Transforming Document Exploration: Using GPT-3 to Create a Chatbot that Answers Your Questions

GPT-3 is the underlying model of ChatGPT. It is an advanced language model that can generate text but it can't find information that is not already in the model. In additon, there are limitation to feed long text to GPT-3. In this article, we will show you how to use a technique called conditional generation to build a chatbot that can help you explore long documents.

Conditional generation is a way of training a language model to understand the context of a document and generate answers to questions about it. We will take the following steps to build the chatbot:

1. Create a corpus from the document.
2. Match the user's question to the relavant passages in the corpus.
3. Create a prompt that includes the context of the question.
4. Send the prompt to GPT-3 to generate an answer.
5. Create a function that combines steps 2-4.
6. Test the chatbot by creating a question-answer loop.
7. Use Gradio to create an interface for the chatbot.

In this exercise, we will build a chatbot to answer the question in an employee handbook.

Create a corpus from the document

First we have prepared a file HKIHRMEmployeeHandbook.txt which is text version of a sample employee handbook created by Hong Kong Institute of Human Resource Management.

To start, we are using a python library called "sentence-transformers" to create a corpus of embeddings from the text document "HKIHRMEmployeeHandbook.txt". The sentence-transformers library is an NLP library that makes use of pre-trained transformer-based models, like BERT, RoBERTa, DistilBERT, and, in this case, multi-qa-mpnet-base-cos-v1, to generate embeddings for each sentence in the corpus. This is useful for our chatbot application as it allows us to find the most relevant passage in our document for the user's query. This will help to provide more accurate and useful answers for the user.

Embeddings are a way to represent text in a numerical form that can be used for comparison and similarity calculation by the model. In this case, we are using sentence-transformers library to convert the text into embeddings. These embeddings are a compact and dense representation of the text, which can be used to compare and measure similarity between different texts.

There are several advantages of using embeddings instead of keyword search when building a chatbot for document exploration:

• Semantic Similarity: Embeddings capture the semantic similarity between sentences, which allows for more accurate and relevant matches between the user's query and the passages in the corpus.
• Handling synonyms and variations: Embeddings can handle synonyms and variations in the query, which are difficult to handle with traditional keyword search. For example, if the user asks "what are the benefits of working overtime?" and the document talks about "compensation for working beyond regular hours", the embeddings will be able to match the query with the relevant passage, even though the words in the query and document are different.
• Handling context: Embeddings can also capture the context in which a word is used, which allows for more accurate matches between the query and the passages in the corpus. For example, "what are the benefits of working overtime?" and "what are the drawbacks of working overtime?" have similar keywords but opposite meaning, embeddings can help to understand the context and give the correct answer.
• Scalability: Embeddings can handle large datasets and can be used to match queries against a large corpus of documents efficiently.

Overall, using embeddings instead of keyword search improves the accuracy and relevance of the chatbot's answers, and provides better user experience.

Match the user's question to the relavant passages in the corpus

Once we have the embeddings, we can find the most similar passages in the document to the given query. The util.semantic_search method to find the most similar passages in the corpus based on the query embedding and the corpus embeddings. The top_k variable controls how many passages to retrieve. After getting the passages from the corpus, they are joined as a whole piece together with linebreak to form the context. The result is as follows:

2.2 Working Hours 3.2 Overtime Compensation Prior approval must be obtained from respective supervisor for working overtime. Overtime for employees at grade 7-10 will be compensated by pay at the following rates: Category 1 a) For non-shift employees, Monday to Friday starting from an hour after normal working hour to 12:00 midnight and Saturday after 2:00 p.m. b) For shift employees, overtime worked beyond his shift 1.5 times hourly rate Overtime for employees at grade 4-6 will be compensated by way of time off. Such compensation leave will only be granted when their department’ s workload permits. Any overtime not compensated in the form of leave by the end of the calendar year will be paid in the following February at the rate of 1.5 times the employee's hourly rate. 3.3 Annual Bonus Employees who work overtime in the following conditions are entitled to claim for meal allowance: - overtime for 3 consecutive hours from Monday to Saturday; - overtime for 6 consecutive hours on Sunday or public holidays Employees who are required to report for duty outside office hours in case of emergency will be entitled to an emergency allowance and compensation leave 4.5 Compensation Leave or Time-off for Overtime Work Employees at grade 4-6 who have worked overtime will be compensated by way of time off. Such compensation leave will only be granted when their department’ s workload permits. Please also refer to “Overtime Compensation” under Section 3.2 of this handbook for details.

You can observed that all passages retrieved are more or less relavant to the overtime question.
 

Create a prompt that includes the context of the question

Then, we can creating a prompt template. The prompt will be created and send for GPT-3 to generate an answer for the query based on the context. The following prompt will be created

Context: 2.2 Working Hours 3.2 Overtime Compensation Prior approval must be obtained from respective supervisor for working overtime. Overtime for employees at grade 7-10 will be compensated by pay at the following rates: Category 1 a) For non-shift employees, Monday to Friday starting from an hour after normal working hour to 12:00 midnight and Saturday after 2:00 p.m. b) For shift employees, overtime worked beyond his shift 1.5 times hourly rate Overtime for employees at grade 4-6 will be compensated by way of time off. Such compensation leave will only be granted when their department’ s workload permits. Any overtime not compensated in the form of leave by the end of the calendar year will be paid in the following February at the rate of 1.5 times the employee's hourly rate. 3.3 Annual Bonus Employees who work overtime in the following conditions are entitled to claim for meal allowance: - overtime for 3 consecutive hours from Monday to Saturday; - overtime for 6 consecutive hours on Sunday or public holidays Employees who are required to report for duty outside office hours in case of emergency will be entitled to an emergency allowance and compensation leave 4.5 Compensation Leave or Time-off for Overtime Work Employees at grade 4-6 who have worked overtime will be compensated by way of time off. Such compensation leave will only be granted when their department’ s workload permits. Please also refer to “Overtime Compensation” under Section 3.2 of this handbook for details. Answer the following question: Q: what should I do if I worked overtime? A:

You can try a few other forms like:

• Answer the following question. If the answer cannot be found in the context, write "I don't know"
• Answer the following question. If the answer cannot be found in the context, rewrite the question that is more related to the context by starting with "Do you want to ask" and then elaborate the answer

Send the prompt to GPT-3 to generate an answer

The following code is using the OpenAI API to generate a response for the prompt using GPT-3. GPT-3 will come up with an answer that fits the context:

If you are an employee at grade 7-10, you should obtain prior approval from your supervisor and you will be compensated by pay at the rate of 1.5 times your hourly rate. If you are an employee at grade 4-6, you will be compensated by way of time off. Such compensation leave will only be granted when your department’s workload permits. Any overtime not compensated in the form of leave by the end of the calendar year will be paid in the following February at the rate of 1.5 times your hourly rate.

Create a function that wraps up everything

Wrapping the code into a function called answer(query) allows for easy reuse of the code and inputting of different queries. This function takes in a query string, and return the response of the API. This makes it easy to input different queries and get an answer without having to repeat the previous steps every time.

Test the chatbot by creating a question-answer loop

Here, we have an infinite loop that repeatedly prompts the user for a query, and exits the loop when the user enters "xxx". You can input multiple queries and get answers without having to restart the code. It allows easy testing of the function by allowing the user to input different queries and see the answers generated by GPT-3. The output will be as follows:

Q: do I get paid if I got sick? A: Yes, employees who are not able to report for duty due to illness are entitled to full pay sick leave, subject to a maximum of the lesser of 120 days or the number of sickness days that the employee has accumulated according to provisions in the Employment Ordinance. Additionally, employees who suffer from injury arising out of and in the course of employment are entitled to compensation in accordance with the Employee Compensation Ordinance. ====================================================================== Q: what should I do if I got sick? A: If I become ill, I should notify my immediate supervisor as soon as possible and provide a medical certificate if necessary. I am entitled to full pay sick leave, subject to a maximum of 120 days or the number of sickness days I have accumulated according to the Employment Ordinance. If I have to leave during office hours due to sickness or personal reasons, I must obtain prior approval from either my supervisor, department manager or the Human Resources Department. ====================================================================== Q: xxx

Use Gradio to create an interface for the chatbot

Finally, let's use gradio library to create a (GUI) for the answer(query) function defined earlier: Now you have a more user friendly GUI to ask the questions:

Try it out in Colab: