This post was originally published in Chatbot's Life
Self-service is undoubtedly an essential part of the customer journey with clear benefits for both the business and its customers. The main component of any self-service strategy is a well structured help center, where customers can find answers to some of their questions without having to get in touch with a customer support agent.
One of the problems of having a help center is that customers often can’t find or (won’t look for) the answer they need, so they go ahead and contact support anyway. Existing solutions that match and suggest knowledge base articles developed by AI vendors active in the customer service space come with a high and variable runtime cost as most vendors would charge per ticket “deflected”.
In this article I will show you how to build a web contact form on Zendesk that uses state-of-the-art AI to suggest relevant articles based on the customer question. The form adds an extra layer of deflection to be used when the customer either doesn’t find what they need or goes straight to the “Get help” button.
Building your own solution for deflection comes with many benefits:
minimum running costs, at a fraction of existing customer service AI vendors (web app hosting + Dialogflow API pricing at 0.2 cents/ticket!)
state of the art AI technology provided by Google
custom build for your own needs, you control your customers experience
The only caveat is that you will need some initial software development resources to implement the contact form and integrate it with Dialogflow. The steps and instructions presented below will help you minimize this cost.
This tutorial is based on the Zendesk platform but with minor modifications can be used with any other helpdesk that offers a programmatic way (API) of creating a new ticket on behalf of the customer. Examples include but are not limited to: Freshdesk, Salesforce, Intercom or Helpscout.
The complete code is available here.
What are you building
Starting from the custom ticket form illustrated in this article, you will add a deflection layer before the user can submit a new ticket. I recommend you go through the steps from the article to understand the basics of creating a new ticket using Zendesk’s Requests API.
The new contact form should perform the following tasks:
Ask the user for the description of the problem
Send the description to Dialogflow to get the most relevant articles from your knowledge base
Display the most relevant articles (if any) and invite the user to contact support if he couldn’t find an answer
If user chooses to contact support, show the create ticket form where the user can submit a new ticket
Before you begin
You should have the following to get started:
Zendesk Support account (any plan)
Google cloud account with Dialogflow API enabled (see instructions here)
Python 3 and packages: flask, requests, dialogflow
You also need basic Python skills and API knowledge (i.e. how to call a REST API from Python).
Tasks covered in this tutorial
build and train a virtual agent in Dialogflow
create the question form (user submits his question)
create the articles page (user sees the most relevant articles to his question)
create the custom contact form (user submits a new ticket)
build the Dialogflow integration
Build and train a virtual agent in Dialogflow
If many of your customers’ questions can be answered by help center articles you are already in a good place with self-service. Now you need to help the customer help himself. So, when a customer is reaching out with a question, you have to quickly find the help center article that answers that question and send it to him. This can be done manually by agents or automatically using AI.
Let’s start by creating a new agent in the Dialogflow console. Think of it as a new (virtual) member of your team, that can be trained to understand certain customer questions and suggest relevant articles. In the console, select “Create new agent” from the left drop-down menu, give it a name (let’s say “Faq_bot”) and leave all the other fields the same (detailed instructions about creating a new agent). This will also create a new project with Google Cloud Platform where the new agent will be allocated to. Go to the settings page and note down the Project ID (instructions here) — you will need this later.
In order to be able to authenticate the Dialogflow API calls, you need to create and download the API credentials. Follow these instructions to generate the service account key file (.json). Make sure the new credentials are for the same Google Project noted down in the previous step. Keep the file at hand, you will need it later.
In this tutorial, you will train the new agent to answer two types of questions that e-commerce merchants are frequently getting: “Where do you ship?” and “Cancel my order”. You need to create one intent for each question type. An intent categorizes a customer’s intention so that when the customer enters a description of the question, Dialogflow matches the description to the best intent in your agent (more about intents here and instructions how to create one here).
Let’s create the first intent and call it “where_do_you_ship”. Now you need to add training phrases or different ways customers can ask the “Where do you ship” question. These are few examples:
I would like to ask if you ship your products to Morocco?
Can you deliver to the republic of Ireland
Do you trade in Australia?
Can I order from Italy?
Would you post to France?
As you can see, there are many different ways of saying the same thing — a notable feature of human language which makes it difficult for machines to understand. Add as many examples as you can here, and make sure it covers a wide range of expressions. The more training data and the more diverse it is, the better — see this guide on AI for customer support. You will come back to this step many times in the future when you’ll notice that certain customer expressions are not understood by the Dialogflow agent.
After teaching the virtual agent how to understand this question, let’s define the response: what the API will return when this intent is matched. You will be calling the agent from a custom web app, so you want the agent to return a format that an app can understand, like a json response. To do this, go to the “Responses” section, remove the default text response, click on “Add responses” and select “Custom Payload”. Add the following json content there pointing to a kb article that includes the answer to the intent question:
{
"title": "Where do you ship?",
"url": "https://example.zendesk.com/hc/en-us/articles/Where-do-you-ship-to"
}Save and add a second intent named “cancel_my_order” with the following training phrases:
Cancel my order
I have just placed an order that I would like to stop please
I don’t want the two books I’ve just ordered anymore
Can I cancel the order please
I want to cancel my order right now
Add another json response for this intent following the same instructions as above:
{
"title": "How do I cancel my order?",
"url": "https://example.zendesk.com/hc/en-us/articles/I-want-to-cancel-my-order"
}Save again and wait for the agent training to finish. Now you’re ready to test the new agent by entering a few expressions in the right hand side panel and check the results. Phrase the question a bit differently than in the training examples, let’s say: “Cancel all my orders”.
Create the question form
The new contact form will be developed in Flask, a lightweight Python library for writing web applications. Create a new folder for this project with the following structure:
Next, add all the UI style details into this file:
static/css/form_styles.css
#ticket_form { font-family: Verdana, Helvetica, sans-serif; font-size: 10pt;
color: dimgray; width: 320px; border: 1px solid lightgray; padding: 0 8px; }
.title { font-family: Arial, Helvetica, sans-serif; font-weight: bold; padding: 12px 0 0; }
form ul { list-style-type: none; padding-left: 0; }
form li { margin: 0 0 6px 0; }
textarea { vertical-align: top; border-color: lightgray; }
.field { width: 100%; }
.register { font-size: 8pt; color: gray; padding: 8px 0; }
.feedback { font-size: 8pt; padding: 8px 0 0; }
.text-normal { font-size: 10pt; }
.margin-top { margin-top: 8px; }
.margin-top-lg { margin-top: 24px; }
.margin-left { margin-left: 8px; }
.margin-bottom { margin-bottom: 8px; }The first step in the user interface is to invite the customer to enter a description of the question or problem he has. Let’s create the templates/ask_question.html file that displays this form:
templates/ask_question.html
<link rel="stylesheet" href="/static/css/form_styles.css">
<div id="ticket_form">
<div class="title">How can we help?</div>
<form action="ask_question" method="post">
<ul>
<li>
<textarea placeholder="ask a question or describe the problem you need help with"
name="description" rows="6" class="field" required></textarea>
</li>
<li><input type="submit" value="Find answers"></li>
</ul>
</form>
</div>The file app.py will run your main Flask application and includes the following code:
app.py
from flask import Flask, render_template, request
app = Flask(__name__)
app = Flask(__name__)
@app.route('/ask_question',methods = ['POST', 'GET'])
def ask_question():
if request.method == 'POST':
# Get the issue description from the form data
description = request.form.get('description')
# do something with the description ...
return render_template('ask_question.html')
if __name__ == '__main__':
app.run()In this file you will also define the first Flask route, which is a way to tell Python to assign an URL to a function. Here, all requests to the ‘/ask_question’ URL will be handled by the ask_question() function which renders and returns the ask_question.html file created above.
Let’s run the first version of the app. From the main project folder execute the following command:
python app.pythen open this link in your browser: http://localhost:5000/ask_question.
You should see this page:
Congratulations, you completed the first task! When the user enters his issue description and clicks on “Find answers”, the browser sends a new POST request to the server using the same URL (/ask_question). At the moment it doesn’t do anything with it, but you will change that in the following section.
Create the articles page
After the user submits his question, you need to find the relevant articles that might answer it and display them in the next page.
Create the new articles page templates/articles.html and add this code:
templates/articles.html
<link rel="stylesheet" href="/static/css/form_styles.css">
<div id="ticket_form">
<div class="title">Query: {{ description }}</div>
<form action="contact_form" method="post">
<input type="hidden" name="description" value="{{ description }}"/>
<div class="text-normal margin-top">
{% if articles %}
<p>We found some articles that might help:</p>
<ul >
{% for article in articles %}
<li class="text-normal margin-left">
<a href="{{article['url']}}" target="_blank">
{{article['title']}}
</a>
</li>
{% endfor %}
</ul>
{% else %}
<p>
We couldn't find any articles for your query.
Please get in touch if you need help.
</p>
{% endif %}
</div>
<div class="margin-top-lg margin-left">
<input type="button" onclick="location.href='/ask_question';" value="Go back">
<input class="margin-left" type="submit" value="Contact support">
</div>
</form>
</div>This file is using the Jinja template format (included with Flask) to display the content dynamically. There are two situations to cover: when relevant articles are found, display them, otherwise let the user know that no articles are matching his description. In both cases, invite the user to either create a new ticket or go back and enter a new question.
To display the articles page you need to handle the case when a POST is sent to /ask_question in the function /ask_question() of app.py:
app.py
from flask import Flask, render_template, request
from dialogflow import get_suggestions
app = Flask(__name__)
@app.route('/ask_question',methods = ['POST', 'GET'])
def ask_question():
if request.method == 'POST':
# Get the issue description from the form data
description = request.form.get('description')
# ask dialogflow for suggestions
articles = get_suggestions(description)
# render the articles page
return render_template('articles.html', articles=articles, description=description)
return render_template('ask_question.html')
# ... before __main__After getting the issue description, the next step is to access the Dialogflow API and ask for suggestions. You’ll do this in a separate function called get_suggestions(). Let’s define the function in a new file dialogflow.py located in the main project folder. For now the function only returns a dummy json response as the complete Dialogflow integration will be done later on:
dialogflow.py
def get_suggestions(text):
return [{
"url": "http://www.example.com",
"title": "Suggestion example",
"confidence": 0.9
}]Let’s test the app now. Stop and start the web server again, then go to http://localhost:5000/ask_question. Enter a question (for example “Do you ship to Canada?”) and click find answers. You should see a page like the one below:
Well done! You’re half-way through now to have a fully functional contact form that uses AI to deflect tickets.
Create the ticket form
If the user can’t find his answer in the suggested articles, he needs to have the option to contact the customer support team. For this he will enter his email address, the ticket subject and the description.
Let’s do the form in this new html template file:
templates/articles.html
<link rel="stylesheet" href="/static/css/form_styles.css">
<div id="ticket_form">
<div class="title">Submit a ticket</div>
{% if feedback %}
<div class="feedback">{{feedback}}</div>
{% endif %}
<form action="create_ticket" method="post">
<ul>
<li>
<input type="email"
placeholder="your email address"
name="email" class="field" required>
</li>
<li class="margin-top">
<input type="text" placeholder="subject"
name="subject" class="field" required>
</li>
<li>
<textarea placeholder="ask a question or describe the problem you need help with"
name="description" rows="6" class="field" required>{{ description }}</textarea>
</li>
<li>
<input type="submit" value="Submit">
</li>
</ul>
</form>
</div>Then add a new endpoint in the app.py file:
app.py
# ... after ask_question()
@app.route('/contact_form',methods = ['POST', 'GET'])
def contact_form():
description = ""
# if POST is called, fill in the description automatically
if request.method == 'POST':
description = request.form.get('description')
return render_template('ticket_form.html', description=description)
# ... before __main__Let’s reduce the customer effort as much as possible, so when the user clicks on “Contact support” in the articles page, the ticket description will be automatically filled in on his behalf from the initial question he entered before.
Once the customer submits the new ticket form, the app needs to use Zendesk’s API to create a ticket on his behalf. Let’s add a new route for this in the app.py file. Here you are calling the requests endpoint as instructed in this article. You also need to add your Zendesk subdomain and API token, which is used to authenticate the API calls to your Zendesk account.
app.py
import json # add new imports
import requests
from flask import Flask, render_template, request
from dialogflow import get_suggestions
app = Flask(__name__)
# add your Zendesk API token and subdomain here
ZENDESK_SUBDOMAIN = 'your zendesk subdomain'
ZENDESK_API_TOKEN = 'your api token'
# ... after contact_form()
@app.route('/create_ticket',methods = ['POST'])
def create_ticket():
success, feedback = True, None
# Get the form data
subject = request.form.get('subject')
description = request.form.get('description')
email = request.form.get('email')
# define the request body
data = {'request': {'subject': subject, 'comment': {'body': description}}}
ticket = json.dumps(data)
# Make the API request to create a new ticket
user = email + '/token'
headers = {'content-type': 'application/json'}
api_url = 'https://' + ZENDESK_SUBDOMAIN + '.zendesk.com/api/v2/requests.json'
r = requests.post(
api_url,
data=ticket,
auth=(user, ZENDESK_API_TOKEN),
headers=headers
)
if r.status_code != 201:
success = False
if r.status_code == 401 or 422:
feedback = 'Could not authenticate you. Check your email address or register.'
else:
feedback = 'Problem with the request. Status ' + str(r.status_code)
# the feedback template will let the user now about the outcome of this action
return render_template('feedback.html', success=success, feedback=feedback)
# ... before __main__Finally, you need to let the user know about the outcome of his action, if a ticket was created successfully or not. Let’s do this in a new template html file:
templates/feedback.html
<link rel="stylesheet" href="/static/css/form_styles.css">
<div id="ticket_form">
{% if success %}
<div class="title">Thank you!</div>
<div class="text-normal margin-top">
Your ticket was created. We will be in touch soon.
</div>
{% else %}
<div class="text-normal">{{ feedback}}</div>
{% endif %}
<div class="margin-top margin-bottom">
<input type="button" onclick="location.href='/ask_question';"
value="Start again">
</div>
</div>Save everything and start again the web app on the command line ( python app.py ). Go to http://localhost:5000/ask_question, enter a question (i.e. Do you ship to Canada?), click “Find answers”, then select to contact support. You should see this page, with the description field already filled in:
Add an email address and subject, then click Submit. You should see this feedback page and a new ticket will be created in your helpdesk.
You did great! You are almost done.
Integrate Dialogflow
In this final step you will complete the articles suggestions page implemented earlier on by calling the Dialogflow API and get the relevant articles suggestions given the customer description of the problem. You will need the project ID and the credentials file generated in the first step (build the virtual agent).
Change the dialogflow.py file created earlier on and implement the get_suggestions() function that calls the API:
dialogflow.py
import os
import random
import dialogflow_v2 as dialogflow
# path to the Google API credentials file
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="google cloud credentials file"
# Google project ID from the agent settings page
PROJECT_ID = "google project id"
# the language your agent is trained on
LANGUAGE_CODE = "en-US"
session_client = dialogflow.SessionsClient()
def get_suggestions(text, session_id=None):
# if no session, create a random session id (default behaviour)
if session_id is None:
session_id = str(random.randint(0, 10000))
# get the dialogflow session
session = session_client.session_path(PROJECT_ID, session_id)
# create a new input using the text description
text_input = dialogflow.types.TextInput(text=text, language_code=LANGUAGE_CODE)
query_input = dialogflow.types.QueryInput(text=text_input)
# call the Dialogflow API to ask for suggested articles given text in input
response = session_client.detect_intent(session=session, query_input=query_input)
result = response.query_result
# if the matching intent is fallback, no suggested articles were found
if result.intent.is_fallback or len(result.fulfillment_messages) == 0:
return None
# return the list of suggested articles as a list of dict
articles = []
for msg in result.fulfillment_messages:
fields = msg.payload.fields
articles.append({"url": fields["url"].string_value,
"title": fields["title"].string_value,
"confidence":
result.intent_detection_confidence})
return articles if len(articles)>0 else NoneThe function takes a string as an input and goes through the following steps:
Create a new Dialogflow session. A session is used to track the conversation with a user over multiple dialog turns. In this example you don’t need to track turns, therefore you will create a new session for each API call (session_id=None)
Call the API to detect the intent for the given input.
If the Fallback intent is returned, the agent didn’t recognized the customer description of the problem, so return None
If at least one non-fallback intent is returned, add all fulfillment messages to the list to be returned. Here messages are the json objects you have added in the first step of this tutorial as an intent response.
Each article also includes the confidence value, representing a number between 0 and 1 indicating how confident the AI is in making this suggestion (the higher the better). You can use this value with a threshold to increase the accuracy of suggestions and only return those with high confidence levels.
Let’s see the results now. Start again the web app on the command line ( python app.py ). Go to http://localhost:5000/ask_question, enter the question “Do you ship to Canada?” and click “Find answers”. You should see the page below including a reference to the article answering this question.
Congratulations! You’ve just built your first ticket deflection contact form using AI!
Considerations
In this tutorial you’ve learned how to create a smart contact form that is using AI to suggest relevant articles to customers based on their issue description. All this comes in a few lines of code and at (almost) no cost.
As you have seen, it is easy to get started with a simple AI solution for customer support that deflects tickets. The part that requires most work is in building a complex model that understands and answers many customer questions. As explained in this article, having the right training data (the customer phrases labeled for each question) is crucial to the success of any AI project. Same is having good explanatory articles that addresses those questions.
There are several ways to expand the contact form above, some of which I’m highlighting below:
Customize the look and feel of your form to suit your brand and design style guide
Add a threshold to the confidence level returned by the Dialogflow API with each response, to improve accuracy
For certain questions, on the articles page show the answer directly on the page rather than inviting the user to click on a link
If you’re getting a large number of email tickets in addition to contact form tickets, you can use a similar approach to automatically send article suggestions when a new email from a customer is received in your helpdesk. You can achieve this through Zendesk’s targets and triggers (trigger the Dialogflow API call for each new ticket received on the email channel).
For inquiries that require some level of troubleshooting, create a more complex conversation flow that guides the user step by step in resolving his issue.
I am certain there are many more other ways to extend the functionality presented in this tutorial.
I’d love to hear your own ideas! What else you think would benefit your customers and your support team?
You can download the complete code from here.
Swifteq is a software development company providing tailored software solutions for customer support teams. Feel free to get in touch if you need our help.