cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Flow Basics: Natural Language Processing

Flow Basics: Natural Language Processing

The Flow NLP engine allows your chatbot to understand natural language. The two tasks the NLP can perform is Intent recognition and Entity extraction. Intent recognition can be used to map a user utterance to a predefined bot reply. Entity extraction is used to extract specific data from a user utterance. For these tasks to be successful you'll need to add training data.

Intent classification

The core of well functioning conversational AI is intent classification. Intent classification is the process of understanding what a user means by the text they type.

For example, the user utterance I want to fly from Amsterdam to San Francisco can be mapped to the intent of booking a flight.

Our Natural Language Processing (NLP) takes care of intent classification but in order to function it needs to be trained with examples that need to be provided by the conversational AI developer.

Adding a Text Trigger lets you train an intent. When you click the Train button you can add examples for this intent.

 

 

Tut_hello.gif

Adding more examples enables the classifier to make better predictions. Always make sure you add a least ten examples that are diverse in form, but consistent in meaning. When you have a sufficient amount of examples you can click Save. This will save your examples and initiate the training process.

Note: Best practice is at least 10 examples

When you add fewer than 5 examples, only exact matching will be used.

Reusing intents

A great way to speed up the development process is reusing intents. This also improves the quality of intent classification.

 

flows-reusing-intents1.png

Every intent you create automatically becomes part of you intent library. You can drag and drop any existing intent in other flows to give different answers depending on the context.

Tip: Never create duplicate intents

Make sure you do not create semantically similar intents: if two intents are very similar, the probability of misclassification rises. In that case the best practice is to create a more general intent and then ask follow up questions.

Reuse example

Start by creating an intent to understand greetings (hi, hello, hi there, hey, etc). After training this intent with examples, reuse it by dragging and dropping it from the library.

Now, if a user greets, the system will reply Hi! How can I help you?. Next, if the user greets again. The system knows that he already has greeted, so it can reply with Hello again

Follow up intents

The NLP engine works best if you provide clear examples. In some cases you can also train more generic intents.

For example, if a users says the closest without any context, it doesn't mean much. But if the user asks it after I'm searching for a place to park my car, then it becomes relevant. The AI engine makes it possible to create and train these intents in context, and then reuse them in other places.

So, if you have a flow on finding parking spaces, training the followup intent 'the closest' makes sense. You can then easily reuse this intent in a flow on finding restaurants or toilets.

Confusion

When designing your flows it's good to think about the different use cases your bot will serve. One of the biggest challenges is keeping your project from offering too many identical intents that will confuse the NLP.

This can be best explained with an example.

Example

Imagine a company providing support for customers having problems with phones and dish washers. Business wise it might mean each product would result in a different answer. A broken phone might be picked up, while a broken dish washer requires a repair man to make an appointment,

Naturally we tend to split these repair questions into different flows, introducing a number of intents for repair questions regarding phones and dishwashers.

The training data for the intents might look like:

Repair phone intent

  • Hi, I have a broken screen on my new phone, can you fix this?
  • My phone is broken, I want it repaired
  • The phone of my wife hampers and it wont start anymore
  • ...

Repair dishwasher intent

  • Hi, I have a broken door on my new dishwasher, can you fix this?
  • My dishwasher is broken, I want it repaired
  • The dishwasher of my wife hampers and it wont start anymore
  • ...
 

intent-duplicate-intent2.png

Can you tell the difference? The problem is this will result in confusing the NLP because the example sentences are to similar and would make classification unreliable.

Merge intents by subject

One way to solve this issue is by combining the two intents into a single repair intent:

Repair intent

  • Hi, I have a broken screen on my new phone, can you fix this?
  • My phone is broken, I want it repaired
  • The dishwasher of my wife hampers and it wont start anymore
  • ...
 

intent-one-intent3.png

With a single intent you would mark the products as entity types. Next you can use the extracted product within reply actions and conditions to differentiate the follow up flows.

Explanation

The NLP engine interprets sentences differently than humans.

To us I have an issue with my mother-in-law has a totally different meaning than I might have some problem with the television. To the system both sentences are about someone having a problem with a certain object (no pun intended).

Since we cannot prevent this from happening, it's best to create intents based on subjects.

Read more

Other best practices

Balance the number of examples across intents

Try not to create multiple intents with just 10 examples and a number of intents with over 100 example sentences. It's better to keep intents balanced with around an equal amount of training examples.

The infamous Hello intent

When there is an intent with many short examples (often a 'hello' or greeting intent), short user utterances that are unclear are often misclassified as belonging to this 'hello' intent. Often it can be solved by turning off NLP for this intent.

Limit the number of entry intents

More intents results in a lower degree of classification accuracy, because the probability of choosing the wrong intent increases. Consider keeping the number of intents that can be matched at the entry level of a flow to a bare minimum. If your use case requires hundreds of entry intents, try to merge as many intents into the the same topic.

Keep your intents clean.

On the other hand, make sure you don't merge intents that are not semantically similar. Keep your intents clean and clear!

 

Flow Basics: Natural Language Processing

The Flow NLP engine allows your chatbot to understand natural language. The two tasks the NLP can perform is Intent recognition and Entity extraction. Intent recognition can be used to map a user utterance to a predefined bot reply. Entity extraction is used to extract specific data from a user utterance. For these tasks to be successful you'll need to add training data.

Intent classification

The core of well functioning conversational AI is intent classification. Intent classification is the process of understanding what a user means by the text they type.

For example, the user utterance I want to fly from Amsterdam to San Francisco can be mapped to the intent of booking a flight.

Our Natural Language Processing (NLP) takes care of intent classification but in order to function it needs to be trained with examples that need to be provided by the conversational AI developer.

Adding a Text Trigger lets you train an intent. When you click the Train button you can add examples for this intent.

 

 

Tut_hello.gif

Adding more examples enables the classifier to make better predictions. Always make sure you add a least ten examples that are diverse in form, but consistent in meaning. When you have a sufficient amount of examples you can click Save. This will save your examples and initiate the training process.

Note: Best practice is at least 10 examples

When you add fewer than 5 examples, only exact matching will be used.

Reusing intents

A great way to speed up the development process is reusing intents. This also improves the quality of intent classification.

 

flows-reusing-intents1.png

Every intent you create automatically becomes part of you intent library. You can drag and drop any existing intent in other flows to give different answers depending on the context.

Tip: Never create duplicate intents

Make sure you do not create semantically similar intents: if two intents are very similar, the probability of misclassification rises. In that case the best practice is to create a more general intent and then ask follow up questions.

Reuse example

Start by creating an intent to understand greetings (hi, hello, hi there, hey, etc). After training this intent with examples, reuse it by dragging and dropping it from the library.

Now, if a user greets, the system will reply Hi! How can I help you?. Next, if the user greets again. The system knows that he already has greeted, so it can reply with Hello again

Follow up intents

The NLP engine works best if you provide clear examples. In some cases you can also train more generic intents.

For example, if a users says the closest without any context, it doesn't mean much. But if the user asks it after I'm searching for a place to park my car, then it becomes relevant. The AI engine makes it possible to create and train these intents in context, and then reuse them in other places.

So, if you have a flow on finding parking spaces, training the followup intent 'the closest' makes sense. You can then easily reuse this intent in a flow on finding restaurants or toilets.

Confusion

When designing your flows it's good to think about the different use cases your bot will serve. One of the biggest challenges is keeping your project from offering too many identical intents that will confuse the NLP.

This can be best explained with an example.

Example

Imagine a company providing support for customers having problems with phones and dish washers. Business wise it might mean each product would result in a different answer. A broken phone might be picked up, while a broken dish washer requires a repair man to make an appointment,

Naturally we tend to split these repair questions into different flows, introducing a number of intents for repair questions regarding phones and dishwashers.

The training data for the intents might look like:

Repair phone intent

  • Hi, I have a broken screen on my new phone, can you fix this?
  • My phone is broken, I want it repaired
  • The phone of my wife hampers and it wont start anymore
  • ...

Repair dishwasher intent

  • Hi, I have a broken door on my new dishwasher, can you fix this?
  • My dishwasher is broken, I want it repaired
  • The dishwasher of my wife hampers and it wont start anymore
  • ...
 

intent-duplicate-intent2.png

Can you tell the difference? The problem is this will result in confusing the NLP because the example sentences are to similar and would make classification unreliable.

Merge intents by subject

One way to solve this issue is by combining the two intents into a single repair intent:

Repair intent

  • Hi, I have a broken screen on my new phone, can you fix this?
  • My phone is broken, I want it repaired
  • The dishwasher of my wife hampers and it wont start anymore
  • ...
 

intent-one-intent3.png

With a single intent you would mark the products as entity types. Next you can use the extracted product within reply actions and conditions to differentiate the follow up flows.

Explanation

The NLP engine interprets sentences differently than humans.

To us I have an issue with my mother-in-law has a totally different meaning than I might have some problem with the television. To the system both sentences are about someone having a problem with a certain object (no pun intended).

Since we cannot prevent this from happening, it's best to create intents based on subjects.

Read more

Other best practices

Balance the number of examples across intents

Try not to create multiple intents with just 10 examples and a number of intents with over 100 example sentences. It's better to keep intents balanced with around an equal amount of training examples.

The infamous Hello intent

When there is an intent with many short examples (often a 'hello' or greeting intent), short user utterances that are unclear are often misclassified as belonging to this 'hello' intent. Often it can be solved by turning off NLP for this intent.

Limit the number of entry intents

More intents results in a lower degree of classification accuracy, because the probability of choosing the wrong intent increases. Consider keeping the number of intents that can be matched at the entry level of a flow to a bare minimum. If your use case requires hundreds of entry intents, try to merge as many intents into the the same topic.

Keep your intents clean.

On the other hand, make sure you don't merge intents that are not semantically similar. Keep your intents clean and clear!

 

Flow Basics: Natural Language Processing

The Flow NLP engine allows your chatbot to understand natural language. The two tasks the NLP can perform is Intent recognition and Entity extraction. Intent recognition can be used to map a user utterance to a predefined bot reply. Entity extraction is used to extract specific data from a user utterance. For these tasks to be successful you'll need to add training data.

Intent classification

The core of well functioning conversational AI is intent classification. Intent classification is the process of understanding what a user means by the text they type.

For example, the user utterance I want to fly from Amsterdam to San Francisco can be mapped to the intent of booking a flight.

Our Natural Language Processing (NLP) takes care of intent classification but in order to function it needs to be trained with examples that need to be provided by the conversational AI developer.

Adding a Text Trigger lets you train an intent. When you click the Train button you can add examples for this intent.

 

 

Tut_hello.gif

Adding more examples enables the classifier to make better predictions. Always make sure you add a least ten examples that are diverse in form, but consistent in meaning. When you have a sufficient amount of examples you can click Save. This will save your examples and initiate the training process.

Note: Best practice is at least 10 examples

When you add fewer than 5 examples, only exact matching will be used.

Reusing intents

A great way to speed up the development process is reusing intents. This also improves the quality of intent classification.

 

flows-reusing-intents1.png

Every intent you create automatically becomes part of you intent library. You can drag and drop any existing intent in other flows to give different answers depending on the context.

Tip: Never create duplicate intents

Make sure you do not create semantically similar intents: if two intents are very similar, the probability of misclassification rises. In that case the best practice is to create a more general intent and then ask follow up questions.

Reuse example

Start by creating an intent to understand greetings (hi, hello, hi there, hey, etc). After training this intent with examples, reuse it by dragging and dropping it from the library.

Now, if a user greets, the system will reply Hi! How can I help you?. Next, if the user greets again. The system knows that he already has greeted, so it can reply with Hello again

Follow up intents

The NLP engine works best if you provide clear examples. In some cases you can also train more generic intents.

For example, if a users says the closest without any context, it doesn't mean much. But if the user asks it after I'm searching for a place to park my car, then it becomes relevant. The AI engine makes it possible to create and train these intents in context, and then reuse them in other places.

So, if you have a flow on finding parking spaces, training the followup intent 'the closest' makes sense. You can then easily reuse this intent in a flow on finding restaurants or toilets.

Confusion

When designing your flows it's good to think about the different use cases your bot will serve. One of the biggest challenges is keeping your project from offering too many identical intents that will confuse the NLP.

This can be best explained with an example.

Example

Imagine a company providing support for customers having problems with phones and dish washers. Business wise it might mean each product would result in a different answer. A broken phone might be picked up, while a broken dish washer requires a repair man to make an appointment,

Naturally we tend to split these repair questions into different flows, introducing a number of intents for repair questions regarding phones and dishwashers.

The training data for the intents might look like:

Repair phone intent

  • Hi, I have a broken screen on my new phone, can you fix this?
  • My phone is broken, I want it repaired
  • The phone of my wife hampers and it wont start anymore
  • ...

Repair dishwasher intent

  • Hi, I have a broken door on my new dishwasher, can you fix this?
  • My dishwasher is broken, I want it repaired
  • The dishwasher of my wife hampers and it wont start anymore
  • ...
 

intent-duplicate-intent2.png

Can you tell the difference? The problem is this will result in confusing the NLP because the example sentences are to similar and would make classification unreliable.

Merge intents by subject

One way to solve this issue is by combining the two intents into a single repair intent:

Repair intent

  • Hi, I have a broken screen on my new phone, can you fix this?
  • My phone is broken, I want it repaired
  • The dishwasher of my wife hampers and it wont start anymore
  • ...
 

intent-one-intent3.png

With a single intent you would mark the products as entity types. Next you can use the extracted product within reply actions and conditions to differentiate the follow up flows.

Explanation

The NLP engine interprets sentences differently than humans.

To us I have an issue with my mother-in-law has a totally different meaning than I might have some problem with the television. To the system both sentences are about someone having a problem with a certain object (no pun intended).

Since we cannot prevent this from happening, it's best to create intents based on subjects.

Read more

Other best practices

Balance the number of examples across intents

Try not to create multiple intents with just 10 examples and a number of intents with over 100 example sentences. It's better to keep intents balanced with around an equal amount of training examples.

The infamous Hello intent

When there is an intent with many short examples (often a 'hello' or greeting intent), short user utterances that are unclear are often misclassified as belonging to this 'hello' intent. Often it can be solved by turning off NLP for this intent.

Limit the number of entry intents

More intents results in a lower degree of classification accuracy, because the probability of choosing the wrong intent increases. Consider keeping the number of intents that can be matched at the entry level of a flow to a bare minimum. If your use case requires hundreds of entry intents, try to merge as many intents into the the same topic.

Keep your intents clean.

On the other hand, make sure you don't merge intents that are not semantically similar. Keep your intents clean and clear!

 
Version history
Last update:
‎06-16-2021 10:01 AM
Updated by:
Contributors