Intent Detention is really bad

Description

Hello Team,

I am newbie in RASA NLU and have started exploring for my POC. I have lot of doubts in intent & entity detention.

With below example my model intent is detecting wrong? I still don’t understand how dietclassifer detect intent, does it detect intent based on matched keywords or based on meaning?

When user ask question about ‘can you bring me cup of water’ it is detecting as get_batch_status intent? i feel its detecting because my one of training example contains ‘can you please’. I am very confused on it, how to move on it.

can anyone please advice me on it, whether i have to

  1. modify training examples? if yes how should i take care?

  2. Can we try custom intent classifer (any pre-trained model/ LLM’s/ gpt 3.5) model instead of DietClassifier? if yes how to do it? (or) what should be next?

Please someone help me with this

This is my nlu.yml:

  • intent: get_batch_status examples: |

    • What is the current status of the batch?

    • Can you provide an update on the batch status?

    • Please share the latest status of the batch.

    • Could you provide information about the current batch status?

    • What’s the current status of the batch?

    • I’d like to know the current batch status. Could you share that?

    • Can you give me an overview of the batch status at the moment?

    • What’s the current update on the batch status?

    • What is the current batch status?

    • What’s the estimated time of completion for the batch?

    • Can you tell me when my job will be completed?

    • How long will it take to finish this job?

    • I need to know the Estimated Time of Completion (ETC) for this job.

    • What’s the expected completion time for my job?

    • When can I expect this job to be done?

    • Tell me the estimated completion time.

    • Give me an idea of when this will be finished.

    • Provide the Estimated Time of Completion (ETC) for my job.

    • What is the Estimated Time of Completion (ETC)?

    • What’s the progress of the batch?

    • Do you have an update on the batch’s status?

    • How much longer until my job is finished?

    • When is my task expected to be completed?

    • Give me a status report on the batch.

    • What’s my batch status?

  • intent: inform examples: |

    • ABCS

    • lcrse

    • lcedr

    • Lcrtr

    • CDBoM

    • Mas63iem

    • csmwed

    • MCOre

    • SECpeieD

  • intent: fallback examples: |

    • fallback

    • I don’t understand

    • Can you repeat that?

This is stories.yml

  • story: Greet, Get Batch status details from Form steps:

    • intent: greet

    • action: utter_greet

    • intent: get_batch_status

    • action: batch_details_form

    • active_loop: batch_details_form

    • slot_was_set:

      • requested_slot: appcode

    • intent: inform

    • slot_was_set:

      • appcode: ‘ABC’

    • action: utter_ask_wait

    • action: action_show_app_batch_status

This is my Config.yml

pipeline:

  • name: SpacyNLP model: “en_core_web_md-3.4.1\en_core_web_md\en_core_web_md-3.4.1” case_sensitive: False

  • name: SpacyTokenizer “intent_tokenization_flag”: False “intent_split_symbol”: “_” “token_pattern”: None

  • name: SpacyFeaturizer pooling: mean

  • name: RegexFeaturizer “case_sensitive”: True “use_word_boundaries”: True

  • name: LexicalSyntacticFeaturizer

  • name: CountVectorsFeaturizer

  • name: CountVectorsFeaturizer analyzer: char_wb min_ngram: 1 max_ngram: 4

  • name: SpacyEntityExtractor dimensions: [ “PERSON”, “GPE” ]

  • name: DIETClassifier epochs: 200 constrain_similarities: true

  • name: EntitySynonymMapper

  • name: ResponseSelector epochs: 100 constrain_similarities: true

  • name: FallbackClassifier threshold: 0.7 ambiguity_threshold: 0.1

policies:

  • name: MemoizationPolicy

  • name: RulePolicy core_fallback_threshold: 0.3 core_fallback_action_name: “action_default_fallback” enable_fallback_prediction: True

  • name: UnexpecTEDIntentPolicy max_history: 50 epochs: 100

  • name: TEDPolicy max_history: 50 epochs: 200 constrain_similarities: true

Activity

Show:

Details

Assignee

Reporter

Rasa Open Source Version

Rasa SDK Version

Python version

3.9

Operating System

Windows

Difficulty

Hard

Created January 11, 2024 at 3:05 PM
Updated January 11, 2024 at 3:05 PM