NLU Intents Classification with LLM

NLU intents classification with LLM enables intent classification using a large language model (LLM). This model automatically learns from flow intents and training phrases, reducing the need for manual classification. It is built into the Flow Predict Engine, enhancing accuracy and adaptability in intent recognition.

NOTE: This feature is available as technology preview in DRUID 8.10 and higher. NLU intents classification with LLM currently does not support Named Entity Recognition (NER).

Supported LLM providers

You can choose from the following large language model (LLM) providers:

  • AzureOpenAI
  • OpenAI
  • Google
  • Mistral
  • Mesolitica – MaLLaM - This LLM provider is available in DRUID 9.1 and higher.
  • AWS Bedrock - This LLM provider is available in DRUID 9.5 and higher. Contact your DRUID representative to activate it on your tenant.

DRUID-dedicated LLM resources

  • Druid Becus 3.0 / 1.0 (Proprietary LLM)
  • Azure OpenAI - gpt-4o-mini

  • AwsBedrock - mistral-large-2407-v1:0

  • Google Vertex AI

If you want to use DRUID-dedicated LLM resources, contact your sales representative to activate them for your tenant.

How It Works

When NLU intents classification with LLM is configured, the system leverages a LLM for intent classification using two key prompts:

  • System prompt: Instructs the LLM to classify a user query by scoring provided intents based on relevance, adhering to strict JSON formatting and predefined scoring rules while considering both user-supplied and system intents.
  • User prompt: Provides the intent list and user query, ensuring the model has the necessary context for classification.

Configure NLU Intents Classification with LLM

NOTE: Before you start, make sure you have an LLM resource set up in the Druid Portal. For more information, see Create LLM Resources.

To configure NLU intents classification with LLM, follow these steps:

  1. Navigate to NLU > Configurations > Intents tab.
  2. Click on Thresholds and parameters.
  3. Add the NLU.NER.Classification.ModelType parameter and set it to LLM.

    4. Click the link that appears in the message below the NLP parameters.

    The NLU Intents Classification with LLM expands.

  4. From the Endpoint Type field, select one of the following options:
    • Druid if you have an LLM subscription with Druid.
    • My resource if you are using your own generative endpoint.
  5. Select the Model Name. In the dropdown, you will see all your resources created in Administration > LLM Resources.

  6. For both prompts, click the Set default button. The default Druid prompts will be automatically filled in.
    7. HINT: Since this feature is in technology preview, you can experiment with the prompts, provided their JSON structure remains unchanged. In an upcoming release, the prompts will become non-editable.
  7. Click Save to apply the NLU configuration.
  8. Train the bot.

To configure NLU intents classification with LLM, follow these steps:

  1. Navigate to NLU > Configurations > Intents tab.
  2. Click on Thresholds and parameters.
  3. Add the NLU.NER.Classification.ModelType parameter and set it to LLM.

    4. Click the link that appears in the message below the NLP parameters. The NLU Intents Classification with LLM expands.

  4. From the Endpoint Type field, select Druid if you have a LLM subscription with DRUID or Custom if you're using your own generative endpoint.
  5. Select the Client Type.
  6. For custom endpoints, enter the URL of the generative API.
  7. Provide the secret key generated in your generative account.
  8. Specify the generative model name.
  9. If you selected Google as Client Type, enter the Location of the location for a Vertex AI and your Google Project Id.
    10. NOTE: Google Vertex AI is available in DRUID 8.13 and higher.
  10. For both prompts, click the Set default button. The default DRUID prompts will be automatically filled in.
    11. HINT: Since this feature is in technology preview, you can experiment with the prompts, provided their JSON structure remains unchanged. In an upcoming release, the prompts will become non-editable.
  11. Click Save to apply the NLU configuration.
  12. Train the bot.

Once configured, the model uses the two prompts to classify user intents automatically.

IMPORTANT! When using NLU intents classification with LLM, it’s important to separate natural language training phrases from technical commands. To avoid introducing noise into intent detection, move any technical training phrases—such as exact keywords, system triggers, or command-like inputs — into the Commands section of the respective flow. These phrases will then be matched exactly, without affecting the NLP model's understanding of user intent. Commands are available in DRUID 8.14 and higher.