Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Bot Producing Random and Incorrect Responses ("Ol", "O", "A") - Issue Report #906

Open
3 of 4 tasks
grupocopa opened this issue Dec 16, 2024 · 0 comments
Open
3 of 4 tasks
Labels
bug Something isn't working status: needs triage New issues that have not yet been reviewed or categorized.

Comments

@grupocopa
Copy link

Did you check docs and existing issues?

  • I have read all the NeMo-Guardrails docs
  • I have updated the package to the latest version before submitting this issue
  • (optional) I have used the develop branch
  • I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

Python 3.10.11

Operating system/version

Windows 11

NeMo-Guardrails version (if you must use a specific version and not the latest

0.11.0

Describe the bug

nemo_guardrails_a_error.txt
intents_inside_awnser_error.txt
nemo_guardrails_a_error.txt
Subject: Bot Producing Random and Incorrect Responses ("Ol", "O", "A") - Issue Report

Hello NVIDIA Developer Community,

We are using the NVIDIA NeMo Guardrails framework in a system where a user-facing product interacts with a bot hosted via HTTP requests on a separate backend. While the bot operates as expected initially, it eventually starts producing random, incorrect responses such as "Ol", "O", or completely blank outputs.

This issue persists until we reset the hosting server and thread ID, which temporarily restores normal behavior. However, the need for frequent resets significantly impacts the reliability of our system. When the system resets, all goes well for a short period of time, and the problems surges again and persists until the next reset.

Example Behavior

At an undefined point during operation, instead of returning coherent responses to user inputs, the bot begins responding with seemingly random or nonsensical outputs like:

  • "Ol"
  • "O"
  • Empty or blank responses
  • Repeated system-level warnings and unrelated logs
  • Sending literally intents to user

Details

  • The logs for the affected sessions will be attached to this post for clarity and further investigation.
  • The issue occurs consistently after the bot has been running for some time, though the exact trigger remains unclear.

Attached Logs

For better understanding, we are attaching a selection of logs captured during sessions where these issues occurred. The attached logs highlight examples of:

  1. Incorrect or nonsensical responses.
  2. Warnings and anomalies recorded by the system.

We would greatly appreciate any insights or suggestions from the community regarding the cause and potential resolution for this issue.
nemo_guardrails_a_error.txt
intents_inside_awnser_error.txt

Thank you for your support!

Steps To Reproduce

I don't know. The server is going working well in 1-2 days, and then start to have those problems and errors.

Expected Behavior

The server completely normal like when it started.

Actual Behavior

Bot awnsering random, incorrect or blank responses.

@grupocopa grupocopa added bug Something isn't working status: needs triage New issues that have not yet been reviewed or categorized. labels Dec 16, 2024
@grupocopa grupocopa changed the title bug: Bot Producing Random and Incorrect Responses ("Ol", "O") - Issue Report bug: Bot Producing Random and Incorrect Responses ("Ol", "O", "A") - Issue Report Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working status: needs triage New issues that have not yet been reviewed or categorized.
Projects
None yet
Development

No branches or pull requests

1 participant