Random Prompt Injection - Ghost User Entries

[FarVision2]

I don't know how else to describe it.

I'm sitting there looking at the window doing nothing and the user prompt will enter all on its own, it'll be something like 'thank you for watching' or '6 radishes' or parts of ingredients of a recipe or a random letter.

I have been using
gpt-4o-mini
and
gemini-2.0-flash-exp

So I don't recall exactly when it happened. Normally I would blame a squirrely new experimental API except there's really no reasoning for a user injection to occur at all. And I've never had it happen with any other tool using their API

I'll try to see if there is an API log on the Gemini side.

[FarVision2]

It's getting worse. I have never seen such a thing. As I'm sitting there hitting the pause agent button:

User message
감사합니다.
(google translate:Thank You)

The user has provided a message in Korean, which I should ignore based on the current behavioral rules.
I need to proceed with the next steps based on the previous context, which was to check the output of the code execution.
The previous code execution was intended to send a test prompt to the front end and check the response.
I need to check the output of the code execution to see if the test was successful and if there were any errors.

User message
오늘도 시청해주셔서 감사합니다.

User message
ご覧いただきありがとうございます。

User message
ご覧いただきありがとうございます。

User message
ご視聴ありがとうございました

The user has provided a Japanese greeting and some context about previous interactions.
I need to acknowledge the greeting and then proceed with the next steps based on the previous context.
The previous context indicates that the front end for the Gemini 2.0 Flash Experimental Model has been successfully built and is accessible at http://0.0.0.0:8000/index.html.
I should now verify the functionality of the front end by sending a test prompt and checking the response.

The user has provided another Japanese greeting, which indicates they are acknowledging the previous action.
I should acknowledge the greeting and then proceed to check the output of the previous code execution.
The previous code execution was intended to send a test prompt to the front end and check the response.
I need to check the output of the code execution to see if the test was successful and if there were any errors.

Non-Stop every 10 seconds or so. I had been using the docker image all day yesterday and it started towards the later half of the day. Restarted via the front end. Had to eventually shut the whole thing down.

Is there such a thing as prompt poisoning? Or is there a vulnerability in the image?

I'm going to run it from the CLI now and see how it goes.

[FarVision2]

Depends on the API you're using I guess

…

On Fri, Dec 27, 2024, 8:37 AM YourTendency ***@***.***> wrote: How do I get the framework to actually be uncensored On Wed, Dec 25, 2024 at 6:07 PM Steve Shelby ***@***.***> wrote: > I don't know how else to describe it. > > I'm sitting there looking at the window doing nothing and the user prompt > will enter all on its own, it'll be something like 'thank you for watching' > or '6 radishes' or parts of ingredients of a recipe or a random letter. > > I have been using > gpt-4o-mini > and > gemini-2.0-flash-exp > > So I don't recall exactly when it happened. Normally I would blame a > squirrely new experimental API except there's really no reasoning for a > user injection to occur at all. And I've never had it happen with any other > tool using their API > > I'll try to see if there is an API log on the Gemini side. > > — > Reply to this email directly, view it on GitHub > <#282>, or unsubscribe > < https://github.com/notifications/unsubscribe-auth/BKRPGIMTLPUZI55JCDN3XV32HM3DJAVCNFSM6AAAAABUGNGKVGVHI2DSMVQWIX3LMV43ASLTON2WKOZSG42TSMJRHAYDOMQ> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > — Reply to this email directly, view it on GitHub <#282 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4IUSELWAA5Q5O7JTD3BRE32HVJZNAVCNFSM6AAAAABUGNGKVGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRTG4YDSOBZGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[3clyp50]

Hi, this is interesting, can you share a section of the chat log if able? There's clearly risks of injection but that is only from the agent itself or who has the credentials in its possess (in the worst scenario).