fix(openai): wrap streamed response

[Yun-Kim]

Fixes #9077.

This PR changes the way we patch streamed completion/chat responses in the OpenAI integration. Previously we had been returning generator functions that yielded the value of each chunk. However, this was incorrect as OpenAIStream/OpenAIAsyncStream objects can be used as context managers as of openai>=1.6.0, which means returning a generator function to replace OpenAIStream would break user applications that tried to access stream responses as context managers:

with openai.OpenAI().completions.create(..., stream=True) as response:
    for chunk in response:
        print(chunk)

There is no introduced/changed functionality other than the fact that we now returned a wrapped TracedOpenAIStream object instead of the previous traced generator function.

Note: This PR may seem much larger than it really is, we also updated the tested versions of OpenAI so the riot requirement lockfiles have been changed.

Checklist

• Change(s) are motivated and described in the PR description
• Testing strategy is described if automated tests are not included in the PR
• Risks are described (performance impact, potential for breakage, maintainability)
• Change is maintainable (easy to change, telemetry, documentation)
• Library release note guidelines are followed or label changelog/no-changelog is set
• Documentation is included (in-code, generated user docs, public corp docs)
• Backport labels are set (if applicable)
• If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

• Title is accurate
• All changes are related to the pull request's stated goal
• Description motivates each change
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Change is maintainable (easy to change, telemetry, documentation)
• Release note makes sense to a user of the library
• Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: yunkim/openai-fix-streamed-responses
Commit report: 0182cc7
Test service: dd-trace-py

✅ 0 Failed, 118389 Passed, 57257 Skipped, 4h 26m 34.21s Total duration (4h 58m 54.85s time saved)
❄️ 1 New Flaky

New Flaky Tests (1)

• test_tracer_hooks - test_processor.py - Last Failure

  Expand for error

   STDERR: Expected [] got [b'sent 389B in 2.34969s to http://localhost:9126/v0.5/traces\n']
  

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-05-21 20:21:00

Comparing candidate commit 1fa5dcb in PR branch yunkim/openai-fix-streamed-responses with baseline commit 52b0c59 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 201 metrics, 9 unstable metrics.

[lievan]

Looks good!

I think we also need to implement __next__ in TracedOpenAIStream/AsyncStream

Currently getting:

    chunk1 = next(resp)
TypeError: 'TracedOpenAIStream' object is not an iterator

[github-actions]

The backport to 2.8 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.8 2.8
# Navigate to the new working tree
cd .worktrees/backport-2.8
# Create a new branch
git switch --create backport-9321-to-2.8
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0e8a2579af9b8d28e7c8c39ece51c506417708b0
# Push it to GitHub
git push --set-upstream origin backport-9321-to-2.8
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.8

Then, create a pull request where the base branch is 2.8 and the compare/head branch is backport-9321-to-2.8.

[github-actions]

The backport to 2.9 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.9 2.9
# Navigate to the new working tree
cd .worktrees/backport-2.9
# Create a new branch
git switch --create backport-9321-to-2.9
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0e8a2579af9b8d28e7c8c39ece51c506417708b0
# Push it to GitHub
git push --set-upstream origin backport-9321-to-2.9
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.9

Then, create a pull request where the base branch is 2.9 and the compare/head branch is backport-9321-to-2.9.

[github-actions]

The backport to 2.8 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.8 2.8
# Navigate to the new working tree
cd .worktrees/backport-2.8
# Create a new branch
git switch --create backport-9321-to-2.8
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0e8a2579af9b8d28e7c8c39ece51c506417708b0
# Push it to GitHub
git push --set-upstream origin backport-9321-to-2.8
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.8

Then, create a pull request where the base branch is 2.8 and the compare/head branch is backport-9321-to-2.8.

[github-actions]

The backport to 2.9 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.9 2.9
# Navigate to the new working tree
cd .worktrees/backport-2.9
# Create a new branch
git switch --create backport-9321-to-2.9
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0e8a2579af9b8d28e7c8c39ece51c506417708b0
# Push it to GitHub
git push --set-upstream origin backport-9321-to-2.9
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.9

Then, create a pull request where the base branch is 2.9 and the compare/head branch is backport-9321-to-2.9.