Recap for October 23rd, 2024 - Semantic Kernel Office Hours (US/EMEA)

[sophialagerkranspandey]

Semantic Kernel Office Hours for US/EMEA - October 16th, 2024 recording: https://aka.ms/SKOfficeHoursRecordings

If you weren't able to attend office hours but still have questions, please post them here: https://github.com/microsoft/semantic-kernel/discussions

Meeting notes:

• Office Hours Announcements:

• Sophia announced the weekly office hours schedule, including the new .NET release with structured outputs and the Python package update for streaming OpenAI systems. She also mentioned updates to the documentation and the backlog by bucket.

  • Office Hours Schedule: Sophia reminded everyone about the weekly office hours from 8:00 to 9:00 AM Pacific Time and the APAC office hours on the second Wednesday of every month from 4:00 to 5:00 PM Pacific Time.
  • .NET Release: Sophia highlighted the new .NET release, emphasizing the added example of structured outputs with Azure OpenAI. She recommended that attendees review the release details.
  • Python Package Update: Sophia reiterated the Python package update, which includes support for streaming OpenAI systems and entropic function calling updates. She encouraged attendees to explore the new features.
  • Documentation Updates: Sophia mentioned ongoing updates to the documentation, including vector search and other pieces. She pointed out the updated table of contents and provided a link in the chat.
  • Backlog and Roadmap: Sophia informed attendees about the backlog by bucket, which includes different items and the roadmap. She encouraged attendees to review it for a better understanding of the team's plans.

• DAPR Runtime Updates:

  • Ben demonstrated updates to the DAPR runtime, including a demo on how to run it and add processes to a program. He highlighted the simplicity of adding processes to DAPR and the stateful nature of the steps.
  • DAPR Runtime Demo: Ben demonstrated the DAPR runtime, showing how to run it and add processes to a program. He provided a step-by-step walkthrough of the demo, including the code required to add processes to DAPR.
  • Adding Processes: Ben explained that adding processes to DAPR requires minimal code, primarily involving the addition of actors needed to run processes. He highlighted the simplicity of the process.
  • Stateful Steps: Ben showcased the stateful nature of the steps in DAPR, explaining how initial state can be set up and maintained across process runs. He provided examples of stateful steps and their behavior during execution.
  • Demo Execution: Ben ran the demo, showing the logs and the process execution. He explained the significance of stateful steps and how they retain state across multiple runs.

• Process Framework and Semantic Kernel: Garry inquired about the connection between the process framework and semantic kernel. Ben and Evan explained that the process framework is built around kernel functions, making it easy to organize processes and use AI to direct the process.

• Framework Connection: Ben explained that the process framework is built around kernel functions, which are the units of work in the semantic kernel. This integration allows for easy organization of processes and the use of AI to direct the process flow.

• Kernel Functions: Ben emphasized that kernel functions are central to the process framework, enabling the creation of processes that leverage AI and integrations. This approach simplifies the development of complex workflows.

• AI Integration: Evan added that the process framework allows for the use of AI to help decide the next steps in a process. This includes function calling within the process to determine the appropriate flow based on AI insights.

• Multi-Agent Support: Evan highlighted that the process framework supports multi-agent scenarios, where agents can perform tasks, create artifacts, and review work. This capability enhances the flexibility and power of the framework.

• Messaging Integration with Cloud Events: Garry asked about integrating existing messaging platforms with the process framework. Ben mentioned that they plan to integrate with cloud events, which is used by many messaging platforms, to facilitate communication.

• Cloud Events Integration: Ben explained that the process framework will integrate with cloud events, an industry standard used by many messaging platforms. This integration will facilitate communication between the process framework and existing messaging systems.

• Lightweight Communication: Ben mentioned that the integration with cloud events will be lightweight, allowing for seamless communication without the need to build extensive infrastructure. This approach aims to simplify the integration process.

• Telemetry and Streaming: Brett asked about tracking token usage in streaming scenarios. Dmytro explained that token usage information comes with the last chunk in streaming and offered to create an example for logging in streaming scenarios.

• Token Usage Tracking: Dmytro explained that in streaming scenarios, token usage information is included in the last chunk of the stream. This information can be used to track token usage effectively.

• Logging Example: Dmytro offered to create an example for logging token usage in streaming scenarios, similar to the existing example for non-streaming scenarios. This example will help users implement effective logging for streaming.

• Telemetry Integration: Brett mentioned using Azure App Insights for telemetry, including token usage and function duration. Dmytro confirmed that the built-in telemetry in semantic kernel supports both streaming and non-streaming scenarios.
  - Content Filter Results Issue:

• Brett reported an issue with the content filter results for prompt class missing after upgrading to 1.24.1. Mark and Roger clarified that the issue might be related to the Azure OpenAI SDK and recommended aligning the SDK versions.

• SDK Alignment: Mark and Roger clarified that the issue might be related to the Azure OpenAI SDK. They recommended aligning the SDK versions used in the project with the versions supported by semantic kernel to resolve the issue.

• Beta Dependency: Roger explained that the latest release of semantic kernel depends on the beta version of the Azure OpenAI SDK to avoid breaking changes. This dependency ensures continued support for experimental features like agents.

• Handling Long Task Lists in Chat Completion: Rasmus asked about managing long task lists in chat completion without losing history. Chris suggested implementing a custom history producer interface and mentioned plans to address this in the backlog.

• Date Handling in Prompts: Rasmus inquired about handling dates in prompts, particularly for overdue tasks. Mark and Evan suggested creating an issue for further investigation and mentioned the importance of ensuring the agent knows the current date.

• Workshop on Semantic Kernel: Jason announced a workshop in Boston focusing on getting started with semantic kernel, including RAG scenarios and the agent framework. He sought reference workshops and examples to build a cohesive agenda.

• Text Chunking Libraries: Romeo asked about additional libraries for text chunking in semantic kernel. Evan recommended the Kernel Memory library, which supports various chunking methods and is widely used.

[davidldennison]

🚀