AutoGen meets SWE_bench

[skzhang1]

Why are these changes needed?

This PR provides a basic implementation for SWE_bench benchmark (https://www.swebench.com). This PR is still on working and not ready for review.

• Specific packages pre-requirement
• Documentations (notebook, blogpost)
• Clean the codes.

🌹 Acknowledgement: The code is greatly on the basis of swe-agent.

Please add more @Hk669 if you have other suggestions. @Hk669 will mainly take response for it based on this draft PR.

Related issue number

N/A

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[gitguardian]

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.

---

^{_{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}}

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 284 lines in your changes missing coverage. Please review.

Project coverage is 19.97%. Comparing base (6279247) to head (03d1afa).
Report is 91 commits behind head on main.

Files	Patch %	Lines
autogen/agentchat/contrib/swebench_agent.py	0.00%	263 Missing ⚠️
autogen/agentchat/contrib/swebench_utils.py	0.00%	21 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2933       +/-   ##
===========================================
- Coverage   32.90%   19.97%   -12.94%     
===========================================
  Files          94       97        +3     
  Lines       10235    10953      +718     
  Branches     2193     2509      +316     
===========================================
- Hits         3368     2188     -1180     
- Misses       6580     8614     +2034     
+ Partials      287      151      -136     

Flag	Coverage Δ
unittests	19.93% <0.00%> (-12.98%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ekzhu]

@skzhang1 would you like to continue working on this draft PR?

[skzhang1]

@ekzhu Hi, I may not working on it recently. This PR provides a basic implementation and it is workable. I hope other people in the community could further work on it based on this PR. @Hk669

[ekzhu]

Thanks. Do you think we can close this and revisit the idea once the new version is merged? It can be part of agbench

[Hk669]

Thanks. Do you think we can close this and revisit the idea once the new version is merged? It can be part of agbench

Sure @ekzhu , we can revisit after the new version. agbench