Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cerebras Integration #3585

Merged
merged 4 commits into from
Sep 30, 2024
Merged

Conversation

henrytwo
Copy link
Contributor

@henrytwo henrytwo commented Sep 30, 2024

Why are these changes needed?

Add integration for Cerebras, which provides super low-latency, high speed LLM inference. Currently Llama 3.1-8B/70B are supported.

Tool calling examples have been provided in this PR, which also function with streaming. Additionally, token cost calculations have been implemented.

This PR mirrors the changes from: autogenhub#48

Checks

@henrytwo henrytwo mentioned this pull request Sep 30, 2024
3 tasks
@henrytwo
Copy link
Contributor Author

cc: @jackgerrits @afourney

@gagb gagb requested review from gagb and afourney September 30, 2024 18:52
@henrytwo
Copy link
Contributor Author

Anyone know why these files that I didn't touch are failing the linter? https://github.com/microsoft/autogen/actions/runs/11111974198/job/30874062565?pr=3585#step:6:193

The first few seem legit, but I didn't touch any HTML stuff

@henrytwo henrytwo force-pushed the henrytu/cerebras-integration branch from 3ea6961 to 9a25348 Compare September 30, 2024 19:02
Copy link
Member

@afourney afourney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look great, and consistent with how other client providers are implemented. I do not have means to test the changes end-to-end, but what I see looks good. I believe there is some small format fixed that are needed to pass the test, but on substance this looks good.

@jackgerrits
Copy link
Member

Ah the HTML thing is a line ending issue in main I am pretty sure, I'll open a PR for it

@henrytwo
Copy link
Contributor Author

Ah the HTML thing is a line ending issue in main I am pretty sure, I'll open a PR for it

Is there something I can do to fix it locally? It's actually breaking my git right now 😢

@jackgerrits
Copy link
Member

jackgerrits commented Sep 30, 2024

Ah the HTML thing is a line ending issue in main I am pretty sure, I'll open a PR for it

Is there something I can do to fix it locally? It's actually breaking my git right now 😢

Yeah I had the same thing happen to me... I'm not sure. I'll have what I think is the fix merged in a few mins (just waiting on an approval)

@henrytwo henrytwo force-pushed the henrytu/cerebras-integration branch from 9a25348 to 6bc31ad Compare September 30, 2024 19:21
@jackgerrits
Copy link
Member

Cool, looks like the HTML file issue is gone, just some other formatting issues to be resolved then we can merge this

@henrytwo
Copy link
Contributor Author

@jackgerrits can you approve the workflow please? I ran pre-commit locally to format the files

@gagb gagb enabled auto-merge September 30, 2024 21:10
@gagb gagb added this pull request to the merge queue Sep 30, 2024
Merged via the queue into microsoft:main with commit 3fdf8de Sep 30, 2024
36 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants