Skip to content

Conversation

@f14-bertolotti
Copy link
Contributor

Summary

This PR introduces a minimal refactoring to ensure that vLLM shuts down gracefully during cleanup. Without this change, users would encounter the following error at the end of an evaluation:

RuntimeError: There is no current event loop in thread 'MainThread'.

Additionally, I removed the @cache decorator, as it was not functioning correctly and resulted in:

TypeError: 'coroutine' object is not iterable

Co-authored-by: Francesco Bertolotti <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants