We support streaming for the chat completions API. Limitations: We only support text prompts and completions. We haven't tested every model, but are confident that all models work. We have NOT tests on the Assistants API.

We support streaming for chat completions API. Limitations: We only support text prompts and completions. We estimate token usage (the Microsoft API doesn't yet send token usage in streaming responses)

We support the Anthropic streaming API. We only support text prompts using the messages api. We have tested the endpoint https://api.anthropic.com/v1/messages. Anthropic support token usage in their streaming api.

We support the Gemini streaming api. We have tested on the https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:streamGenerateContent?alt=sse&key={{API-KEY}}. Gemeni supports token usage in their API.

We will expand the tested scenarios as we achieve them here.

Coming soon: Amazon Bedrock. What others do you need? Send us a message and we'll add support.