The home page for the client code packages is at https://www.npmjs.com/package/llmasaservice-client
You can install it using NPM or Yarn -
npm i llmasaservice-client
This allows you to use the streaming responses with all of the benefits of LLMAsAService (failover, caching, PII redaction, customer token budgets, etc) from your application.
import { useLLM } from 'useLLM';
...
// get the project_id from the Embed page in the control panel ...
const {send, response, idle} = useLLM(
{project_id: "[your LLMAsAService project id]"});
const handleChatClick = () => {
send("What is 1+1="); // calls the LLMs for a streaming response };
// the response is streamed back and can be shown where needed
return (
<div>
<button onClick={handleChatClick} disabled={!idle}> Call Chat </button>
<div>{response}</div>
</div>);