Identifying customers making LLM calls from your applications helps -

Set different token policies to different customers. For example, how many tokens they can use when trialing your software
Limit token usage if malicious actors try and use your LLM features
Turn on and off some customers based on product levels
Shed some lower-tier customers in peak times or during LLM vendor outages (shedding means making LLM unavailable for certain users)
Track analytics and usage by customer.
Set the geographic restrictions to what service regions this customer needs to use. For example, only EU hosting locations.

Identifying the Customer

When creating the useLLM hook, in addition to your specific project_id, you can optionally pass a unique customer_id and customer_name.

We suggest the customer_id be a UUID or whatever unique identifier you use in your user database, and the name is just for reference when we show the customer to you for managing them. We expect that the customer will be a company, not an individual user, but that is up to you by choosing what customer_id you send (a company id for multiple individual users, or individual user ids)

const { send, response, idle } = useLLM(
    { project_id: [your project] 

     // optional - identifies your customer in the control panel 
     customer: { 
       customer_id: "123", 
       customer_name: "Our Best Customer, Inc.", 
     } 
   });

How we use these fields -

If this is the first time that customer_id has made a call, then a new customer is created in your project with the name specified. These new customers will get the project defaults for tokens and load shedding tier. You defined these in the project settings -

If the project settings were as shown, the new customer gets 50000 tokens and is set to the middle load shedding tier.

If YOU DON'T want new customers to have any access until approved, set the "New Customer Initial Token Count" to 0 or blank. In this case, those customers will get a "No tokens" message until you add tokens to their account.

Managing Customers

Customer can be viewed from the Customers page.

You can edit the customer details and click save.