> ## Documentation Index
> Fetch the complete documentation index at: https://docs.millis.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Web SDK

> Integrate Millis AI's voice agent capabilities directly into your web applications and browser extensions.

## Installation

Install the SDK with npm:

```
npm install @millisai/web-sdk
```

## Usage

Here’s how to quickly set up a voice agent in your web application:

### 1. Import the SDK:

```js theme={null}
import Millis from '@millisai/web-sdk';
```

### 2. Initialize the Client:

```js theme={null}
const msClient = Millis.createClient({publicKey: 'your_public_key', endPoint?: 'region-based-endpoint'});
```

Obtain your public key from your [Millis AI Playground](https://app.millis.ai/settings/keys)

Learn more about which `endPoint` to use [HERE](https://millisai.mintlify.app/core-concepts/regions).

### 3. Start a Conversation:

Starting from version 1.0.15, use the following format to initiate a call:

```js theme={null}
msClient.start({
  agent: {
    agent_id: agentId,          // Optionally pass agent_id
    agent_config: {}            // Optionally pass agent_config
  },
  metadata: {},                 // Optional metadata for personalized context
  include_metadata_in_prompt: true/false,  // Optional flag to include metadata in prompt
  session_continuation: { // Optional session ID for continuation
    session_id: "<previous session id>"
  }
});
```

#### Using a Predefined Agent

First, create a voice agent from the [Playground](https://app.millis.ai/agents). Then, start a conversation with your agent using the following code:

```js theme={null}
msClient.start({
  agent: {
    agent_id: <agent-id>
  }
});
```

Replace `agent-id` with the ID of your agent obtained from the Playground.

The `metadata` is optional. You can pass any additional data to the session, which we will forward to your custom LLM and function webhooks.
If you provide `metadata`, you can make it available to the agent by setting `include_metadata_in_prompt` to `true`. This will include the metadata in the agent's system prompt, allowing the agent to use the data during the conversation.

#### Dynamically Creating a Temporary Voice Agent

You can also dynamically create a temporary voice agent with custom configurations using the code below:

```js theme={null}
msClient.start({
  agent: {
    agent_config: {
      prompt: "You're a helpful assistant.", // Example prompt
      voice: {
        provider: "elevenlabs", // Voice provider
        voice_id: "voice-id" // Replace 'voice-id' with the ID of the desired voice
      },
      language: "<language_code>", // optional - use language code such as en, es
      tools: [
        {
          name: "get_user_data",
          description: "",
          webhook: "https://...",
          header: {
            "Content-Type": "application/json",
            "Authorization": ""
          },
          params: [
            {
              name: "",
              type: "string" | "number" | "boolean",
              description: "",
              required: true
            }
          ]
        }
      ], // Replace with actual function calls you need
      custom_llm_websocket: "wss://...", // optional - enable custom llm
      llm: "", // optional - choose llm model. Ex: gpt-4o, llama-3-70b
    }
  }
});
```

To obtain the voice\_id, use this API to acquire the complete list of voices: [https://api-west.millis.ai:8080/voices](https://api-west.millis.ai:8080/voices)

#### Overriding Agent configuration

When both `agent_id` and `agent_config` are provided, the session will use the configuration associated with `agent_id` but will override it with any settings provided in `agent_config`. This option allows for minor modifications to the agent’s default configuration on a per-session basis.

#### Additional Parameters

##### Metadata

* **`metadata`**: Optional field to pass any additional information that may personalize the conversation. It can be used by the agent if `include_metadata_in_prompt` is set to `true`.

##### Including Metadata in Prompt

* **`include_metadata_in_prompt`**: Boolean flag (`true` or `false`). If `true`, the metadata provided will be included in the prompt to give context to the agent.

##### Session Continuation

* **`session_continuation`**: Provide `session_id` from a previous session to enable continuity in conversation. This allows the agent to reference previous interactions.

### 4. Stop a Conversation:

```js theme={null}
msClient.stop();
```

### 5. Setup event listener:

```js theme={null}
    msClient.on("onopen", () => {
      // When the client connected to the server
    });

    msClient.on("onready", () => {
      // When the conversation is ready
    });

    msClient.on("onaudio", (audio: Uint8Array) => {
      // Incoming audio chunk
    });

    msClient.on("analyzer", (analyzer: AnalyserNode) => {
      // AnalyserNode that you can use for audio animation
    });

    msClient.on("onclose", (event) => {
      // When the connection is closed
    });

    msClient.on("onerror", (error) => {
      // An error occurred
    });
```

## Event List

### `onopen`

**Description:** Emitted when the WebSocket connection is successfully opened.

**Callback Signature:**

```typescript theme={null}
'onopen': () => void;
```

### `onready`

**Description:** Emitted when the client is ready to start processing audio or other tasks.

**Callback Signature:**

```typescript theme={null}
'onready': () => void;
```

### `onsessionended`

**Description:** Emitted when a session has ended.

**Callback Signature:**

```typescript theme={null}
'onsessionended': () => void;
```

### `onaudio`

**Description:** Emitted when audio data is received.

**Callback Signature:**

```typescript theme={null}
'onaudio': (audio: Uint8Array) => void;
```

**Parameters:**

* `audio` - The received audio data in `Uint8Array` format.

### `onresponsetext`

**Description:** Emitted when agent's response text is received.

**Callback Signature:**

```typescript theme={null}
'onresponsetext': (text: string, payload: { is_final?: boolean }) => void;
```

**Parameters:**

* `text` - The received response text.
* `payload` - An object containing additional information.
  * `is_final` (optional) - A boolean indicating if the response text is final.

### `ontranscript`

**Description:** Emitted when user's transcript text is received.

**Callback Signature:**

```typescript theme={null}
'ontranscript': (text: string, payload: { is_final?: boolean }) => void;
```

**Parameters:**

* `text` - The received transcript text.
* `payload` - An object containing additional information.
  * `is_final` (optional) - A boolean indicating if the transcript text is final.

### `onfunction`

**Description:** Emitted when agent triggered a function call.

**Callback Signature:**

```typescript theme={null}
'onfunction': (text: string, payload: { name: string, params: object }) => void;
```

**Parameters:**

* `text` - Empty.
* `payload` - Information about the triggered function.
  * `name` - The function name.
  * `params` - The params being used in the function call.

### `analyzer`

**Description:** Emitted with an `AnalyserNode` for agent's audio analysis.

**Callback Signature:**

```typescript theme={null}
'analyzer': (analyzer: AnalyserNode) => void;
```

**Parameters:**

* `analyzer` - The `AnalyserNode` used for audio analysis.

### `useraudioready`

**Description:** Emitted when user audio is ready for processing.

**Callback Signature:**

```typescript theme={null}
'useraudioready': (data: { analyser: AnalyserNode, stream: MediaStream }) => void;
```

**Parameters:**

* `data` - An object containing audio-related information.
  * `analyser` - The `AnalyserNode` for user's audio analysis.
  * `stream` - The `MediaStream` containing the user's audio data.

### `onlatency`

**Description:** Emitted to report latency information for debugging purpose.

**Callback Signature:**

```typescript theme={null}
'onlatency': (latency: number) => void;
```

**Parameters:**

* `latency` - The measured latency in milliseconds.

### `onclose`

**Description:** Emitted when the WebSocket connection is closed.

**Callback Signature:**

```typescript theme={null}
'onclose': (event: CloseEvent) => void;
```

**Parameters:**

* `event` - The `CloseEvent` containing details about the WebSocket closure.

### `onerror`

**Description:** Emitted when an error occurs in the WebSocket connection.

**Callback Signature:**

```typescript theme={null}
'onerror': (error: Event) => void;
```

**Parameters:**

* `error` - The `Event` containing details about the error.

***

### Example Usage

Here's an example of how to listen to these events in the `Client` class:

```typescript theme={null}
const client = new Client(config);

client.on('onopen', () => {
  console.log('WebSocket connection opened.');
});

client.on('onready', () => {
  console.log('Client is ready.');
});

client.on('onsessionended', () => {
  console.log('Session ended.');
});

client.on('onaudio', (audio) => {
  console.log('Audio received:', audio);
});

client.on('onresponsetext', (text, payload) => {
  console.log('Response text:', text, 'Payload:', payload);
});

client.on('ontranscript', (text, payload) => {
  console.log('Transcript:', text, 'Payload:', payload);
});

client.on('analyzer', (analyzer) => {
  console.log('Analyzer node:', analyzer);
});

client.on('useraudioready', (data) => {
  console.log('User audio ready:', data);
});

client.on('onlatency', (latency) => {
  console.log('Latency:', latency);
});

client.on('onclose', (event) => {
  console.log('WebSocket connection closed:', event);
});

client.on('onerror', (error) => {
  console.error('WebSocket error:', error);
});
```

## Support

If you encounter any issues or have questions, please reach out to us directly at [thach@millis.ai](mailto:thach@millis.ai).