InferenceProvider

Agent Runtimes

---

Agent Runtimes / types/inference / InferenceProvider

Interface: InferenceProvider

Defined in: types/inference.ts:139

Abstract inference provider interface

Properties

name

readonly name: string

Defined in: types/inference.ts:141

Provider name for identification

Methods

cancelRequest()

cancelRequest(requestId?): void

Defined in: types/inference.ts:163

Cancel an ongoing request

Parameters

requestId?

string

Returns

void

---

executeBackendTool()?

optional executeBackendTool(toolName, args, options?): Promise<ToolExecutionResult<unknown>>

Defined in: types/inference.ts:168

Execute a backend tool (for hybrid execution)

Parameters

toolName

string

args

Record<string, unknown>

options?

InferenceRequestOptions

Returns

Promise<ToolExecutionResult<unknown>>

---

getConfig()

getConfig(): InferenceProviderConfig

Defined in: types/inference.ts:182

Get provider configuration

Returns

InferenceProviderConfig

---

isAvailable()

isAvailable(): boolean

Defined in: types/inference.ts:177

Check if provider is available/configured

Returns

boolean

---

sendMessage()

sendMessage(messages, options?): Promise<InferenceResponse>

Defined in: types/inference.ts:146

Send a message and get a response (non-streaming)

Parameters

messages

ChatMessage[]

options?

InferenceRequestOptions

Returns

Promise<InferenceResponse>

---

streamMessage()

streamMessage(messages, options?, onEvent?): Promise<InferenceResponse>

Defined in: types/inference.ts:154

Send a message with streaming response

Parameters

messages

ChatMessage[]

options?

InferenceRequestOptions

onEvent?

StreamEventHandler

Returns

Promise<InferenceResponse>