Creates an AI agent that conducts voice conversations using automatic speech recognition (ASR), large language models (LLMs), and text-to-speech (TTS) synthesis. The agent processes caller speech in real-time, generates contextually appropriate responses, and can execute custom functions to interact with external systems and databases through SignalWire AI Gateway (SWAIG).

Since the prompt configuration is central to AI agent behavior, it is recommended to read the Prompting Best Practices guide.

Properties

ai

objectRequired

An object that defines an AI agent for conducting voice conversations. Accepts the following properties to configure the agent’s prompt, behavior, functions, language support, and other settings.

ai.prompt

objectRequired

Defines the AI agent’s personality, goals, behaviors, and instructions for handling conversations. The prompt establishes how the agent should interact with callers, what information it should gather, and how it should respond to various scenarios.

It is recommended to write prompts using markdown formatting as LLMs better understand structured content. Additionally it is recommended to read the Prompting Best Practices guide.

ai.global_data

object

A key-value object for storing data that persists throughout the AI session. Can be set initially in the SWML script or modified during the conversation using the set_global_data action.

The global_data object is accessible everywhere in the AI session: prompts, AI parameters, and SWML returned from SWAIG functions. Access properties using template strings (e.g ${global_data.property_name})

ai.hints

string[] | object[]

Provide an array of strings and/or objects to guide the AI’s pronunciation and understanding of specific words or phrases. Words that can commonly be mispronounced can be added to the hints to help the AI speak more accurately.

Hints as strings: Each string in the array gives the AI context on how to interpret certain words. For example, if a user says Toni and the hint is Tony, the AI understands that the user said Tony.

Hints as objects: An array of objects with the properties below to customize how the AI handles specific words.

hints[].hint

stringRequired

The hint to match. This will match the string exactly as provided.

hints[].pattern

stringRequired

A regular expression to match the hint against. This will ensure that the hint has a valid matching pattern before being replaced.

hints[].replace

stringRequired

The text to replace the hint with. This will replace the portion of the hint that matches the pattern.

hints[].ignore_case

booleanDefaults to false

If true, the hint will be matched in a case-insensitive manner. Defaults to false.

ai.languages

object[]

An array of JSON objects defining supported languages in the conversation.

See languages for more details.

ai.params

object

A JSON object containing parameters as key-value pairs.

See params for more details.

ai.post_prompt

object

The final set of instructions and configuration settings to send to the agent.

post_prompt.text

stringRequired

The instructions to send to the agent.

post_prompt.temperature

numberDefaults to 1.0

Randomness setting. Float value between 0.0 and 1.5. Closer to 0 will make the output less random.

post_prompt.top_p

numberDefaults to 1.0

Randomness setting. Alternative to temperature. Float value between 0.0 and 1.0. Closer to 0 will make the output less random.

post_prompt.confidence

numberDefaults to 0.6

Threshold to fire a speech-detect event at the end of the utterance. Float value between 0.0 and 1.0. Decreasing this value will reduce the pause after the user speaks, but may introduce false positives.

post_prompt.presence_penalty

numberDefaults to 0

Aversion to staying on topic. Float value between -2.0 and 2.0. Positive values increase the model’s likelihood to talk about new topics.

post_prompt.frequency_penalty

numberDefaults to 0

Aversion to repeating lines. Float value between -2.0 and 2.0. Positive values decrease the model’s likelihood to repeat the same line verbatim.

ai.post_prompt_url

string

The URL to which to send status callbacks and reports. Authentication can also be set in the url in the format of username:password@url. See post_prompt_url callback below.

ai.pronounce

object[]

An array of objects to clarify the AI’s pronunciation of certain words or expressions.

pronounce[].replace

stringRequired

The expression to replace.

pronounce[].with

stringRequired

The phonetic spelling of the expression.

pronounce[].ignore_case

booleanDefaults to true

Whether the pronunciation replacement should ignore case.

ai.SWAIG

object

An array of JSON objects to create user-defined functions/endpoints that can be executed during the dialogue.

See SWAIG for more details.

post_prompt_url callback

SignalWire will make a request to the post_prompt_url with the following parameters:

action

string

Action that prompted this request. The value will be “post_conversation”.

ai_end_date

integer

Timestamp indicating when the AI session ended.

ai_session_id

string

A unique identifier for the AI session.

ai_start_date

integer

Timestamp indicating when the AI session started.

app_name

string

Name of the application that originated the request.

call_answer_date

integer

Timestamp indicating when the call was answered.

call_end_date

integer

Timestamp indicating when the call ended.

call_id

string

ID of the call.

call_log

object

The complete log of the call, as a JSON object.

call_log.content

string

Content of the call log entry.

call_log.role

string

Role associated with the call log entry (e.g., “system”, “assistant”, “user”).

call_start_date

integer

Timestamp indicating when the call started.

caller_id_name

string

Name associated with the caller ID.

caller_id_num

string

Number associated with the caller ID.

content_disposition

string

Disposition of the content.

content_type

string

Type of content. The value will be text/swaig.

conversation_id

string

A unique identifier for the conversation thread, if configured via the AI parameters.

post_prompt_data

object

The answer from the AI agent to the post_prompt. The object contains the three following fields.

post_prompt_data.parsed

object

If a JSON object is detected within the answer, it is parsed and provided here.

post_prompt_data.raw

string

The raw data answer from the AI agent.

post_prompt_data.substituted

string

The answer from the AI agent, excluding any JSON.

project_id

string

ID of the Project.

space_id

string

ID of the Space.

SWMLVars

object

A collection of variables related to SWML.

swaig_log

object

A log related to SWAIG functions.

total_input_tokens

integer

Represents the total number of input tokens.

total_output_tokens

integer

Represents the total number of output tokens.

version

string

Version number.

Post prompt callback request example

Below is a JSON example of the callback request that is sent to the post_prompt_url:

{

  "total_output_tokens": 119,

  "caller_id_name": "[CALLER_NAME]",

  "SWMLVars": {

    "ai_result": "success",

    "answer_result": "success"

  },

  "call_start_date": 1694541295773508,

  "project_id": "[PROJECT_ID]",

  "call_log": [\
\
    {\
\
      "content": "[AI INITIAL PROMPT/INSTRUCTIONS]",\
\
      "role": "system"\
\
    },\
\
    {\
\
      "content": "[AI RESPONSE]",\
\
      "role": "assistant"\
\
    },\
\
    {\
\
      "content": "[USER RESPONSE]",\
\
      "role": "user"\
\
    }\
\
  ],

  "ai_start_date": 1694541297950440,

  "call_answer_date": 1694541296799504,

  "version": "2.0",

  "content_disposition": "Conversation Log",

  "conversation_id": "[CONVERSATION_ID]",

  "space_id": "[SPACE_ID]",

  "app_name": "swml app",

  "swaig_log": [\
\
    {\
\
      "post_data": {\
\
        "content_disposition": "SWAIG Function",\
\
        "conversation_id": "[CONVERSATION_ID]",\
\
        "space_id": "[SPACE_ID]",\
\
        "meta_data_token": "[META_DATA_TOKEN]",\
\
        "app_name": "swml app",\
\
        "meta_data": {},\
\
        "argument": {\
\
          "raw": "{\n  \"target\": \"[TRANSFER_TARGET]\"\n}",\
\
          "substituted": "",\
\
          "parsed": [\
\
            {\
\
              "target": "[TRANSFER_TARGET]"\
\
            }\
\
          ]\
\
        },\
\
        "call_id": "[CALL_ID]",\
\
        "content_type": "text/swaig",\
\
        "ai_session_id": "[AI_SESSION_ID]",\
\
        "caller_id_num": "[CALLER_NUMBER]",\
\
        "caller_id_name": "[CALLER_NAME]",\
\
        "project_id": "[PROJECT_ID]",\
\
        "purpose": "Use to transfer to a target",\
\
        "argument_desc": {\
\
          "type": "object",\
\
          "properties": {\
\
            "target": {\
\
              "description": "the target to transfer to",\
\
              "type": "string"\
\
            }\
\
          }\
\
        },\
\
        "function": "transfer",\
\
        "version": "2.0"\
\
      },\
\
      "command_name": "transfer",\
\
      "epoch_time": 1694541334,\
\
      "command_arg": "{\n  \"target\": \"[TRANSFER_TARGET]\"\n}",\
\
      "url": "https://example.com/here",\
\
      "post_response": {\
\
        "action": [\
\
          {\
\
            "say": "This is a say message!"\
\
          },\
\
          {\
\
            "SWML": {\
\
              "sections": {\
\
                "main": [\
\
                  {\
\
                    "connect": {\
\
                      "to": "+1XXXXXXXXXX"\
\
                    }\
\
                  }\
\
                ]\
\
              },\
\
              "version": "1.0.0"\
\
            }\
\
          },\
\
          {\
\
            "stop": true\
\
          }\
\
        ],\
\
        "response": "transferred to [TRANSFER_TARGET], the call has ended"\
\
      }\
\
    }\
\
  ],

  "total_input_tokens": 5627,

  "caller_id_num": "[CALLER_NUMBER]",

  "call_id": "[CALL_ID]",

  "call_end_date": 1694541335435503,

  "content_type": "text/swaig",

  "action": "post_conversation",

  "post_prompt_data": {

    "substituted": "[SUMMARY_MESSAGE_PLACEHOLDER]",

    "parsed": [],

    "raw": "[SUMMARY_MESSAGE_PLACEHOLDER]"

  },

  "ai_end_date": 1694541335425164,

  "ai_session_id": "[AI_SESSION_ID]"

}

Responding to post prompt requests

The response to the callback request should be a JSON object with the following parameters:

{

  "response": "ok"

}

Examples

Minimal AI agent

YAMLJSON

version: 1.0.0

sections:

  main:

    - answer: {}

    - ai:

        prompt:

          text: "You are a customer service agent. Answer questions about account status and billing."

Hints

YAMLJSON

ai:

  hints:

  - Tony

  - hint: swimmel

    pattern: swimmel

    replace: SWML

Pronounce

YAMLJSON

version: 1.0.0

sections:

  main:

    - ai:

        prompt:

          text: |

            You are an expert in the GIF file format. Tell the user whatever they'd like to know in this

            field.

        pronounce:

          - replace: GIF

            with: jif