🌟 Gemini Multimodal Live API Extension with RTC

agent.theten.ai

Try Google Gemini Multimodal Live API with realtime vision and realtime screenshare detection capabilities, it is a ready-to-use extension, along with powerful tools like Weather Check and Web Search integrated perfectly into TEN Agent.

TEN Agent Usecases

Ready-to-use Extensions

TEN Agent Playground in Local Environment

Prerequisites

macOS: Docker setting on Apple Silicon

For Apple Silicon Macs, uncheck "Use Rosetta for x86/amd64 emulation" in Docker settings. Note: This may result in slower build times on ARM, but performance will be normal when deployed to x64 servers.

Next step

1. Create .env file

cp ./.env.example ./.env

2. Setup Agora App ID and App Certificate in .env

AGORA_APP_ID=
AGORA_APP_CERTIFICATE=

3. Start agent development containers

docker compose up -d

4. Enter container

docker exec -it ten_agent_dev bash

5. Build agent

task use

6. Start the web server

task run

7. Edit playground settings

Open the playground at localhost:3000 to configure your agent.

1. Select a graph type (e.g. Voice Agent, Realtime Agent)
2. Choose a corresponding module
3. Select an extension and configure its API key settings

Running Gemini Realtime Extension

Open the playground at localhost:3000.

1. Select voice_assistant_realtime graph
2. Choose Gemini Realtime module
3. Select v2v extension and enter Gemini API key

TEN Agent Components

Stay Tuned

Before we get started, be sure to star our repository and get instant notifications for all new releases!

Join Community

• Discord: Ideal for sharing your applications and engaging with the community.
• GitHub Discussion: Perfect for providing feedback and asking questions.
• GitHub Issues: Best for reporting bugs and proposing new features. Refer to our contribution guidelines for more details.
• (https://img.shields.io/twitter/follow/TenFramework?logo=X&color=%20%23f5f5f5): Great for sharing your agents and interacting with the community.

Star History

Code Contributors

Contribution Guidelines

Contributions are welcome! Please read the contribution guidelines first.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.