- Updated BoxAudioCodec constructor to include parameters for input gain, reference gain channel, and reference gain.
- Modified EnableInput method to set gain for the reference channel if applicable.
- Adjusted CustomAudioCodec initialization to utilize new gain parameters for improved audio processing.
- Added PressToTalkMcpTool to manage press-to-talk and click-to-talk modes.
- Updated button event handling to support listening state transitions based on press-to-talk status.
- Initialized press-to-talk tool during board setup for enhanced user interaction.
* feat: migrate firmware to xiaozhi-fonts 2.0.0
* fix: restart scrolling subtitles on text change
* docs: add AGENTS.md for project overview, architecture, rules, and commands
- Introduced AGENTS.md to document the XiaoZhi voice-assistant firmware architecture, required rules for development, board configuration process, and validation steps.
- Included detailed sections on project structure, commands for building and testing, and authoritative documentation references.
* feat: finalize glyph push protocol
* fix: remove stale emoji font include
* build: use xiaozhi-fonts 2.0.0
* fix: decouple custom fonts from glyph push
* refactor: improve wake word invocation handling
- Introduced BeginWakeWordInvoke method to streamline the wake word processing flow.
- Updated HandleWakeWordDetectedEvent to utilize the new method, ensuring proper state transitions and scheduling.
- Enhanced error handling to prevent the device from getting stuck in the connecting state.
- Refactored audio channel management to improve responsiveness during wake word detection.
---------
Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
* Migrate board builds to ESP-IDF 6.0.1
* Document upstream IDF 6 CI blockers
* Use IDF 6 patched registry components
* Fix board-specific ESP-IDF 6 build errors
* Fix remaining ESP-IDF 6 board builds
* Document final ESP-IDF 6 matrix results [skip ci]
* update IDF 6 component releases
* rebase IDF 6 migration and use upstream SSCMA
* Enhance audio input management in AudioService
- Introduced AS_EVENT_AUDIO_INPUT_STOP_REQUEST to manage audio input stopping more effectively.
- Updated AudioService::Start() to clear the new stop request event.
- Modified AudioService::AudioInputTask() to handle the stop request and ensure proper input disabling.
- Adjusted AdcPdmAudioCodec::Start() to reflect lazy input opening, ensuring start/stop operations run in the same task.
* Refactor audio codec management and configuration
- Added output_device_opened_ flag to track the state of the output device in Es8388 and Es8389 codecs.
- Updated EnableOutput method to prevent unnecessary device reopening and added mute functionality to manage audio output more effectively.
- Enhanced error handling in Es8311AudioCodec by ensuring proper closure and deletion of the codec device.
- Adjusted sample rates in board configurations for atk-dnesp32s3-box2 to 24000 Hz and introduced AUDIO_CODEC_USE_MCLK for improved clock management.
* Update build configurations and documentation for ESP-IDF 6 compatibility
- Added container specification for the build workflow using espressif/idf:v6.0.1.
- Updated the version of the espressif/esp_video component to ^2.3.0.
- Enhanced documentation to clarify the support status of ESP32-P4 variants, specifying that legacy Rev < 3 variants are excluded from the IDF 6 matrix.
- Adjusted board configurations to include IDF version constraints and necessary SDK configurations for legacy support.
- Improved handling of YUV formats in EspVideo to maintain compatibility with previous versions.
* Implement playback drained event handling in Application
- Added MAIN_EVENT_PLAYBACK_DRAINED to manage playback state transitions.
- Introduced callbacks for playback drained events in AudioService to trigger listening state changes.
- Refactored Application::Run() to handle deferred listening start based on playback queue status.
- Enhanced audio processing logic to prevent audio truncation during state changes.
- Updated related methods to ensure proper wake word detection configuration during listening mode.
* Fix variant selection shell in CI
* Update project version to 2.4.0 and adjust component dependencies
- Bump project version in CMakeLists.txt to 2.4.0.
- Change espressif/esp_video component version to ^2.0.1 in idf_component.yml.
- Modify AUDIO_INPUT_REFERENCE setting in config.h to false for m5stack-core-s3.
- Remove unnecessary infinite loops in xmini_c3_board.cc during initialization.
---------
Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
- Implemented a new function, IsEs8311Present(), to verify the presence of the ES8311 codec by checking its chip ID registers instead of relying solely on I2C bus probing.
- Updated error logging to provide clearer messages when the ES8311 codec is not detected.
- Added a continuous logging loop to indicate waiting for the boot button to be pressed during initialization.
* Update QQ group contact in README.md
* Update dependencies in idf_component.yml and modify AFE audio processor configuration
- Updated versions for several dependencies in idf_component.yml to improve compatibility and performance.
- Changed AFE audio processor initialization to use low-cost mode and adjusted AEC settings for enhanced audio processing.
* Update audio dependencies in idf_component.yml
- Downgraded versions for espressif/esp_audio_effects and espressif/esp_audio_codec to improve compatibility with existing components.
* refactor(audio): unify speech processing engines
---------
Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
- Renamed CONFIG_XIAOZHI_USE_ETHERNET to CONFIG_XIAOZHI_NETWORK_ETHERNET for clarity.
- Updated CMakeLists.txt and Kconfig.projbuild to reflect the new configuration option.
- Adjusted conditional compilation in system_info.cc and esp32-p4-nano.cc to use the new network configuration.
- Added a rename file to maintain compatibility with existing sdkconfig files.
* Enable WebSocket control server for electron-bot
* Add servo control features to ElectronBot
- Introduced new actions for servo movement and sequences, including ACTION_SERVO_MOVE and ACTION_SERVO_SEQUENCE.
- Implemented methods for clamping servo positions and applying oscillation effects.
- Added GetServoPositions method to retrieve current servo angles.
- Enhanced README.md with detailed AI command examples and servo capabilities.
These changes improve the flexibility and control of the ElectronBot's movements, allowing for more complex actions and better integration with AI functionalities.
* Refactor servo control in ElectronBot
- Removed unnecessary home action logic during servo sequences to improve action fluidity.
- Updated README.md to clarify the action parameters for hand movements, specifically the amplitude for flapping actions.
- Introduced clamping functions for servo angles and amplitudes to ensure safe operation within defined limits.
These changes enhance the control and safety of the ElectronBot's servo movements, allowing for more precise and natural actions.
* Enhance Otto Robot camera support by adding configuration for OV3660. Updated config.h to define camera types and GPIO settings, modified config.json to include new camera options, and refactored otto_robot.cc for improved camera detection and initialization logic.
* fix: 移除 OttoEmojiDisplay 构造函数中的 SetTheme 调用以修复 LoadProhibited 崩溃
Made-with: Cursor
* refactor: improve audio service error handling and codec timeout management
- Updated AudioService to prevent input task termination on read timeout, introducing a delay instead.
- Enhanced NoAudioCodec to implement a read timeout for I2S channel reads.
- Adjusted WebSocketControlServer to set a control port for improved socket management.
- Added manufacturer information to the config.json for waveshare ESP32-Touch-LCD-3.5.
* fix(otto): WebSocket direct clients not receiving MCP responses
When a browser connects directly to the WebSocket control server (port
8080) and sends a JSON-RPC request, the MCP response was routed through
Application::SendMcpMessage -> protocol_->SendMcpMessage, which sends it
to the cloud protocol channel. As a result, the direct WebSocket client
never received the response, while the WeChat mini-program could because
it communicates via the cloud.
Fix:
- Add BroadcastMessage() to WebSocketControlServer, using
httpd_queue_work + httpd_ws_send_frame_async to asynchronously
send responses back to all connected clients on port 8080
- Add RegisterMcpBroadcastCallback() to Application, allowing an
additional MCP send callback to be registered; SendMcpMessage()
now invokes it alongside the cloud protocol
- Register the broadcast callback in OttoRobot after the WebSocket
server starts successfully
Also add WebSocket direct-connect API documentation to README.md
with complete JSON-RPC 2.0 command examples.
* fix(otto-robot): migrate camera backend and set safe dark default theme
- Migrate `otto-robot` camera backend from `EspVideo` to `Esp32Camera` to improve capture stability after reboot/power cycle.
- Keep runtime sensor detection for both OV2640 and OV3660, and rename PID macros with `OTTO_` prefix to avoid symbol conflicts.
- Configure camera output as `RGB565 + FRAMESIZE_240X240` to match the 240x240 display.
- Rotate OV2640 output by 180 degrees (`VFlip + HMirror`) for correct orientation.
- Simplify `otto-robot` camera sdkconfig options by keeping only:
- `CONFIG_CAMERA_OV2640=y`
- `CONFIG_CAMERA_OV3660=y`
- Move Otto default dark theme setup into `OttoEmojiDisplay::SetupUI()` (after base UI init) to avoid boot-time LVGL null-object crash caused by calling `SetTheme()` too early.
* feat(motion): smooth action ending and adaptive homing on otto/electron-bot
Improve servo motion transitions to reduce abrupt returns to home pose.
Replace linear interpolation with ease-out movement, make homing duration adaptive to angle delta, and skip intermediate homing when queued actions are pending to keep multi-action sequences fluid.
* fix(electron-bot): show chat subtitle on display
ElectronEmojiDisplay overrode SetupChatLabel() to delete the parent's
chat_message_label_ (which lives inside bottom_bar_) and recreate it on the
bottom-layer container_ without alignment. As a result the real subtitle label
was hidden behind emoji_box_/top_bar_, while SetChatMessage() only toggled the
now-empty bottom_bar_, so chat subtitles were never visible.
Reuse the parent label created in bottom_bar_ (matching otto-robot) and only
switch to the dark theme in SetupUI(), removing the custom SetupChatLabel().
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(electron-bot): restore status icons on standby and clean up display
- SetStatus() did not re-show network_label_/battery_label_ when returning
to STANDBY, so the WiFi and battery icons stayed hidden after listening or
speaking. Restore them on standby to match otto-robot.
- Remove the unused InitializeElectronEmojis() method and its declaration.
- Drop unused includes (vector, assets.h, emoji_collection.h, lvgl_image.h)
and add the missing trailing newline in the header.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(esp32-s3-touch-amoled-1.8): keep wake word always-on while charging
The previous logic in GetBatteryLevel() used a static last_discharging
initialized to false. If the device booted on the charger (discharging
already false), the equality check never fired and PowerSaveTimer kept
running, so the device entered light sleep after 60s and stopped wake
word detection — defeating the always-on-while-charging behavior.
Move the tracker to a member with a tri-state initial value (-1 =
unknown) so the very first GetBatteryLevel() tick reconciles the timer
state regardless of boot conditions. Add ESP_LOGI on each transition
so the always-on state is visible in the logs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(esp32-s3-touch-amoled-1.8): keep PowerSaveTimer ticking while charging
The previous patch disabled PowerSaveTimer entirely on the charger,
which also disabled the screen dim and sleep emoji that fire from
OnEnterSleepMode after 60s of idle. Since this board constructs the
timer with cpu_max_freq=-1, it never enters CPU light sleep nor
disables wake word detection in OnEnterSleepMode anyway — the only
charger-incompatible action is the 5-minute pmic_->PowerOff() in
OnShutdownRequest.
Keep the timer running unconditionally and gate just the shutdown
callback on last_discharging_. Also call WakeUp() on the
charging→battery transition so an unplug after 5+ idle minutes
doesn't trigger an immediate power-off.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Revert "fix(esp32-s3-touch-amoled-1.8): keep PowerSaveTimer ticking while charging"
This reverts commit d7a9b619eb.
* feat(esp32-s3-touch-amoled-1.8): add wake word heartbeat while charging
Brings back the display dim + sleepy emoji UX while plugged in by
keeping PowerSaveTimer running and gating just the shutdown callback
on charging state. To work around the wake word stalling after long
idle (~22 min observed in testing), add a periodic bounce timer that
restarts wake word detection every 5 minutes while idle on the
charger. Skips the bounce mid-conversation, on battery, or when wake
word is already disabled.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(esp32-s3-touch-amoled-1.8): bounce wake word on sleep mode entry
Hardware testing shows wake word detection stops responding shortly
after PowerSaveTimer enters sleep mode (display dim + sleepy emoji).
The 5-minute periodic heartbeat doesn't help — the failure happens
well before the first bounce.
Add a preemptive bounce in OnEnterSleepMode while charging, and
shorten the periodic heartbeat to 2 minutes as a safety net for any
later stalls.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(esp32-s3-touch-amoled-1.8): use standalone dim timer on charger
Hardware testing showed that the previous approach (keeping
PowerSaveTimer active with wake-word bounces on the charger) still
left wake word unresponsive after sleep mode entered — even with a
preemptive bounce in OnEnterSleepMode and a 2-min periodic heartbeat,
the AFE / audio pipeline kept stalling, and the user had to wake the
device with the boot button.
Switch to a different split:
- On charger: PowerSaveTimer is fully disabled. Its sleep callbacks
never fire, so wake word detection is undisturbed. A standalone
dim_timer (1s tick) handles the brightness + sleepy-emoji UX
independently — on idle 60s it dims, on activity (conversation,
button) it restores via Application::CanEnterSleepMode() polling.
- On battery: PowerSaveTimer runs as before (60s sleep callbacks,
5min auto-shutdown), dim_timer stays stopped.
Also drops the wake-word bounce timer entirely — without
PowerSaveTimer's sleep mode firing on the charger there's nothing to
recover from.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* Enhance Otto Robot camera support by adding configuration for OV3660. Updated config.h to define camera types and GPIO settings, modified config.json to include new camera options, and refactored otto_robot.cc for improved camera detection and initialization logic.
* fix: 移除 OttoEmojiDisplay 构造函数中的 SetTheme 调用以修复 LoadProhibited 崩溃
Made-with: Cursor
* refactor: improve audio service error handling and codec timeout management
- Updated AudioService to prevent input task termination on read timeout, introducing a delay instead.
- Enhanced NoAudioCodec to implement a read timeout for I2S channel reads.
- Adjusted WebSocketControlServer to set a control port for improved socket management.
- Added manufacturer information to the config.json for waveshare ESP32-Touch-LCD-3.5.
* fix(otto): WebSocket direct clients not receiving MCP responses
When a browser connects directly to the WebSocket control server (port
8080) and sends a JSON-RPC request, the MCP response was routed through
Application::SendMcpMessage -> protocol_->SendMcpMessage, which sends it
to the cloud protocol channel. As a result, the direct WebSocket client
never received the response, while the WeChat mini-program could because
it communicates via the cloud.
Fix:
- Add BroadcastMessage() to WebSocketControlServer, using
httpd_queue_work + httpd_ws_send_frame_async to asynchronously
send responses back to all connected clients on port 8080
- Add RegisterMcpBroadcastCallback() to Application, allowing an
additional MCP send callback to be registered; SendMcpMessage()
now invokes it alongside the cloud protocol
- Register the broadcast callback in OttoRobot after the WebSocket
server starts successfully
Also add WebSocket direct-connect API documentation to README.md
with complete JSON-RPC 2.0 command examples.
* fix(otto-robot): migrate camera backend and set safe dark default theme
- Migrate `otto-robot` camera backend from `EspVideo` to `Esp32Camera` to improve capture stability after reboot/power cycle.
- Keep runtime sensor detection for both OV2640 and OV3660, and rename PID macros with `OTTO_` prefix to avoid symbol conflicts.
- Configure camera output as `RGB565 + FRAMESIZE_240X240` to match the 240x240 display.
- Rotate OV2640 output by 180 degrees (`VFlip + HMirror`) for correct orientation.
- Simplify `otto-robot` camera sdkconfig options by keeping only:
- `CONFIG_CAMERA_OV2640=y`
- `CONFIG_CAMERA_OV3660=y`
- Move Otto default dark theme setup into `OttoEmojiDisplay::SetupUI()` (after base UI init) to avoid boot-time LVGL null-object crash caused by calling `SetTheme()` too early.
* feat(motion): smooth action ending and adaptive homing on otto/electron-bot
Improve servo motion transitions to reduce abrupt returns to home pose.
Replace linear interpolation with ease-out movement, make homing duration adaptive to angle delta, and skip intermediate homing when queued actions are pending to keep multi-action sequences fluid.
Xiaoxia