AI Robotics Developer — Autonomous AI Agent, Personality, Memory & Real-Time Interaction (Jetson Orin NX)

I'm looking for an experienced AI/robotics developer to help complete an autonomous AI-powered robot running on an NVIDIA Jetson Orin NX 16GB. This is not a project to build a simple voice assistant or chatbot. The goal is to create a robot with a consistent personality, long-term memory, emotional behavior, independent decision-making, and natural interactions, capable of reacting to its environment in real time. The robot is a character with an established personality, history, and relationship with another humanoid robot. Its behavior should reflect that identity through dialogue, voice, visual expressions, and physical movements. Current Development Status The project is already partially implemented. The following components are installed and operational: Whisper (STT): Speech recognition is implemented and working. Qwen (LLM): A local language model is installed and capable of generating conversational responses. Piper (TTS): Local text-to-speech is implemented. Basic conversational pipeline: The robot can already recognize speech, generate an answer, and speak it aloud. The main issue is that the responses are not yet consistently aligned with the robot's intended personality. The current implementation also needs further development to become a truly autonomous, responsive character rather than a basic question-and-answer system. The goal is to improve and extend the existing system, not unnecessarily rebuild components that already work. Main Features to Develop 1. Personality and Autonomous Intelligence Implement a persistent personality system based on an existing character profile. Maintain consistent personality traits, speaking style, opinions, and emotional reactions. Develop an internal emotional state that influences speech and physical behavior. Enable independent reasoning and decision-making. Allow the robot to decide when to speak, react, initiate conversations, or remain silent. Generate context-aware humor and sarcasm naturally, without repetitive scripted responses. Maintain consistent behavior throughout long conversations and across restarts. 2. Long-Term Memory and Character History Implement persistent short-term and long-term memory. Import a pre-existing fictional character history containing events that occurred before the robot's first activation. Differentiate between objective story history and the robot's subjective knowledge and memories. Remember conversations, events, relationships, and significant experiences. Retrieve relevant memories without overloading the LLM context. Support manual memory editing, deletion, and protection. Implement memory consolidation and a dream-generation system during charging or inactivity, keeping simulated dreams separate from factual memories. 3. Autonomous Conversation and Interruption Handling Implement natural conversational turn-taking. Enable the robot to detect and respond to interruptions while speaking. Develop an interruption-priority system: the robot should not necessarily stop speaking every time someone interrupts. Allow the robot to pause, resume, continue, or interrupt based on context, emotion, and conversational importance. Support independent conversational initiative rather than requiring a question or wake word for every interaction. 4. Vision, Hearing, and Attention Integrate camera-based face, person, and object tracking. Integrate the ReSpeaker XVF3800 four-microphone array for sound detection and direction. Develop an attention manager combining visual and audio information. Enable the robot to automatically orient toward speakers or relevant visual events. Prioritize competing stimuli and avoid unnecessary attention switching. Implement fast reactions independently of the LLM. 5. Physical Movement and Emotional Expression Integrate the Pimoroni Servo 2040 controller. Support eight servos controlling the quadruped's legs, with additional servos planned for eye movement. Implement basic locomotion, including standing, sitting, walking, and turning. Integrate an animated eye and physical mechanical iris for expressive behavior. Synchronize eye, iris, and body movements with speech, emotions, and attention. Ensure smooth movements, safe servo limits, and appropriate failure recovery. 6. Voice and Low-Latency Optimization Optimize the existing STT → LLM → TTS pipeline. Reduce the delay between the end of the user's speech and the robot's spoken response. Explore streaming LLM output and streaming TTS. Improve natural speech delivery, emotion, timing, and expressiveness. Maintain responsiveness while vision, audio, reasoning, and motor control operate simultaneously. 7. Cloud AI Integration and Comparison In addition to the fully local system, I want to implement an optional cloud-based AI pipeline for comparison.

Requirements

Integrate a cloud LLM as an alternative to the local Qwen model. Evaluate cloud-based TTS as an alternative to the local voice system. Allow switching between local and cloud processing. Maintain the same personality, character memory, and behavior logic regardless of which backend is used. Compare response latency, conversational quality, personality consistency, voice quality, resource usage, and operating costs. Benchmark both solutions under realistic conditions. The cloud system is intended for testing and comparison. The robot must remain fully functional locally without internet access. 8. Architecture, Reliability, and Testing Work with or extend the existing ROS 2 architecture. Keep hardware reflexes separate from slower AI reasoning. Ensure the language model generates high-level behavioral intentions rather than directly controlling servos. Implement appropriate watchdogs, safety limits, and failure handling. Maintain a modular and maintainable codebase. Measure system performance, including STT latency, LLM first-token latency, TTS startup latency, and total conversational response time. Provide documentation, source code, and reproducible performance benchmarks.