Voice-enabled conversation bots powered by Large Language Models (LLMs) offer promising opportunities to enhance communication accessibility and companionship for older adults. However, current systems often overlook age-related speech characteristics and interaction preferences. This study presents an adaptive conversation bot system that adjusts voice synthesis based on user-specific speech features such as pitch and speech rate. To test the hypothesis that speech-based voice adaptation improves usability and user experience, we develop predictive models using the Mozilla Common Voice Delta Segment 19.0 dataset. We also benchmark three compact LLMs, Mistral 7B, Llama 2 7B, and Llama 3.1 8B, based on latency, throughput, and memory usage to identify a model suitable for real-time interaction. Given the sensitivity of the target population, we conducted a pilot user study with younger adult participants, comparing a baseline and an adapted version of the conversation bot across multiple usability criteria. Results of user evaluation showed that the adaptive version improved clarity, comfort, and naturalness, supporting the effectiveness of the proposed approach for voice-enabled conversation systems.