hermes(voice): fix HHermesProcessed + first-sentence-stop; orb mark, lang selector
Real root cause (confirmed against the live build-24 DOM): the caption and TTS extraction fell back to turn.textContent whenever a settle-frame race left no readable answer segment, scraping the avatar letter, author name and 'Processed 13s' chip - and that truncated reply made TTS speak only the first segment then drop to Listening even with the mic muted (the muted-mic first-sentence-stop). Extraction now prefers each answer segment's data-raw-text, else the answer .msg-body only (excluding thinking/tool/worklog/role chrome), and the textContent fallback is gone; a genuinely mid-flight reply retries briefly so the whole thing is read before the overlay drains. Also: the app's own caduceus mark embedded in the conversation orb as a subtle watermark; a corner language selector (Auto + en/es/ru) that forces both the STT hint and the reply voice; and a thinking affordance after 2.5s of dead time. New response probe + extraction test prove caption==body-only and that every sentence reaches the TTS queue. 278 voice-lane tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BvMSXH8VH2tMWXanb8SJdf
This commit is contained in:
jenkins
parent
13a058a5c8
commit
3fd760de0e
@ -427,6 +427,43 @@
|
||||
opacity: 0.85;
|
||||
}
|
||||
|
||||
/* FIX 2: the Hermes caduceus mark, a static centred watermark inside the orb.
|
||||
Sits above the core wash but below no energy layer's motion — it never
|
||||
animates on its own and never scales past the orb, so the breathing/sweep/
|
||||
wave animations always read over it. Monochrome via currentColor at low
|
||||
opacity so it stays legible against every state tint without competing with
|
||||
the accent colour. Scales with the orb because it is inset-positioned. */
|
||||
.voice-conversation-orb-mark {
|
||||
position: absolute;
|
||||
inset: 24%;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
pointer-events: none;
|
||||
color: rgba(233, 244, 255, 0.9);
|
||||
opacity: 0.16;
|
||||
mix-blend-mode: screen;
|
||||
transition: opacity 200ms ease-out;
|
||||
}
|
||||
|
||||
.voice-conversation-orb-mark svg {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
display: block;
|
||||
filter: drop-shadow(0 0 6px rgba(var(--voice-accent), 0.3));
|
||||
}
|
||||
|
||||
/* Speaking/thinking lift the watermark a touch so it feels alive with the turn,
|
||||
still far below any level that would mask the animation. */
|
||||
.voice-conversation[data-voice-state="speaking"] .voice-conversation-orb-mark,
|
||||
.voice-conversation[data-voice-state="thinking"] .voice-conversation-orb-mark {
|
||||
opacity: 0.2;
|
||||
}
|
||||
|
||||
.voice-conversation.is-muted .voice-conversation-orb-mark {
|
||||
opacity: 0.1;
|
||||
}
|
||||
|
||||
/* Idle listening: slow breathing. */
|
||||
.voice-conversation[data-voice-state="listening"] .voice-conversation-orb-halo {
|
||||
animation: voice-conversation-breathe 4.4s ease-in-out infinite;
|
||||
@ -540,6 +577,117 @@
|
||||
background: rgba(235, 135, 88, 0.14);
|
||||
}
|
||||
|
||||
/* FIX 3: unobtrusive language chooser in the top-right corner. A small frosted
|
||||
globe that opens a compact Auto + supported-language menu; selecting forces
|
||||
both the reply voice and the STT hint for the session. */
|
||||
.voice-conversation-lang {
|
||||
position: absolute;
|
||||
top: calc(16px + env(safe-area-inset-top, 0px));
|
||||
right: calc(16px + env(safe-area-inset-right, 0px));
|
||||
z-index: 2;
|
||||
}
|
||||
|
||||
.voice-conversation-lang-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
padding: 0;
|
||||
color: rgba(226, 238, 250, 0.72);
|
||||
background: rgba(255, 255, 255, 0.06);
|
||||
border: 1px solid rgba(var(--voice-accent), 0.28);
|
||||
border-radius: 50%;
|
||||
cursor: pointer;
|
||||
transition: color 160ms ease, background 160ms ease, border-color 160ms ease;
|
||||
}
|
||||
|
||||
.voice-conversation-lang-btn:hover {
|
||||
color: rgba(240, 248, 255, 0.95);
|
||||
background: rgba(255, 255, 255, 0.12);
|
||||
}
|
||||
|
||||
.voice-conversation-lang-btn:focus-visible {
|
||||
outline: 2px solid rgb(var(--voice-accent));
|
||||
outline-offset: 2px;
|
||||
}
|
||||
|
||||
/* Forced (non-Auto) language: a small accent dot marks the active override. */
|
||||
.voice-conversation-lang-btn.is-forced {
|
||||
color: rgb(var(--voice-accent));
|
||||
border-color: rgba(var(--voice-accent), 0.7);
|
||||
}
|
||||
|
||||
.voice-conversation-lang-btn.is-forced::after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
top: 2px;
|
||||
right: 2px;
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
background: rgb(var(--voice-accent));
|
||||
box-shadow: 0 0 6px rgba(var(--voice-accent), 0.8);
|
||||
}
|
||||
|
||||
.voice-conversation-lang-menu {
|
||||
position: absolute;
|
||||
top: 46px;
|
||||
right: 0;
|
||||
min-width: 148px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 2px;
|
||||
padding: 6px;
|
||||
background: rgba(16, 22, 36, 0.96);
|
||||
border: 1px solid rgba(var(--voice-accent), 0.28);
|
||||
border-radius: 14px;
|
||||
box-shadow: 0 18px 44px rgba(0, 0, 0, 0.5);
|
||||
backdrop-filter: blur(8px);
|
||||
}
|
||||
|
||||
.voice-conversation-lang-menu[hidden] {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.voice-conversation-lang-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 10px;
|
||||
font: inherit;
|
||||
font-size: 14px;
|
||||
font-weight: 550;
|
||||
text-align: left;
|
||||
color: rgba(231, 241, 251, 0.9);
|
||||
background: transparent;
|
||||
border: 0;
|
||||
border-radius: 9px;
|
||||
padding: 9px 12px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.voice-conversation-lang-item:hover {
|
||||
background: rgba(255, 255, 255, 0.08);
|
||||
}
|
||||
|
||||
.voice-conversation-lang-item:focus-visible {
|
||||
outline: 2px solid rgb(var(--voice-accent));
|
||||
outline-offset: -2px;
|
||||
}
|
||||
|
||||
/* The selected option carries a trailing check. */
|
||||
.voice-conversation-lang-item[aria-checked="true"] {
|
||||
color: #fff;
|
||||
background: rgba(var(--voice-accent), 0.16);
|
||||
}
|
||||
|
||||
.voice-conversation-lang-item[aria-checked="true"]::after {
|
||||
content: "✓";
|
||||
font-size: 13px;
|
||||
color: rgb(var(--voice-accent));
|
||||
}
|
||||
|
||||
.voice-conversation.is-muted .voice-conversation-orb-halo,
|
||||
.voice-conversation.is-muted .voice-conversation-orb-core {
|
||||
animation: none;
|
||||
@ -551,6 +699,38 @@
|
||||
filter: saturate(0.35) brightness(0.8);
|
||||
}
|
||||
|
||||
/* FIX 4: "working…" affordance. Armed by atlas-voice.js (.is-awaiting) only
|
||||
after the model has been Thinking a beat with no reply text yet, so dead time
|
||||
reads as alive. A slow orb shimmer plus animated dots trailing the state
|
||||
caption — cheap, CSS-only, and fully suppressed under reduced motion by the
|
||||
blanket rule below. */
|
||||
.voice-conversation.is-awaiting .voice-conversation-orb-halo {
|
||||
animation: voice-conversation-await-shimmer 2.6s ease-in-out infinite;
|
||||
}
|
||||
|
||||
.voice-conversation.is-awaiting .voice-conversation-state::after {
|
||||
content: "";
|
||||
display: inline-block;
|
||||
width: 1.4em;
|
||||
margin-left: 0.15em;
|
||||
text-align: left;
|
||||
vertical-align: bottom;
|
||||
animation: voice-conversation-await-dots 1.4s steps(4, end) infinite;
|
||||
}
|
||||
|
||||
@keyframes voice-conversation-await-shimmer {
|
||||
0%, 100% { opacity: 0.6; filter: blur(10px); }
|
||||
50% { opacity: 0.92; filter: blur(13px); }
|
||||
}
|
||||
|
||||
@keyframes voice-conversation-await-dots {
|
||||
0% { content: ""; }
|
||||
25% { content: "·"; }
|
||||
50% { content: "··"; }
|
||||
75% { content: "···"; }
|
||||
100% { content: ""; }
|
||||
}
|
||||
|
||||
@keyframes voice-conversation-breathe {
|
||||
0%, 100% { opacity: 0.55; transform: scale(calc(0.96 + (var(--conversation-level) * 0.3))); }
|
||||
50% { opacity: 0.85; transform: scale(calc(1.05 + (var(--conversation-level) * 0.3))); }
|
||||
|
||||
@ -55,6 +55,12 @@
|
||||
}catch(_){return '';}
|
||||
})();
|
||||
const reducedMotion=window.matchMedia?window.matchMedia('(prefers-reduced-motion: reduce)'):{matches:false};
|
||||
// FIX 2: the app's own Hermes caduceus mark (static/favicon.svg), embedded
|
||||
// as a centred watermark inside the conversation orb. Rendered monochrome via
|
||||
// currentColor at low opacity so it reads across every state tint
|
||||
// (idle/listening/transcribing/thinking/speaking) without ever obscuring the
|
||||
// energy animation, and it carries no animation of its own (reduced-motion safe).
|
||||
const HERMES_MARK_SVG='<svg viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg" focusable="false" aria-hidden="true"><g transform="translate(-12.326 -15.707) scale(0.07072)"><path fill="currentColor" fill-rule="evenodd" d="M630.5 961.9 C634.9 960.7 638.5 957.9 640.5 953.9 C642.5 950.1 643.3 865.1 641.4 864.3 C640 863.8 623.9 872.5 618.2 876.8 C616.4 878.2 613.8 881.2 612.5 883.5 L610 887.7 610 918.4 C610 951.8 610.2 953.1 615.7 958.3 C618.3 960.8 622.4 962.7 625.5 962.9 C626 963 628.3 962.5 630.5 961.9 Z M596 913 C596.8 911.5 596.6 909.4 595.4 904.8 C592.1 892.1 595.4 881.4 605.5 872.1 C612.9 865.4 621.2 860.6 641.1 851.4 C681.3 832.9 691.1 827.1 704.5 813.6 C724.9 793.1 730 768.6 718.9 745.5 C714.9 737.4 705.5 727.4 696.7 722.1 L691.1 718.8 678.3 722.6 C671.3 724.6 664.9 726.8 664.2 727.3 C663.5 727.9 663 730.6 663 734.1 C663 739.9 663.1 740 666.8 741.9 C672.9 745 680.6 752.6 683.4 758.2 C688.9 769.3 686 781.3 675.3 791 C666.4 799.1 662.1 801.7 631.3 817.2 C598.7 833.5 587.2 840.5 578.9 849 C565.9 862.3 561.8 880.1 568.3 894.4 C574.4 907.7 592.4 919.8 596 913 Z M579.8 832.2 C582.7 830.2 586.8 827.3 589 825.8 C592.9 823.1 593 822.9 593 817.2 L593 811.4 586.6 807.2 C578.4 801.8 572.5 795.2 568.7 787.2 C566 781.5 565.8 780.1 566.2 773.5 C566.8 764.5 569.4 759.6 577.8 751.8 C589.1 741.4 603.5 735.3 666 714.8 C687.7 707.6 710.2 699.7 715.9 697.2 C741.9 685.8 757.8 670 764.5 648.7 C765.9 644.4 767 639.2 767 637.1 C767 631.5 768.1 631 777.9 631.6 C801.3 633.2 819.4 623.2 829 603.6 C831.2 599.2 833.5 593.4 834.2 590.7 C835.3 586.4 835.2 585.8 833.4 584 C831.5 582.1 830.8 582 814 583 C804.4 583.5 789.8 584.1 781.5 584.2 L766.5 584.5 766.2 577.9 C766 573.4 766.3 571 767.2 570.2 C767.9 569.6 778.4 568.4 790.4 567.6 C835.7 564.3 849.7 561.9 862.2 555.3 C878.5 546.7 889.5 529.3 893 506.4 C894.1 499.3 894 498.6 892.3 496.8 C890.4 494.9 890 495 865.4 500.4 C838.7 506.3 789.4 516.1 776.8 518.1 C766.3 519.7 766 519.5 766 511.2 C766 507.2 766.5 503.7 767.2 502.8 C767.9 502 771.2 500.7 774.5 500.1 C788.7 497.1 852.9 480.7 868.5 476 C877.9 473.2 889.8 468.8 895 466.3 C903 462.5 905.8 460.4 913.1 453.1 C922.9 443.2 928.3 433.9 932.5 419.5 C935.4 409.5 937.6 393.5 936.6 389.7 C935.3 384.3 933.5 384.5 914.3 392.1 C879.6 406 825.4 423.1 754.6 442.4 C728.5 449.6 719.1 452.5 717.2 454.2 C711.9 459.2 712 457.7 712 516.7 C712 551.6 711.6 572.9 710.9 575.2 C709.6 580 703.8 585.7 699.1 587 C697.1 587.5 689.9 588.2 683 588.6 C674 589.1 670 589.7 668.8 590.8 C667.2 592.1 667 594.3 667 608 C667 621.6 667.2 624.1 668.8 625.8 C670.4 627.8 671.8 627.9 694.9 628.2 C710.4 628.4 719.8 628.9 720.8 629.6 C723.1 631.2 720.6 639.8 715.9 646.9 C706.4 661.1 694.2 667.1 631 689 C581.4 706.1 563.9 713.8 550.3 724.7 C512.3 755 518.4 806 563.1 831.3 C567.7 833.9 572.2 836 573.1 836 C573.9 836 577 834.3 579.8 832.2 Z M626.3 807.4 C633.5 803.8 640.2 800.1 641.1 799.4 C642.5 798.1 642.7 794.2 642.5 766.5 C642.5 749.2 642.1 734.7 641.7 734.4 C641 733.7 613.6 743.6 611.2 745.3 C610.3 746 610 754.1 610 779.5 C610 797.7 610.3 813 610.7 813.3 C611.8 814.5 612.7 814.1 626.3 807.4 Z M571.1 699.2 L584.5 693.4 584.8 685.5 C585 681.2 584.7 677.3 584.1 676.7 C583.6 676.2 579.9 674.7 575.8 673.3 C567.5 670.5 554.7 664.2 548.4 660 C538.6 653.2 530.5 640.3 531.2 632.6 L531.5 629.5 541 628.9 C546.2 628.5 557.7 628.2 566.5 628.1 C577.6 628 583 627.6 583.8 626.8 C585.5 625.1 585.5 591.6 583.8 590.3 C583.1 589.7 576.5 589 569.1 588.6 C555.2 587.9 550.4 586.7 546.1 582.7 C541 578 541 578.1 541 518.8 C541 466.4 540.9 463.3 539 459.3 C536.5 453.7 533.3 451.9 518.7 448.1 C442.4 428 363.2 402.9 330.3 388.3 C322.4 384.9 322 384.8 319.5 386.4 C317.3 387.9 317 388.7 317 393.9 C317 397.1 317.7 403.5 318.5 408.1 C324.3 439.3 338.4 458 364.5 469.2 C374.6 473.5 396.4 479.7 441.3 491 C464.8 496.9 484.7 502.3 485.5 503 C486.6 503.9 487 506.3 487 511.2 C487 519.5 486.6 519.8 476.7 518.1 C461.5 515.5 412.5 505.6 391.5 500.9 C379.4 498.2 368.3 495.7 366.7 495.3 C364.8 494.9 363.3 495.3 361.9 496.6 C360 498.3 359.9 499.2 360.5 505.4 C362.5 526 373.6 544.9 388.9 554 C401.6 561.5 417.9 564.4 468 568.1 C477.1 568.7 485.1 569.7 485.8 570.3 C487.5 571.7 487.4 582.4 485.6 583.9 C484.2 585.1 473.4 584.7 432.5 582.4 C422.2 581.8 421.4 581.9 419.7 583.8 C417.9 585.8 417.9 586.1 419.5 591.6 C424.1 607.4 434.2 620.5 446.4 626.5 C455.1 630.8 461.3 632 474.1 632 C479.5 632 484.1 632.4 484.4 632.9 C484.8 633.4 485.6 637.4 486.4 641.7 C489.7 660.4 500.4 676.9 516.5 688 C526.3 694.7 549.1 704.8 555.1 704.9 C556.5 705 563.7 702.4 571.1 699.2 Z M628.9 678.9 C635.3 676.6 641 674.2 641.5 673.5 C643.1 671.6 642.5 576.3 640.9 574.4 C639.4 572.5 613.1 572.3 611.2 574.2 C610.3 575.1 610 588.7 610 629 C610 658.5 610.3 683 610.7 683.4 C611.8 684.5 616 683.5 628.9 678.9 Z M636.1 556 C654.1 552.6 669.6 536.9 673.6 517.8 C677.9 497.7 668.2 476.3 650 465.8 C636.2 457.8 615.8 458 601.7 466.3 C595.3 470.1 586.1 480.5 582.7 487.8 C570 514.9 585.4 547.4 614.7 555.5 C620.6 557.1 629 557.3 636.1 556 Z"/></g></svg>';
|
||||
const ERROR_VISIBLE_MS=3200;
|
||||
const STREAMING_CAPABILITY_URL='/api/voice/streaming/capability';
|
||||
const TTS_STREAM_URL='/api/tts/stream';
|
||||
@ -117,7 +123,28 @@
|
||||
// detection-less turn), consumed as the NEXT streaming STT session's bias
|
||||
// and as the thinking-cue locale. Cleared on activate/deactivate.
|
||||
let sessionLanguage='';
|
||||
// Forced conversation-mode language (FIX 3): when non-empty it overrides both
|
||||
// the streaming STT language hint and the reply-TTS voice, superseding
|
||||
// auto-detection until the user picks Auto again. Session-scoped; no storage.
|
||||
let forcedLanguage='';
|
||||
// Bounded retry budget for the completion (isFinal) pump: the STREAM_DONE
|
||||
// callback can beat the settle re-render that stamps the answer onto
|
||||
// data-raw-text, so a transient empty read must never tear a live reply down.
|
||||
let finalizeAttempts=0;
|
||||
const TTS_LANGUAGES=['en','ru','es'];
|
||||
// FIX 3: conversation-mode language chooser. Auto (default) plus the languages
|
||||
// the private voice map supports — kept in lockstep with
|
||||
// dockerfiles/hermes-jetson-tts-server.py LANGUAGE_VOICE_MAP (en → amy,
|
||||
// es → claude, ru → irina). Selecting one FORCES both the reply TTS voice and
|
||||
// the streaming STT language hint for the rest of the hands-free session (via
|
||||
// forcedLanguage), overriding auto-detection until Auto is chosen again.
|
||||
const CONVERSATION_LANGUAGES=[
|
||||
{code:'',label:'Auto',sublabel:'Detect'},
|
||||
{code:'en',label:'English',sublabel:'English'},
|
||||
{code:'es',label:'Español',sublabel:'Spanish'},
|
||||
{code:'ru',label:'Русский',sublabel:'Russian'},
|
||||
];
|
||||
const GLOBE_ICON_SVG='<svg viewBox="0 0 20 20" width="18" height="18" fill="none" stroke="currentColor" stroke-width="1.5" aria-hidden="true" focusable="false"><circle cx="10" cy="10" r="7.25"/><path d="M2.9 10h14.2M10 2.75c1.9 2 2.9 4.6 2.9 7.25S11.9 15.25 10 17.25C8.1 15.25 7.1 12.65 7.1 10S8.1 4.75 10 2.75z"/></svg>';
|
||||
const originalAutoRead=window.autoReadLastAssistant;
|
||||
const originalApplyPreference=window._applyVoiceModePref;
|
||||
|
||||
@ -212,11 +239,35 @@
|
||||
// on deactivate, keeps no storage, and is silently skipped in environments
|
||||
// without a usable DOM (headless contract probes).
|
||||
let conversation=null;
|
||||
// FIX 4: when the model has been thinking for a beat with no reply text yet,
|
||||
// the overlay shows a subtle animated "working…" affordance so the dead time
|
||||
// reads as alive rather than frozen. Purely a client progress hint — it never
|
||||
// fabricates spoken acknowledgements (that is the model's job).
|
||||
let awaitingTimer=null;
|
||||
const AWAITING_AFFORDANCE_MS=2500;
|
||||
|
||||
function conversationUsable(){
|
||||
return !!(typeof document!=='undefined'&&document&&document.body&&typeof document.createElement==='function');
|
||||
}
|
||||
|
||||
function clearAwaitingAffordance(){
|
||||
if(awaitingTimer){window.clearTimeout(awaitingTimer);awaitingTimer=null;}
|
||||
if(conversation&&conversation.root.classList) conversation.root.classList.remove('is-awaiting');
|
||||
}
|
||||
|
||||
function updateAwaitingAffordance(next){
|
||||
// Arm the affordance only while Thinking and only after the grace window;
|
||||
// any other state, or the first scrap of reply text, disarms it.
|
||||
clearAwaitingAffordance();
|
||||
if(!conversation||next!=='thinking') return;
|
||||
awaitingTimer=window.setTimeout(function(){
|
||||
awaitingTimer=null;
|
||||
if(conversation&&conversation.root.dataset.voiceState==='thinking'&&conversation.root.classList){
|
||||
conversation.root.classList.add('is-awaiting');
|
||||
}
|
||||
},AWAITING_AFFORDANCE_MS);
|
||||
}
|
||||
|
||||
function conversationNode(tag,className,attributes){
|
||||
const node=document.createElement(tag);
|
||||
if(className) node.className=className;
|
||||
@ -228,6 +279,7 @@
|
||||
if(!conversation) return;
|
||||
conversation.root.dataset.voiceState=next;
|
||||
conversation.stateEl.textContent=customLabel||STATE_LABELS[next]||'';
|
||||
updateAwaitingAffordance(next);
|
||||
}
|
||||
|
||||
function updateCaptionRegion(element,text,limit){
|
||||
@ -255,7 +307,11 @@
|
||||
}
|
||||
|
||||
function setConversationAssistantCaption(text){
|
||||
if(conversation) updateCaptionRegion(conversation.assistantCaption,text,9000);
|
||||
if(!conversation) return;
|
||||
// The first scrap of reply text means the model is no longer silently
|
||||
// thinking: disarm the "working…" affordance immediately (FIX 4).
|
||||
if(text&&String(text).trim()) clearAwaitingAffordance();
|
||||
updateCaptionRegion(conversation.assistantCaption,text,9000);
|
||||
}
|
||||
|
||||
function setConversationPlaying(playing){
|
||||
@ -291,12 +347,15 @@
|
||||
if(!conversation) return;
|
||||
if(event.key==='Escape'){
|
||||
if(event.preventDefault) event.preventDefault();
|
||||
// Escape peels one layer at a time: an open language menu closes first
|
||||
// (focus returns to its button), and only a second Escape exits the mode.
|
||||
if(conversation.langMenu&&!conversation.langMenu.hidden){closeLanguageMenu(true);return;}
|
||||
deactivate(true);
|
||||
return;
|
||||
}
|
||||
if(event.key==='Tab'){
|
||||
// Minimal focus trap across the two overlay controls.
|
||||
const stops=[conversation.muteBtn,conversation.exitBtn];
|
||||
// Minimal focus trap across the overlay controls (language, mute, exit).
|
||||
const stops=[conversation.langBtn,conversation.muteBtn,conversation.exitBtn].filter(Boolean);
|
||||
const current=stops.indexOf(document.activeElement);
|
||||
const index=current<0?(event.shiftKey?0:stops.length-1):current;
|
||||
const next=stops[(index+(event.shiftKey?stops.length-1:1))%stops.length];
|
||||
@ -305,6 +364,93 @@
|
||||
}
|
||||
}
|
||||
|
||||
function closeLanguageMenu(focusButton){
|
||||
if(!conversation||!conversation.langMenu) return;
|
||||
conversation.langMenu.hidden=true;
|
||||
conversation.langMenu.setAttribute('hidden','');
|
||||
if(conversation.langBtn){
|
||||
conversation.langBtn.setAttribute('aria-expanded','false');
|
||||
if(focusButton&&conversation.langBtn.focus) conversation.langBtn.focus();
|
||||
}
|
||||
}
|
||||
|
||||
function openLanguageMenu(){
|
||||
if(!conversation||!conversation.langMenu) return;
|
||||
conversation.langMenu.hidden=false;
|
||||
conversation.langMenu.removeAttribute('hidden');
|
||||
if(conversation.langBtn) conversation.langBtn.setAttribute('aria-expanded','true');
|
||||
// Focus the currently selected option so keyboard users land on it.
|
||||
const items=conversation.langItems||[];
|
||||
const active=items.filter(function(item){return item.getAttribute('aria-checked')==='true';})[0]||items[0];
|
||||
if(active&&active.focus) active.focus();
|
||||
}
|
||||
|
||||
function toggleLanguageMenu(){
|
||||
if(!conversation||!conversation.langMenu) return;
|
||||
if(conversation.langMenu.hidden) openLanguageMenu(); else closeLanguageMenu(true);
|
||||
}
|
||||
|
||||
function reflectLanguageSelection(){
|
||||
if(!conversation||!conversation.langItems) return;
|
||||
conversation.langItems.forEach(function(item){
|
||||
const selected=(item.getAttribute('data-lang')||'')===forcedLanguage;
|
||||
item.setAttribute('aria-checked',selected?'true':'false');
|
||||
});
|
||||
if(conversation.langBtn){
|
||||
const current=CONVERSATION_LANGUAGES.filter(function(entry){return entry.code===forcedLanguage;})[0];
|
||||
const label=current?current.label:'Auto';
|
||||
conversation.langBtn.setAttribute('aria-label','Conversation language: '+label);
|
||||
conversation.langBtn.setAttribute('title','Language — '+label);
|
||||
if(conversation.langBtn.classList){
|
||||
if(forcedLanguage) conversation.langBtn.classList.add('is-forced');
|
||||
else conversation.langBtn.classList.remove('is-forced');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function selectConversationLanguage(code){
|
||||
// Force (or, for '', release back to auto-detection) the session language.
|
||||
forcedLanguage=code||'';
|
||||
reflectLanguageSelection();
|
||||
// Re-bias the open streaming STT session immediately when it is safe — i.e.
|
||||
// it has heard nothing yet — so the forced hint applies to the very next
|
||||
// utterance instead of only the one after it. Mirrors the auto-switch path.
|
||||
if(active&&captureActive&&streamingStt&&typeof streamingStt.latestPartial==='function'&&!streamingStt.latestPartial()){
|
||||
startListening(generation,{preserveDisplay:true});
|
||||
}
|
||||
}
|
||||
|
||||
function buildLanguageControl(){
|
||||
const wrap=conversationNode('div','voice-conversation-lang');
|
||||
const btn=conversationNode('button','voice-conversation-lang-btn',{
|
||||
type:'button','aria-haspopup':'menu','aria-expanded':'false',
|
||||
'aria-label':'Conversation language: Auto',title:'Language — Auto',
|
||||
});
|
||||
btn.innerHTML=GLOBE_ICON_SVG;
|
||||
const menu=conversationNode('div','voice-conversation-lang-menu',{role:'menu','aria-label':'Conversation language',hidden:''});
|
||||
menu.hidden=true;
|
||||
const items=CONVERSATION_LANGUAGES.map(function(entry){
|
||||
const item=conversationNode('button','voice-conversation-lang-item',{
|
||||
type:'button',role:'menuitemradio','data-lang':entry.code,
|
||||
'aria-checked':entry.code===forcedLanguage?'true':'false',
|
||||
});
|
||||
item.textContent=entry.label;
|
||||
item.addEventListener('click',function(){
|
||||
selectConversationLanguage(entry.code);
|
||||
closeLanguageMenu(true);
|
||||
});
|
||||
menu.appendChild(item);
|
||||
return item;
|
||||
});
|
||||
btn.addEventListener('click',function(event){
|
||||
if(event&&event.stopPropagation) event.stopPropagation();
|
||||
toggleLanguageMenu();
|
||||
});
|
||||
wrap.appendChild(btn);
|
||||
wrap.appendChild(menu);
|
||||
return {wrap:wrap,btn:btn,menu:menu,items:items};
|
||||
}
|
||||
|
||||
function openConversationOverlay(){
|
||||
if(conversation||!conversationUsable()) return;
|
||||
try{
|
||||
@ -315,6 +461,10 @@
|
||||
orb.appendChild(conversationNode('span','voice-conversation-orb-halo'));
|
||||
orb.appendChild(conversationNode('span','voice-conversation-orb-core'));
|
||||
orb.appendChild(conversationNode('span','voice-conversation-orb-ring'));
|
||||
// FIX 2: Hermes mark watermark, centred in the orb beneath the energy layers.
|
||||
const orbMark=conversationNode('span','voice-conversation-orb-mark',{'aria-hidden':'true'});
|
||||
orbMark.innerHTML=HERMES_MARK_SVG;
|
||||
orb.appendChild(orbMark);
|
||||
const stateEl=conversationNode('div','voice-conversation-state',{'aria-hidden':'true'});
|
||||
const captions=conversationNode('div','voice-conversation-captions',{'aria-live':'polite'});
|
||||
// Each caption is its own bounded, independently scrollable region
|
||||
@ -333,6 +483,8 @@
|
||||
exitBtn.textContent='Exit voice mode';
|
||||
controls.appendChild(muteBtn);
|
||||
controls.appendChild(exitBtn);
|
||||
const lang=buildLanguageControl();
|
||||
root.appendChild(lang.wrap);
|
||||
root.appendChild(orb);
|
||||
root.appendChild(stateEl);
|
||||
root.appendChild(captions);
|
||||
@ -340,8 +492,16 @@
|
||||
muteBtn.addEventListener('click',function(){if(conversation) setConversationMuted(!conversation.muted);});
|
||||
exitBtn.addEventListener('click',function(){deactivate(true);});
|
||||
root.addEventListener('keydown',conversationKeydown);
|
||||
// A tap anywhere outside the language menu closes it (never deactivates).
|
||||
root.addEventListener('click',function(event){
|
||||
if(!conversation||!conversation.langMenu||conversation.langMenu.hidden) return;
|
||||
const target=event&&event.target;
|
||||
const inWrap=target&&typeof target.closest==='function'&&target.closest('.voice-conversation-lang');
|
||||
if(!inWrap) closeLanguageMenu(false);
|
||||
});
|
||||
document.body.appendChild(root);
|
||||
conversation={root:root,orb:orb,stateEl:stateEl,userCaption:userCaption,assistantCaption:assistantCaption,muteBtn:muteBtn,exitBtn:exitBtn,muted:false};
|
||||
conversation={root:root,orb:orb,stateEl:stateEl,userCaption:userCaption,assistantCaption:assistantCaption,muteBtn:muteBtn,exitBtn:exitBtn,langBtn:lang.btn,langMenu:lang.menu,langItems:lang.items,muted:false};
|
||||
reflectLanguageSelection();
|
||||
syncConversationOverlay(state);
|
||||
if(root.focus) root.focus();
|
||||
}catch(_){
|
||||
@ -353,6 +513,7 @@
|
||||
}
|
||||
|
||||
function removeConversationOverlay(){
|
||||
clearAwaitingAffordance();
|
||||
const overlay=conversation;
|
||||
conversation=null;
|
||||
if(!overlay) return;
|
||||
@ -645,6 +806,8 @@
|
||||
removeConversationOverlay();
|
||||
clearSttLanguage();
|
||||
sessionLanguage='';
|
||||
forcedLanguage='';
|
||||
finalizeAttempts=0;
|
||||
suppressAutoRead=false;
|
||||
pendingStitch=null;
|
||||
lastSentTranscript=null;
|
||||
@ -688,6 +851,7 @@
|
||||
// capture so the next utterance starts a clean streaming turn.
|
||||
thinkingSession=null;
|
||||
thinkingTurnId='';
|
||||
finalizeAttempts=0;
|
||||
clearSttLanguage();
|
||||
stopResponseObserver();
|
||||
cancelThinkingCues();
|
||||
@ -729,37 +893,108 @@
|
||||
return !!(typeof segment.querySelector==='function'&&segment.querySelector('.provider-error-details'));
|
||||
}
|
||||
|
||||
// Ground-truth DOM contract (verified against the live build-24 bundle,
|
||||
// ui.js renderMessages / messages.js ensureAssistantRow):
|
||||
//
|
||||
// <div class="msg-row assistant-turn" data-role="assistant">
|
||||
// <div class="msg-role assistant">
|
||||
// <div class="role-icon assistant">H</div> ← avatar letter
|
||||
// <span class="msg-role-name">Hermes</span> ← author name
|
||||
// </div>
|
||||
// <div class="assistant-turn-blocks">
|
||||
// <div class="tool-worklog-group">… "Processed 13s" …</div> ← worklog chip
|
||||
// <div class="assistant-segment assistant-segment-worklog-source"
|
||||
// hidden aria-hidden="true" data-raw-text="…">…</div> ← folded interim
|
||||
// <div class="assistant-segment" data-raw-text="ANSWER">
|
||||
// <div class="thinking-card">…</div> ← reasoning (optional)
|
||||
// <div class="msg-body">ANSWER</div> ← the spoken answer
|
||||
// </div>
|
||||
// </div>
|
||||
// </div>
|
||||
//
|
||||
// The settled renderer stamps the CLEAN answer text (no avatar, no author, no
|
||||
// "Processed Ns" chip, no reasoning) onto every answer segment's data-raw-text
|
||||
// — the exact field the app's own autoReadLastAssistant() reads (ui.js:~8891).
|
||||
// Reasoning uses .thinking-card-body, tool activity uses .tool-call-group-body
|
||||
// and the worklog "Processed Ns" chip lives in the worklog group beside the
|
||||
// segments — so .msg-body is answer-only, and the concatenated data-raw-text of
|
||||
// the visible answer segments is the whole reply. The prior round's fallback to
|
||||
// turn.textContent scraped the avatar "H" + "Hermes" + "Processed 13s" whenever
|
||||
// a settle-frame race left no readable answer segment; that fallback is gone.
|
||||
const NON_ANSWER_SEGMENT_CLASSES=['assistant-segment-worklog-source','assistant-segment-anchor'];
|
||||
// A .msg-body is the spoken answer ONLY when it is not nested inside reasoning,
|
||||
// tool, worklog, run-status, error or role chrome. The live renderer uses
|
||||
// distinct classes for each, but this guard keeps chrome out of the caption and
|
||||
// TTS even if a future build nests a body differently.
|
||||
const CHROME_CONTAINER_SELECTOR='.thinking-card,.agent-activity-thinking,.tool-call-group,.tool-worklog-group,.tool-group,.tool-card,.wl-reason,.process-wakeup-card,.provider-error-details,.msg-role,.assistant-run-status,.assistant-segment-worklog-source';
|
||||
|
||||
function segmentHasClass(segment,name){
|
||||
if(!segment) return false;
|
||||
if(segment.classList&&typeof segment.classList.contains==='function') return segment.classList.contains(name);
|
||||
return typeof segment.className==='string'&&segment.className.split(/\s+/).indexOf(name)>=0;
|
||||
}
|
||||
|
||||
function segmentIsAnswer(segment){
|
||||
// A visible answer segment: not folded into the worklog, not an anchor
|
||||
// scaffold, not hidden, not an error/system envelope.
|
||||
if(!segment) return false;
|
||||
if(segmentIsHidden(segment)) return false;
|
||||
if(segmentIsError(segment)) return false;
|
||||
for(let i=0;i<NON_ANSWER_SEGMENT_CLASSES.length;i+=1){
|
||||
if(segmentHasClass(segment,NON_ANSWER_SEGMENT_CLASSES[i])) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function answerBodiesText(segment){
|
||||
// Concatenate ONLY answer .msg-body text, excluding any body that sits inside
|
||||
// reasoning / tool / worklog / status chrome.
|
||||
if(!segment||typeof segment.querySelectorAll!=='function') return '';
|
||||
const bodies=segment.querySelectorAll('.msg-body');
|
||||
if(!bodies||!bodies.length) return '';
|
||||
const parts=[];
|
||||
Array.prototype.forEach.call(bodies,function(body){
|
||||
if(body.classList&&typeof body.classList.contains==='function'&&body.classList.contains('process-wakeup-body')) return;
|
||||
if(typeof body.closest==='function'&&body.closest(CHROME_CONTAINER_SELECTOR)) return;
|
||||
const value=typeof body.textContent==='string'?body.textContent:'';
|
||||
if(value) parts.push(value);
|
||||
});
|
||||
return parts.join('\n');
|
||||
}
|
||||
|
||||
function readSegmentBody(segment){
|
||||
if(!segment) return '';
|
||||
if(segment.dataset&&typeof segment.dataset.rawText==='string') return segment.dataset.rawText;
|
||||
// Scrape only message BODY text — never the avatar letter, author name,
|
||||
// "Processed Ns" worklog chips or footer controls that share the row.
|
||||
if(typeof segment.querySelectorAll==='function'){
|
||||
const bodies=segment.querySelectorAll('.msg-body');
|
||||
if(bodies&&bodies.length){
|
||||
return Array.prototype.map.call(bodies,function(body){return body.textContent||'';}).join('\n');
|
||||
}
|
||||
}
|
||||
return typeof segment.textContent==='string'?segment.textContent:'';
|
||||
// Prefer the renderer-stamped clean answer text; while a turn is still
|
||||
// streaming the live segment has no data-raw-text yet, so read its answer
|
||||
// .msg-body directly. Never fall through to the segment's whole textContent.
|
||||
if(segment.dataset&&typeof segment.dataset.rawText==='string'&&segment.dataset.rawText!=='') return segment.dataset.rawText;
|
||||
return answerBodiesText(segment);
|
||||
}
|
||||
|
||||
function readAssistantTurn(turn){
|
||||
// {text, error}: body-only text of every visible answer segment in the
|
||||
// turn, plus whether any segment is an error/system envelope.
|
||||
// {text, error}: the concatenated human-visible ANSWER text of every visible
|
||||
// answer segment of the turn — never the avatar letter, the "Hermes" author
|
||||
// name, a "Processed Ns" worklog chip, reasoning or tool chrome — plus whether
|
||||
// any segment is an error/system envelope. Returns '' (NEVER the row's
|
||||
// textContent) when the turn has no readable answer segment, so a tool-only or
|
||||
// not-yet-answered settle frame can never leak chrome into the caption or the
|
||||
// one-ahead TTS synthesizer.
|
||||
if(!turn) return {text:'',error:false};
|
||||
let error=false;
|
||||
const parts=[];
|
||||
const segments=(typeof turn.querySelectorAll==='function')?turn.querySelectorAll('.assistant-segment'):null;
|
||||
if(segments&&segments.length){
|
||||
Array.prototype.forEach.call(segments,function(segment){
|
||||
if(segmentIsHidden(segment)) return;
|
||||
if(segmentIsError(segment)){error=true;return;}
|
||||
if(!segmentIsAnswer(segment)) return;
|
||||
const value=readSegmentBody(segment);
|
||||
if(value&&value.trim()) parts.push(value.trim());
|
||||
});
|
||||
}else if(segmentIsError(turn)){
|
||||
error=true;
|
||||
}else{
|
||||
}else if(segmentIsAnswer(turn)){
|
||||
// The matched node was itself a lone answer segment (a leaf's
|
||||
// querySelectorAll returns nothing): read its own stamped answer text.
|
||||
const value=readSegmentBody(turn);
|
||||
if(value&&value.trim()) parts.push(value.trim());
|
||||
}
|
||||
@ -1062,7 +1297,7 @@
|
||||
socket.onopen=function(){
|
||||
// Bias recognition toward the session's sticky language the moment the
|
||||
// user has audibly switched; 'auto' remains the cold-start default.
|
||||
sendJson({type:'start',turn_id:turnId,format:'pcm_s16le',sample_rate:16000,language:sessionLanguage||'auto'});
|
||||
sendJson({type:'start',turn_id:turnId,format:'pcm_s16le',sample_rate:16000,language:forcedLanguage||sessionLanguage||'auto'});
|
||||
flush();
|
||||
};
|
||||
socket.onmessage=function(event){
|
||||
@ -2264,9 +2499,35 @@
|
||||
}
|
||||
const text=response.text;
|
||||
if(!text){
|
||||
if(isFinal){thinkingSession=null;thinkingTurnId='';clearSttLanguage();stopResponseObserver();cancelThinkingCues();restartSoon(token,250);}
|
||||
if(isFinal){
|
||||
// First-sentence-stop guard: once a reply has begun speaking, the
|
||||
// completion callback can fire a frame before the settle re-render
|
||||
// re-stamps the answer onto data-raw-text, making this read transiently
|
||||
// empty. Tearing down here would drop every unspoken sentence and jump
|
||||
// to "Listening" after sentence one (even with the mic muted). While a
|
||||
// reply is genuinely mid-flight (something was queued/spoken) retry for
|
||||
// up to ~1s so the WHOLE reply is read, then close the queue cleanly so
|
||||
// the buffered audio drains instead of a hard restart. A truly empty
|
||||
// completion — no reply produced at all — still tears down immediately,
|
||||
// so it never holds the speaking state open across the next utterance.
|
||||
const midReply=!!(speechTurn&&(speechTurn.consumed>0||speechTurn.queue.length||(speechTurn.waiters&&speechTurn.waiters.length)));
|
||||
if(midReply&&finalizeAttempts<8&&active&&token===generation&&(state==='thinking'||state==='speaking')){
|
||||
finalizeAttempts+=1;
|
||||
window.setTimeout(function(){pumpAssistantResponse(token,true);},120);
|
||||
return;
|
||||
}
|
||||
finalizeAttempts=0;
|
||||
if(midReply){
|
||||
finishSpeechQueue(speechTurn);
|
||||
stopResponseObserver();
|
||||
thinkingSession=null;thinkingTurnId='';clearSttLanguage();cancelThinkingCues();
|
||||
return;
|
||||
}
|
||||
thinkingSession=null;thinkingTurnId='';clearSttLanguage();stopResponseObserver();cancelThinkingCues();restartSoon(token,250);
|
||||
}
|
||||
return;
|
||||
}
|
||||
finalizeAttempts=0;
|
||||
setConversationAssistantCaption(text);
|
||||
// Any answer text, even a not-yet-speakable partial clause, owns the audio
|
||||
// timeline from this point forward. A cue must never overlap or become part
|
||||
@ -2281,9 +2542,11 @@
|
||||
// falls back to the reply-text heuristic. A correction on a
|
||||
// detection-less turn also re-biases the sticky session language so
|
||||
// the next streaming STT turn recognizes the switch promptly.
|
||||
const resolved=strongReplyLanguage(text)||turn.sttLanguage||detectReplyLanguage(text);
|
||||
// A user-forced language (FIX 3) wins over every auto signal: the reply is
|
||||
// spoken by the chosen voice regardless of script evidence or detection.
|
||||
const resolved=forcedLanguage||strongReplyLanguage(text)||turn.sttLanguage||detectReplyLanguage(text);
|
||||
if(resolved) turn.language=resolved;
|
||||
if(!turn.sttLanguage&&resolved) sessionLanguage=resolved;
|
||||
if(!forcedLanguage&&!turn.sttLanguage&&resolved) sessionLanguage=resolved;
|
||||
}
|
||||
if(text.length<turn.sourceText.length||!text.startsWith(turn.sourceText)){
|
||||
// Renderers can revise the still-unspoken tail. Already-spoken text is
|
||||
@ -2365,6 +2628,8 @@
|
||||
clearErrorTimer();
|
||||
clearSttLanguage();
|
||||
sessionLanguage='';
|
||||
forcedLanguage='';
|
||||
finalizeAttempts=0;
|
||||
modeBtn.classList.add('active');
|
||||
toast('Hands-free private voice mode on');
|
||||
openConversationOverlay();
|
||||
@ -2438,5 +2703,20 @@
|
||||
}
|
||||
}
|
||||
|
||||
// Behaviour-neutral instrumentation surface: pure references to the internal
|
||||
// extraction and one-ahead chunking helpers so a deterministic node probe can
|
||||
// drive them against a realistic rendered assistant turn (never touched by the
|
||||
// running app). Regression-locks the caption/TTS extraction contract.
|
||||
if(typeof window!=='undefined'){
|
||||
window.__atlasVoiceInternals={
|
||||
readAssistantTurn:readAssistantTurn,
|
||||
collectAssistantResponse:collectAssistantResponse,
|
||||
currentAssistantText:currentAssistantText,
|
||||
adaptiveChunks:adaptiveChunks,
|
||||
sentenceEnd:sentenceEnd,
|
||||
cleanForSpeech:cleanForSpeech,
|
||||
};
|
||||
}
|
||||
|
||||
initialize();
|
||||
})();
|
||||
|
||||
@ -882,6 +882,59 @@ scenarios.sticky_language_biases_next_stt_session = async () => {
|
||||
return { startLanguages: harness.servers.map((socket) => socket.startLanguage) };
|
||||
};
|
||||
|
||||
// FIX 3: choosing a language in the conversation overlay FORCES both the
|
||||
// streaming STT hint and the reply TTS voice for the whole session, overriding
|
||||
// auto-detection, until Auto is chosen again.
|
||||
scenarios.language_override_forces_stt_and_voice = async () => {
|
||||
const harness = makeHarness();
|
||||
await harness.start();
|
||||
const overlay = harness.overlay();
|
||||
if (!overlay) return { overlayPresent: false };
|
||||
const langWrap = overlay.children.find(
|
||||
(child) => String(child.className).indexOf('voice-conversation-lang') >= 0,
|
||||
);
|
||||
const menu = langWrap ? langWrap.children.find(
|
||||
(child) => String(child.className).indexOf('voice-conversation-lang-menu') >= 0,
|
||||
) : null;
|
||||
const items = menu ? menu.children : [];
|
||||
const ru = items.find((item) => item.getAttribute('data-lang') === 'ru');
|
||||
const auto = items.find((item) => item.getAttribute('data-lang') === '');
|
||||
const langBtn = langWrap ? langWrap.children.find(
|
||||
(child) => String(child.className).indexOf('voice-conversation-lang-btn') >= 0,
|
||||
) : null;
|
||||
if (!ru) return { overlayPresent: true, ruItemPresent: false };
|
||||
// Force Russian.
|
||||
ru.click();
|
||||
const forcedChecked = ru.getAttribute('aria-checked');
|
||||
const autoChecked = auto ? auto.getAttribute('aria-checked') : null;
|
||||
const btnForced = langBtn ? String(langBtn.className).indexOf('is-forced') >= 0 : false;
|
||||
// Run a turn with an English reply: STT hint and TTS voice must both be 'ru'.
|
||||
await harness.silence(300);
|
||||
await harness.speak('alpha', 1300);
|
||||
await harness.silence(1500);
|
||||
await harness.tick(400);
|
||||
harness.setAssistantReply('A short English reply. It has two sentences.');
|
||||
harness.completeResponse();
|
||||
await harness.tick(600);
|
||||
const startLanguages = harness.servers.map((socket) => socket.startLanguage);
|
||||
const ttsLanguages = harness.ttsCalls.map((call) => call.language);
|
||||
// Release back to Auto.
|
||||
if (auto) auto.click();
|
||||
const releasedChecked = auto ? auto.getAttribute('aria-checked') : null;
|
||||
const btnForcedAfterAuto = langBtn ? String(langBtn.className).indexOf('is-forced') >= 0 : false;
|
||||
return {
|
||||
overlayPresent: true,
|
||||
ruItemPresent: true,
|
||||
forcedChecked,
|
||||
autoChecked,
|
||||
btnForced,
|
||||
startLanguages,
|
||||
ttsLanguages,
|
||||
releasedChecked,
|
||||
btnForcedAfterAuto,
|
||||
};
|
||||
};
|
||||
|
||||
(async () => {
|
||||
const output = {};
|
||||
for (const name of Object.keys(scenarios)) {
|
||||
|
||||
402
testing/probes/hermes_voice_response_probe.js
Normal file
402
testing/probes/hermes_voice_response_probe.js
Normal file
@ -0,0 +1,402 @@
|
||||
// Deterministic assistant-response EXTRACTION probe for
|
||||
// dockerfiles/hermes-webui-atlas-voice.js.
|
||||
//
|
||||
// Round 3's caption/TTS extraction was validated only against a single synthetic
|
||||
// {dataset:{rawText}} node, so it never exercised the REAL rendered assistant
|
||||
// turn and shipped a turn.textContent fallback that scraped the avatar "H", the
|
||||
// "Hermes" author name and the "Processed 13s" worklog chip into the
|
||||
// conversation caption and the one-ahead TTS synthesizer ("HHermesProcessed
|
||||
// 13s"), and truncated multi-sentence replies to their first sentence.
|
||||
//
|
||||
// This probe builds a FAITHFUL rendered turn — matching the live build-24 DOM
|
||||
// (ui.js _createAssistantTurn / renderMessages, messages.js ensureAssistantRow):
|
||||
//
|
||||
// <div class="msg-row assistant-turn" data-role="assistant">
|
||||
// <div class="msg-role assistant">
|
||||
// <div class="role-icon assistant">H</div>
|
||||
// <span class="msg-role-name">Hermes</span>
|
||||
// </div>
|
||||
// <div class="assistant-turn-blocks">
|
||||
// <div class="tool-worklog-group">…Processed 13s…</div>
|
||||
// <div class="assistant-segment assistant-segment-worklog-source"
|
||||
// hidden aria-hidden="true" data-raw-text="…interim…">…</div>
|
||||
// <div class="assistant-segment" data-raw-text="ANSWER">
|
||||
// <div class="thinking-card"><div class="msg-body">REASONING</div></div>
|
||||
// <div class="msg-body">ANSWER</div>
|
||||
// </div>
|
||||
// </div>
|
||||
// </div>
|
||||
//
|
||||
// with a real querySelectorAll / closest / matches implementation, then drives
|
||||
// the actual exported extraction + one-ahead chunker and asserts:
|
||||
// (i) caption/TTS text is the answer BODY only — never "H"/"Hermes"/"Processed"
|
||||
// (ii) every sentence of a multi-sentence reply reaches the chunk queue
|
||||
// (iii) a pre-settle worklog-only frame extracts to '' (so the final pump
|
||||
// retries rather than finalizing garbage into "Listening")
|
||||
//
|
||||
// node hermes_voice_response_probe.js <path-to-atlas-voice.js>
|
||||
'use strict';
|
||||
|
||||
const fs = require('fs');
|
||||
const vm = require('vm');
|
||||
|
||||
const SCRIPT_PATH = process.argv[2];
|
||||
if (!SCRIPT_PATH) {
|
||||
throw new Error('usage: hermes_voice_response_probe.js <atlas-voice.js>');
|
||||
}
|
||||
const SOURCE = fs.readFileSync(SCRIPT_PATH, 'utf8');
|
||||
|
||||
// ── Faithful minimal DOM ────────────────────────────────────────────────────
|
||||
// Supports exactly what the extraction touches: className/classList, dataset
|
||||
// (backed by data-* attributes so `[data-raw-text]` selects), hidden, recursive
|
||||
// textContent, appendChild/children/parentNode, setAttribute/getAttribute, and
|
||||
// querySelector(All)/closest/matches over comma-separated compound selectors of
|
||||
// tag / .class / [attr] / [attr="value"] terms (no combinators — none are used).
|
||||
|
||||
function parseSelector(selector) {
|
||||
return String(selector).split(',').map((group) => {
|
||||
const term = group.trim();
|
||||
const parts = [];
|
||||
const re = /([.#]?[\w-]+)|\[([\w-]+)(?:([~|^$*]?=)"?([^"\]]*)"?)?\]/g;
|
||||
let m;
|
||||
while ((m = re.exec(term))) {
|
||||
if (m[1]) {
|
||||
if (m[1][0] === '.') parts.push({ kind: 'class', value: m[1].slice(1) });
|
||||
else if (m[1][0] === '#') parts.push({ kind: 'id', value: m[1].slice(1) });
|
||||
else parts.push({ kind: 'tag', value: m[1].toLowerCase() });
|
||||
} else if (m[2]) {
|
||||
parts.push({ kind: 'attr', name: m[2], op: m[3] || null, value: m[4] });
|
||||
}
|
||||
}
|
||||
return parts;
|
||||
}).filter((parts) => parts.length);
|
||||
}
|
||||
|
||||
class Node {
|
||||
constructor(tag) {
|
||||
this.tag = String(tag || 'div').toLowerCase();
|
||||
this.className = '';
|
||||
this.attributes = new Map();
|
||||
this.children = [];
|
||||
this.parentNode = null;
|
||||
this.hidden = false;
|
||||
this._text = '';
|
||||
const self = this;
|
||||
this.dataset = new Proxy({}, {
|
||||
get(_t, key) {
|
||||
if (typeof key !== 'string') return undefined;
|
||||
const attr = 'data-' + key.replace(/[A-Z]/g, (c) => '-' + c.toLowerCase());
|
||||
return self.attributes.has(attr) ? self.attributes.get(attr) : undefined;
|
||||
},
|
||||
set(_t, key, value) {
|
||||
const attr = 'data-' + String(key).replace(/[A-Z]/g, (c) => '-' + c.toLowerCase());
|
||||
self.attributes.set(attr, String(value));
|
||||
return true;
|
||||
},
|
||||
has(_t, key) {
|
||||
const attr = 'data-' + String(key).replace(/[A-Z]/g, (c) => '-' + c.toLowerCase());
|
||||
return self.attributes.has(attr);
|
||||
},
|
||||
});
|
||||
this.style = { setProperty() {}, removeProperty() {}, getPropertyValue() { return ''; }, display: '' };
|
||||
this.listeners = new Map();
|
||||
}
|
||||
|
||||
get classList() {
|
||||
const el = this;
|
||||
return {
|
||||
add(name) { const s = new Set(el.className.split(/\s+/).filter(Boolean)); s.add(name); el.className = [...s].join(' '); },
|
||||
remove(name) { el.className = el.className.split(/\s+/).filter((v) => v && v !== name).join(' '); },
|
||||
contains(name) { return el.className.split(/\s+/).indexOf(name) >= 0; },
|
||||
toggle(name, force) { if (force === undefined ? this.contains(name) : !force) this.remove(name); else this.add(name); },
|
||||
};
|
||||
}
|
||||
|
||||
setAttribute(name, value) {
|
||||
this.attributes.set(name, String(value));
|
||||
if (name === 'class') this.className = String(value);
|
||||
if (name === 'hidden') this.hidden = true;
|
||||
}
|
||||
getAttribute(name) {
|
||||
if (name === 'class') return this.className || null;
|
||||
return this.attributes.has(name) ? this.attributes.get(name) : null;
|
||||
}
|
||||
appendChild(child) { child.parentNode = this; this.children.push(child); return child; }
|
||||
addEventListener(type, cb) { this.listeners.set(type, cb); }
|
||||
removeEventListener(type) { this.listeners.delete(type); }
|
||||
|
||||
set textContent(value) { this._text = String(value); this.children = []; }
|
||||
get textContent() {
|
||||
if (this.children.length) return this.children.map((c) => c.textContent).join('');
|
||||
return this._text;
|
||||
}
|
||||
|
||||
_matchesTerm(parts) {
|
||||
return parts.every((p) => {
|
||||
if (p.kind === 'class') return this.classList.contains(p.value);
|
||||
if (p.kind === 'tag') return this.tag === p.value;
|
||||
if (p.kind === 'id') return this.getAttribute('id') === p.value;
|
||||
if (p.kind === 'attr') {
|
||||
if (!this.attributes.has(p.name)) return false;
|
||||
if (!p.op) return true;
|
||||
return this.attributes.get(p.name) === p.value;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
}
|
||||
matches(selector) { return parseSelector(selector).some((parts) => this._matchesTerm(parts)); }
|
||||
|
||||
_walk(out) { for (const c of this.children) { out.push(c); c._walk(out); } return out; }
|
||||
querySelectorAll(selector) {
|
||||
const groups = parseSelector(selector);
|
||||
return this._walk([]).filter((node) => groups.some((parts) => node._matchesTerm(parts)));
|
||||
}
|
||||
querySelector(selector) { const all = this.querySelectorAll(selector); return all.length ? all[0] : null; }
|
||||
closest(selector) {
|
||||
const groups = parseSelector(selector);
|
||||
let node = this;
|
||||
while (node) { if (groups.some((parts) => node._matchesTerm(parts))) return node; node = node.parentNode; }
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function el(tag, className, attrs, text) {
|
||||
const node = new Node(tag);
|
||||
if (className) node.setAttribute('class', className);
|
||||
if (attrs) Object.keys(attrs).forEach((k) => node.setAttribute(k, attrs[k]));
|
||||
if (text !== undefined) node.textContent = text;
|
||||
return node;
|
||||
}
|
||||
|
||||
// A role header (avatar "H" + "Hermes") — the source of the "HHermes" leak.
|
||||
function roleHeader() {
|
||||
const role = el('div', 'msg-role assistant');
|
||||
role.appendChild(el('div', 'role-icon assistant', null, 'H'));
|
||||
role.appendChild(el('span', 'msg-role-name', null, 'Hermes'));
|
||||
return role;
|
||||
}
|
||||
// The worklog "Processed 13s" chip — the source of the "Processed 13s" leak.
|
||||
function worklogGroup() {
|
||||
const group = el('div', 'tool-worklog-group tool-call-group', { 'data-anchor-scene-owner': '1' });
|
||||
const summary = el('button', 'tool-call-group-summary tool-worklog-summary');
|
||||
summary.appendChild(el('span', 'tool-call-group-label', null, 'Processed'));
|
||||
summary.appendChild(el('span', 'tool-call-group-duration', null, ' 13s'));
|
||||
group.appendChild(summary);
|
||||
const body = el('div', 'tool-call-group-body tool-worklog-body', { hidden: 'hidden' });
|
||||
body.appendChild(el('div', 'wl-reason', null, 'internal tool reasoning that must never be spoken'));
|
||||
group.appendChild(body);
|
||||
return group;
|
||||
}
|
||||
function answerSegment(rawText, bodyText, opts) {
|
||||
const seg = el('div', 'assistant-segment', { 'data-msg-idx': String((opts && opts.idx) || 1) });
|
||||
if (rawText !== null && rawText !== undefined) seg.setAttribute('data-raw-text', rawText);
|
||||
if (opts && opts.live) seg.setAttribute('data-live-assistant', '1');
|
||||
// Reasoning rendered INSIDE the segment uses .thinking-card-body, but pin the
|
||||
// chrome exclusion by nesting a stray .msg-body inside a thinking card too.
|
||||
if (opts && opts.reasoning) {
|
||||
const card = el('div', 'thinking-card');
|
||||
card.appendChild(el('div', 'msg-body', null, 'REASONING: ' + opts.reasoning));
|
||||
seg.appendChild(card);
|
||||
}
|
||||
if (bodyText !== null && bodyText !== undefined) seg.appendChild(el('div', 'msg-body', null, bodyText));
|
||||
return seg;
|
||||
}
|
||||
function assistantTurn(segments, opts) {
|
||||
const turn = el('div', 'msg-row assistant-turn', { 'data-role': 'assistant', 'data-session-id': 'session-1' });
|
||||
if (opts && opts.live) turn.setAttribute('id', 'liveAssistantTurn');
|
||||
turn.appendChild(roleHeader());
|
||||
const blocks = el('div', 'assistant-turn-blocks');
|
||||
if (opts && opts.worklog) blocks.appendChild(worklogGroup());
|
||||
segments.forEach((s) => blocks.appendChild(s));
|
||||
turn.appendChild(blocks);
|
||||
return turn;
|
||||
}
|
||||
|
||||
// ── vm harness ──────────────────────────────────────────────────────────────
|
||||
const documentRoot = el('div', 'msgInner');
|
||||
function makeControl(id) {
|
||||
const node = new Node('div');
|
||||
node.setAttribute('id', id);
|
||||
return node;
|
||||
}
|
||||
const controls = {
|
||||
btnVoiceMode: makeControl('btnVoiceMode'),
|
||||
voiceModeBar: makeControl('voiceModeBar'),
|
||||
voiceModeIndicator: makeControl('voiceModeIndicator'),
|
||||
voiceModeLabel: makeControl('voiceModeLabel'),
|
||||
msg: makeControl('msg'),
|
||||
};
|
||||
|
||||
const documentStub = {
|
||||
baseURI: 'https://chat.test/',
|
||||
body: new Node('body'),
|
||||
getElementById: (id) => controls[id] || (id === 'liveAssistantTurn' ? documentRoot.querySelector('#liveAssistantTurn') : null),
|
||||
querySelectorAll: (selector) => documentRoot.querySelectorAll(selector),
|
||||
querySelector: (selector) => documentRoot.querySelector(selector),
|
||||
createElement: (tag) => new Node(tag),
|
||||
addEventListener() {},
|
||||
removeEventListener() {},
|
||||
};
|
||||
|
||||
const sandbox = {
|
||||
console,
|
||||
window: null,
|
||||
document: documentStub,
|
||||
navigator: { mediaDevices: { getSupportedConstraints: () => ({}), getUserMedia: async () => ({ getTracks: () => [], getAudioTracks: () => [] }) } },
|
||||
MediaRecorder: function MediaRecorder() {},
|
||||
matchMedia: () => ({ matches: false, addEventListener() {}, removeEventListener() {} }),
|
||||
localStorage: { getItem: () => null, setItem() {}, removeItem() {} },
|
||||
crypto: { getRandomValues(b) { for (let i = 0; i < b.length; i += 1) b[i] = i + 1; return b; } },
|
||||
location: { protocol: 'https:', host: 'chat.test', href: 'https://chat.test/' },
|
||||
fetch: () => Promise.reject(new Error('no network in probe')),
|
||||
setTimeout: () => 0,
|
||||
clearTimeout: () => {},
|
||||
setInterval: () => 0,
|
||||
clearInterval: () => {},
|
||||
queueMicrotask: (fn) => Promise.resolve().then(fn),
|
||||
S: { session: { session_id: 'session-1' }, busy: false, activeStreamId: null },
|
||||
URL,
|
||||
Math, JSON, String, Number, Boolean, Error, Array, Object, Set, Map,
|
||||
parseInt, parseFloat, isNaN, Date,
|
||||
};
|
||||
sandbox.window = sandbox;
|
||||
sandbox.globalThis = sandbox;
|
||||
sandbox.window.matchMedia = sandbox.matchMedia;
|
||||
sandbox.window.MediaRecorder = sandbox.MediaRecorder;
|
||||
|
||||
vm.createContext(sandbox);
|
||||
vm.runInContext(SOURCE, sandbox, { filename: 'atlas-voice.js' });
|
||||
|
||||
const internals = sandbox.window.__atlasVoiceInternals;
|
||||
if (!internals) {
|
||||
throw new Error('atlas-voice.js did not expose __atlasVoiceInternals (early return? missing DOM stub)');
|
||||
}
|
||||
|
||||
function setTurn(turn) {
|
||||
documentRoot.children = [];
|
||||
if (turn) documentRoot.appendChild(turn);
|
||||
}
|
||||
|
||||
// Mirror pumpAssistantResponse's one-ahead consume loop over a full reply: a
|
||||
// partial pass (isFinal=false) then the completion pass (isFinal=true) + tail
|
||||
// flush. Proves every sentence is chunked, not just sentence one.
|
||||
function pumpChunks(text) {
|
||||
let consumed = 0;
|
||||
let first = true;
|
||||
const chunks = [];
|
||||
for (let pass = 0; pass < 12; pass += 1) {
|
||||
const remaining = text.slice(consumed).replace(/^\s+/, '');
|
||||
const skipped = text.slice(consumed).length - remaining.length;
|
||||
const ex = internals.adaptiveChunks(remaining, false, first);
|
||||
if (!ex.chunks.length) break;
|
||||
first = false;
|
||||
consumed += skipped + ex.consumed;
|
||||
chunks.push(...ex.chunks);
|
||||
}
|
||||
// completion pass
|
||||
{
|
||||
const remaining = text.slice(consumed).replace(/^\s+/, '');
|
||||
const skipped = text.slice(consumed).length - remaining.length;
|
||||
const ex = internals.adaptiveChunks(remaining, true, first);
|
||||
if (ex.chunks.length) { consumed += skipped + ex.consumed; chunks.push(...ex.chunks); }
|
||||
const tail = text.slice(consumed).trim();
|
||||
if (tail) { chunks.push(tail); consumed = text.length; }
|
||||
}
|
||||
return { chunks, consumed, covered: chunks.join(' ') };
|
||||
}
|
||||
|
||||
const results = {};
|
||||
|
||||
// (1) Finalized multi-segment turn: caption/TTS is answer-only, all sentences chunked.
|
||||
{
|
||||
const answer = 'Sentence one is here. Sentence two follows it. And sentence three concludes.';
|
||||
const turn = assistantTurn([
|
||||
answerSegment('earlier interim answer', 'earlier interim answer', { idx: 0 }),
|
||||
answerSegment(answer, answer, { idx: 1, reasoning: 'let me think' }),
|
||||
], { worklog: true });
|
||||
// fold the interim segment into the worklog exactly as the settle renderer does
|
||||
const interim = turn.querySelectorAll('.assistant-segment')[0];
|
||||
interim.setAttribute('class', 'assistant-segment assistant-segment-worklog-source');
|
||||
interim.setAttribute('aria-hidden', 'true');
|
||||
interim.hidden = true;
|
||||
setTurn(turn);
|
||||
const extracted = internals.collectAssistantResponse();
|
||||
const pumped = pumpChunks(extracted.text);
|
||||
results.finalized_multi_segment = {
|
||||
text: extracted.text,
|
||||
error: extracted.error,
|
||||
leaksAvatar: /HHermes|Hermes/.test(extracted.text),
|
||||
leaksProcessed: /Processed|13s/.test(extracted.text),
|
||||
leaksReasoning: /REASONING/.test(extracted.text),
|
||||
leaksInterim: /interim/.test(extracted.text),
|
||||
chunkCount: pumped.chunks.length,
|
||||
chunkConsumedAll: pumped.consumed === extracted.text.length,
|
||||
sentenceOnePresent: /one/.test(pumped.covered),
|
||||
sentenceTwoPresent: /two/.test(pumped.covered),
|
||||
sentenceThreePresent: /three/.test(pumped.covered),
|
||||
};
|
||||
}
|
||||
|
||||
// (2) Pre-settle worklog-only frame (the STREAM_DONE-beats-settle race): the
|
||||
// answer segment is not rendered yet — role header + "Processed 13s" only.
|
||||
// Must extract to '' so the final pump RETRIES instead of speaking chrome.
|
||||
{
|
||||
const turn = assistantTurn([], { worklog: true });
|
||||
setTurn(turn);
|
||||
const extracted = internals.collectAssistantResponse();
|
||||
results.presettle_worklog_only = {
|
||||
text: extracted.text,
|
||||
isEmpty: extracted.text === '',
|
||||
error: extracted.error,
|
||||
};
|
||||
}
|
||||
|
||||
// (3) Live streaming segment (no data-raw-text yet) — read via .msg-body.
|
||||
{
|
||||
const partial = 'The reply is still streaming right now';
|
||||
const turn = assistantTurn([answerSegment(null, partial, { idx: 1, live: true })], { worklog: false, live: true });
|
||||
setTurn(turn);
|
||||
const extracted = internals.collectAssistantResponse();
|
||||
results.live_streaming_reads_body = {
|
||||
text: extracted.text,
|
||||
matches: extracted.text === partial,
|
||||
leaksAvatar: /Hermes/.test(extracted.text),
|
||||
};
|
||||
}
|
||||
|
||||
// (4) Reasoning + worklog chrome nested with stray .msg-body must be excluded.
|
||||
{
|
||||
const answer = 'Only this answer body should be spoken aloud.';
|
||||
const seg = answerSegment(null, answer, { idx: 1, reasoning: 'hidden chain of thought' });
|
||||
const turn = assistantTurn([seg], { worklog: true });
|
||||
setTurn(turn);
|
||||
const extracted = internals.collectAssistantResponse();
|
||||
results.chrome_excluded = {
|
||||
text: extracted.text,
|
||||
matches: extracted.text === answer,
|
||||
leaksReasoning: /REASONING|chain of thought/.test(extracted.text),
|
||||
};
|
||||
}
|
||||
|
||||
// (5) Streaming growth then finalize: extraction grows monotonically to the full
|
||||
// reply — the queue keeps receiving sentences until the turn is final.
|
||||
{
|
||||
const two = 'First sentence. Second sentence.';
|
||||
const four = 'First sentence. Second sentence. Third sentence. Fourth sentence.';
|
||||
const liveTurn = assistantTurn([answerSegment(null, two, { idx: 1, live: true })], { live: true });
|
||||
setTurn(liveTurn);
|
||||
const partial = internals.collectAssistantResponse().text;
|
||||
const settledTurn = assistantTurn([answerSegment(four, four, { idx: 1 })], { worklog: true });
|
||||
setTurn(settledTurn);
|
||||
const full = internals.collectAssistantResponse().text;
|
||||
const pumped = pumpChunks(full);
|
||||
results.streaming_growth = {
|
||||
partial,
|
||||
full,
|
||||
grows: full.length > partial.length && full.startsWith(partial),
|
||||
allFourChunked: /First/.test(pumped.covered) && /Second/.test(pumped.covered) && /Third/.test(pumped.covered) && /Fourth/.test(pumped.covered),
|
||||
consumedAll: pumped.consumed === full.length,
|
||||
};
|
||||
}
|
||||
|
||||
process.stdout.write(JSON.stringify(results, null, 2) + '\n');
|
||||
@ -256,9 +256,95 @@ def test_reply_language_stickiness_source_contract():
|
||||
source = VOICE_SCRIPT.read_text(encoding="utf-8")
|
||||
|
||||
assert "let sessionLanguage=''" in source
|
||||
assert "language:sessionLanguage||'auto'" in source
|
||||
# A user-forced conversation-mode language (FIX 3) overrides the sticky
|
||||
# auto-detected hint for the streaming STT session.
|
||||
assert "language:forcedLanguage||sessionLanguage||'auto'" in source
|
||||
assert "function strongReplyLanguage(text)" in source
|
||||
assert "function detectReplyLanguage(text)" in source
|
||||
assert "scheduleThinkingCues(token,language||sessionLanguage,thinkingTurnId)" in source
|
||||
# The sticky language resets with each hands-free session.
|
||||
assert source.count("sessionLanguage='';") >= 2
|
||||
|
||||
|
||||
def test_conversation_language_override_forces_stt_and_voice(probe_results):
|
||||
"""FIX 3: picking Russian in the overlay forces BOTH the streaming STT hint
|
||||
and the reply TTS voice for the session (overriding auto-detection), and Auto
|
||||
releases the override."""
|
||||
scenario = probe_results["language_override_forces_stt_and_voice"]
|
||||
assert scenario["overlayPresent"] is True
|
||||
assert scenario["ruItemPresent"] is True
|
||||
# Selection state is reflected accessibly (menuitemradio aria-checked).
|
||||
assert scenario["forcedChecked"] == "true"
|
||||
assert scenario["autoChecked"] == "false"
|
||||
assert scenario["btnForced"] is True
|
||||
# Every STT session opened after the override carries the forced hint, and an
|
||||
# ENGLISH reply is still spoken by the Russian voice — auto-detection is
|
||||
# fully overridden.
|
||||
assert scenario["startLanguages"][1:] == ["ru", "ru", "ru"]
|
||||
assert scenario["ttsLanguages"] == ["ru", "ru"]
|
||||
# Auto releases the override.
|
||||
assert scenario["releasedChecked"] == "true"
|
||||
assert scenario["btnForcedAfterAuto"] is False
|
||||
|
||||
|
||||
def test_conversation_language_selector_source_and_style_contract():
|
||||
"""FIX 3: the corner language control is present, accessible, session-only
|
||||
(no localStorage), and lists Auto + the server voice-map languages."""
|
||||
source = VOICE_SCRIPT.read_text(encoding="utf-8")
|
||||
css = (ROOT / "dockerfiles" / "hermes-webui-atlas-voice.css").read_text(encoding="utf-8")
|
||||
|
||||
assert "let forcedLanguage=''" in source
|
||||
assert "const CONVERSATION_LANGUAGES=[" in source
|
||||
# Auto + en/es/ru mirror hermes-jetson-tts-server.py LANGUAGE_VOICE_MAP.
|
||||
for code in ("{code:'',label:'Auto'", "code:'en'", "code:'es'", "code:'ru'"):
|
||||
assert code in source
|
||||
assert "function selectConversationLanguage(code)" in source
|
||||
assert "aria-haspopup" in source and "'aria-expanded':'false'" in source
|
||||
assert "role:'menuitemradio'" in source
|
||||
# Forced language overrides both STT (start hint) and the reply voice.
|
||||
assert "language:forcedLanguage||sessionLanguage||'auto'" in source
|
||||
assert "const resolved=forcedLanguage||strongReplyLanguage(text)" in source
|
||||
# Session-only: the forced language never touches localStorage.
|
||||
assert "localStorage" not in source.split("CONVERSATION_LANGUAGES", 1)[1].split("function openConversationOverlay", 1)[0]
|
||||
# Escape peels the menu before exiting; the language button joins the trap.
|
||||
assert "if(conversation.langMenu&&!conversation.langMenu.hidden){closeLanguageMenu(true);return;}" in source
|
||||
assert "[conversation.langBtn,conversation.muteBtn,conversation.exitBtn]" in source
|
||||
for token in (".voice-conversation-lang-btn", ".voice-conversation-lang-menu", "menuitemradio"):
|
||||
pass # menu roles live in JS; assert CSS hooks below
|
||||
for token in (".voice-conversation-lang", ".voice-conversation-lang-btn", ".voice-conversation-lang-menu", ".voice-conversation-lang-item"):
|
||||
assert token in css
|
||||
|
||||
|
||||
def test_conversation_orb_hermes_mark_source_and_style_contract():
|
||||
"""FIX 2: the Hermes caduceus mark is embedded in the orb as a static,
|
||||
low-opacity watermark that respects reduced motion and never animates."""
|
||||
source = VOICE_SCRIPT.read_text(encoding="utf-8")
|
||||
css = (ROOT / "dockerfiles" / "hermes-webui-atlas-voice.css").read_text(encoding="utf-8")
|
||||
|
||||
assert "const HERMES_MARK_SVG=" in source
|
||||
assert 'fill-rule="evenodd"' in source # the real favicon caduceus path
|
||||
assert "'voice-conversation-orb-mark'" in source
|
||||
assert "orbMark.innerHTML=HERMES_MARK_SVG" in source
|
||||
assert ".voice-conversation-orb-mark" in css
|
||||
# Monochrome via currentColor at low opacity; carries no animation of its own.
|
||||
assert "color: rgba(233, 244, 255, 0.9)" in css
|
||||
mark_rule = css.split(".voice-conversation-orb-mark {", 1)[1].split("}", 1)[0]
|
||||
assert "animation" not in mark_rule
|
||||
assert "opacity: 0.16" in mark_rule
|
||||
|
||||
|
||||
def test_conversation_thinking_working_affordance_contract():
|
||||
"""FIX 4: a delayed 'working…' affordance during silent Thinking, CSS-driven
|
||||
and reduced-motion aware, with no fabricated spoken acknowledgement."""
|
||||
source = VOICE_SCRIPT.read_text(encoding="utf-8")
|
||||
css = (ROOT / "dockerfiles" / "hermes-webui-atlas-voice.css").read_text(encoding="utf-8")
|
||||
|
||||
assert "const AWAITING_AFFORDANCE_MS=2500" in source
|
||||
assert "function updateAwaitingAffordance(next)" in source
|
||||
assert "conversation.root.classList.add('is-awaiting')" in source
|
||||
# The affordance disarms the instant any reply text arrives.
|
||||
assert "if(text&&String(text).trim()) clearAwaitingAffordance();" in source
|
||||
assert ".voice-conversation.is-awaiting" in css
|
||||
# Reduced motion suppresses every overlay animation, including this one.
|
||||
rm_block = css.split("@media (prefers-reduced-motion: reduce)", 1)[1]
|
||||
assert "animation: none !important" in rm_block
|
||||
|
||||
100
testing/tests/test_hermes_voice_extraction.py
Normal file
100
testing/tests/test_hermes_voice_extraction.py
Normal file
@ -0,0 +1,100 @@
|
||||
"""Assistant-response extraction contract for conversation-mode captions + TTS.
|
||||
|
||||
Round 3's caption/TTS extraction was validated only against a single synthetic
|
||||
``{dataset:{rawText}}`` node, so it never exercised the REAL rendered assistant
|
||||
turn and shipped a ``turn.textContent`` fallback that scraped the avatar "H", the
|
||||
"Hermes" author name and the "Processed 13s" worklog chip into the conversation
|
||||
caption and the one-ahead TTS synthesizer ("HHermesProcessed 13s"), and truncated
|
||||
multi-sentence replies to their first sentence.
|
||||
|
||||
``hermes_voice_response_probe.js`` builds a faithful rendered turn matching the
|
||||
live build-24 DOM (ui.js ``_createAssistantTurn`` / ``renderMessages``, messages.js
|
||||
``ensureAssistantRow``) with a real ``querySelectorAll`` / ``closest`` and drives the
|
||||
actual exported extraction + one-ahead chunker. These tests assert the three
|
||||
symptoms are fixed and locked.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
VOICE_SCRIPT = ROOT / "dockerfiles" / "hermes-webui-atlas-voice.js"
|
||||
PROBE = ROOT / "testing" / "probes" / "hermes_voice_response_probe.js"
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def results() -> dict:
|
||||
node = shutil.which("node")
|
||||
if not node:
|
||||
pytest.skip("node is required to drive the response extraction contract")
|
||||
completed = subprocess.run(
|
||||
[node, str(PROBE), str(VOICE_SCRIPT)],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
)
|
||||
assert completed.returncode == 0, completed.stderr
|
||||
return json.loads(completed.stdout)
|
||||
|
||||
|
||||
def test_caption_is_answer_body_only(results):
|
||||
"""Symptom 1: caption/TTS text must be the answer body, never row chrome."""
|
||||
scenario = results["finalized_multi_segment"]
|
||||
assert scenario["text"] == (
|
||||
"Sentence one is here. Sentence two follows it. And sentence three concludes."
|
||||
)
|
||||
assert scenario["leaksAvatar"] is False, "avatar 'H' / 'Hermes' leaked into caption"
|
||||
assert scenario["leaksProcessed"] is False, "'Processed 13s' worklog chip leaked"
|
||||
assert scenario["leaksReasoning"] is False, "reasoning leaked into the spoken answer"
|
||||
assert scenario["leaksInterim"] is False, "folded interim segment leaked"
|
||||
assert scenario["error"] is False
|
||||
|
||||
|
||||
def test_every_sentence_reaches_the_tts_queue(results):
|
||||
"""Symptom 2: the one-ahead chunker enqueues the WHOLE reply, not sentence one."""
|
||||
scenario = results["finalized_multi_segment"]
|
||||
assert scenario["chunkConsumedAll"] is True, "chunker left part of the reply unspoken"
|
||||
assert scenario["sentenceOnePresent"] is True
|
||||
assert scenario["sentenceTwoPresent"] is True
|
||||
assert scenario["sentenceThreePresent"] is True
|
||||
assert scenario["chunkCount"] >= 3
|
||||
|
||||
|
||||
def test_presettle_worklog_only_extracts_empty(results):
|
||||
"""Symptom 3 root: a pre-settle worklog-only frame must extract to '' so the
|
||||
completion pump retries rather than finalizing 'HHermesProcessed 13s' into
|
||||
'Listening' after the first sentence."""
|
||||
scenario = results["presettle_worklog_only"]
|
||||
assert scenario["isEmpty"] is True, f"chrome leaked pre-settle: {scenario['text']!r}"
|
||||
assert scenario["error"] is False
|
||||
|
||||
|
||||
def test_live_streaming_reads_message_body(results):
|
||||
"""While streaming, the live segment has no data-raw-text yet — the answer
|
||||
still comes from its .msg-body, never the row textContent."""
|
||||
scenario = results["live_streaming_reads_body"]
|
||||
assert scenario["matches"] is True
|
||||
assert scenario["leaksAvatar"] is False
|
||||
|
||||
|
||||
def test_reasoning_and_worklog_chrome_excluded(results):
|
||||
scenario = results["chrome_excluded"]
|
||||
assert scenario["matches"] is True
|
||||
assert scenario["leaksReasoning"] is False
|
||||
|
||||
|
||||
def test_extraction_grows_to_full_reply(results):
|
||||
"""Extraction grows monotonically from the streaming partial to the full
|
||||
settled reply, and every settled sentence reaches the queue."""
|
||||
scenario = results["streaming_growth"]
|
||||
assert scenario["grows"] is True
|
||||
assert scenario["allFourChunked"] is True
|
||||
assert scenario["consumedAll"] is True
|
||||
Loading…
x
Reference in New Issue
Block a user