Commonwealth of Australia

1. FIELD OF THE INVENTION

The present invention relates to clinical documentation systems. More specifically, the invention relates to an automated system for capturing doctor-patient consultations through ambient voice recording, processing the consultation transcripts using artificial intelligence, and generating structured clinical documentation in SOAP format (Subjective, Objective, Assessment, Plan) with integrated prescriptions, medical orders, and country-specific healthcare system compliance.

2. BACKGROUND ART

Healthcare professionals globally, including in Australia, face significant administrative burdens related to clinical documentation. Australian general practitioners spend 2-3 hours daily on documentation tasks, contributing to practitioner burnout and reducing time available for direct patient care.

Existing documentation solutions include:

• Traditional Electronic Health Records (EHR): Require manual data entry with no automation, resulting in time-consuming documentation workflows.
• Basic Speech-to-Text Systems: Convert speech to text without medical context understanding or clinical structure, requiring extensive manual editing by clinicians.
• US-Focused AI Scribes: Systems designed for the United States healthcare environment lack integration with Australian-specific requirements including Medicare Benefits Schedule (MBS), Pharmaceutical Benefits Scheme (PBS), and Therapeutic Goods Administration (TGA) approved medication databases.
• Single-Language Systems: Fail to support Australia's multicultural patient population speaking languages other than English.

No existing system combines: (1) ambient voice capture of natural doctor-patient conversations; (2) multilingual transcription supporting over 100 languages; (3) AI-powered extraction of clinical entities (symptoms, diagnoses, medications); (4) automated generation of structured SOAP format clinical notes; (5) integration with Australian healthcare systems (MBS billing codes, PBS medication codes); (6) real-time generation of medical orders and prescriptions; and (7) compliance with Australian privacy regulations.

3. SUMMARY OF THE INVENTION

The present invention provides a clinical documentation system comprising:

• An ambient audio capture module utilizing web browser speech recognition APIs (Web Speech API) to capture doctor-patient conversations in real-time without requiring manual recording activation;
• A continuous speech recognition engine with speaker diarization capabilities to distinguish between clinician speech and patient speech;
• A multilingual transcription system supporting over 100 languages including Mandarin, Arabic, Vietnamese, Cantonese, Italian, Greek, Hindi, and Punjabi, utilizing language-specific speech recognition models;
• An AI enhancement module that processes raw transcripts to: (a) correct speech recognition errors; (b) enhance medical terminology accuracy; (c) detect and handle language switching during consultations; (d) apply clinical context understanding;
• A natural language processing (NLP) engine that analyzes transcripts to extract structured clinical information including: chief complaints, symptoms, medical history, physical examination findings, vital signs, and treatment discussions;
• An automated SOAP note generation module utilizing large language models (LLMs) to create structured clinical documentation comprising:
  • Subjective section (patient-reported symptoms and history)
  • Objective section (examination findings and observations)
  • Assessment section (diagnoses and differential diagnoses)
  • Plan section (treatment recommendations and follow-up)
• A medical orders generation system that automatically creates structured orders for laboratory tests and imaging studies based on clinical need;
• A prescription generation module that produces detailed medication orders including drug names, dosages, routes of administration, frequencies, durations, and total quantities;
• An Australian healthcare integration module that:
  • Suggests appropriate MBS item numbers for consultations and procedures
  • Identifies PBS-listed medications and provides PBS codes
  • Calculates Medicare rebate amounts
  • Ensures medication compliance with TGA approvals
• An Indian healthcare integration module (alternative implementation) that:
  • Follows CDSCO and ICMR guidelines
  • Suggests medications from the National List of Essential Medicines (NLEM)
  • Provides Jan Aushadhi generic alternatives with pricing in INR
• A database storage system for persisting clinical sessions including transcripts, generated notes, consultation duration, and metadata;
• A real-time user interface displaying live transcription with visual indicators for ongoing speech capture and processing status.

Technical Advantages:

• Reduces clinical documentation time by 60-75% compared to manual entry;
• Supports over 100 languages enabling care for multicultural patient populations;
• Generates structured SOAP notes automatically from conversational speech;
• Integrates seamlessly with Australian Medicare billing and PBS prescribing systems;
• Provides real-time transcription allowing clinicians to verify accuracy during consultations;
• Eliminates post-consultation documentation burden, allowing immediate generation of complete clinical records;
• Ensures consistency in clinical documentation structure and completeness.

4. DETAILED DESCRIPTION OF THE INVENTION

4.1 System Architecture

The system comprises client-side and server-side components operating in concert. The client application executes within a web browser environment, utilizing native browser APIs for audio capture. Server-side processing occurs on cloud infrastructure (preferably AWS Sydney region for Australian data sovereignty compliance).

4.2 Ambient Audio Capture Module

The audio capture module implements the W3C Web Speech API, specifically the SpeechRecognition interface (or vendor-prefixed webkitSpeechRecognition). The system configures the recognition interface with the following parameters:

• Continuous Mode: Set to true, allowing ongoing capture throughout the consultation without manual intervention.
• Interim Results: Enabled to provide real-time transcription feedback to clinicians.
• Language Code: Dynamically set based on user-selected or auto-detected language (e.g., "en-US", "zh-CN", "ar-SA").
• Maximum Alternatives: Set to retrieve multiple recognition alternatives for improved accuracy selection.

The audio capture operates "ambient" in nature, meaning once activated, it continuously listens and transcribes without requiring clinician interaction, mimicking the presence of a human medical scribe.

4.3 Continuous Speech Recognition with Speaker Diarization

The speech recognition engine processes audio streams in real-time, distinguishing between:

1. Interim Results: Partial transcriptions displayed immediately for real-time feedback;
2. Final Results: Confirmed transcription segments with higher confidence scores, permanently added to the consultation transcript.

While the browser API provides basic transcription, the system implements post-processing speaker attribution logic to identify and label speech segments as originating from clinician or patient based on contextual analysis of medical terminology density and sentence structure patterns.

4.4 Multilingual Support and Language Detection

The system maintains a database of supported languages with the following attributes for each:

• ISO 639-1/BCP 47 language code (e.g., "en-US", "hi-IN", "ar-SA")
• Display name in English
• Native name in the language's own script
• Flag emoji for visual identification
• Right-to-left (RTL) text direction indicator
• Voice input/output support status

The system includes over 100 language configurations covering major languages spoken in Australia's multicultural communities. Users select their preferred language before consultation start, which configures both the speech recognition language model and subsequent AI processing language context.

An automatic language detection feature monitors transcription content and identifies when patients switch languages mid-conversation, triggering translation services to maintain documentation in the clinician's preferred language while preserving clinical meaning.

4.5 AI Transcript Enhancement Module

Raw speech recognition output often contains errors due to:

• Fast speech causing word concatenation
• Slow speech causing word fragmentation
• Accented pronunciation variations
• Medical terminology misrecognition
• Homophone confusion (e.g., "patient" vs "patience")

To address these issues, the system implements a post-processing AI enhancement module that:

1. Analyzes raw transcript segments using large language models (LLMs) with medical domain knowledge;
2. Corrects speech recognition errors by applying contextual understanding of clinical conversations;
3. Enhances medical terminology accuracy, converting layman terms to proper clinical nomenclature;
4. Normalizes drug names, dosages, and medical conditions to standard formats;
5. Applies proper punctuation and capitalization;
6. Detects and translates language-switched segments back to the primary documentation language;
7. Generates confidence scores and logs corrections made for quality assurance;
8. Identifies clinical terms enhanced for audit trails.

The enhancement process typically completes within 1-2 seconds after speech finalization, providing near-real-time corrected transcription.

4.6 Natural Language Processing and Clinical Entity Extraction

After transcript enhancement, the NLP engine analyzes the complete consultation transcript to extract structured clinical information. This process utilizes LLMs trained on medical conversations to identify:

• Chief Complaint: The primary reason for the patient's visit;
• History of Present Illness (HPI): Timeline and progression of current symptoms;
• Symptoms: Individual reported symptoms with associated characteristics (onset, duration, severity);
• Physical Examination Findings: Clinician observations and examination results;
• Vital Signs: Blood pressure, heart rate, temperature, respiratory rate, oxygen saturation;
• Diagnoses: Primary diagnosis and differential diagnoses with associated reasoning;
• Treatment Plan: Recommended interventions, medications, procedures;
• Follow-up Instructions: Return visit timing and instructions.

4.7 Automated SOAP Note Generation

The SOAP note generation module creates structured clinical documentation following the universally recognized SOAP format:

Subjective Section:

Contains patient-reported information including chief complaint, history of present illness, past medical history mentions, and symptom descriptions. The AI extracts this information from patient statements within the transcript, filtering clinician questions and focusing on patient responses.

Objective Section:

Captures clinician observations including physical examination findings, vital signs mentioned during the consultation, and clinical observations. The system distinguishes objective findings from subjective symptoms by analyzing source attribution and statement structure.

Assessment Section:

Presents the clinician's diagnostic assessment including:

• Primary diagnosis with supporting rationale
• Differential diagnoses ranked by probability
• Clinical reasoning for each diagnostic consideration
• Supporting evidence from history, examination, or prior investigations

Plan Section:

Outlines treatment strategy including:

• Immediate interventions
• Ongoing management recommendations
• Lifestyle modifications
• Monitoring parameters
• Follow-up timing and instructions

4.8 Medical Orders Generation System

The system automatically generates structured medical orders based on clinical assessment. Each order comprises:

• Order Type: Classification as laboratory test or imaging study;
• Order Name: Specific test or study requested (e.g., "Complete Blood Count", "Chest X-Ray");
• Clinical Rationale: Reason for ordering based on clinical presentation;
• MBS Item Number (Australian Implementation): Appropriate Medicare Benefits Schedule item code for billing and reimbursement.

4.9 Prescription Generation Module

The prescription module creates detailed medication orders from treatment discussions captured in the transcript. Each prescription includes:

• Medication Name: Generic or brand name as discussed;
• Dosage: Strength per administration unit (e.g., "500mg");
• Route of Administration: Oral, intravenous, intramuscular, subcutaneous, topical, inhalation, rectal, or other;
• Frequency: Timing description (e.g., "twice daily", "every 8 hours");
• Times Per Day: Numerical frequency for clarity;
• Duration: Number of days treatment should continue;
• Total Quantity: Total number of units to be dispensed;
• Instructions: Special administration instructions (e.g., "take with food", "take on empty stomach").

4.10 Australian Healthcare System Integration

For Australian implementation, the system includes specialized integration modules:

4.10.1 Medicare Billing Integration:

The system analyzes consultation characteristics to suggest appropriate MBS billing:

• Consultation Type Classification: Determines if consultation qualifies as standard, long, complex, or specialized based on duration, complexity, and components documented;
• MBS Item Number Selection: Suggests appropriate item numbers (e.g., 23 for standard consultation, 36 for long consultation, 44 for complex consultation);
• Rebate Calculation: Provides Medicare rebate amount for selected item number;
• Procedure Item Codes: For procedures performed during consultation, suggests relevant MBS codes.

4.10.2 PBS Medication Integration:

The prescription module identifies PBS-listed medications and provides:

• PBS Code: Pharmaceutical Benefits Scheme code for the medication;
• PBS Listed Status: Boolean indicator whether medication is PBS subsidized;
• Therapeutic Group: PBS therapeutic category;
• Authority Requirements: Indication if authority prescription is required for PBS subsidy.

4.10.3 TGA Compliance:

Prescribed medications are cross-referenced against TGA-approved medicines to ensure regulatory compliance for Australian prescribing.

4.11 Indian Healthcare System Integration (Alternative)

For Indian market implementation, the system provides alternative healthcare system integration:

• CDSCO Compliance: Ensures medications comply with Central Drugs Standard Control Organization approvals;
• NLEM Medication Prioritization: Suggests medications from the National List of Essential Medicines;
• Jan Aushadhi Generics: For prescribed branded medications, provides Jan Aushadhi Kendra generic alternatives;
• INR Pricing: Displays approximate medication costs in Indian Rupees;
• ICMR Guidelines: Treatment recommendations align with Indian Council of Medical Research clinical practice guidelines;
• India-Specific Conditions: Enhanced detection and treatment protocols for prevalent conditions like dengue, malaria, tuberculosis, and typhoid.

4.12 Database Persistence

The system stores clinical sessions in a structured database (AmbientSession entity) with the following schema:

• Patient identifier (reference to patient record)
• Visit type (Consultation, Follow-up, Emergency, Routine Checkup)
• Complete transcript text
• Generated SOAP note (stored as structured JSON)
• Medical orders array
• Prescriptions array
• Session status (draft, processing, completed)
• Duration in seconds
• Timestamp metadata (creation, last update)
• Clinician identifier

4.13 User Interface Components

The system provides a web-based user interface comprising:

• Recording Control Panel: Start/stop buttons for audio capture with visual indicator of recording status and elapsed time display;
• Live Transcript View: Scrollable pane displaying ongoing transcription with real-time updates as speech is recognized;
• Language Selector: Dropdown menu for selecting consultation language from supported options;
• Country/Region Selector: Toggle for Australian vs Indian healthcare system integration;
• Processing Indicator: Visual feedback during AI analysis of transcript;
• Generated Note Display: Tabbed interface showing SOAP note, orders, and prescriptions;
• Copy/Export Functions: Buttons for copying specific sections or exporting complete documentation.