OBJECTIVES: Patients often receive health care from multiple organizations. Privacy Preserving Record Linkage (PPRL) is a technology for linking patient records without releasing personally identifiable information. We compared a commercial PPRL tool that uses the XGBoost machine learning algorithm with Care Everywhere (CE), a widely used rule-based patient linkage module. MATERIALS AND METHODS: We matched the complete patient populations from Cedars-Sinai Health System and University of California, Los Angeles (UCLA) Health using the XGBoost PPRL tool at each of 3 score thresholds (98, 95, and 90), reflecting stricter vs more permissive matching. We compared PPRL matches with CE matches for the cohort of 849 157 patients who had been queried by CE from UCLA to Cedars-Sinai over 18 months. To classify proposed matches as false, uncertain or correct matches, 2 reviewers manually reviewed a random sample of 1200 patients representing each category of matches. RESULTS: Care Everywhere matched 18% of the cohort, whereas PPRL matched 9%, 27%, and 29% of the cohort using the 98, 95, and 90 thresholds, respectively. Projecting the false match rates from the manual review to the original populations, precision for CE was 99.6% (95% CI, 97.8%-100%). Precision for PPRL was 100% (95% CI, 99.2%-100%), 99.4% (95% CI, 97.4%-99.9%), and 98.7% (95% CI, 96.5%-99.4%) at the 3 thresholds, respectively. Using CE and PPRL matches together as a proxy gold standard, recall for CE was 61.5% (95% CI, 60.3%-61.9%) and for PPRL was 30.6% (95% CI, 30.3%-30.7%), 92.2% (95% CI, 90.2%-92.7%), and 96.8% (95% CI, 94.6%-97.5%) at each threshold, respectively. CONCLUSIONS: The precision and recall of PPRL matching differed substantially across the available match thresholds. Compared with the rule-based system, PPRL at the 95 threshold had 50% higher recall with similar precision. Privacy Preserving Record Linkage holds promise for improving research, but users must choose the precision vs recall needed for their application.

Using 2023-2024 U.S. National Health Interview Survey data, we found that digital health literacy (dHL) mediated nearly half of the difference in telehealth use between Latino adults with non-English and English language preference. These findings identify dHL as a modifiable mechanism linking linguistic and digital access barriers, underscoring the need for multilingual, inclusive, and equitable telehealth design.

OBJECTIVES: We report on findings from a meeting convened by the American College of Medical Informatics (ACMI) to characterize aspects of the patient experience that could be improved using informatics. MATERIALS AND METHODS: The American College of Medical Informatics fellows were invited to share their experiences as patients and suggest informatics approaches that may improve the patient experience. RESULTS: We identified 4 themes: (1) getting the right care, (2) data sharing and data interoperability, (3) guiding low-cost evaluations, and (4) predictive analytics. DISCUSSION: Despite widespread adoption of health IT, patient experiences remain far from optimal. CONCLUSION: The American College of Medical Informatics fellows identified informatics approaches, applications, and research areas that have the potential to improve patient experiences with health care systems.

OBJECTIVE: We aimed to develop a distributed, immutable, and highly available cross-cloud blockchain system to facilitate federated data analysis activities among multiple institutions. MATERIALS AND METHODS: We preprocessed 9166 COVID-19 Structured Query Language (SQL) code, summary statistics, and user activity logs, from the GitHub repository of the Reliable Response Data Discovery for COVID-19 (R2D2) Consortium. The repository collected local summary statistics from participating institutions and aggregated the global result to a COVID-19-related clinical query, previously posted by clinicians on a website. We developed both on-chain and off-chain components to store/query these activity logs and their associated queries/results on a blockchain for immutability, transparency, and high availability of research communication. We measured run-time efficiency of contract deployment, network transactions, and confirmed the accuracy of recorded logs compared to a centralized baseline solution. RESULTS: The smart contract deployment took 4.5 s on an average. The time to record an activity log on blockchain was slightly over 2 s, versus 5-9 s for baseline. For querying, each query took on an average less than 0.4 s on blockchain, versus around 2.1 s for baseline. DISCUSSION: The low deployment, recording, and querying times confirm the feasibility of our cross-cloud, blockchain-based federated data analysis system. We have yet to evaluate the system on a larger network with multiple nodes per cloud, to consider how to accommodate a surge in activities, and to investigate methods to lower querying time as the blockchain grows. CONCLUSION: Blockchain technology can be used to support federated data analysis among multiple institutions.

OBJECTIVE: Patient portals are increasingly used to recruit patients in research studies, but communication response rates remain low without tactics such as financial incentives or manual outreach. We evaluated a new method of study enrollment by embedding a study information sheet and HIPAA authorization form (HAF) into the patient portal preCheck-in (where patients report basic information like allergies). MATERIALS AND METHODS: Eligible patients who enrolled received an after-visit patient-reported outcomes survey through the patient portal. No additional recruitment/messaging efforts were made. RESULTS: A total of 386 of 843 patients completed preCheck-in, 308 of whom signed the HAF and enrolled in the study (37% enrollment rate). Of 93 patients who were eligible to receive the after-visit survey, 45 completed it (48% completion rate). CONCLUSION: Enrollment and survey completion rates were higher than what is typically seen with recruitment by patient portal messaging, suggesting that preCheck-in recruitment can enhance research study recruitment and warrants further investigation.

OBJECTIVE: Clinical decision support (CDS) alerts may improve health care quality but "alert fatigue" can reduce provider responsiveness. We analyzed how the introduction of competing alerts affected provider adherence to a single depression screening alert. MATERIALS AND METHODS: We analyzed the audit data from all occurrences of a CDS alert at a large academic health system. For patients who screen positive for depression during ambulatory visits, a noninterruptive alert was presented, offering a number of relevant documentation actions. Alert adherence was defined as the selection of any option offered within the alert. We assessed the effect of competing clinical guidance alerts presented during the same encounter and the total of all CDS alerts that the same provider had seen in the prior 90 days, on the probability of depression screen alert adherence, adjusting for physician and patient characteristics. RESULTS: The depression alert fired during 55 649 office visits involving 418 physicians and 40 474 patients over 41 months. After adjustment, physicians who had seen the most alerts in the prior 90 days were much less likely to respond (adjusted OR highest-lowest quartile, 0.38; 95% CI 0.35-0.42; P < .001). Competing alerts in the same visit further reduced the likelihood of adherence only among physicians in the middle two quartiles of alert exposure in the prior 90 days. CONCLUSIONS: Adherence to a noninterruptive depression alert was strongly associated with the provider's cumulative alert exposure over the past quarter. Health systems should monitor providers' recent alert exposure as a measure of alert fatigue.

OBJECTIVE: To utilize, in an individual and institutional privacy-preserving manner, electronic health record (EHR) data from 202 hospitals by analyzing answers to COVID-19-related questions and posting these answers online. MATERIALS AND METHODS: We developed a distributed, federated network of 12 health systems that harmonized their EHRs and submitted aggregate answers to consortia questions posted at https://www.covid19questions.org. Our consortium developed processes and implemented distributed algorithms to produce answers to a variety of questions. We were able to generate counts, descriptive statistics, and build a multivariate, iterative regression model without centralizing individual-level data. RESULTS: Our public website contains answers to various clinical questions, a web form for users to ask questions in natural language, and a list of items that are currently pending responses. The results show, for example, that patients who were taking angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers, within the year before admission, had lower unadjusted in-hospital mortality rates. We also showed that, when adjusted for, age, sex, and ethnicity were not significantly associated with mortality. We demonstrated that it is possible to answer questions about COVID-19 using EHR data from systems that have different policies and must follow various regulations, without moving data out of their health systems. DISCUSSION AND CONCLUSIONS: We present an alternative or a complement to centralized COVID-19 registries of EHR data. We can use multivariate distributed logistic regression on observations recorded in the process of care to generate results without transferring individual-level data outside the health systems.

OBJECTIVE: The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity. MATERIALS AND METHODS: Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site. RESULTS: The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review. DISCUSSION: We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions. CONCLUSIONS: We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Objective: To evaluate provider responses to a narrowly targeted "Best Practice Advisory" (BPA) alert for the intensification of blood pressure medications for persons with diabetes before and after implementation of a "chart closure" hard stop, which is non-interruptive but demands an action or dismissal before the chart can be closed. Materials and Methods: We designed a BPA that fired alerts within an electronic health record (EHR) system during outpatient encounters for patients with diabetes when they had elevated blood pressures and were not on angiotensin receptor blocking medications. The BPA alerts were implemented in eight primary care practices within UCLA Health. We compared data on provider responses to the alerts before and after implementing a "chart closure" hard stop, and we conducted chart reviews to adjudicate each alert's appropriateness. Results: Providers responded to alerts more often after the "chart closure" hard stop was implemented (P < .001). Among 284 alert firings over 16 months, we judged 107 (37.7%) to be clinically unnecessary or inappropriate based on chart review. Among the remainder, which represent clear opportunities for treatment, providers ordered the indicated medication more often (41% vs 75%) after the "chart closure" hard stop was implemented (P = .001). Discussion: The BPA alerts for diabetes and blood pressure control achieved relatively high specificity. The "chart closure" hard stop improved provider attention to the alerts and was effective at getting patients treated when they needed it. Conclusion: Targeting specific omitted medication classes can produce relatively specific alerts that may reduce alert fatigue, and using a "chart closure" hard stop may prompt providers to take action without excessively disrupting their workflow.

OBJECTIVE: We sought to assess the potential of a widely available source of electronic medication data to prevent medication history errors and resultant inpatient order errors. METHODS: We used admission medication history (AMH) data from a recent clinical trial that identified 1017 AMH errors and 419 resultant inpatient order errors among 194 hospital admissions of predominantly older adult patients on complex medication regimens. Among the subset of patients for whom we could access current Surescripts electronic pharmacy claims data (SEPCD), two pharmacists independently assessed error severity and our main outcome, which was whether SEPCD (1) was unrelated to the medication error; (2) probably would not have prevented the error; (3) might have prevented the error; or (4) probably would have prevented the error. RESULTS: Seventy patients had both AMH errors and current, accessible SEPCD. SEPCD probably would have prevented 110 (35%) of 315 AMH errors and 46 (31%) of 147 resultant inpatient order errors. When we excluded the least severe medication errors, SEPCD probably would have prevented 99 (47%) of 209 AMH errors and 37 (61%) of 61 resultant inpatient order errors. SEPCD probably would have prevented at least one AMH error in 42 (60%) of 70 patients. CONCLUSION: When current SEPCD was available for older adult patients on complex medication regimens, it had substantial potential to prevent AMH errors and resultant inpatient order errors, with greater potential to prevent more severe errors. Further study is needed to measure the benefit of SEPCD in actual use at hospital admission.

This article describes the patient-centered Scalable National Network for Effectiveness Research (pSCANNER), which is part of the recently formed PCORnet, a national network composed of learning healthcare systems and patient-powered research networks funded by the Patient Centered Outcomes Research Institute (PCORI). It is designed to be a stakeholder-governed federated network that uses a distributed architecture to integrate data from three existing networks covering over 21 million patients in all 50 states: (1) VA Informatics and Computing Infrastructure (VINCI), with data from Veteran Health Administration's 151 inpatient and 909 ambulatory care and community-based outpatient clinics; (2) the University of California Research exchange (UC-ReX) network, with data from UC Davis, Irvine, Los Angeles, San Francisco, and San Diego; and (3) SCANNER, a consortium of UCSD, Tennessee VA, and three federally qualified health systems in the Los Angeles area supplemented with claims and health information exchange data, led by the University of Southern California. Initial use cases will focus on three conditions: (1) congestive heart failure; (2) Kawasaki disease; (3) obesity. Stakeholders, such as patients, clinicians, and health service researchers, will be engaged to prioritize research questions to be answered through the network. We will use a privacy-preserving distributed computation model with synchronous and asynchronous modes. The distributed system will be based on a common data model that allows the construction and evaluation of distributed multivariate models for a variety of statistical analyses.

OBJECTIVE: To identify key principles for establishing a national clinical decision support (CDS) knowledge sharing framework. MATERIALS AND METHODS: As part of an initiative by the US Office of the National Coordinator for Health IT (ONC) to establish a framework for national CDS knowledge sharing, key stakeholders were identified. Stakeholders' viewpoints were obtained through surveys and in-depth interviews, and findings and relevant insights were summarized. Based on these insights, key principles were formulated for establishing a national CDS knowledge sharing framework. RESULTS: Nineteen key stakeholders were recruited, including six executives from electronic health record system vendors, seven executives from knowledge content producers, three executives from healthcare provider organizations, and three additional experts in clinical informatics. Based on these stakeholders' insights, five key principles were identified for effectively sharing CDS knowledge nationally. These principles are (1) prioritize and support the creation and maintenance of a national CDS knowledge sharing framework; (2) facilitate the development of high-value content and tooling, preferably in an open-source manner; (3) accelerate the development or licensing of required, pragmatic standards; (4) acknowledge and address medicolegal liability concerns; and (5) establish a self-sustaining business model. DISCUSSION: Based on the principles identified, a roadmap for national CDS knowledge sharing was developed through the ONC's Advancing CDS initiative. CONCLUSION: The study findings may serve as a useful guide for ongoing activities by the ONC and others to establish a national framework for sharing CDS knowledge and improving clinical care.

OBJECTIVE: Alert fatigue represents a common problem associated with the use of clinical decision support systems in electronic health records (EHR). This problem is particularly profound with drug-drug interaction (DDI) alerts for which studies have reported override rates of approximately 90%. The objective of this study is to report consensus-based recommendations of an expert panel on DDI that can be safely made non-interruptive to the provider's workflow, in EHR, in an attempt to reduce alert fatigue. METHODS: We utilized an expert panel process to rate the interactions. Panelists had expertise in medicine, pharmacy, pharmacology and clinical informatics, and represented both academic institutions and vendors of medication knowledge bases and EHR. In addition, representatives from the US Food and Drug Administration and the American Society of Health-System Pharmacy contributed to the discussions. RESULTS: Recommendations and considerations of the panel resulted in the creation of a list of 33 class-based low-priority DDI that do not warrant being interruptive alerts in EHR. In one institution, these accounted for 36% of the interactions displayed. DISCUSSION: Development and customization of the content of medication knowledge bases that drive DDI alerting represents a resource-intensive task. Creation of a standardized list of low-priority DDI may help reduce alert fatigue across EHR. CONCLUSIONS: Future efforts might include the development of a consortium to maintain this list over time. Such a list could also be used in conjunction with financial incentives tied to its adoption in EHR.

OBJECTIVE: To develop a set of high-severity, clinically significant drug-drug interactions (DDIs) for use in electronic health records (EHRs). METHODS: A panel of experts was convened with the goal of identifying critical DDIs that should be used for generating medication-related decision support alerts in all EHRs. Panelists included medication knowledge base vendors, EHR vendors, in-house knowledge base developers from academic medical centers, and both federal and private agencies involved in the regulation of medication use. Candidate DDIs were assessed by the panel based on the consequence of the interaction, severity levels assigned to them across various medication knowledge bases, availability of therapeutic alternatives, monitoring/management options, predisposing factors, and the probability of the interaction based on the strength of evidence available in the literature. RESULTS: Of 31 DDIs considered to be high risk, the panel approved a final list of 15 interactions. Panelists agreed that this list represented drugs that are contraindicated for concurrent use, though it does not necessarily represent a complete list of all such interacting drug pairs. For other drug interactions, severity may depend on additional factors, such as patient conditions or timing of co-administration. DISCUSSION: The panel provided recommendations on the creation, maintenance, and implementation of a central repository of high severity interactions. CONCLUSIONS: A set of highly clinically significant drug-drug interactions was identified, for which warnings should be generated in all EHRs. The panel highlighted the complexity of issues surrounding development and implementation of such a list.

Developing effective clinical decision support (CDS) systems for the highly complex and dynamic domain of clinical medicine is a serious challenge for designers. Poor usability is one of the core barriers to adoption and a deterrent to its routine use. We reviewed reports describing system implementation efforts and collected best available design conventions, procedures, practices and lessons learned in order to provide developers a short compendium of design goals and recommended principles. This targeted review is focused on CDS related to medication prescribing. Published reports suggest that important principles include consistency of design concepts across networked systems, use of appropriate visual representation of clinical data, use of controlled terminology, presenting advice at the time and place of decision making and matching the most appropriate CDS interventions to clinical goals. Specificity and contextual relevance can be increased by periodic review of trigger rules, analysis of performance logs and maintenance of accurate allergy, problem and medication lists in health records in order to help avoid excessive alerting. Developers need to adopt design practices that include user-centered, iterative design and common standards based on human-computer interaction (HCI) research methods rooted in ethnography and cognitive science. Suggestions outlined in this report may help clarify the goals of optimal CDS design but larger national initiatives are needed for systematic application of human factors in health information technology (HIT) development. Appropriate design strategies are essential for developing meaningful decision support systems that meet the grand challenges of high-quality healthcare.

OBJECTIVE: To evaluate the ability of the structure and code sets specified in the National Council for Prescription Drug Programs Structured and Codified Sig Format to represent ambulatory electronic prescriptions. DESIGN: We parsed the Sig strings from a sample of 20,161 de-identified ambulatory e-prescriptions into variables representing the fields of the Structured and Codified Sig Format. A stratified random sample of these representations was then reviewed by a group of experts. For codified Sig fields, we attempted to map the actual words used by prescribers to the equivalent terms in the designated terminology. MEASUREMENTS: Proportion of prescriptions that the Format could fully represent; proportion of terms used that could be mapped to the designated terminology. RESULTS: The fields defined in the Format could fully represent 95% of Sigs (95% CI 93% to 97%), but ambiguities were identified, particularly in representing multiple-step instructions. The terms used by prescribers could be codified for only 60% of dose delivery methods, 84% of dose forms, 82% of vehicles, 95% of routes, 70% of sites, 33% of administration timings, and 93% of indications. LIMITATIONS: The findings are based on a retrospective sample of ambulatory prescriptions derived mostly from primary care physicians. CONCLUSION: The fields defined in the Format could represent most of the patient instructions in a large prescription sample, but prior to its mandatory adoption, further work is needed to ensure that potential ambiguities are addressed and that a complete set of terms is available for the codified fields.

OBJECTIVE To compare the experiences of e-prescribing users and nonusers regarding prescription safety and workload and to assess the use of information from two e-prescribing standards (for medication history and formulary and benefit information), as they are implemented. DESIGN Cross-sectional survey of physicians who either had installed or were awaiting installation of one of two commercial e-prescribing systems. MEASUREMENTS Perceptions about medication history and formulary and benefit information among all respondents, and among e-prescribing users, experiences with system usability, job performance impact, and amount of e-prescribing. RESULTS Of 395 eligible physicians, 228 (58%) completed the survey. E-prescribers (n = 139) were more likely than non-e-prescribers (n = 89) to perceive that they could identify clinically important drug-drug interactions (83 versus 67%, p = 0.004) but not that they could identify prescriptions from other providers (65 versus 60%, p = 0.49). They also perceived no significant difference in calls about drug coverage problems (76 versus 71% reported getting 10 or fewer such calls per week; p = 0.43). Most e-prescribers reported high satisfaction with their systems, but 17% had stopped using the system and another 46% said they sometimes reverted to handwriting for prescriptions that they could write electronically. The volume of e-prescribing was correlated with perceptions that it enhanced job performance, whereas quitting was associated with perceptions of poor usability. CONCLUSIONS E-prescribing users reported patient safety benefits but they did not perceive the enhanced benefits expected from using standardized medication history or formulary and benefit information. Additional work is needed for these standards to have the desired effects.

OBJECTIVE: To compare the functional capabilities being offered by commercial ambulatory electronic prescribing systems with a set of expert panel recommendations. DESIGN: A descriptive field study of ten commercially available ambulatory electronic prescribing systems, each of which had established a significant market presence. Data were collected from vendors by telephone interview and at sites where the systems were functioning through direct observation of the systems and through personal interviews with prescribers and technical staff. MEASUREMENTS: The capabilities of electronic prescribing systems were compared with 60 expert panel recommendations for capabilities that would improve patient safety, health outcomes, or patients' costs. Each recommended capability was judged as having been implemented fully, partially, or not at all by each system to which the recommendation applied. Vendors' claims about capabilities were compared with the capabilities found in the site visits. RESULTS: On average, the systems fully implemented 50% of the recommended capabilities, with individual systems ranging from 26% to 64% implementation. Only 15% of the recommended capabilities were not implemented by any system. Prescribing systems that were part of electronic health records (EHRs) tended to implement more recommendations. Vendors' claims about their systems' capabilities had a 96% sensitivity and a 72% specificity when site visit findings were considered the gold standard. CONCLUSIONS: The commercial electronic prescribing marketplace may not be selecting for capabilities that would most benefit patients. Electronic prescribing standards should include minimal functional capabilities, and certification of adherence to standards may need to take place where systems are installed and operating.

OBJECTIVE: Electronic prescribing (e-prescribing) may substantially improve health care quality and efficiency, but the available systems are complex and their heterogeneity makes comparing and evaluating them a challenge. The authors aimed to develop a conceptual framework for anticipating the effects of alternative designs for outpatient e-prescribing systems. DESIGN: Based on a literature review and on telephone interviews with e-prescribing vendors, the authors identified distinct e-prescribing functional capabilities and developed a conceptual framework for evaluating e-prescribing systems' potential effects based on their capabilities. Analyses of two commercial e-prescribing systems are presented as examples of applying the conceptual framework. MEASUREMENTS: Major e-prescribing functional capabilities identified and the availability of evidence to support their specific effects. RESULTS: The proposed framework for evaluating e-prescribing systems is organized using a process model of medication management. Fourteen e-prescribing functional capabilities are identified within the model. Evidence is identified to support eight specific effects for six of the functional capabilities. The evidence also shows that a functional capability with generally positive effects can be implemented in a way that creates unintended hazards. Applying the framework involves identifying an e-prescribing system's functional capabilities within the process model and then assessing the effects that could be expected from each capability in the proposed clinical environment. CONCLUSION: The proposed conceptual framework supports the integration of available evidence in considering the full range of effects from e-prescribing design alternatives. More research is needed into the effects of specific e-prescribing functional alternatives. Until more is known, e-prescribing initiatives should include provisions to monitor for unintended hazards.

OBJECTIVE: The aim of this study was to determine whether physician offices located in high-minority and low-income neighborhoods have different levels of access to information technology than offices located in lower-minority and higher-income areas. DESIGN: A cross-sectional survey was conducted of pediatrics, family medicine, and general practice offices in Orange County, California. Survey data were linked with community demographic data from the 2000 Census using a geographical information system. RESULTS: Of 307 offices surveyed, 141 responded (46%). Offices located in high-minority and high-poverty areas were as likely to respond as other offices. Among responding offices, 94% had a computer, 77% had Web access, 29% had broadband Internet access, and 53% used computerized scheduling and billing systems. Offices located in minority and low-income communities had equivalent access to each technology. Offices in communities with larger proportions of Hispanics were less likely to have practice Web pages, but other uses of the Internet were not associated with practice location. Offices reported high levels of interest in online clinical systems but also high levels of concern about these systems' usability and confidentiality. Offices with Web access and those with practice management systems expressed greater interest in online clinical systems but also greater levels of concern about usability and confidentiality. These attitudes were equivalent among offices in different communities. CONCLUSION: Primary care offices located in poor and minority communities in a large, suburban county had high levels of access to and interest in Web-based systems. Physicians' offices may therefore provide a venue for online services aimed at improving health outcomes for poor and minority communities. Research is needed in other geographic regions to determine the generalizability of these findings.

Consenting visitors to a health survey Web site were randomly assigned to a "matrix" presentation or an "expanded" presentation of survey response options. Among 4,208 visitors to the site over 3 months, 1,615 (38 percent) participated by giving consent and completing the survey. During a pre-trial period, when consent was not required, 914 of 1,667 visitors (55 percent) participated (odds ratio 1.9, P<0.0001). Mean response times were 5.07 minutes for the matrix format and 5.22 minutes for the expanded format (P=0.16). Neither health status scores nor alpha reliability coefficients were substantially influenced by the survey format, but health status scores varied with age and gender as expected from U.S. population norms. In conclusion, presenting response options in a matrix format may not substantially speed survey completion. This study demonstrates a method for rapidly evaluating interface design alternatives using anonymous Web volunteers who have provided informed consent.

OBJECTIVE: Development of methods for building concept models to support structured data entry and image retrieval in chest radiography. DESIGN: An organizing model for chest-radiographic reporting was built by analyzing manually a set of natural-language chest-radiograph reports. During model building, clinician-informaticians judged alternative conceptual structures according to four criteria: content of clinically relevant detail, provision for semantic constraints, provision for canonical forms, and simplicity. The organizing model was applied in representing three sample reports in their entirety. To explore the potential for automatic model discovery, the representation of one sample report was compared with the noun phrases derived from the same report by the CLARIT natural-language processing system. RESULTS: The organizing model for chest-radiographic reporting consists of 62 concept types and 17 relations, arranged in an inheritance network. The broadest types in the model include finding, anatomic locus, procedure, attribute, and status. Diagnoses are modeled as a subtype of finding. Representing three sample reports in their entirety added 79 narrower concept types. Some CLARIT noun phrases suggested valid associations among subtypes of finding, status, and anatomic locus. CONCLUSIONS: A manual modeling process utilizing explicitly stated criteria for making modeling decisions produced an organizing model that showed consistency in early testing. A combination of top-down and bottom-up modeling was required. Natural-language processing may inform model building, but algorithms that would replace manual modeling were not discovered. Further progress in modeling will require methods for objective model evaluation and tools for formalizing the model-building process.

The Canon Group is an informal organization of medical informatics researchers who are working on the problem of developing a "deeper" representation formalism for use in exchanging data and developing applications. Individuals in the group represent experts in such areas as knowledge representation and computational linguistics, as well as in a variety of medical subdisciplines. All share the view that current mechanisms for the characterization of medical phenomena are either inadequate (limited or rigid) or idiosyncratic (useful for a specific application but incapable of being generalized or extended). The Group proposes to focus on the design of a general schema for medical-language representation including the specification of the resources and associated procedures required to map language (including standard terminologies) into representations that make all implicit relations "visible," reveal "hidden attributes," and generally resolve ambiguous or vague references. The Group is proceeding by examining large numbers of texts (records) in medical sub-domains to identify candidate "concepts" and by attempting to develop general rules and representations for elements such as attributes and values so that all concepts may be expressed uniformly.