INTRODUCTION: The rapid advancement of artificial intelligence (AI) has led to significant transformations in health and healthcare. As AI technologies continue to evolve, there is an urgent need to establish a unified framework that guides the design, implementation, and evaluation of AI-driven interventions across individual and population health contexts. APPROACH: In response to this need, the National Academy of Medicine (NAM) has initiated the development of an AI code of conduct (AICC) through its Digital Health Action Collaborative. This code of conduct is grounded in shared principles and commitments, aiming to actualize ethical and effective AI practices within the broader health and healthcare ecosystem. Given its specialized expertise and insight, the biomedical informatics (BMI) community plays a pivotal role in shaping and applying these guidelines. RECOMMENDATIONS: We, as members of the AICC Steering Committee and the NAM Digital Health Action Collaborative, urge BMI educators, researchers, and practitioners to engage actively in refining and implementing the AICC. This involvement is critical to ensuring that the code is robust, applicable, and continuously improved to meet the evolving challenges facing health and healthcare.

OBJECTIVE: To assess whether HL7 Fast Healthcare Interoperability Resources (FHIR) can underpin a fully standards-based, end-to-end digital research architecture, demonstrate it in a live study, and quantify its benefits for interoperability and development efficiency. MATERIALS AND METHODS: We designed a generalizable standards-based architecture to accelerate digital health research relying on FHIR as the sole transactional model throughout a participant research lifecycle starting from API-based study discovery to results. It was instantiated for People Heart Study, a real-world digital health cardiovascular-risk assessment study with its protocol transformed into FHIR resources (eligibility, consent, tasks, and results). Evaluation examined workflow coverage, validator conformance across independent servers, and points requiring custom extensions or app logic. RESULTS: The architecture was implemented using cloud managed FHIR stores including an illustrative public research discovery API for first-/third-party apps. A participant-facing iOS app was published on the App Store. Our evaluation reveals that 6 of 10 research app workflows could be executed entirely from FHIR artifacts; 2 were partially standards-driven and 2 remained limited requiring custom development. All FHIR resources passed structural, semantic validation with minimal custom extension usage and terminology integrity issues. DISCUSSION: Our approach addresses persistent challenges in digital health research by enhancing data interoperability, minimizing redundant development, and supporting the full research lifecycle. The architecture aligns with national priorities and complements healthcare standardization efforts. CONCLUSION: By leveraging FHIR, our architecture enables generalizability, interoperability, and reuse across diverse digital health research contexts, transforming study design into data modeling rather than software development, and fostering a more inclusive and agile digital health ecosystem.

OBJECTIVE: To evaluate the real-world performance of the SMART/HL7 Bulk Fast Health Interoperability Resources (FHIR) Access Application Programming Interface (API), developed to enable push button access to electronic health record data on large populations, and required under the 21st Century Cures Act Rule. MATERIALS AND METHODS: We used an open-source Bulk FHIR Testing Suite at 5 healthcare sites from April to September 2023, including 4 hospitals using electronic health records (EHRs) certified for interoperability, and 1 Health Information Exchange (HIE) using a custom, standards-compliant API build. We measured export speeds, data sizes, and completeness across 6 types of FHIR. RESULTS: Among the certified platforms, Oracle Cerner led in speed, managing 5-16 million resources at over 8000 resources/min. Three Epic sites exported a FHIR data subset, achieving 1-12 million resources at 1555-2500 resources/min. Notably, the HIE's custom API outperformed, generating over 141 million resources at 12 000 resources/min. DISCUSSION: The HIE's custom API showcased superior performance, endorsing the effectiveness of SMART/HL7 Bulk FHIR in enabling large-scale data exchange while underlining the need for optimization in existing EHR platforms. Agility and scalability are essential for diverse health, research, and public health use cases. CONCLUSION: To fully realize the interoperability goals of the 21st Century Cures Act, addressing the performance limitations of Bulk FHIR API is critical. It would be beneficial to include performance metrics in both certification and reporting processes.

OBJECTIVE: To address challenges in large-scale electronic health record (EHR) data exchange, we sought to develop, deploy, and test an open source, cloud-hosted app "listener" that accesses standardized data across the SMART/HL7 Bulk FHIR Access application programming interface (API). METHODS: We advance a model for scalable, federated, data sharing and learning. Cumulus software is designed to address key technology and policy desiderata including local utility, control, and administrative simplicity as well as privacy preservation during robust data sharing, and artificial intelligence (AI) for processing unstructured text. RESULTS: Cumulus relies on containerized, cloud-hosted software, installed within a healthcare organization's security envelope. Cumulus accesses EHR data via the Bulk FHIR interface and streamlines automated processing and sharing. The modular design enables use of the latest AI and natural language processing tools and supports provider autonomy and administrative simplicity. In an initial test, Cumulus was deployed across 5 healthcare systems each partnered with public health. Cumulus output is patient counts which were aggregated into a table stratifying variables of interest to enable population health studies. All code is available open source. A policy stipulating that only aggregate data leave the institution greatly facilitated data sharing agreements. DISCUSSION AND CONCLUSION: Cumulus addresses barriers to data sharing based on (1) federally required support for standard APIs, (2) increasing use of cloud computing, and (3) advances in AI. There is potential for scalability to support learning across myriad network configurations and use cases.

OBJECTIVE: The 21st Century Cures Act Final Rule requires that certified electronic health records (EHRs) be able to export a patient's full set of electronic health information (EHI). This requirement becomes more powerful if EHI exports use interoperable application programming interfaces (APIs). We sought to advance the ecosystem, instantiating policy desiderata in a working reference implementation based on a consensus design. MATERIALS AND METHODS: We formulate a model for interoperable, patient-controlled, app-driven access to EHI exports in an open source reference implementation following the Argonaut FHIR Accelerator consensus implementation guide for EHI Export. RESULTS: The reference implementation, which asynchronously provides EHI across an API, has three central components: a web application for patients to request EHI exports, an EHI server to respond to requests, and an administrative export management web application to manage requests. It leverages mandated SMART on FHIR/Bulk FHIR APIs. DISCUSSION: A patient-controlled app enabling full EHI export from any EHR across an API could facilitate national-scale patient-directed information exchange. We hope releasing these tools sparks engagement from the health IT community to evolve the design, implement and test in real-world settings, and develop patient-facing apps. CONCLUSION: To advance regulatory innovation, we formulate a model that builds on existing requirements under the Cures Act Rule and takes a step toward an interoperable, scalable approach, simplifying patient access to their own health data; supporting the sharing of clinical data for both improved patient care and medical research; and encouraging the growth of an ecosystem of third-party applications.

Precision medicine promises to revolutionize treatment, shifting therapeutic approaches from the classical one-size-fits-all to those more tailored to the patient's individual genomic profile, lifestyle and environmental exposures. Yet, to advance precision medicine's main objective-ensuring the optimum diagnosis, treatment and prognosis for each individual-investigators need access to large-scale clinical and genomic data repositories. Despite the vast proliferation of these datasets, locating and obtaining access to many remains a challenge. We sought to provide an overview of available patient-level datasets that contain both genotypic data, obtained by next-generation sequencing, and phenotypic data-and to create a dynamic, online catalog for consultation, contribution and revision by the research community. Datasets included in this review conform to six specific inclusion parameters that are: (i) contain data from more than 500 human subjects; (ii) contain both genotypic and phenotypic data from the same subjects; (iii) include whole genome sequencing or whole exome sequencing data; (iv) include at least 100 recorded phenotypic variables per subject; (v) accessible through a website or collaboration with investigators and (vi) make access information available in English. Using these criteria, we identified 30 datasets, reviewed them and provided results in the release version of a catalog, which is publicly available through a dynamic Web application and on GitHub. Users can review as well as contribute new datasets for inclusion (Web: https://avillachlab.shinyapps.io/genophenocatalog/; GitHub: https://github.com/hms-dbmi/GenoPheno-CatalogShiny).

BACKGROUND: Whole exome sequencing (WES) is widely adopted in clinical and research settings; however, one of the practical concerns is the potential false negatives due to incomplete breadth and depth of coverage for several exons in clinically implicated genes. In some cases, a targeted gene panel testing may be a dependable option to ascertain true negatives for genomic variants in known disease-associated genes. We developed a web-based tool to quickly gauge whether all genes of interest would be reliably covered by WES or whether targeted gene panel testing should be considered instead to minimize false negatives in candidate genes. RESULTS: WEScover is a novel web application that provides an intuitive user interface for discovering breadth and depth of coverage across population-scale WES datasets, searching either by phenotype, by targeted gene panel(s) or by gene(s). Moreover, the application shows metrics from the Genome Aggregation Database to provide gene-centric view on breadth of coverage. CONCLUSIONS: WEScover allows users to efficiently query genes and phenotypes for the coverage of associated exons by WES and recommends use of panel tests for the genes with potential incomplete coverage by WES.

OBJECTIVE: Multimodal automated phenotyping (MAP) is a scalable, high-throughput phenotyping method, developed using electronic health record (EHR) data from an adult population. We tested transportability of MAP to a pediatric population. MATERIALS AND METHODS: Without additional feature engineering or supervised training, we applied MAP to a pediatric population enrolled in a biobank and evaluated performance against physician-reviewed medical records. We also compared performance of MAP at the pediatric institution and the original adult institution where MAP was developed, including for 6 phenotypes validated at both institutions against physician-reviewed medical records. RESULTS: MAP performed equally well in the pediatric setting (average AUC 0.98) as it did at the general adult hospital system (average AUC 0.96). MAP's performance in the pediatric sample was similar across the 6 specific phenotypes also validated against gold-standard labels in the adult biobank. CONCLUSIONS: MAP is highly transportable across diverse populations and has potential for wide-scale use.

The 21st Century Cures Act, passed in 2016, and the Final Rules it called for create a roadmap for enabling patient access to their electronic health information. The set of data to be made available, as determined by the Office of the National Coordinator for Health IT through the US Core Data for Interoperability expansion process, will impact the value creation of this improved data liquidity. In this commentary, we look at the potential for significant value creation from USCDI in the context of clinical bioinformatics research and advocate for the research community's involvement in the USCDI process to propel this value creation forward. We also describe 1 mechanism-using existing required APIs for full data export capabilities-that could pragmatically enable this value creation at minimal additional technical lift beyond the current regulatory requirements.

The Office of National Coordinator for Health Information Technology final rule implementing the interoperability and information blocking provisions of the 21st Century Cures Act requires support for two SMART (Substitutable Medical Applications, Reusable Technologies) application programming interfaces (APIs) and instantiates Health Level Seven International (HL7) Fast Healthcare Interoperability Resources (FHIR) as a lingua franca for health data. We sought to assess the current state and near-term plans for the SMART/HL7 Bulk FHIR Access API implementation across organizations including electronic health record vendors, cloud vendors, public health contractors, research institutions, payors, FHIR tooling developers, and other purveyors of health information technology platforms. We learned that many organizations not required through regulation to use standardized bulk data are rapidly implementing the API for a wide array of use cases. This may portend an unprecedented level of standardized population-level health data exchange that will support an apps and analytics ecosystem. Feedback from early adopters on the API's limitations and unsolved problems in the space of population health are highlighted.

OBJECTIVE: The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity. MATERIALS AND METHODS: Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site. RESULTS: The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review. DISCUSSION: We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions. CONCLUSIONS: We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Under the 21st Century Cures Act and the Office of the National Coordinator for Health Information Technology (ONC) rule implementing its interoperability provisions, a patient's rights to easily request and obtain digital access to portions of their medical records are now supported by both technology and policy. Data, once directed by a patient to leave a Health Insurance Portability and Accountability Act-covered health entity and enter a consumer app, will usually fall under Federal Trade Commission oversight. Because the statutory authority of the ONC does not extend to health data protection, there is not yet regulation to specifically address privacy protections for consumer apps. A technologically feasible workflow that could be widely adopted and permissible under ONC's rule, involves using the SMART on FHIR OAuth authorization routine to present standardized information about app behavior. This approach would not bias the patient in a way that triggers penalties under information blocking provisions of the rule.

OBJECTIVE: Real-world data (RWD) are increasingly used for pharmacoepidemiology and regulatory innovation. Our objective was to compare adverse drug event (ADE) rates determined from two RWD sources, electronic health records and administrative claims data, among children treated with drugs for pulmonary hypertension. MATERIALS AND METHODS: Textual mentions of medications and signs/symptoms that may represent ADEs were identified in clinical notes using natural language processing. Diagnostic codes for the same signs/symptoms were identified in our electronic data warehouse for the patients with textual evidence of taking pulmonary hypertension-targeted drugs. We compared rates of ADEs identified in clinical notes to those identified from diagnostic code data. In addition, we compared putative ADE rates from clinical notes to those from a healthcare claims dataset from a large, national insurer. RESULTS: Analysis of clinical notes identified up to 7-fold higher ADE rates than those ascertained from diagnostic codes. However, certain ADEs (eg, hearing loss) were more often identified in diagnostic code data. Similar results were found when ADE rates ascertained from clinical notes and national claims data were compared. DISCUSSION: While administrative claims and clinical notes are both increasingly used for RWD-based pharmacovigilance, ADE rates substantially differ depending on data source. CONCLUSION: Pharmacovigilance based on RWD may lead to discrepant results depending on the data source analyzed. Further work is needed to confirm the validity of identified ADEs, to distinguish them from disease effects, and to understand tradeoffs in sensitivity and specificity between data sources.

OBJECTIVE: The study sought to design, pilot, and evaluate a federated data completeness tracking system (CTX) for assessing completeness in research data extracted from electronic health record data across the Accessible Research Commons for Health (ARCH) Clinical Data Research Network. MATERIALS AND METHODS: The CTX applies a systems-based approach to design workflow and technology for assessing completeness across distributed electronic health record data repositories participating in a queryable, federated network. The CTX invokes 2 positive feedback loops that utilize open source tools (DQe-c and Vue) to integrate technology and human actors in a system geared for increasing capacity and taking action. A pilot implementation of the system involved 6 ARCH partner sites between January 2017 and May 2018. RESULTS: The ARCH CTX has enabled the network to monitor and, if needed, adjust its data management processes to maintain complete datasets for secondary use. The system allows the network and its partner sites to profile data completeness both at the network and partner site levels. Interactive visualizations presenting the current state of completeness in the context of the entire network as well as changes in completeness across time were valued among the CTX user base. DISCUSSION: Distributed clinical data networks are complex systems. Top-down approaches that solely rely on technology to report data completeness may be necessary but not sufficient for improving completeness (and quality) of data in large-scale clinical data networks. Improving and maintaining complete (high-quality) data in such complex environments entails sociotechnical systems that exploit technology and empower human actors to engage in the process of high-quality data curating. CONCLUSIONS: The CTX has increased the network's capacity to rapidly identify data completeness issues and empowered ARCH partner sites to get involved in improving the completeness of respective data in their repositories.

OBJECTIVE: This paper outlines the implementation of a comprehensive clinical pharmacogenomics (PGx) service within a pediatric teaching hospital and the integration of clinical decision support in the electronic health record (EHR). MATERIALS AND METHODS: An approach to clinical decision support for medication ordering and dispensing driven by documented PGx variant status in an EHR is described. A web-based platform was created to automatically generate a clinical report from either raw assay results or specified diplotypes, able to parse and combine haplotypes into an interpretation for each individual and compared to the reference lab call for accuracy. RESULTS: Clinical decision support rules built within an EHR provided guidance to providers for 31 patients (100%) who had actionable PGx variants and were written for interacting medications. A breakdown of the PGx alerts by practitioner service, and alert response for the initial cohort of patients tested is described. In 90% (355/394) of the cases, thiopurine methyltranferase genotyping was ordered pre-emptively. DISCUSSION: This paper outlines one approach to implementing a clinical PGx service in a pediatric teaching hospital that cares for a heterogeneous patient population. There is a focus on incorporation of PGx clinical decision support rules and a program to standardize report text within the electronic health record with subsequent exploration of clinician behavior in response to the alerts. CONCLUSION: The incorporation of PGx data at the time of prescribing and dispensing, if done correctly, has the potential to impact the incidence of adverse drug events, a significant cause of morbidity and mortality.

We have developed an interface to serve patient data from Informatics for Integrating Biology and the Bedside (i2b2) repositories in the Fast Healthcare Interoperability Resources (FHIR) format, referred to as a SMART-on-FHIR cell. The cell serves FHIR resources on a per-patient basis, and supports the "substitutable" modular third-party applications (SMART) OAuth2 specification for authorization of client applications. It is implemented as an i2b2 server plug-in, consisting of 6 modules: authentication, REST, i2b2-to-FHIR converter, resource enrichment, query engine, and cache. The source code is freely available as open source. We tested the cell by accessing resources from a test i2b2 installation, demonstrating that a SMART app can be launched from the cell that accesses patient data stored in i2b2. We successfully retrieved demographics, medications, labs, and diagnoses for test patients. The SMART-on-FHIR cell will enable i2b2 sites to provide simplified but secure data access in FHIR format, and will spur innovation and interoperability. Further, it transforms i2b2 into an apps platform.

OBJECTIVE: One promise of nationwide adoption of electronic health records (EHRs) is the availability of data for large-scale clinical research studies. However, because the same patient could be treated at multiple health care institutions, data from only a single site might not contain the complete medical history for that patient, meaning that critical events could be missing. In this study, we evaluate how simple heuristic checks for data "completeness" affect the number of patients in the resulting cohort and introduce potential biases. MATERIALS AND METHODS: We began with a set of 16 filters that check for the presence of demographics, laboratory tests, and other types of data, and then systematically applied all 216 possible combinations of these filters to the EHR data for 12 million patients at 7 health care systems and a separate payor claims database of 7 million members. RESULTS: EHR data showed considerable variability in data completeness across sites and high correlation between data types. For example, the fraction of patients with diagnoses increased from 35.0% in all patients to 90.9% in those with at least 1 medication. An unrelated claims dataset independently showed that most filters select members who are older and more likely female and can eliminate large portions of the population whose data are actually complete. DISCUSSION AND CONCLUSION: As investigators design studies, they need to balance their confidence in the completeness of the data with the effects of placing requirements on the data on the resulting patient cohort.

OBJECTIVE: Reinventing data extraction from electronic health records (EHRs) to meet new analytical needs is slow and expensive. However, each new data research network that wishes to support its own analytics tends to develop its own data model. Joining these different networks without new data extraction, transform, and load (ETL) processes can reduce the time and expense needed to participate. The Informatics for Integrating Biology and the Bedside (i2b2) project supports data network interoperability through an ontology-driven approach. We use i2b2 as a hub, to rapidly reconfigure data to meet new analytical requirements without new ETL programming. MATERIALS AND METHODS: Our 12-site National Patient-Centered Clinical Research Network (PCORnet) Clinical Data Research Network (CDRN) uses i2b2 to query data. We developed a process to generate a PCORnet Common Data Model (CDM) physical database directly from existing i2b2 systems, thereby supporting PCORnet analytic queries without new ETL programming. This involved: a formalized process for representing i2b2 information models (the specification of data types and formats); an information model that represents CDM Version 1.0; and a program that generates CDM tables, driven by this information model. This approach is generalizable to any logical information model. RESULTS: Eight PCORnet CDRN sites have implemented this approach and generated a CDM database without a new ETL process from the EHR. This enables federated querying within the CDRN and compatibility with the national PCORnet Distributed Research Network. DISCUSSION: We have established a way to adapt i2b2 to new information models without requiring changes to the underlying data. Eight Scalable Collaborative Infrastructure for a Learning Health System sites vetted this methodology, resulting in a network that, at present, supports research on 10 million patients' data. CONCLUSION: New analytical requirements can be quickly and cost-effectively supported by i2b2 without creating new data extraction processes from the EHR.

OBJECTIVE: In early 2010, Harvard Medical School and Boston Children's Hospital began an interoperability project with the distinctive goal of developing a platform to enable medical applications to be written once and run unmodified across different healthcare IT systems. The project was called Substitutable Medical Applications and Reusable Technologies (SMART). METHODS: We adopted contemporary web standards for application programming interface transport, authorization, and user interface, and standard medical terminologies for coded data. In our initial design, we created our own openly licensed clinical data models to enforce consistency and simplicity. During the second half of 2013, we updated SMART to take advantage of the clinical data models and the application-programming interface described in a new, openly licensed Health Level Seven draft standard called Fast Health Interoperability Resources (FHIR). Signaling our adoption of the emerging FHIR standard, we called the new platform SMART on FHIR. RESULTS: We introduced the SMART on FHIR platform with a demonstration that included several commercial healthcare IT vendors and app developers showcasing prototypes at the Health Information Management Systems Society conference in February 2014. This established the feasibility of SMART on FHIR, while highlighting the need for commonly accepted pragmatic constraints on the base FHIR specification. CONCLUSION: In this paper, we describe the creation of SMART on FHIR, relate the experience of the vendors and developers who built SMART on FHIR prototypes, and discuss some challenges in going from early industry prototyping to industry-wide production use.

BACKGROUND: Precision cancer medicine (PCM) will require ready access to genomic data within the clinical workflow and tools to assist clinical interpretation and enable decisions. Since most electronic health record (EHR) systems do not yet provide such functionality, we developed an EHR-agnostic, clinico-genomic mobile app to demonstrate several features that will be needed for point-of-care conversations. METHODS: Our prototype, called Substitutable Medical Applications and Reusable Technology (SMART)® PCM, visualizes genomic information in real time, comparing a patient's diagnosis-specific somatic gene mutations detected by PCR-based hotspot testing to a population-level set of comparable data. The initial prototype works for patient specimens with 0 or 1 detected mutation. Genomics extensions were created for the Health Level Seven® Fast Healthcare Interoperability Resources (FHIR)® standard; otherwise, the prototype is a normal SMART on FHIR app. RESULTS: The PCM prototype can rapidly present a visualization that compares a patient's somatic genomic alterations against a distribution built from more than 3000 patients, along with context-specific links to external knowledge bases. Initial evaluation by oncologists provided important feedback about the prototype's strengths and weaknesses. We added several requested enhancements and successfully demonstrated the app at the inaugural American Society of Clinical Oncology Interoperability Demonstration; we have also begun to expand visualization capabilities to include cancer specimens with multiple mutations. DISCUSSION: PCM is open-source software for clinicians to present the individual patient within the population-level spectrum of cancer somatic mutations. The app can be implemented on any SMART on FHIR-enabled EHRs, and future versions of PCM should be able to evolve in parallel with external knowledge bases.

Mobile health (mHealth) customers shopping for applications (apps) should be aware of app privacy practices so they can make informed decisions about purchase and use. We sought to assess the availability, scope, and transparency of mHealth app privacy policies on iOS and Android. Over 35,000 mHealth apps are available for iOS and Android. Of the 600 most commonly used apps, only 183 (30.5%) had privacy policies. Average policy length was 1755 (SD 1301) words with a reading grade level of 16 (SD 2.9). Two thirds (66.1%) of privacy policies did not specifically address the app itself. Our findings show that currently mHealth developers often fail to provide app privacy policies. The privacy policies that are available do not make information privacy practices transparent to users, require college-level literacy, and are often not focused on the app itself. Further research is warranted to address why privacy policies are often absent, opaque, or irrelevant, and to find a remedy.

BACKGROUND AND OBJECTIVE: Upgrades to electronic health record (EHR) systems scheduled to be introduced in the USA in 2014 will advance document interoperability between care providers. Specifically, the second stage of the federal incentive program for EHR adoption, known as Meaningful Use, requires use of the Consolidated Clinical Document Architecture (C-CDA) for document exchange. In an effort to examine and improve C-CDA based exchange, the SMART (Substitutable Medical Applications and Reusable Technology) C-CDA Collaborative brought together a group of certified EHR and other health information technology vendors. MATERIALS AND METHODS: We examined the machine-readable content of collected samples for semantic correctness and consistency. This included parsing with the open-source BlueButton.js tool, testing with a validator used in EHR certification, scoring with an automated open-source tool, and manual inspection. We also conducted group and individual review sessions with participating vendors to understand their interpretation of C-CDA specifications and requirements. RESULTS: We contacted 107 health information technology organizations and collected 91 C-CDA sample documents from 21 distinct technologies. Manual and automated document inspection led to 615 observations of errors and data expression variation across represented technologies. Based upon our analysis and vendor discussions, we identified 11 specific areas that represent relevant barriers to the interoperability of C-CDA documents. CONCLUSIONS: We identified errors and permissible heterogeneity in C-CDA documents that will limit semantic interoperability. Our findings also point to several practical opportunities to improve C-CDA document quality and exchange in the coming years.

We describe the architecture of the Patient Centered Outcomes Research Institute (PCORI) funded Scalable Collaborative Infrastructure for a Learning Healthcare System (SCILHS, http://www.SCILHS.org) clinical data research network, which leverages the $48 billion dollar federal investment in health information technology (IT) to enable a queryable semantic data model across 10 health systems covering more than 8 million patients, plugging universally into the point of care, generating evidence and discovery, and thereby enabling clinician and patient participation in research during the patient encounter. Central to the success of SCILHS is development of innovative 'apps' to improve PCOR research methods and capacitate point of care functions such as consent, enrollment, randomization, and outreach for patient-reported outcomes. SCILHS adapts and extends an existing national research network formed on an advanced IT infrastructure built with open source, free, modular components.

BACKGROUND AND OBJECTIVE: Malfunctions or poor usability of devices measuring glucose or delivering insulin are reportable to the FDA. Manufacturers submit 99.9% of these reports. We test online social networks as a complementary source to traditional FDA reporting of device-related adverse events. METHODS: Participatory surveillance of members of a non-profit online social network, TuDiabetes.org, from October 2011 to September 2012. Subjects were volunteers from a group within TuDiabetes, actively engaged online in participatory surveillance. They used the free TuAnalyze app, a privacy-preserving method to report detailed clinical information, available through the network. Network members were polled about finger-stick blood glucose monitors, continuous glucose monitors, and insulin delivery devices, including insulin pumps and insulin pens. RESULTS: Of 549 participants, 75 reported device-related adverse events, nearly half (48.0%) requiring intervention from another person to manage the event. Only three (4.0%) of these were reported by participants to the FDA. All TuAnalyze reports contained outcome information compared with 22% of reports to the FDA. Hypoglycemia and hyperglycemia were experienced by 48.0% and 49.3% of participants, respectively. DISCUSSION: Members of an online community readily engaged in participatory surveillance. While polling distributed online populations does not yield generalizable, denominator-based rates, this approach can characterize risk within online communities using a bidirectional communication channel that enables reach-back and intervention. CONCLUSIONS: Engagement of distributed communities in social networks is a viable complementary approach to traditional public health surveillance for adverse events related to medical devices.

OBJECTIVE: DNA samples are often processed and sequenced in facilities external to the point of collection. These samples are routinely labeled with patient identifiers or pseudonyms, allowing for potential linkage to identity and private clinical information if intercepted during transmission. We present a cryptographic scheme to securely transmit externally generated sequence data which does not require any patient identifiers, public key infrastructure, or the transmission of passwords. MATERIALS AND METHODS: This novel encryption scheme cryptographically protects participant sequence data using a shared secret key that is derived from a unique subset of an individual's genetic sequence. This scheme requires access to a subset of an individual's genetic sequence to acquire full access to the transmitted sequence data, which helps to prevent sample mismatch. RESULTS: We validate that the proposed encryption scheme is robust to sequencing errors, population uniqueness, and sibling disambiguation, and provides sufficient cryptographic key space. DISCUSSION: Access to a set of an individual's genotypes and a mutually agreed cryptographic seed is needed to unlock the full sequence, which provides additional sample authentication and authorization security. We present modest fixed and marginal costs to implement this transmission architecture. CONCLUSIONS: It is possible for genomics researchers who sequence participant samples externally to protect the transmission of sequence data using unique features of an individual's genetic sequence.

OBJECTIVE: Registries are a well-established mechanism for obtaining high quality, disease-specific data, but are often highly project-specific in their design, implementation, and policies for data use. In contrast to the conventional model of centralized data contribution, warehousing, and control, we design a self-scaling registry technology for collaborative data sharing, based upon the widely adopted Integrating Biology & the Bedside (i2b2) data warehousing framework and the Shared Health Research Information Network (SHRINE) peer-to-peer networking software. MATERIALS AND METHODS: Focusing our design around creation of a scalable solution for collaboration within multi-site disease registries, we leverage the i2b2 and SHRINE open source software to create a modular, ontology-based, federated infrastructure that provides research investigators full ownership and access to their contributed data while supporting permissioned yet robust data sharing. We accomplish these objectives via web services supporting peer-group overlays, group-aware data aggregation, and administrative functions. RESULTS: The 56-site Childhood Arthritis & Rheumatology Research Alliance (CARRA) Registry and 3-site Harvard Inflammatory Bowel Diseases Longitudinal Data Repository now utilize i2b2 self-scaling registry technology (i2b2-SSR). This platform, extensible to federation of multiple projects within and between research networks, encompasses >6000 subjects at sites throughout the USA. DISCUSSION: We utilize the i2b2-SSR platform to minimize technical barriers to collaboration while enabling fine-grained control over data sharing. CONCLUSIONS: The implementation of i2b2-SSR for the multi-site, multi-stakeholder CARRA Registry has established a digital infrastructure for community-driven research data sharing in pediatric rheumatology in the USA. We envision i2b2-SSR as a scalable, reusable solution facilitating interdisciplinary research across diseases.

BACKGROUND: We sought to measure population-level adherence to antihyperlipidemics, antihypertensives, and oral hypoglycemics, and to develop a model for early identification of subjects at high risk of long-term poor adherence. METHODS: Prescription-filling data for 2 million subjects derived from a payor's insurance claims were used to evaluate adherence to three chronic drugs over 1 year. We relied on patterns of prescription fills, including the length of gaps in medication possession, to measure adherence among subjects and to build models for predicting poor long-term adherence. RESULTS: All prescription fills for a specific drug were sequenced chronologically into drug eras. 61.3% to 66.5% of the prescription patterns contained medication gaps >30 days during the first year of drug use. These interrupted drug eras include long-term discontinuations, where the subject never again filled a prescription for any drug in that category in the dataset, which represent 23.7% to 29.1% of all drug eras. Among the prescription-filling patterns without large medication gaps, 0.8% to 1.3% exhibited long-term poor adherence. Our models identified these subjects as early as 60 days after the first prescription fill, with an area under the curve (AUC) of 0.81. Model performance improved as the predictions were made at later time-points, with AUC values increasing to 0.93 at the 120-day time-point. CONCLUSIONS: Dispensed medication histories (widely available in real time) are useful for alerting providers about poorly adherent patients and those who will be non-adherent several months later. Efforts to use these data in point of care and decision support facilitating patient are warranted.

OBJECTIVE: The Substitutable Medical Applications, Reusable Technologies (SMART) Platforms project seeks to develop a health information technology platform with substitutable applications (apps) constructed around core services. The authors believe this is a promising approach to driving down healthcare costs, supporting standards evolution, accommodating differences in care workflow, fostering competition in the market, and accelerating innovation. MATERIALS AND METHODS: The Office of the National Coordinator for Health Information Technology, through the Strategic Health IT Advanced Research Projects (SHARP) Program, funds the project. The SMART team has focused on enabling the property of substitutability through an app programming interface leveraging web standards, presenting predictable data payloads, and abstracting away many details of enterprise health information technology systems. Containers--health information technology systems, such as electronic health records (EHR), personally controlled health records, and health information exchanges that use the SMART app programming interface or a portion of it--marshal data sources and present data simply, reliably, and consistently to apps. RESULTS: The SMART team has completed the first phase of the project (a) defining an app programming interface, (b) developing containers, and (c) producing a set of charter apps that showcase the system capabilities. A focal point of this phase was the SMART Apps Challenge, publicized by the White House, using http://www.challenge.gov website, and generating 15 app submissions with diverse functionality. CONCLUSION: Key strategic decisions must be made about the most effective market for further disseminating SMART: existing market-leading EHR vendors, new entrants into the EHR market, or other stakeholders such as health information exchanges.

OBJECTIVE: To foster informed decision-making about health social networking (SN) by patients and clinicians, the authors evaluated the quality/safety of SN sites' policies and practices. DESIGN: Multisite structured observation of diabetes-focused SN sites. Measurements 28 indicators of quality and safety covering: (1) alignment of content with diabetes science and clinical practice recommendations; (2) safety practices for auditing content, supporting transparency and moderation; (3) accessibility of privacy policies and the communication and control of privacy risks; and (4) centralized sharing of member data and member control over sharing. RESULTS: Quality was variable across n=10 sites: 50% were aligned with diabetes science/clinical practice recommendations with gaps in medical disclaimer use (30% have) and specification of relevant glycosylated hemoglobin levels (0% have). Safety was mixed with gaps in external review approaches (20% used audits and association links) and internal review approaches (70% use moderation). Internal safety review offers limited protection: misinformation about a diabetes 'cure' was found on four moderated sites. Of nine sites with advertising, transparency was missing on five; ads for unfounded 'cures' were present on three. Technological safety was poor with almost no use of procedures for secure data storage and transmission; only three sites support member controls over personal information. Privacy policies' poor readability impedes risk communication. Only three sites (30%) demonstrated better practice. Limitations English-language diabetes sites only. CONCLUSION: The quality/safety of diabetes SN is variable. Observed better practice suggests improvement is feasible. Mechanisms for improvement are recommended that engage key stakeholders to balance autonomy, community ownership, conditions for innovation, and consumer protection.

OBJECTIVE: To improve identification of pertussis cases by developing a decision model that incorporates recent, local, population-level disease incidence. DESIGN: Retrospective cohort analysis of 443 infants tested for pertussis (2003-7). MEASUREMENTS: Three models (based on clinical data only, local disease incidence only, and a combination of clinical data and local disease incidence) to predict pertussis positivity were created with demographic, historical, physical exam, and state-wide pertussis data. Models were compared using sensitivity, specificity, area under the receiver-operating characteristics (ROC) curve (AUC), and related metrics. RESULTS: The model using only clinical data included cyanosis, cough for 1 week, and absence of fever, and was 89% sensitive (95% CI 79 to 99), 27% specific (95% CI 22 to 32) with an area under the ROC curve of 0.80. The model using only local incidence data performed best when the proportion positive of pertussis cultures in the region exceeded 10% in the 8-14 days prior to the infant's associated visit, achieving 13% sensitivity, 53% specificity, and AUC 0.65. The combined model, built with patient-derived variables and local incidence data, included cyanosis, cough for 1 week, and the variable indicating that the proportion positive of pertussis cultures in the region exceeded 10% 8-14 days prior to the infant's associated visit. This model was 100% sensitive (p<0.04, 95% CI 92 to 100), 38% specific (p<0.001, 95% CI 33 to 43), with AUC 0.82. CONCLUSIONS: Incorporating recent, local population-level disease incidence improved the ability of a decision model to correctly identify infants with pertussis. Our findings support fostering bidirectional exchange between public health and clinical practice, and validate a method for integrating large-scale public health datasets with rich clinical data to improve decision-making and public health.

Personally controlled health records (PCHRs) enable patients to store, manage, and share their own health data, and promise unprecedented consumer access to medical information. To deploy a PCHR in the pediatric population requires crafting of access and security policies, tailored to a record that is not only under patient control, but one that may also be accessed by parents, guardians, and third-party entities. Such hybrid control of health information requires careful consideration of both the PCHR vendor's access policies, as well as institutional policies regulating data feeds to the PCHR, to ensure that the privacy and confidentiality of each user is preserved. Such policies must ensure compliance with legal mandates to prevent unintended disclosures and must preserve the complex interactions of the patient-provider relationship. Informed by our own operational involvement in the implementation of the Indivo PCHR, we provide a framework for understanding and addressing the challenges posed by child, adolescent, and family access to PCHRs.

OBJECTIVE: Unstructured electronic information sources, such as news reports, are proving to be valuable inputs for public health surveillance. However, staying abreast of current disease outbreaks requires scouring a continually growing number of disparate news sources and alert services, resulting in information overload. Our objective is to address this challenge through the HealthMap.org Web application, an automated system for querying, filtering, integrating and visualizing unstructured reports on disease outbreaks. DESIGN: This report describes the design principles, software architecture and implementation of HealthMap and discusses key challenges and future plans. MEASUREMENTS: We describe the process by which HealthMap collects and integrates outbreak data from a variety of sources, including news media (e.g., Google News), expert-curated accounts (e.g., ProMED Mail), and validated official alerts. Through the use of text processing algorithms, the system classifies alerts by location and disease and then overlays them on an interactive geographic map. We measure the accuracy of the classification algorithms based on the level of human curation necessary to correct misclassifications, and examine geographic coverage. RESULTS: As part of the evaluation of the system, we analyzed 778 reports with HealthMap, representing 87 disease categories and 89 countries. The automated classifier performed with 84% accuracy, demonstrating significant usefulness in managing the large volume of information processed by the system. Accuracy for ProMED alerts is 91% compared to Google News reports at 81%, as ProMED messages follow a more regular structure. CONCLUSION: HealthMap is a useful free and open resource employing text-processing algorithms to identify important disease outbreak information through a user-friendly interface.

Over the past year, several payers, employers, and commercial vendors have announced personal health record projects. Few of these are widely deployed and few are fully integrated into ambulatory or hospital-based electronic record systems. The earliest adopters of personal health records have many lessons learned that can inform these new initiatives. We present three case studies--MyChart at Palo Alto Medical Foundation, PatientSite at Beth Israel Deaconess Medical Center, and Indivo at Children's Hospital Boston. We describe our implementation challenges from 1999 to 2007 and postulate the evolving challenges we will face over the next five years.

OBJECTIVE: To determine the accuracy of self-reported information from patients and families for use in a disease surveillance system. DESIGN: Patients and their parents presenting to the emergency department (ED) waiting room of an urban, tertiary care children's hospital were asked to use a Self-Report Tool, which consisted of a questionnaire asking questions related to the subjects' current illness. MEASUREMENTS: The sensitivity and specificity of three data sources for assigning patients to disease categories was measured: the ED chief complaint, physician diagnostic coding, and the completed Self-Report Tool. The gold standard metric for comparison was a medical record abstraction. RESULTS: A total of 936 subjects were enrolled. Compared to ED chief complaints, the Self-Report Tool was more than twice as sensitive in identifying respiratory illnesses (Rate ratio [RR]: 2.10, 95% confidence interval [CI] 1.81-2.44), and dermatological problems (RR: 2.23, 95% CI 1.56-3.17), as well as significantly more sensitive in detecting fever (RR: 1.90, 95% CI 1.67-2.17), gastrointestinal problems (RR: 1.10, 95% CI 1.00-1.20), and injuries (RR: 1.16, 95% CI 1.08-1.24). Sensitivities were also significantly higher when the Self-Report Tool performance was compared to diagnostic codes, with a sensitivity rate ratio of 4.42 (95% CI 3.45-5.68) for fever, 1.70 (95% CI 1.49-1.93) for respiratory problems, 1.15 (95% CI 1.04-1.27) for gastrointestinal problems, 2.02 (95% CI 1.42-2.87) for dermatologic problems, and 1.06 (95% CI 1.01-1.11) for injuries. CONCLUSIONS: Disease category assignment based on patient-reported information was significantly more sensitive in correctly identifying a disease category than data currently used by national and regional disease surveillance systems.

OBJECTIVE: Broadly, to create a bidirectional communication link between public health surveillance and clinical practice. Specifically, to measure the impact of integrating public health surveillance data into an existing clinical prediction rule. We incorporate data about recent local trends in meningitis epidemiology into a prediction model differentiating aseptic from bacterial meningitis. DESIGN AND MEASUREMENTS: Retrospective analysis of a cohort of all 696 children with meningitis admitted to a large urban pediatric hospital from 1992 to 2000. We modified a published bacterial meningitis score by adding a new epidemiological context adjustor variable. We examined 540 possible rules for this adjustor, varying both the number of aseptic meningitis cases that needed to be seen, and the recent time window in which they were seen. We performed sensitivity analyses with each of 540 possibilities in order to identify the optimal rule--namely, the one that included the most cases of aseptic meningitis without missing additional cases of bacterial meningitis, as compared with the published prediction model. We used bootstrap methods to validate this new score. RESULTS: The optimal rule was found to be: "at least four cases of aseptic meningitis in the previous 10 days." The epidemiological context adjustor based on surveillance of recent cases of meningitis allowed the correct identification of an additional 47 cases (7%) of aseptic meningitis without missing any additional cases of bacterial meningitis. The epidemiological context adjustor was validated, showing significance in 84% of 1,000 bootstrap samples. CONCLUSION: Epidemiological contextual information can improve the performance of a clinical prediction rule. We provide a methodological framework for leveraging regional surveillance data to improve medical decision-making.

OBJECTIVE: This study sought to define a scalable architecture to support the National Health Information Network (NHIN). This architecture must concurrently support a wide range of public health, research, and clinical care activities. STUDY DESIGN: The architecture fulfils five desiderata: (1) adopt a distributed approach to data storage to protect privacy, (2) enable strong institutional autonomy to engender participation, (3) provide oversight and transparency to ensure patient trust, (4) allow variable levels of access according to investigator needs and institutional policies, (5) define a self-scaling architecture that encourages voluntary regional collaborations that coalesce to form a nationwide network. RESULTS: Our model has been validated by a large-scale, multi-institution study involving seven medical centers for cancer research. It is the basis of one of four open architectures developed under funding from the Office of the National Coordinator of Health Information Technology, fulfilling the biosurveillance use case defined by the American Health Information Community. The model supports broad applicability for regional and national clinical information exchanges. CONCLUSIONS: This model shows the feasibility of an architecture wherein the requirements of care providers, investigators, and public health authorities are served by a distributed model that grants autonomy, protects privacy, and promotes participation.

In this report, we describe the Automated Epidemiological Geotemporal Integrated Surveillance system (AEGIS), developed for real-time population health monitoring in the state of Massachusetts. AEGIS provides public health personnel with automated near-real-time situational awareness of utilization patterns at participating healthcare institutions, supporting surveillance of bioterrorism and naturally occurring outbreaks. As real-time public health surveillance systems become integrated into regional and national surveillance initiatives, the challenges of scalability, robustness, and data security become increasingly prominent. A modular and fault tolerant design helps AEGIS achieve scalability and robustness, while a distributed storage model with local autonomy helps to minimize risk of unauthorized disclosure. The report includes a description of the evolution of the design over time in response to the challenges of a regional and national integration environment.

OBJECTIVE: The use of spatially based methods and algorithms in epidemiology and surveillance presents privacy challenges for researchers and public health agencies. We describe a novel method for anonymizing individuals in public health data sets by transposing their spatial locations through a process informed by the underlying population density. Further, we measure the impact of the skew on detection of spatial clustering as measured by a spatial scanning statistic. DESIGN: Cases were emergency department (ED) visits for respiratory illness. Baseline ED visit data were injected with artificially created clusters ranging in magnitude, shape, and location. The geocoded locations were then transformed using a de-identification algorithm that accounts for the local underlying population density. MEASUREMENTS: A total of 12,600 separate weeks of case data with artificially created clusters were combined with control data and the impact on detection of spatial clustering identified by a spatial scan statistic was measured. RESULTS: The anonymization algorithm produced an expected skew of cases that resulted in high values of data set k-anonymity. De-identification that moves points an average distance of 0.25 km lowers the spatial cluster detection sensitivity by less than 4% and lowers the detection specificity less than 1%. CONCLUSION: A population-density-based Gaussian spatial blurring markedly decreases the ability to identify individuals in a data set while only slightly decreasing the performance of a standardly used outbreak detection tool. These findings suggest new approaches to anonymizing data for spatial epidemiology and surveillance.

As the public interest in consumer-driven electronic health care applications rises, so do concerns about the privacy and security of these applications. Achieving a balance between providing the necessary security while promoting user acceptance is a major obstacle in large-scale deployment of applications such as personal health records (PHRs). Robust and reliable forms of authentication are needed for PHRs, as the record will often contain sensitive and protected health information, including the patient's own annotations. Since the health care industry per se is unlikely to succeed at single-handedly developing and deploying a large scale, national authentication infrastructure, it makes sense to leverage existing hardware, software, and networks. This report proposes a new model for authentication of users to health care information applications, leveraging wireless mobile devices. Cell phones are widely distributed, have high user acceptance, and offer advanced security protocols. The authors propose harnessing this technology for the strong authentication of individuals by creating a registration authority and an authentication service, and examine the problems and promise of such a system.

Despite progress in creating standardized clinical data models and interapplication protocols, the goal of creating a lifelong health care record remains mired in the pragmatics of interinstitutional competition, concerns about privacy and unnecessary disclosure, and the lack of a nationwide system for authenticating and authorizing access to medical information. The authors describe the architecture of a personally controlled health care record system, PING, that is not institutionally bound, is a free and open source, and meets the policy requirements that the authors have previously identified for health care delivery and population-wide research.

Syndromic surveillance refers to methods relying on detection of individual and population health indicators that are discernible before confirmed diagnoses are made. In particular, prior to the laboratory confirmation of an infectious disease, ill persons may exhibit behavioral patterns, symptoms, signs, or laboratory findings that can be tracked through a variety of data sources. Syndromic surveillance systems are being developed locally, regionally, and nationally. The efforts have been largely directed at facilitating the early detection of a covert bioterrorist attack, but the technology may also be useful for general public health, clinical medicine, quality improvement, patient safety, and research. This report, authored by developers and methodologists involved in the design and deployment of the first wave of syndromic surveillance systems, is intended to serve as a guide for informaticians, public health managers, and practitioners who are currently planning deployment of such systems in their regions.

As those who attend both the AMIA symposia and the annual meetings of the Academy for Health Services Research and Health Policy (formerly, the Association for Health Services Research) can attest, there has been far too little productive research collaboration between the fields of informatics and health services research. Gaps in coordination among informaticians and health services researchers are particularly ironic in that the opportunities for productive collaboration have never been greater. Many integrated delivery systems and insurers now have access to databases that combine clinical and claims data and are applicable to a wide range of research questions. Many new information technologies and systems are being applied in the health arena. Yet there is a dearth of the expertise required to exploit the research potential of the databases or to undertake credible evaluations of the impact of the new technologies and systems. In January 2000, the National Library of Medicine (NLM) and the Agency for Healthcare Research and Quality (AHRQ) cosponsored a training workshop with the title “Medical Informatics and Health Services Research: Bridging the Gap.” The workshop focused on the need to increase the pool of investigators trained and motivated to work on the challenging research issues that lie at the intersection of medical informatics and health services research.* Workshop participants, who were primarily program directors and faculty at NLM- and AHRQ-funded pre- and postdoctoral training programs† for informatics or health services research, met at the NLM for two days.‡ Leaders in the field of informatics and health services research gave framing presentations on the need for research that encompasses both the informatics and health services research perspectives, the contributions that each field can make to the other, and the current state of research training in the two fields. Small breakout sessions were held to discuss what competencies were needed for research that spans the methodologies of informatics and health services research; options for developing and delivering new curriculum content; and strategies for fostering collaboration between existing informatics and health services research training programs and possibly developing new joint programs. The workshop concluded with the development of prioritized recommendations for future action. This issue of JAMIA contains a group of papers emanating from the workshop. Mandl and Lee1 present the case for greater integration of health services research and informatics, focusing on the need for dual training at the clinical, health systems, and policy levels. Examples in their paper address the benefits of existing collaborations between health services researchers and informaticians and the expanding opportunities for fruitful collaboration. Shortliffe and Garber2 present lessons learned from the highly successful collaboration between the Stanford programs in these two fields. Finally, Corn, Rudzinski, and Cahn3 summarize recommendations from the workshop and describe follow-on activities planned and under way at NLM and AHRQ.

During the 2001 AMIA Annual Symposium, the Anesthesia, Critical Care, and Emergency Medicine Working Group hosted the Roundtable on Bioterrorism Detection. Sixty-four people attended the roundtable discussion, during which several researchers discussed public health surveillance systems designed to enhance early detection of bioterrorism events. These systems make secondary use of existing clinical, laboratory, paramedical, and pharmacy data or facilitate electronic case reporting by clinicians. This paper combines case reports of six existing systems with discussion of some common techniques and approaches. The purpose of the roundtable discussion was to foster communication among researchers and promote progress by 1) sharing information about systems, including origins, current capabilities, stages of deployment, and architectures; 2) sharing lessons learned during the development and implementation of systems; and 3) exploring cooperation projects, including the sharing of software and data. A mailing list server for these ongoing efforts may be found at http://bt.cirg.washington.edu.

Reams of data pertaining directly to the core health services research mission are accumulating in large-scale organizational and clinical information systems. Health services researchers who grasp the structure of information systems and databases and the function of software applications can use existing data more effectively, assist in establishing new databases, and develop new tools to survey populations and collect data. At the same time, informaticians are needed who can structure databases that serve the needs of health service research and who can design and evaluate applications that effectively improve health care delivery. As long as health services researchers and informaticians work in separate spheres, however, opportunities to use data from health care encounters to improve care, expand knowledge, and develop more effective policies will be missed. This paper provides a brief exploration of 1) existing successful collaborations between health services researchers and informaticians and 2) needs and opportunities for additional joint work in several core research areas.