In the article, we explore the feasibility of monitoring outpatients using Fitbit Charge HR wristbands and the potential of machine learning models to predict clinical deterioration (readmissions and death) among outpatients discharged from the hospital. We developed and piloted a data collection system in a clinical study that involved 25 heart failure patients recently discharged. The results demonstrated the feasibility of continuously monitoring outpatients using wristbands. We observed high levels of patient compliance in wearing the wristbands regularly and satisfactory yield, latency, and reliability of data collection from the wristbands to a cloud-based database. Finally, we explored a set of machine learning models to predict deterioration based on the Fitbit data. Through fivefold cross-validation, K nearest neighbor achieved the highest accuracy of 0.8667 for identifying patients at risk of deterioration using the data collected from the beginning of the monitoring. Machine learning models based on multimodal data (step, sleep, and heart rate) significantly outperformed the traditional clinical approach based on LACE index. Moreover, our proposed Weighted Samples One-Class SVM model with estimated confidence can reach high accuracy (0.9635) for predicting the deterioration using data collected within a sliding window, which indicates the potential for allowing timely intervention.

Objective: To review and analyze the literature to determine whether wearable technologies can predict health outcomes. Materials and methods: We queried Ovid Medline 1946 -, Embase 1947 -, Scopus 1823 -, the Cochrane Library, clinicaltrials.gov 1997 - April 17, 2018, and IEEE Xplore Digital Library and Engineering Village through April 18, 2018, for studies utilizing wearable technology in clinical outcome prediction. Studies were deemed relevant to the research question if they involved human subjects, used wearable technology that tracked a health-related parameter, and incorporated data from wearable technology into a predictive model of mortality, readmission, and/or emergency department (ED) visits. Results: Eight unique studies were directly related to the research question, and all were of at least moderate quality. Six studies developed models for readmission and two for mortality. In each of the eight studies, data obtained from wearable technology were predictive of or significantly associated with the tracked outcome. Discussion: Only eight unique studies incorporated wearable technology data into predictive models. The eight studies were of moderate quality or higher and thereby provide proof of concept for the use of wearable technology in developing models that predict clinical outcomes. Conclusion: Wearable technology has significant potential to assist in predicting clinical outcomes, but needs further study. Well-designed clinical trials that incorporate data from wearable technology into clinical outcome prediction models are required to realize the opportunities of this advancing technology.

Clinical study found that early detection and intervention are essential for preventing clinical deterioration in patients, for patients both in intensive care units (ICU) as well as in general wards but under real-time data sensing (RDS). In this paper, we develop an integrated data mining approach to give early deterioration warnings for patients under real-time monitoring in ICU and RDS.

This paper presents the design, deployment, and empirical study of a wireless clinical monitoring system that collects pulse and oxygen saturation readings from patients. The primary contribution of this paper is an in-depth clinical trial that assesses the feasibility of wireless sensor networks for patient monitoring in general hospital units. We present a detailed analysis of the system reliability from a long term hospital deployment over seven months involving 41 patients in a step-down cardiology unit. The network achieved high reliability (median 99.68%, range 95.21% -- 100%). The overall reliability of the system was dominated by sensing reliability of the pulse oximeters (median 80.85%, range 0.46% -- 97.69%). Sensing failures usually occurred in short bursts, although longer periods were also present due to sensor disconnections. We show that the sensing reliability could be significantly improved through oversampling and by implementing a disconnection alarm system that incurs minimal intervention cost. A retrospective data analysis indicated that the system provided sufficient temporal resolution to support the detection of clinical deterioration in three patients who suffered from significant clinical events including transfer to Intensive Care Units. These results indicate the feasibility and promise of using wireless sensor networks for continuous patient monitoring and clinical deterioration detection in general hospital units.

There are limited data on adverse drug event rates in pediatrics. The authors describe the implementation and evaluation of an automated surveillance system modified to detect adverse drug events (ADEs) in pediatric patients. The authors constructed an automated surveillance system to screen admissions to a large pediatric hospital. Potential ADEs identified by the system were reviewed by medication safety pharmacists and a physician and scored for causality and severity. Over the 6 month study period, 6,889 study children were admitted to the hospital for a total of 40,250 patient-days. The ADE surveillance system generated 1226 alerts, which yielded 160 true ADEs. This represents a rate of 2.3 ADEs per 100 admissions or 4 per 1,000 patient-days. Medications most frequently implicated were diuretics, antibiotics, immunosuppressants, narcotics, and anticonvulsants. The composite positive predictive value of the ADE surveillance system was 13%. Automated surveillance can be an effective method for detecting ADEs in hospitalized children.

The quality and safety of health care leaves much to be desired.1,2 Automated clinical decision support (CDS) tools embedded in clinical information systems (CISs) such as computer provider order entry (CPOE) and electronic health records (EHR) applications have the potential to improve care and should be part of any comprehensive approach to improve quality.3,4,5,6 Medication prescribing is a component of health care with well documented quality and safety problems that can be improved by CDS.7,8,9 Medication-related CDS requires that pharmaceutical knowledge be represented in a computable, explicit and unambiguous form. Creating an automated representation of medical knowledge often is the most time consuming step in the development of a CDS system and is known as the “knowledge acquisition bottleneck.”10 For a time, it was hoped that the move toward explicit guidelines in medicine would decrease the knowledge acquisition effort,11 but that has not happened.12 Experiments on data sharing from over a decade ago have not progressed.13 As a result, just a few organizations, primarily academic medical centers, are creating rules and benefiting from CDS,14 but most health care organizations do not have the expertise or resources to create such knowledge bases themselves. One potential solution to the problem of access to automated medication-related knowledge is the set of commercial vendors that supply medication-related knowledge bases for pharmacy and prescribing applications. These vendors' products contain such knowledge as drug-drug and drug-disease interactions, minimum and maximum dosing suggestions, drug-allergy cross-sensitivity groupings, and groupings of medications by therapeutic class. Developers of CISs (either vendor-based or “homegrown”), with appropriate licensing, can incorporate commercial knowledge bases into their products. The knowledge base vendor receives a licensing fee for each CIS implementation and can amortize the costs of knowledge creation over several clients. Any given health care organization pays just a fraction of the total knowledge development costs and is spared the effort of developing the knowledge itself. The CIS developers can create CDS features based on the commercial knowledge base, for example, a drug-drug interaction checking feature within a CPOE application. To the end-user of the CIS, it may not be evident that the CIS application and the knowledge contained therein are two distinct components. The CIS developer may provide end-user configuration tools that allow certain decision support features to be turned on and off. However, several end-user organizations have reported that, when used as the basis for CDS within clinical systems, commercial knowledge bases generate excessive number of alerts, many of which are clinically unhelpful.15 Specific examples include overly sensitive drug-allergy interaction checking,16 drug-drug interaction checking,17 and dose limit checks.18 Such frequent alerts are bothersome to practicing clinicians who deem them to be “nuisance” or “nonsense” alerts. Such nuisance alerts are problematic in various ways. First, they are not helpful in the particular circumstance. Second, they degrade clinician confidence in the entire category of alerts (e.g., a clinically irrelevant maximum dose alert can decrease clinicians' confidence in all maximum dose alerts). Third, excessive nuisance alerts can decrease confidence in the alerting system and CDS as a whole and can cause clinicians to ignore clinically relevant alerts. Fourth, inappropriate CDS may cause clinicians to be dissatisfied with the CIS within which the CDS resides.19,20,21 Such dissatisfaction may impede the nation's progress towards the policy goal22 of rapid and broad dissemination of health information technology. Another problem with the use of commercial knowledge bases is that the tools within the CIS that allow the CDS features to be turned on and off may be inadequate to achieve the alerting functionality desired by the end-user. For example, the tools may permit all class-based duplicate medication alerts to be turned off, but the same tools may not have the capability to turn off the alerting feature for some medications (e.g., antibiotics) while leaving it on for others (e.g., potassium preparations). The result of all these problems is that, in at least some instances, decision support features in CISs that are based on commercial medication knowledge bases have not been well received by the clinician community.15,19 Perhaps it should not be surprising that the use of commercial knowledge bases within CISs results in a large number of alerts. Knowledge base vendors and CIS developers may have incentives (legal as well as financial) to point out when any possible breach of reasonable behavior has occurred. A naïve perspective may be that “any information is good” and some parties may not appreciate the costs of false positive alerts and the human factors involved when an excessive number of alerts are generated. Some health care organizations have found that despite the fact that commercial knowledge bases generate a large number of alerts if used in an unfiltered way, the number of alerts can be reduced and the overall value of the alerting functionality can be increased if the behavior of the knowledge base is customized to be in alignment with local preferences (Figure 1). Such organizations have created filters to reduce the total number of alerts generated by the commercial knowledge base. In addition to filters, the organizations also have created automated knowledge structures that allow local preferences to be represented and folded into the alerting logic. Examples of such filters and other local customizations include excluding certain drug-allergy cross-sensitivity rules,16 increasing maximum drug dose levels,18 and filtering out certain drug-drug interactions.23 In addition, these organizations have found that with just a moderate amount of additional knowledge engineering and technical effort, a vendor-supplied knowledge base can be made much more useful.18 Such organizations are leveraging the knowledge base vendor's substantial effort in knowledge engineering, and are customizing it to meet local requirements. Model for Customizing the Behavior of a Vendor Knowledge Base. (KB = knowledge base, CDS = clinical decision support, CPOE = computer provider order entry) Health care organizations that wish to enhance a commercial knowledge base with local customizations face several challenges. First, the organization must have in place the management processes to decide on the specific CDS functionality that it wishes to implement. Second, the organization needs to build the components that can alter the usual behavior of the vendor's knowledge within the CIS. Such components may include filters that prevent certain alerts from firing18 and/or auxiliary knowledge tables that permit additional or different rules to fire. An organization that builds such components must have detailed knowledge of the structure of the vendor knowledge base as well as the architecture of the CIS. The architecture of the CIS must be open. Developing such tools may be impossible if the architecture of the CIS is closed. Third, the health care organization needs to assure that its knowledge base customizations will not be erased or altered when the vendor sends updated versions of the knowledge base. Fourth, because of the absence of knowledge representation and medical concept standards, one health care organization usually cannot share its knowledge base customizations with other organizations.9 Fifth, there may be liability concerns. For example, if a potentially life-saving alert is rendered non-functional by a local customization and a patient suffers harm that might otherwise have been prevented, the CIS developer, the knowledge base vendor and/or the health care provider organization theoretically could be liable.24 Sixth, customizations to knowledge base behavior could complicate legal agreements among knowledge base vendors, the CIS developers, and end-user organizations. Given the foregoing concerns and complexities, it is unlikely that end-user customization of commercial knowledge bases will meaningfully increase the use of CDS. There are, however, some actions that CIS and knowledge base vendors could take to allow end-user organizations to more easily customize knowledge base behavior and realize attendant benefits. We recommend that CIS and knowledge base vendors: Provide tools that allow end-user organizations to more easily customize the native knowledge bases in a way that decreases the frequency of clinically unhelpful alerts.16,17,18 Giving users the ability to customize severity levels of interactions and/or filter out specific classes of interactions would be one place to start. Another desirable customization feature would be to allow end-users to add to the knowledge base, e.g., to add a drug-drug interaction not in the vendor's knowledge base. The end-user customizations should not be affected by successive updates to the commercial products. Provide better tools for browsing the knowledge. Articulate the “knowledge architecture,” i.e., the set of knowledge-related data structures and software services, within which their products operate. It should be clear how the knowledge base and the CIS interact. Design their products so that customizations made by one health care organization can be exported to other health care organizations that use the same products. Adopt standards for knowledge base representation and concept identifiers so that in the long term such customizations could span knowledge base products and CISs. Facilitate communications among organizations so that when one organization has made a set of customizations to a commercial knowledge base that it believes is clinically useful, such customizations can be evaluated and refined by other organizations. We further recommend that: To minimize risk, healthcare organizations create explicit policies and procedures regarding knowledge editing. Such policies should require that rule specifications be documented explicitly and that an identified individual or clinical committee be responsible for the content of the rule.25,26 The organization must assure that the edited knowledge is behaving as intended.27 Research should be targeted towards understanding how best to use commercial knowledge bases for CDS that is well accepted by practicing clinicians Work on sharable standards for knowledge structures and concept identifiers should continue to be supported. If the benefits of CDS are to be more broadly realized, we will need to accelerate the availability of clinically useful knowledge bases. Knowledge base vendors and CIS developers have an important role to play but health care organizations will realize value only if they have the ability to customize the knowledge to be more suited to their local clinical environment. Purchasers of knowledge bases and CISs should encourage the vendors to supply sophisticated knowledge customization tools and to be explicit about the knowledge architecture in which they operate.

A commercial rule base (Cerner Multum) was used to identify medication orders exceeding recommended dosage limits at five hospitals within BJC HealthCare, an integrated health care system. During initial testing, clinical pharmacists determined that there was an excessive number of nuisance and clinically insignificant alerts, with an overall alert rate of 9.2%. A method for customizing the commercial rule base was implemented to increase rule specificity for problematic rules. The system was subsequently deployed at two facilities and achieved alert rates of less than 1%. Pharmacists screened these alerts and contacted ordering physicians in 21% of cases. Physicians made therapeutic changes in response to 38% of alerts presented to them. By applying simple techniques to customize rules, commercial rule bases can be used to rapidly deploy a safety net to screen drug orders for excessive dosages, while preserving the rule architecture for later implementations of more finely tuned clinical decision support.

The literature on the performance evaluation of medical expert system is extensive, yet most of the techniques used in the early stages of system development are inappropriate for deployed expert systems. Because extensive clinical and informatics expertise and resources are required to perform evaluations, efficient yet effective methods of monitoring performance during the long-term maintenance phase of the expert system life cycle must be devised. Statistical process control techniques provide a well-established methodology that can be used to define policies and procedures for continuous, concurrent performance evaluation. Although the field of statistical process control has been developed for monitoring industrial processes, its tools, techniques, and theory are easily transferred to the evaluation of expert systems. Statistical process tools provide convenient visual methods and heuristic guidelines for detecting meaningful changes in expert system performance. The underlying statistical theory provides estimates of the detection capabilities of alternative evaluation strategies. This paper describes a set of statistical process control tools that can be used to monitor the performance of a number of deployed medical expert systems. It describes how p-charts are used in practice to monitor the GermWatcher expert system. The case volume and error rate of GermWatcher are then used to demonstrate how different inspection strategies would perform.