In this paper, we introduce a sensor designed for robotic fingers which can provide information on the displacements induced by external forces. Our sensor uses LEDs to sense the displacement between two plates connected by a transparent elastomer; when a force is applied to the finger, the elastomer displaces and the LED signals change. We show that using LEDs as both light emitters and receivers in this context provides high sensitivity, allowing such an emitter and receiver pairs to detect very small displacements. We characterize the standalone performance of the sensor by testing the ability of a supervised learning model to predict complete force and torque data from its raw signals, and obtain a mean error between 0.05 and 0.07 N across the three directions of force applied to the finger. Our method allows for compact packaging (fitting at the base of a finger) with no amplification electronics, low cost manufacturing, easy integration into a complete hand, and high overload shear forces and bending torques, suggesting future applicability to complete manipulation tasks.

The acoustic response of an object can reveal a lot about its global state, for example its material properties or the extrinsic contacts it is making with the world. In this work, we build an active acoustic sensing gripper equipped with two piezoelectric fingers: one for generating signals, the other for receiving them. By sending an acoustic vibration from one finger to the other through an object, we gain insight into an object’s acoustic properties and contact state. We use this system to classify objects, estimate grasping position, estimate poses of internal structures, and classify the types of extrinsic contacts an object is making with the environment. Using our contact type classification model, we tackle a standard long-horizon manipulation problem: peg insertion. We use a simple simulated transition model based on the performance of our sensor to train an imitation learning policy that is robust to imperfect predictions from the classifier. We finally demonstrate the policy on a UR5 robot with active acoustic sensing as the only feedback. Videos can be found at https://roamlab.github.io/vibecheck.

In this work, we use MEMS microphones as vibration sensors to simultaneously classify texture and estimate contact position and velocity. Vibration sensors are an important facet of both human and robotic tactile sensing, providing fast detection of contact and onset of slip. Microphones are an attractive option for implementing vibration sensing as they offer a fast response and can be sampled quickly, are affordable, and occupy a very small footprint. Our prototype sensor uses only a sparse array (8-9 mm spacing) of distributed MEMS microphones (<$1, 3.76×2.95×1.10 mm) embedded under an elastomer. We use transformer-based architectures for data analysis, taking advantage of the microphones’ high sampling rate to run our models on time-series data as opposed to individual snapshots. This approach allows us to obtain 77.3% average accuracy on 4-class texture classification (84.2% when excluding the slowest drag velocity), 1.8 mm mean error on contact localization, and 5.6 mm/s mean error on contact velocity. We show that the learned texture and localization models are robust to varying velocity and generalize to unseen velocities. We also report that our sensor provides fast contact detection, an important advantage of fast transducers. This investigation illustrates the capabilities one can achieve with a MEMS microphone array alone, leaving valuable sensor real estate available for integration with complementary tactile sensing modalities.

An optical absorbance-based sensor designed to measure the concentration of vital Nitrogen (N), Phosphorus (P) and Potassium (K) nutrients in urban soil was developed. This device was characterized and tested in nine diverse green spaces around New York City’s Morningside Heights neighborhood, including street-tree pits and park spaces. The results show that the sensor can detect at minimum, a 1.4% change in nutrient concentration. Additionally, it was shown that the sensor can operate in various ambient light settings (indoor and outdoor) after calibration. A study of NYC’s green spaces shows that, on average, soil in street-tree pits that supports plant life has 54% more N, 34% more P, and 37% more K than park spaces, respectively. This new sensor technology will enable more detailed monitoring of soil nutrient conditions and thus help promote healthy green spaces in large urban environments.

Plant Spike is an in situ low-cost sensor system that is wireless, miniature, and low powered. It can be seamlessly implanted in subsurface locations across major cities to measure urban soil health. Plant Spike incorporates noncontact soil moisture monitoring, temperature monitoring, light intensity monitoring, advanced power management, and Bluetooth low energy transmit-only communication for transmitting information to a client device. With a novel combination of aggressive power reduction techniques, the system's lifetime is over two years with a 500-mAh battery. By connecting on-board sensors to a single-chip microcontroller, the total component and assembly cost of each module is less than $10. The sensor system has been tested within an urban soil testbed located on Columbia University's Morningside Campus in New York City as well as street tree pits located in Morningside Heights, proving the functionality and robustness of the system. Plant Spike is able to measure temperature and light ranges that are comparable to the fluctuations experienced by soils located within the climate zone of New York City.

Traditional methods to achieve high localization accuracy with tactile sensors usually use a matrix of miniaturized individual sensors distributed on the area of interest. This approach usually comes at a price of increased complexity in fabrication and circuitry, and can be hard to adapt for non planar geometries. We propose to use low cost optic components mounted on the edges of the sensing area to measure how light traveling through an elastomer is affected by touch. Multiple light emitters and receivers provide us with a rich signal set that contains the necessary information to pinpoint both the location and depth of an indentation with high accuracy. We demonstrate sub-millimeter accuracy on location and depth on a 20mm by 20mm active sensing area. Our sensor provides high depth sensitivity as a result of two different modalities in how light is guided through our elastomer. This method results in a low cost, easy to manufacture sensor. We believe this approach can be adapted to cover non-planar surfaces, simplifying future integration in robot skin applications.

Achieving high spatial resolution in contact sensing for robotic manipulation often comes at the price of increased complexity in fabrication and integration. One traditional approach is to fabricate a large number of taxels, each delivering an individual, isolated response to a stimulus. In contrast, we propose a method where the sensor simply consists of a continuous volume of piezoresistive elastomer with a number of electrodes embedded inside. We measure piezoresistive effects between all pairs of electrodes in the set, and count on this rich signal set containing the information needed to pinpoint contact location with high accuracy using regression algorithms. In our validation experiments, we demonstrate submillimeter median accuracy in locating contact on a 10mm by 16mm sensor using only four electrodes (creating six unique pairs). In addition to extracting more information from fewer wires, this approach lends itself to simple fabrication methods and makes no assumptions about the underlying geometry, simplifying future integration on robot fingers.

Numerous energy harvesting wireless devices that will serve as building blocks for the Internet of Things (IoT) are currently under development. However, there is still only limited understanding of the properties of various energy sources and their impact on energy harvesting adaptive algorithms. Hence, we focus on characterizing the kinetic (motion) energy that can be harvested by a wireless node with an IoT form factor and on developing energy allocation algorithms for such nodes. In this paper, we describe methods for estimating harvested energy from acceleration traces. To characterize the energy availability associated with specific human activities (e.g., relaxing, walking, cycling), we analyze a motion dataset with over 40 participants. Based on acceleration measurements that we collected for over 200 hours, we study energy generation processes associated with day-long human routines. We also briefly summarize our experiments with moving objects. We develop energy allocation algorithms that take into account practical IoT node design considerations, and evaluate the algorithms using the collected measurements. Our observations provide insights into the design of motion energy harvesters, IoT nodes, and energy harvesting adaptive algorithms.

This article focuses on a new type of wireless devices in the domain between RFIDs and sensor networks—Energy-Harvesting Active Networked Tags (EnHANTs). Future EnHANTs will be small, flexible, and self-powered devices that can be attached to objects that are traditionally not networked (e.g., books, furniture, toys, produce, and clothing). Therefore, they will provide the infrastructure for various tracking applications and can serve as one of the enablers for the Internet of Things. We present the design considerations for the EnHANT prototypes, developed over the past 4 years. The prototypes harvest indoor light energy using custom organic solar cells, communicate and form multihop networks using ultra-low-power Ultra-Wideband Impulse Radio (UWB-IR) transceivers, and dynamically adapt their communications and networking patterns to the energy harvesting and battery states. We describe a small-scale testbed that uniquely allows evaluating different algorithms with trace-based light energy inputs. Then, we experimentally evaluate the performance of different energy-harvesting adaptive policies with organic solar cells and UWB-IR transceivers. Finally, we discuss the lessons learned during the prototype and testbed design process.

Numerous energy harvesting wireless devices that will serve as building blocks for the Internet of Things (IoT) are currently under development. However, there is still only limited understanding of the properties of various energy sources and their impact on energy harvesting adaptive algorithms. Hence, we focus on characterizing the kinetic (motion) energy that can be harvested by a wireless node with an IoT form factor and on developing energy allocation algorithms for such nodes. In this paper, we describe methods for estimating harvested energy from acceleration traces. To characterize the energy availability associated with specific human activities (e.g., relaxing, walking, cycling), we analyze a motion dataset with over 40 participants. Based on acceleration measurements that we collected for over 200 hours, we study energy generation processes associated with day-long human routines. We also briefly summarize our experiments with moving objects. We develop energy allocation algorithms that take into account practical IoT node design considerations, and evaluate the algorithms using the collected measurements. Our observations provide insights into the design of motion energy harvesters, IoT nodes, and energy harvesting adaptive algorithms.

This paper focuses on a new type of wireless devices in the domain between RFIDs and sensor networks - Energy Harvesting Active Networked Tags (EnHANTs). Future EnHANTs will be small, flexible, and self-powered devices that can be attached to objects that are traditionally not networked (e.g., books, toys, clothing), thereby providing the infrastructure for novel tracking applications. We present the design considerations for the EnHANT prototypes, developed over the past 3 years. The prototypes harvest indoor light energy using custom organic solar cells, communicate and form multihop networks using ultralow-power Ultra-Wideband Impulse Radio (UWB-IR) transceivers, and adapt their communications and networking patterns to the energy harvesting and battery states. We also describe a small scale EnHANTs testbed that uniquely allows evaluating different algorithms with trace-based light energy inputs.

The modern computing landscape increasingly requires a range of skills to successfully integrate complex systems. Project-based learning is used to help students build professional skills. However, it is typically applied to small teams and small efforts. In this paper, we describe our experience in engaging a large number of students in research projects within a multi-year interdisciplinary research effort. The projects expose the students to various disciplines in Electrical Engineering (circuit design, wireless communications, hardware prototyping), Computer Science (embedded systems, algorithm design, networking) and Applied Physics (thin-film battery design, solar cell fabrication). While a student project is usually focused on one discipline area, it requires interaction with at least two other areas. Over 4 years, 115 semester-long projects have been completed. The students were a diverse group of high school, undergraduate, and M.S. Computer Science, Computer Engineering, and Electrical Engineering students. Some of the approaches we have taken to facilitate student learning are real-world system development constraints, regular cross-group meetings, and extensive involvement of Ph.D. students in student mentorship and knowledge transfer. To assess our approaches, we conducted a survey among the participating students. The results demonstrate the effectiveness of our methods. For example, 70% of the students surveyed indicated that working on their research project improved their ability to function on multidisciplinary teams more than coursework, internships, or any other activity.

This studio will combine familiar rapid prototyping tools with unfamiliar materials to demonstrate how to fabricate electronic devices. Participants will design and fabricate a laser cut capacitive touch pad and a printed organic light emitting diode (OLED) display that can be combined to create a unique custom game.

Energy Harvesting Active Networked Tags (EnHANTs) are a new class of devices in the domain between RFIDs and sensor networks. EnHANTs will be small, flexible, and energetically self-reliant. Their development is enabled by advances in ultra-low-power ultra-wideband (UWB) communications and in organic semiconductor-based energy harvesting materials. In this demo, we present UWB-enabled EnHANT prototypes. Each prototype is based on a MICA2 mote integrated with a UWB Transceiver and an energy harvesting module (EHM) that allows demonstrating energy harvesting-adaptive communications. Additional information about EnHANTs is available at [2] and http://enhants.ee.columbia.edu.

Energy Harvesting Active Networked Tags (EnHANTs) will be a new class of devices in the domain between RFIDs and sensor networks. Small, flexible, and energetically self-reliant, EnHANTs will be attached to objects that are traditionally not networked, such as books, furniture, toys, produce, and clothing. More information about the EnHANTs project is available at http://enhants.ee.columbia.edu. In this demo we present a small network of EnHANT prototypes. The current EnHANT prototypes are integrated with novel custom in-house-developed energy harvesting and communications hardware, namely organic solar cells and ultra-wide-band impulse radio (UWB-IR) transceivers. The demo showcases prototypes communicating using the novel UWB-IR transceivers and adapting their communications and networking parameters to the available environmental energy harvested by the organic solar cells.

Many interactive systems use "conventional" silicon- based sensors and electronics that limit their functionality and scalability. Organic, amorphous inorganic, and other "unconventional" electronics are ideal for applications that require mechanical flexibility or large-area sensing. In this studio participants will use simple control electronics and a range of unconventional materials to design and build a custom interactive system that combines organic light-emitting diodes and piezoelectric pressure sensors.

With the convergence of ultra-low-power communications and energy-harvesting technologies, networking self-sustainable ubiquitous devices is becoming feasible. Hence, we have been recently developing new devices, referred to as Energy Harvesting Active Networked Tags (EnHANTs). These small, flexible, and energetically self-reliant tags can be seen as a new class of devices in the domain between RFIDs and sensor networks. EnHANTs are made possible by advances in ultra-lowpower ultra-wideband (UWB) communications and in organic semiconductor-based energy harvesting materials. They will enable novel tracking applications, such as continuous monitoring of objects and locating misplaced items. In this demo, we present phase I EnHANT prototypes. These prototypes are much larger than the envisioned EnHANTs and do not include custom-made UWB and organic electronic components. Yet, they serve as platforms for preliminary experiments and allow demonstrating energy harvesting-adaptive EnHANT communications. Each prototype is based on a MICA2 mote and includes a custom-designed sensor board with a light sensor and a solar cell, which are used to determine the light energy received from the environment. We have also designed a monitoring system which is used in the demo to show how the EnHANT prototypes adjust their communications patterns based on their energy harvesting parameters.

Tired of the same old pre-packaged display modules? Does your project need more visual oomph? Then make your own display! In this studio you will learn how to design and build novel, application-specific displays. We will cover the basic operating principles of many popular displays, including electroluminescent displays, liquid crystal displays, vacuum fluorescent displays, and organic light-emitting diode (OLED) displays. We will demonstrate how you can build displays using simple design tools and fabrication techniques. Finally you will design and build your own screen-printed electroluminescent display.

This paper presents the design challenges posed by a new class of ultra-low-power devices referred to as Energy-Harvesting Active Networked Tags (EnHANTs). EnHANTs are small, flexible, and self-reliant (in terms of energy devices that can be attached to objects that are traditionally not networked (e.g., books, clothing, and produce), thereby providing the infrastructure for various novel tracking applications. Examples of these applications include locating misplaced items, continuous monitoring of objects (items in a store, boxes in transit), and determining locations of disaster survivors. Recent advances in ultra-low-power wireless communications, ultra-wideband (UWB) circuit design, and organic electronic harvesting techniques will enable the realization of EnHANTs in the near future. In order for EnHANTs to rely on harvested energy, they have to spend significantly less energy than Bluetooth, Zigbee, and IEEE 802.15.4a devices. Moreover, the harvesting components and the ultra-low-power physical layer have special characteristics whose implications on the higher layers have yet to be studied (e.g., when using ultra-low-power circuits, the energy required to receive a bit is an order of magnitude higher than the energy required to transmit a bit). These special characteristics pose several new cross-layer research problems. In this paper, we describe the design challenges at the layers above the physical layer, point out relevant research directions, and outline possible starting points for solutions.

Electronic energy disorder associated within amorphous and polycrystaline molecular organic thin film structures strongly affects the macroscopic observable behavior of organic field effect transistors (OFET) and poses practical challenges to implementing OFET circuits. It has been convenient to ignore the detailed physical processes associated with this disorder and model OFET behavior as equivalent to silicon FETs, but such simplifications limit our ability to develop integrated circuit technology as they fail to predict the true integrated OFET behavior. This talk will highlight the evolution of the organic electronic circuit technology and the challenges that lay ahead, emphasizing the need for physically accurate models of device behavior as the cornerstone of any future circuit advancements.