High reliability, availability, and serviceability are critical for modern large-scale computing systems. As an effective error recovery mechanism, checkpointing has been widely used in such systems for their survival from unexpected failures. The conventional checkpointing schemes, however, are time-consuming due to the limited I/O bandwidth between the DRAM-based main memory and the backup storage. To mitigate the checkpoint overhead, we propose a fast local checkpointing scheme by leveraging Multi-Level Cell (MLC) STT-RAM. We take advantage of the unique features of MLC STT-RAM to accelerate local checkpointing. Our experimental results show that the average performance overhead is less than 1% in a multi-programmed four-core process node with a 1-second local checkpoint interval. The evaluation results also demonstrate that using MLC STT-RAM is an energy-efficient solution.

DRAM cells are leaky and need periodic refresh, which hurts system performance and consumes additional energy. With DRAM scaling towards sub-20nm process technology, we expect a significant portion of DRAM cells become weak cells and require a higher refresh rate, resulting in even higher refresh overhead. A possible solution is to selectively refresh those weak cells using a higher frequency but still refresh the majority at the nominal rate. However, how to provide a multi-rate DRAM refresh scheme is not straightforward. Previous work on this topic was built on an obsolete refresh framework and was incompatible to modern DRAM standards, making it challenging to be adopted in practice. In this work, we propose ProactiveDRAM, a novel scheme that allows DRAM proactively guides the timing of weak cell refresh management and reuses memory controllers' capability in command scheduling. ProactiveDRAM offers a smart retention-aware refresh on the DRAM row granularity, and more importantly, it can be built atop any modern DRAM architecture. Our simulation results show that ProactiveDRAM can handle 1% (or even 10%) weak row population with negligible performance and energy overhead1.

DRAM consumes a significant amount of energy in mobile computing devices today. Emerging non-volatile memory such as magnetoresistive memory (MRAM) offers a DRAM alternative and can potentially lead to a more energy-efficient memory system. The MRAM technology is already mature, but considering the memory industry is highly standardized, we are still unable to see any MRAM used in mainstream products. To tackle this problem, we design an LPDDRx-compatible MRAM interface by considering both MRAM pros and cons. Our design solves the pin-compatibility and the performance issues caused by MRAM small page size, and it optimizes the interface protocol by leveraging the MRAM unique feature of non-destructive reads. Combining our techniques, we boost the MRAM performance by 14% and provide a DRAM-swappable MRAM solution with 20% less energy.

Many new nonvolatile memory (NVM) technologies have been heavily studied to replace the power-hungry SRAM/DRAM-based memory hierarchy in today's computers. Among various emerging NVM technologies, Spin-Transfer Torque RAM (STT-RAM) has many benefits, such as fast read latency, low leakage power, and high density, making it a promising candidate for last-level caches (LLCs). 1 However, STT-RAM write operation is expensive. In particular, a long STT-RAM cache write operation might obstruct other cache accesses and result in severe performance degradation. Consequently, how to mitigate STT-RAM write overhead is critical to the success of STT-RAM adoption. In this article, we propose an obstruction-aware cache management policy called OAP. OAP monitors cache traffic, detects LLC-obstructive processes, and differentiates the cache accesses from different processes. Our experiment on a four-core architecture with an 8MB STT-RAM L3 cache shows a 14% performance improvement and 64% energy reduction.

Emerging non-volatile memory technologies such as MRAM are promising design solutions for energy-efficient memory architecture, especially for mobile systems. However, building commodity MRAM by reusing DRAM designs is not straightforward. The existing memory interfaces are incompatible with MRAM small page size, and they fail to leverage MRAM unique properties, causing unnecessary performance and energy overhead. In this article, we propose four techniques to enable and optimize an LPDDRx-compatible MRAM solution: ComboAS to solve the pin incompatibility; DynLat to avoid unnecessary access latencies; and EarlyPA and BufW to further improve performance by exploiting the MRAM unique features of non-destructive read and independent write path. Combining all these techniques together, we boost the MRAM performance by 17% and provide a DRAM-compatible MRAM solution consuming 21% less energy.

Nonvolatile memories (NVMs) have the potential to replace low-level SRAM or eDRAM on-chip caches because NVMs save standby power and provide large cache capacity. However, limited write endurance is a common problem for NVM technologies, and today's cache management might result in unbalanced cache write traffic, causing heavily written cache blocks to fail much earlier than others. Although wear-leveling techniques for NVM-based main memories exist, we cannot simply apply them to NVM-based caches. This is because cache writes have intraset variations as well as interset variations, while writes to main memories only have interset variations. To solve this problem, we propose i 2 WAP, a new cache management policy that can reduce both inter- and intraset write variations. i 2 WAP has two features: Swap-Shift, an enhancement based on existing main memory wear leveling to reduce cache interset write variations, and Probabilistic Set Line Flush, a novel technique to reduce cache intraset write variations. Implementing i 2 WAP only needs two global counters and two global registers. In one of our studies, i 2 WAP can improve the NVM cache lifetime by 75% on average and up to 224%. We also validate that i 2 WAP is effective in systems with different cache configurations and workloads.

Emerging memory technologies are explored as potential alternatives to traditional SRAM/DRAM-based memory architecture in future microprocessor designs. Among various emerging memory technologies, Spin-Torque Transfer RAM (STT-RAM) has the benefits of fast read latency, low leakage power, and high density, and therefore has been investigated as a promising candidate for last-level cache (LLC)1. One of the major disadvantages for STT-RAM is the latency and energy overhead associated with the write operations. In particular, a long-latency write operation to STT-RAM cache may obstruct other cache accesses and result in severe performance degradation. Consequently, mitigation techniques to minimize the write overhead are required in order to successfully adopt this new technology for cache design. In this paper, we propose an obstruction-aware cache management policy called OAP. OAP monitors the cache to periodically detect LLC-obstruction processes, and manage the cache accesses from different processes. The experimental results on a 4-core architecture with an 8MB STT-RAM L3 cache shows that the performance can be improved by 14% on average and up to 42%, with a reduction of energy consumption by 64%2.

Modern computers require large on-chip caches, but the scalability of traditional SRAM and eDRAM caches is constrained by leakage and cell density. Emerging non-volatile memory (NVM) is a promising alternative to build large on-chip caches. However, limited write endurance is a common problem for non-volatile memory technologies. In addition, today's cache management might result in unbalanced write traffic to cache blocks causing heavily-written cache blocks to fail much earlier than others. Unfortunately, existing wear-leveling techniques for NVM-based main memories cannot be simply applied to NVM-based on-chip caches because cache writes have intra-set variations as well as inter-set variations. To solve this problem, we propose i2WAP, a new cache management policy that can reduce both inter- and intra-set write variations. i2WAP has two features: (1) Swap-Shift, an enhancement based on previous main memory wear-leveling to reduce cache inter-set write variations; (2) Probabilistic Set Line Flush, a novel technique to reduce cache intra-set write variations. Implementing i2WAP only needs two global counters and two global registers. By adopting i2WAP, we can improve the lifetime of on-chip non-volatile caches by 75% on average and up to 224%.

MRAM has emerged as one of the most attractive non-volatile solutions due to fast read access, low leakage power, high bit density, and long endurance. However, the high power consumption of write operations remains a barrier to the commercial adoption of MRAM technology. This paper addresses this problem by introducing low-current probabilistic writes (LCPW), a technique that reduces write access energy by lowering the amplitude of the write current pulse. Although low current pulses no longer guarantee successful bit write operations, we propose and evaluate a simple technique to ensure correctness and achieve significant power reduction over a typical MRAM implementation.

Energy-efficient and cost-effective memory hierarchies are needed in the next era of computing. Currently, many emerging non-volatile memory technologies such as PCRAM, STTRAM, and ReRAM, can potentially meet this requirement. It is necessary to have a framework that can quickly find the optimal memory technology choice and the corresponding circuit design style in terms of performance, energy, or cost. In this work, we first build a circuit-architecture joint design space exploration framework by combining RC circuit analysis and ANN-based performance modeling. Then, we use this framework to design and optimize different memory hierarchy levels by adopting new memory technologies such as ReRAM.

Many new memory technologies are available for building future energy-efficient memory hierarchies. It is necessary to have a framework that can quickly find the optimal memory technology at each hierarchy level. In this work, we first build a circuit-architecture joint design space exploration framework by combining RC circuit analysis and Artificial Neural Network (ANN)-based performance modeling. Then, we use this framework to evaluate some emerging nonvolatile memory hierarchies. We demonstrate that a Resistive RAM (ReRAM)-based cache hierarchy on an 8-core Chip-Multiprocessor (CMP) system can achieve a 24% Energy Delay Product (EDP) improvement and a 36% Energy Delay Area Product (EDAP) improvement compared to a conventional hierarchy with SRAM on-chip caches and DRAM main memory.

Technology scaling of SRAM and embedded DRAM is increasingly constrained by limitations such as leakage power and silicon area. Emerging non-volatile memory technologies are considered as the potential SRAM/eDRAM alternatives for last-level caches in terms of energy and area savings. Unfortunately, these non-volatile memory technologies usually have limited write endurance. Even worse, process variation causes some cells to wear out much earlier than others. While state-of-the-art error-tolerant techniques such as ECC can handle transient soft errors, we need a new architecture for non-volatile last-level caches whose reliability is mainly challenged by hard errors. This paper presents Point-and-Discard (PAD), a hard-failure-tolerant architecture for non-volatile caches. PAD has no initial performance penalty and ensures gradual performance overhead with small storage overhead. By adopting PAD, the lifetime of non-volatile caches can be improved by 4.6X over the conventional architecture under a typical process variation condition.

Various new nonvolatile memory (NVM) technologies have emerged recently. Among all the investigated new NVM candidate technologies, spin-torque-transfer memory (STT-RAM, or MRAM), phase-change random-access memory (PCRAM), and resistive random-access memory (ReRAM) are regarded as the most promising candidates. As the ultimate goal of this NVM research is to deploy them into multiple levels in the memory hierarchy, it is necessary to explore the wide NVM design space and find the proper implementation at different memory hierarchy levels from highly latency-optimized caches to highly density- optimized secondary storage. While abundant tools are available as SRAM/DRAM design assistants, similar tools for NVM designs are currently missing. Thus, in this paper, we develop NVSim, a circuit-level model for NVM performance, energy, and area estimation, which supports various NVM technologies, including STT-RAM, PCRAM, ReRAM, and legacy NAND Flash. NVSim is successfully validated against industrial NVM prototypes, and it is expected to help boost architecture-level NVM-related studies.

Phase-change memory (PCM) is an emerging memory technology that has made rapid progress in the recent years, and surpasses other technologies such as FeRAM and MRAM in terms of scalability. Recently, the feasibility of multi-level cell (MLC) for PCM, which enables a cell to store more than one bit of digital data, has also been shown. This new property makes PCM more competitive and considered as the successor of the NAND flash technology, which also has the MLC capability but does not have an easy scaling path to reach higher densities. However, the MLC capability of PCM comes with the penalty of longer programming time and shortened cell lifetime compared to its single-level cell (SLC) mode. Therefore, it suggests an adaptive MLC/SLC reconfigurable PCM design that can exploit the fast SLC access speed and the large MLC capacity with the awareness of workload characteristics and lifetime requirements. In this work, a circuit-level adaptive MLC/SLC PCM array is designed at first, the management policy of MLC/SLC mode is proposed, and finally the performance and lifetime of a novel PCM-based SSD with run-time MLC/SLC reconfiguration ability is evaluated.

Emerging non-volatile memory (NVM) technologies are getting mature in recent years. These emerging NVM technologies have demonstrated great potentials for the universal memory hierarchy design. Among all the technology candidates, resistive random-access memory (RRAM) is considered to be the most promising as it operates faster than phase-change memory (PCRAM), and it has simpler and smaller cell structure than magnetic memory (MRAM or STT-RAM). In contrast to a conventional MOS-accessed memory cell, memristor-based RRAM has the potential of forming a cross-point structure without using access devices, achieving ultra high density. The cross-point structure, however, brings extra challenges to the peripheral circuitry design. In this work, we study the memristor-based RRAM array design and focus on the choices of different peripherals to achieve the best trade-off among performance, energy, and area. In addition, a system-level model is built to estimate the performance, energy, and area values.

In this paper, we propose an energy-efficient 3D-stacked CMP design by both temporally and spatially finegrained tuning of processor cores and caches. In particular, temporally fine-grained DVFS is employed by each core and L2 cache to reduce the dynamic energy consumption, while spatially fine-grained DVS is applied to the cache hierarchy for the leakage energy reduction. Our tuning technique is implemented by integrating an array of on-chip voltage regulators into the original processor. Experimental results show that the proposed design can provide an energy-efficient, direct, and adaptive control to the system, leading to 20% dynamic and 89% leakage energy reductions, and an average of 34% total energy saving compared to the baseline design.

Phase-change memory (PCM) is one of the most promising technologies among emerging non-volatile memories. Recently, the technology of multi-level cell (MLC) for PCM has been developed and a high capacity memory system can be implemented by storing multiple bits in a cell. However, programming MLC PCM involves the program-and-verify scheme. Thus, the energy of programming intermediate states in MLC PCM is considerably larger than that of single-level cell (SLC) PCM. To mitigate the MLC energy overhead, we propose an energy-efficient PCM architecture using data encoding write based on the observation that there are significant value-dependent energy variations in programming MLC PCM. In addition, data comparison write (DCW) is adopted to enhance the effectiveness of the proposed data encoding architecture for MLC PCM. Simulation results show that this encoding architecture achieves 9.6% average energy saving (up to 19.8%) on the plain MLC PCM system, and 12.9% average energy saving (up to 26.7%) on the DCW-adopted MLC PCM system.

Emerging memory technologies such as STT-RAM, PCRAM, and resistive RAM are being explored as potential replacements to existing on-chip caches or main memories for future multi-core architectures. This is due to the many attractive features these memory technologies posses: high density, low leakage, and non-volatility. However, the latency and energy overhead associated with the write operations of these emerging memories has become a major obstacle in their adoption. Previous works have proposed various circuit and architectural level solutions to mitigate the write overhead. In this paper, we study the integration of STT-RAM in a 3D multi-core environment and propose solutions at the on-chip network level to circumvent the write overhead problem in the cache architecture with STT-RAM technology. Our scheme is based on the observation that instead of staggering requests to a write-busy STT-RAM bank, the network should schedule requests to other idle cache banks for effectively hiding the latency. Thus, we prioritize cache accesses to the idle banks by delaying accesses to the STT-RAM cache banks that are currently serving long latency write requests. Through a detailed characterization of the cache access patterns of 42 applications, we propose an efficient mechanism to facilitate such delayed writes to cache banks by (a) accurately estimating the busy time of each cache bank through logical partitioning of the cache layer and (b) prioritizing packets in a router requesting accesses to idle banks. Evaluations on a 3D architecture, consisting of 64 cores and 64 STT-RAM cache banks, show that our proposed approach provides 14% average IPC improvement for multi-threaded benchmarks, 19% instruction throughput benefits for multi-programmed workloads, and 6% latency reduction compared to a recently proposed write buffering mechanism.

The scalability of future Massively Parallel Processing (MPP) systems is being severely challenged by high failure rates. Current centralized Hard Disk Drive (HDD) checkpointing results in overhead of 25% or more at petascale. Since systems become more vulnerable as the node count keeps increasing, novel techniques that enable fast and frequent checkpointing are critical to the future exascale system implementation. In this work, we first introduce one of the emerging nonvolatile memory technologies, Phase-Change Random Access Memory (PCRAM), as a proper candidate of the fast checkpointing device. After a thorough analysis of MPP systems, failure rates and failure sources, we propose a PCRAM-based hybrid local/global checkpointing mechanism which not only provides a faster checkpoint storage, but also boosts the effectiveness of other orthogonal techniques such as incremental checkpointing and background checkpointing. Three variant implementations of the PCRAM-based hybrid checkpointing are designed to be adopted at different stages and to offer a smooth transition from the conventional in-disk checkpointing to the instant in-memory approach. Analyzing the overhead by using a hybrid checkpointing performance model, we show the proposed approach only incurs less than 3% performance overhead on a projected exascale system.

Phase-Change Random Access Memory (PRAM) has become one of the most promising emerging memory technologies, due to its attractive features such as high density, fast access, non-volatility, and good scalability. The physical characteristics of a PRAM cell mainly depend on the material characteristic and the fabrication process. However, the access device and the operating voltage have significant impact on the PRAM performance, energy dissipation, and lifetime. In this paper, we study the design constraints for PRAM memory array, and propose design optimizations of the access device and the circuit operational voltage. The important features of PRAM memory, such as power consumption, read/write stability, speed, as well as lifetime are all considered as the constrained conditions in the proposed optimizations. Experimental results showed that the proposed methodology can provide a reliable design space for the access device and the operating voltage.

The ever increasing die area of Chip Multiprocessors (CMPs) affects manufacturing yield, resulting in higher manufacture cost. Meanwhile, network-on-chip (NoC) has emerged as a promising and scalable solution for interconnecting the cores in CMPs, however it consumes significant portion of the total die area. In this paper, we propose to decouple the interconnect fabric from computing and storage layers, forming a separate layer called Interconnect Service Layer (ISL), in the context of three-dimensional (3D) chip integration. Such decoupling helps reduce the die area for each layer in 3D stacking. ISL itself can integrate multiple super-imposed interconnect topologies. More importantly, ISL can be designed, manufactured, and tested as a separate Intellectual Property (IP) component, which supports multiple designs in the computing and storage layers. The resulting methodology also helps support different manufacturing volume in each die of 3D to reduce the overall manufacturing cost. We demonstrate the proposed methodology with an ISL design example and compare to its 2D and 3D counterparts without ISL support. The results show that 3D design with ISL not only provides significant cost reduction, but also achieves power-performance improvement thanks to the efficient usage of ISL.

The emerging three-dimensional integrated circuit (3D IC) is beneficial for various applications from both area and performance perspectives. While the general trend in processor design has been shifting from multi-core to many-core, questions such as whether 3D integration should be adopted, and how to choose among various design options must be addressed at the early design stage. In order to guide the final design towards a cost-effective direction, system-level cost evaluation is one of the most critical issues to be considered. In this paper, we propose a 3D many-core multiprocessor cost model, which includes wafer, bonding, package, and cooling cost analysis. Using the proposed cost model, we evaluate the optimal partitioning strategies for 16-, 32- and 64-core multiprocessors from the cost point of view.

Phase change memory (PCM) is one of the most promising technology among emerging non-volatile random access memory technologies. Implementing a cache memory using PCM provides many benefits such as high density, non-volatility, low leakage power, and high immunity to soft error. However, its disadvantages such as high write latency, high write energy, and limited write endurance prevent it from being used as a drop-in replacement of an SRAM cache. In this paper, we study a set of techniques to design an energy- and endurance-aware PCM cache. We also modeled the timing, energy, endurance, and area of PCM caches and integrated them into a PCM cache simulator to evaluate the techniques. Experiments show that our PCM cache design can achieve 8% of energy saving and 3.8 years of lifetime compared with a baseline PCM cache having less than a hour of lifetime.

System-in-Package (SiP) and 3D integration are promising technologies to bring more memory onto a microprocessor package to mitigate the "memory wall" problem. In this paper, instead of using them to build caches, we study a heterogenous main memory using both on- and off-package memories providing both fast and high-bandwidth on-package accesses and expandable and low-cost commodity off-package memory capacity. We introduce another layer of address translation coupled with an on-chip memory controller that can dynamically migrate data between off-package and off-package memory either in hardware or with operating system assistance depending on the migration granularity. Our experimental results demonstrate that such design can achieve the average effectiveness of 83% of the ideal case where all memory can be placed in high-speed on-package memory for our simulated benchmarks.

3-D integration technology is emerging as an attractive alternative to increase the transistor count for future chips. The majority of the existing 3-D integrated circuit (IC) research is focused on the performance, power, density, and heterogeneous integration benefits offered by 3-D integration. All such advantages, however, ultimately have to translate into cost evaluation when a design strategy has to be decided. Consequently, system-level cost analysis at early design stages is imperative to decide on whether 3-D integration should be adopted. This paper presents a cost estimation method for 3-D ICs at early design stages and proposes a set of cost models that include wafer cost, 3-D bonding cost, package cost, and cooling cost. The proposed 3-D IC cost estimation method can help designers analyze the cost implication for 3-D ICs during the design space exploration at the early stage, and it enables a cost-driven 3-D IC design flow that can guide the design choice toward a cost-effective direction. Based on the proposed cost estimation method, this paper demonstrates two case studies that explore the cost benefits of 3-D integration for application-specific integrated circuit designs and many-core microprocessor designs style, respectively. Finally, this paper suggests the optimum partitioning strategy for future 3-D IC designs.

Three-dimensional integrated circuit (3D IC) is emerging as an attractive option for overcoming the barriers in interconnect scaling. The majority of the existing 3D IC research is focused on how to take advantage of the performance, power, smaller form-factor, and heterogeneous integration benefits that offered by 3D integration. However, all such advantages ultimately have to translate into cost savings when a design strategy has to be decided: Is 3D integration a cost effective technology for a particular IC design? Consequently, system-level cost analysis at the early design stage is imperative to help the decision making on whether 3D integration should be adopted. In this paper, we study the design estimation method for 3D ICs at the early design stage, and propose a cost analysis model to study the cost implication for 3D ICs, and address the following cost-related problems related to 3D IC design: (1) Do all the benefits of 3D IC design come with a much higher cost? (2) How can 3D integration be achieved in a cost-effective way? (3) Are there any design options to compensate the extra 3D bonding cost? A cost-driven 3D IC design flow is also proposed to guide the design space exploration for 3D ICs toward a cost-effective direction.

Magnetic random access memory (MRAM) is a promising memory technology, which has fast read access, high density, and non-volatility. Using 3D heterogeneous integrations, it becomes feasible and cost-efficient to stack MRAM atop conventional chip multiprocessors (CMPs). However, one disadvantage of MRAM is its long write latency and its high write energy. In this paper, we first stack MRAM-based L2 caches directly atop CMPs and compare it against SRAM counterparts in terms of performance and energy. We observe that the direct MRAM stacking might harm the chip performance due to the aforementioned long write latency and high write energy. To solve this problem, we then propose two architectural techniques: read-preemptive write buffer and SRAM-MRAM hybrid L2 cache. The simulation result shows that our optimized MRAM L2 cache improves performance by 4.91% and reduces power by 73.5%compared to the conventional SRAM L2 cache with the similar area.

Phase-change random access memory (PCRAM) is an emerging memory technology with attractive features, such as fast read access, high density, and non-volatility. Because of these attractive properties, PCRAM is regarded as a promising candidate for future universal memories, and system-level designers could open up new design opportunities by leveraging this new memory technology. However, the majority of the PCRAM research has been at the device level, and system-level design space exploration using PCRAM is still in its infancy due to the lack of high-level modeling tools for PCRAM-based caches and memories. In this paper, we present a PCRAM model, called PCRAMsim, to bridge the gap between the device-level and system-level research on PCRAM technology. The model is validated against industrial PCRAM prototypes. This new PCRAMsim tool is expected to help boost PCRAM-related studies such as next-generation memory subsystems.

Graphics Processing Units (GPUs) offer tremendous computational and processing power. The architecture requires high communication bandwidth and lower latency between computation units and caches. 3D die-stacking technology is a promising approach to meet such requirements. To the best of our knowledge no other study has investigated the implementation of 3D technology in GPUs. In this paper, we study the impact of stacking caches using the 3D technology on GPU performance. We also investigate the benefits of using 3D stacked MRAM on GPUs. Our work includes cost, power, and thermal analysis of the proposed architectural designs. Our results show a 53% geometric mean performance speedup for iso-cycle time architectures and about 19% for iso-cost architectures.

The scalability of future massively parallel processing (MPP) systems is challenged by high failure rates. Current hard disk drive (HDD) checkpointing results in overhead of 25% or more at the petascale. With a direct correlation between checkpoint frequencies and node counts, novel techniques that can take more frequent checkpoints with minimum overhead are critical to implement a reliable exascale system. In this work, we leverage the upcoming Phase-Change Random Access Memory (PCRAM) technology and propose a hybrid local/global checkpointing mechanism after a thorough analysis of MPP systems failure rates and failure sources.

Magnetic Random Access Memory (MRAM) has been considered as a promising memory technology due to many attractive properties. Integrating MRAM with CMOS logic may incur extra manufacture cost, due to its hybrid magnetic-CMOS fabrication process. Stacking MRAM on top of CMOS logics using 3D integration is a way to minimize this cost overhead. In this paper, we discuss the circuit design issues for MRAM, and present the MRAM cache model. Based on the model, we compare MRAM against SRAM and DRAM in terms of area, performance, and energy. Finally we conduct architectural evaluation for 3D microprocessor stacking with MRAM. The experimental results show that MRAM stacking offers competitive IPC performance with a large reduction in power consumption compared to SRAM and DRAM counterparts.