We consider the problem of estimating object pose and shape from an RGB-D image. Our first contribution is to introduce CRISP, a category-agnostic object pose and shape estimation pipeline. The pipeline implements an encoder-decoder model for shape estimation. It uses FiLM-conditioning for implicit shape reconstruction and a DPT-based network for estimating pose-normalized points for pose estimation. As a second contribution, we propose an optimization-based pose and shape corrector that can correct estimation errors caused by a domain gap. Observing that the shape decoder is well behaved in the convex hull of known shapes, we approximate the shape decoder with an active shape model, and show that this reduces the shape correction problem to a constrained linear least squares problem, which can be solved efficiently by an interior point algorithm. Third, we introduce a self-training pipeline to perform self-supervised domain adaptation of CRISP. The self-training is based on a correct-and-certify approach, which leverages the corrector to generate pseudo-labels at test time, and uses them to self-train CRISP. We demonstrate CRISP (and the self-training) on YCBV, SPE3R, and NOCS datasets. CRISP shows high performance on all the datasets. Moreover, our self-training is capable of bridging a large domain gap. Finally, CRISP also shows an ability to generalize to unseen objects. Code, pre-trained models and videos of sample results are available on the project webpage.1

Deep learning plays a critical role in vision-based satellite pose estimation. However, the scarcity of real data from the space environment means that deep models need to be trained using synthetic data, which raises the Sim2Real domain gap problem. A major cause of the Sim2Real gap are novel lighting conditions encountered during test time. Event sensors have been shown to provide some robustness against lighting variations in vision-based pose estimation. However, challenging lighting conditions due to strong directional light can still cause undesirable effects in the output of commercial off-the-shelf event sensors, such as noisy/spurious events and inhomogeneous event densities on the object. Such effects are non-trivial to simulate in software, thus leading to Sim2Real gap in the event domain. To close the Sim2Real gap in event-based satellite pose estimation, the paper proposes a test-time self-supervision scheme with a certifier module. Self-supervision is enabled by an optimisation routine that aligns a dense point cloud of the predicted satellite pose with the event data to attempt to rectify the inaccurately estimated pose. The certifier attempts to verify the corrected pose, and only certified test-time inputs are backpropagated via implicit differentiation to refine the predicted landmarks, thus improving the pose estimates and closing the Sim2Real gap. Results show that the our method outperforms established test-time adaptation schemes.

Deep learning has had remarkable success in robotic perception, but its data-centric nature suffers when it comes to generalizing to ever-changing environments. By contrast, physics-based optimization generalizes better, but it does not perform as well in complicated tasks due to the lack of high-level semantic information and reliance on manual parametric tuning. To take advantage of these two complementary worlds, we present PyPose: a robotics-oriented, PyTorch-based library that combines deep perceptual models with physics-based optimization. PyPose's architecture is tidy and well-organized, it has an imperative style interface and is efficient and user-friendly, making it easy to integrate into real-world robotic applications. Besides, it supports parallel computing of any order gradients of Lie groups and Lie algebras and 2nd-order optimizers, such as trust region methods. Experiments show that PyPose achieves more than 10× speedup in computation compared to the state-of-the-art libraries. To boost future research, we provide concrete examples for several fields of robot learning, including SLAM, planning, control, and inertial navigation.

We consider the problem of minimizing age of information in multihop wireless networks and propose three classes of policies to solve the problem - stationary randomized, age difference, and age debt. For the unicast setting with fixed routes between each source-destination pair, we first develop a procedure to find age optimal Stationary Randomized policies. These policies are easy to implement and allow us to derive closed-form expression for average AoI. Next, for the same unicast setting, we develop a class of heuristic policies, called Age Difference, based on the idea that if neighboring nodes try to reduce their age differential then all nodes will have fresher updates. This approach is useful in practice since it relies only on the local age differential between nodes to make scheduling decisions. Finally, we propose the class of policies called Age Debt, which can handle 1) non-linear AoI cost functions; 2) unicast, multicast and broadcast flows; and 3) no fixed routes specified per flow beforehand. Here, we convert AoI optimization problems into equivalent network stability problems and use Lyapunov drift to find scheduling and routing schemes that stabilize the network. We also provide numerical results comparing our proposed classes of policies with the best known scheduling and routing schemes available in the literature for a wide variety of network settings.

In this article, we consider acertifiableobject pose estimation problem, where—given a partial point cloud of an object—the goal is to not only estimate the object pose, but also provide a certificate of correctness for the resulting estimate. Our first contribution is a general theory of certification for end-to-end perception models. In particular, we introduce the notion of$\zeta$-correctness, which bounds the distance between an estimate and the ground truth. We then show that$\zeta$-correctness can be assessed by implementing two certificates: 1) a certificate ofobservable correctness, which asserts if the model output is consistent with the input data and prior information; and 2) a certificate ofnondegeneracy, which asserts whether the input data are sufficient to compute a unique estimate. Our second contribution is to apply this theory and design a new learning-based certifiable pose estimator. In particular, we proposeC-3PO, a semantic-keypoint-based pose estimation model, augmented with the two certificates, to solve the certifiable pose estimation problem.C-3POalso includes akeypoint corrector, implemented as a differentiable optimization layer, that can correct large detection errors (e.g., due to the sim-to-real gap). Our third contribution is a novel self-supervised training approach that uses our certificate of observable correctness to provide the supervisory signal toC-3POduring training. In it, the model trains only on the observably correct input–output pairs produced in each batch and at each iteration. As training progresses, we see that the observably correct input–output pairs grow, eventually reaching near 100% in many cases. We conduct extensive experiments to evaluate the performance of the corrector, the certification, and the proposed self-supervised training using the ShapeNet and YCB datasets. The experiments show that 1) standard semantic-keypoint-based methods (which constitute the backbone ofC-3PO) outperform more recent alternatives in challenging problem instances; 2)C-3POfurther improves performance and significantly outperforms all the baselines; and 3)C-3PO’s certificates are able to discern correct pose estimates.1

Graph Neural Networks (GNNs) have emerged as a flexible and powerful approach for learning over graphs. Despite this success, existing GNNs are constrained by their local message-passing architecture and are provably limited in their expressive power. In this work, we propose a new GNN architecture – the Neural Tree. The neural tree architecture does not perform message passing on the input graph, but on a tree-structured graph, called the H-tree, that is constructed from the input graph. Nodes in the H-tree correspond to subgraphs in the input graph, and they are reorganized in a hierarchical manner such that the parent of a node in the H-tree always corresponds to a larger subgraph in the input graph. We show that the neural tree architecture can approximate any smooth probability distribution function over an undirected graph. We also prove that the number of parameters needed to achieve an $\epsilon$-approximation of the distribution function is exponential in the treewidth of the input graph, but linear in its size. We prove that any continuous G-invariant/equivariant function can be approximated by a nonlinear combination of such probability distribution functions over G. We apply the neural tree to semi-supervised node classification in 3D scene graphs, and show that these theoretical properties translate into significant gains in prediction accuracy, over the more traditional GNN architectures. We also show the applicability of the neural tree architecture to citation networks with large treewidth, by using a graph sub-sampling technique.

We consider an m server system in which each server can service at most one update packet at a time. The system designer controls (1) scheduling - the order in which the packets get serviced, (2) routing - the server that an arriving update packet joins for service, and (3) the service time distribution with fixed service rate. Given a fixed update generation process, we prove a strong age-delay and age-delay variance tradeoff, wherein, as the average AoI approaches its minimum, the packet delay and its variance approach infinity. In order to prove this result, we consider two special cases of the m server system, namely, a single server system with last come first served with preemptive service and an infinite server system. In both these cases, we derive sufficient conditions to show that three heavy tailed service time distributions, namely Pareto, log-normal, and Weibull, asymptotically minimize the average AoI as their tail gets heavier, and establish the age-delay tradeoff results. We provide an intuitive explanation as to why such a seemingly counter intuitive age-delay tradeoff is natural, and that it should exist in many systems.

Futuristic communication network formed by autonomously operated, unmanned aerial vehicles, has piqued researchers interests in highly mobile wireless networks. Exchanging safety critical information, with low latency and high throughput, in such systems is of paramount importance. We study the broadcast capacity and minimum delay scaling laws for such highly mobile wireless networks, in which each node has to disseminate packets to all other nodes in the network. In particular, we consider a cell partitioned network under an IID mobility model, in which each node chooses a new position at random, every time slot. We derive scaling laws for broadcast capacity and minimum delay as a function of the network size. We propose a simple first-come-first-serve flooding scheme, which nearly achieve both capacity and minimum delay scaling. Thus, in contrast to what has been speculated in the literature, we show that there is nearly no tradeoff between capacity and delay. Our results also show that high mobility does not improve broadcast capacity. Our analysis makes use of the theory of Markov Evolving Graphs (MEGs), and develops two new bounds on flooding time in MEGs by relaxing the previously required expander property assumption. Simulation results verify our analysis, and throw up interesting open problems.

Age of information (AoI) is a recently proposed metric for measuring information freshness. AoI measures the time that elapsed since the last received update was generated. We consider the problem of minimizing average and peak AoI in a wireless networks, consisting of a set of source-destination links, under general interference constraints. When fresh information is always available for transmission, we show that a stationary scheduling policy is peak age optimal. We also prove that this policy achieves average age that is within a factor of two of the optimal average age. In the case where fresh information is not always available, and packet/information generation rate has to be controlled along with scheduling links for transmission, we prove an important separation principle: the optimal scheduling policy can be designed assuming fresh information, and independently, the packet generation rate control can be done by ignoring interference. Peak and average AoI for discrete time G/Ber/1 queue is analyzed for the first time, which may be of independent interest.

Age of information (AoI), defined as the time that elapsed since the last received update was generated, is a newly proposed metric to measure the timeliness of information updates in a network. We consider AoI minimization problem for a network with general interference constraints, and time varying channels. We propose two policies, namely, virtual-queue based policy and age-based policy when the channel state is available to the network scheduler at each time step. We prove that the virtual-queue based policy is nearly optimal, up to a constant additive factor, and the age-based policy is at-most a factor of 4 away from optimality. Comparison with previous work, which derived age optimal policies when channel state information is not available to the scheduler, demonstrates significant improvement in age due to the availability of channel state information. Our analysis relies on the age conservation law and age-square conservation law developed in this paper, which hold more generally and may be of independent interest.

We consider the problem of timely exchange of updates between a central station and a set of ground terminals$V$, via a mobile agent that traverses across the ground terminals along a mobility graph$G = (V, E)$. We design the trajectory of the mobile agent to minimize average-peak and average age of information (AoI), two recently proposed metrics for measuring timeliness of information. We consider randomized trajectories, in which the mobile agent travels from terminal$i$to terminal$j$with probability$P_{i,j}$. For the information gathering problem, we show that a randomized trajectory is average-peak age optimal and factor-$8\mathcal {H}$average age optimal, where$\mathcal {H}$is the mixing time of the randomized trajectory on the mobility graph$G$. We also show that the average age minimization problem is NP-hard. For the information dissemination problem, we prove that the same randomized trajectory is factor-$O(\mathcal {H})$average-peak and average age optimal. Moreover, we propose an age-based trajectory, which utilizes information about current age at terminals, and show that it is factor-2 average age optimal in a symmetric setting.

Age-of-information (AoI) is a newly proposed performance metric of information freshness. It differs from the traditional delay metric, because it is destination centric and measures the time that elapsed since the last received fresh information update was generated at the source. We show that AoI and packet delay differ in a fundamental way in certain systems, i.e. minimizing one can imply maximizing the other. We consider two queueing systems, namely a single server last come first serve queue with preemptive service (LCFSp) and G/G/∞ queue, and show that a heavy tailed service distribution, that results in the worst case packet delay or variance in packet delay, respectively, minimizes AoI. For the specific case of M/G/1 LCFSp and G/G/∞ queue, we also prove that deterministic service, that minimizes packet delay and variance in packet delay, respectively, results in the worst case AoI.

Information freshness and low latency communication is important to many emerging applications. While Age of Information (AoI) serves as a metric of information freshness, packet delay is a traditional metric of communication latency. We prove that there is a natural tradeoff between the AoI and packet delay. We consider a single server system, in which at most one update packet can be serviced at a time. The system designer controls the order in which the packets get serviced and the service time distribution, with a given service rate. We analyze two tradeoff problems that minimize packet delay and the variance in packet delay, respectively, subject to an average age constraint. We prove a strong age-delay and age-delay variance tradeoff, wherein, as the average age approaches its minimum, the delay and its variance approach infinity. We show that the service time distribution that minimizes average age, must necessarily have an unbounded-second moment.

Age of information (AoI) is a recently proposed metric that measures the time elapsed since the generation of the last received information update. We consider the problem of AoI minimization for a network under general interference constraints, and time varying channel. We study the case where the channel statistics are known, but the current channel state is unknown. We propose two scheduling policies, namely, the virtual queue based policy and age-based policy. In the virtual queue based policy, the scheduler schedules links with maximum weighted sum of the virtual queue lengths, while in the age-based policy, the scheduler schedules links with maximum weighted sum of a function of link AoI. We prove that the virtual queue based policy is peak age optimal, up to an additive constant, while the age-based policy is at most factor 4 away from the optimal age. Numerical results suggest that both the proposed policies are, in fact, very close to the optimal.

Age of information (AoI), defined as the time elapsed since the last received update was generated, is a newly proposed metric to measure the timeliness of information updates in a network. We consider AoI minimization problem for a network with general interference constraints, and time varying channels. We propose two policies, namely, virtual-queue based policy and age-based policy when the channel state is available to the network scheduler at each time step. We prove that the virtual-queue based policy is nearly optimal, up to a constant additive factor, and the age-based policy is at-most factor 4 away from optimality. Comparison with previous work, which derived age optimal policies when channel state information is not available to the scheduler, demonstrates a 4 fold improvement in age due to the availability of channel state information.

We study broadcast capacity and minimum delay scaling laws for highly mobile wireless networks, in which each node has to disseminate or broadcast packets to all other nodes in the network. In particular, we consider a cell partitioned network under the simplified independent and identically distributed (IID) mobility model, in which each node chooses a new cell at random every time slot. We derive scaling laws for broadcast capacity and minimum delay as a function of the cell size. We propose a simple first-come-first-serve (FCFS) flooding scheme that nearly achieves both capacity and minimum delay scaling. Our results show that high mobility does not improve broadcast capacity, and that both capacity and delay improve with increasing cell sizes. In contrast to what has been speculated in the literature we show that there is (nearly) no tradeoff between capacity and delay. Our analysis makes use of the theory of Markov Evolving Graphs (MEGs) and develops two new bounds on flooding time in MEGs by relaxing the previously required expander property assumption.

An opportunistic, rate-adaptive system exploits multi-user diversity by selecting the best node, which has the highest channel power gain, and adapting the data rate to selected node's channel gain. Since channel knowledge is local to a node, we propose using a distributed, low-feedback timer backoff scheme to select the best node. It uses a mapping that maps the channel gain, or, in general, a real-valued metric, to a timer value. The mapping is such that timers of nodes with higher metrics expire earlier. Our goal is to maximize the system throughput when rate adaptation is discrete, as is the case in practice. To improve throughput, we use a pragmatic selection policy, in which even a node other than the best node can be selected. We derive several novel, insightful results about the optimal mapping and develop an algorithm to compute it. These results bring out the inter-relationship between the discrete rate adaptation rule, optimal mapping, and selection policy. We also extensively benchmark the performance of the optimal mapping with several timer and opportunistic multiple access schemes considered in the literature, and demonstrate that the developed scheme is effective in many regimes of interest.

The distributed, low-feedback, timer scheme is used in several wireless systems to select the best node from the available nodes. In it, each node sets a timer as a function of a local preference number called a metric, and transmits a packet when its timer expires. The scheme ensures that the timer of the best node, which has the highest metric, expires first. However, it fails to select the best node if another node transmits a packet within Δ s of the transmission by the best node. We derive the optimal metric-to-timer mappings for the practical scenario where the number of nodes is unknown. We consider two cases in which the probability distribution of the number of nodes is either known a priori or is unknown. In the first case, the optimal mapping maximizes the success probability averaged over the probability distribution. In the second case, a robust mapping maximizes the worst case average success probability over all possible probability distributions on the number of nodes. Results reveal that the proposed mappings deliver significant gains compared to the mappings considered in the literature.

In a communication system in which K nodes communicate with a central sink node, the following problem of selection often occurs. Each node maintains a preference number called a metric, which is not known to other nodes. The sink node must find the `best' node with the largest metric. The local nature of the metrics requires the selection process to be distributed. Further, the selection needs to be fast in order to increase the fraction of time available for data transmission using the selected node and to handle time-varying environments. While several selection schemes have been proposed in the literature, each has its own shortcomings. We propose a novel, distributed selection scheme that generalizes the best features of the timer scheme, which requires minimal feedback but does not guarantee successful selection, and the splitting scheme, which requires more feedback but guarantees successful selection. The proposed scheme introduces several new ideas into the design of the timer and splitting schemes. It explicitly accounts for feedback overheads and guarantees selection of the best node. We analyze and optimize the performance of the scheme and show that it is scalable, reliable, and fast. We also present new insights about the optimal timer scheme.

Opportunistic selection is a practically appealing technique that is used in multi-node wireless systems to maximize throughput, implement proportional fairness, etc. However, selection is challenging since the information about a node's channel gains is often available only locally at each node and not centrally. We propose a novel multiple access-based distributed selection scheme that generalizes the best features of the timer scheme, which requires minimal feedback but does not always guarantee successful selection, and the fast splitting scheme, which requires more feedback but guarantees successful selection. The proposed scheme's design explicitly accounts for feedback time overheads unlike the conventional splitting scheme and guarantees selection of the user with the highest metric unlike the timer scheme. We analyze and minimize the average time including feedback required by the scheme to select. With feedback overheads, the proposed scheme is scalable and considerably faster than several schemes proposed in the literature. Furthermore, the gains increase as the feedback overhead increases.