The rapid advancements in Vision-Language Models (VLMs) have shown great potential in tackling mathematical reasoning tasks that involve visual context. Unlike humans who can reliably apply solution steps to similar problems with minor modifications, we found that state-of-the-art VLMs like GPT-4o can consistently fail in these scenarios, revealing limitations in their mathematical reasoning capabilities. In this paper, we investigate the **mathematical reasoning robustness** in VLMs and evaluate how well these models perform under different variants of the same question, such as changes in visual numerical values or function graphs. While several vision-based math benchmarks have been developed to assess VLMs' problem-solving capabilities, these benchmarks contain only static sets of problems and cannot easily evaluate mathematical reasoning robustness. To fill this gap, we introduce **DynaMath**, a dynamic visual math benchmark designed for in-depth assessment of VLMs. **DynaMath** includes 501 high-quality, multi-topic *seed* questions, *each represented as a Python program*. Those programs are carefully designed and annotated to enable the automatic generation of a much larger set of *concrete* questions, including many different types of visual and textual variations. **DynaMath** allows us to evaluate the generalization ability of VLMs, by assessing their performance under varying input conditions of a seed question. We evaluated 14 state-of-the-art VLMs with 5,010 generated concrete questions (10 per seed question). Our results show that the worst-case model accuracy, defined as the percentage of correctly answered seed questions in all 10 variants, is significantly lower than the average-case accuracy. In addition, many models show high consistency in answering these questions -- the incorrectness of a certain variant of a seed question is not only due to inherent randomness. Our analysis emphasizes the need to study the robustness of VLMs' reasoning abilities, and **DynaMath** provides valuable insights to guide the development of more reliable models for mathematical reasoning.

Modern engineering, spanning electrical, mechanical, aerospace, civil, and computer disciplines, stands as a cornerstone of human civilization and the foundation of our society. However, engineering design poses a fundamentally different challenge for large language models (LLMs) compared with traditional textbook-style problem solving or factual question answering. Although existing benchmarks have driven progress in areas such as language understanding, code synthesis, and scientific problem solving, real-world engineering design demands the synthesis of domain knowledge, navigation of complex trade-offs, and management of the tedious processes that consume much of practicing engineers' time. Despite these shared challenges across engineering disciplines, no benchmark currently captures the unique demands of engineering design work. In this work, we introduce EngDesign, an Engineering Design benchmark that evaluates LLMs' abilities to perform practical design tasks across nine engineering domains. Unlike existing benchmarks that focus on factual recall or question answering, EngDesign uniquely emphasizes LLMs' ability to synthesize domain knowledge, reason under constraints, and generate functional, objective-oriented engineering designs. Each task in EngDesign represents a real-world engineering design problem, accompanied by a detailed task description specifying design goals, constraints, and performance requirements. EngDesign pioneers a simulation-based evaluation paradigm that moves beyond textbook knowledge to assess genuine engineering design capabilities and shifts evaluation from static answer checking to dynamic, simulation-driven functional verification, marking a crucial step toward realizing the vision of engineering Artificial General Intelligence (AGI).

Jailbreaks on large language models (LLMs) have recently received increasing attention. For a comprehensive assessment of LLM safety, it is essential to consider jailbreaks with diverse attributes, such as contextual coherence and sentiment/stylistic variations, and hence it is beneficial to study controllable jailbreaking, i.e. how to enforce control on LLM attacks. In this paper, we formally formulate the controllable attack generation problem, and build a novel connection between this problem and controllable text generation, a well-explored topic of natural language processing. Based on this connection, we adapt the Energy-based Constrained Decoding with Langevin Dynamics (COLD), a state-of-the-art, highly efficient algorithm in controllable text generation, and introduce the COLD-Attack framework which unifies and automates the search of adversarial LLM attacks under a variety of control requirements such as fluency, stealthiness, sentiment, and left-right-coherence. The controllability enabled by COLD-Attack leads to diverse new jailbreak scenarios which not only cover the standard setting of generating fluent (suffix) attack with continuation constraint, but also allow us to address new controllable attack settings such as revising a user query adversarially with paraphrasing constraint, and inserting stealthy attacks in context with position constraint. Our extensive experiments on various LLMs (Llama-2, Mistral, Vicuna, Guanaco, GPT-3.5, and GPT-4) show COLD-Attack's broad applicability, strong controllability, high success rate, and attack transferability. Our code is available at https://github.com/Yu-Fangxu/COLD-Attack.

Direct policy search has been widely applied in modern reinforcement learning and continuous control. However, the theoretical properties of direct policy search on nonsmooth robust control synthesis have not been fully understood. The optimal $\mathcal{H}_\infty$ control framework aims at designing a policy to minimize the closed-loop $\mathcal{H}_\infty$ norm, and is arguably the most fundamental robust control paradigm. In this work, we show that direct policy search is guaranteed to find the global solution of the robust $\mathcal{H}_\infty$ state-feedback control design problem. Notice that policy search for optimal $\mathcal{H}_\infty$ control leads to a constrained nonconvex nonsmooth optimization problem, where the nonconvex feasible set consists of all the policies stabilizing the closed-loop dynamics. We show that for this nonsmooth optimization problem, all Clarke stationary points are global minimum. Next, we identify the coerciveness of the closed-loop $\mathcal{H}_\infty$ objective function, and prove that all the sublevel sets of the resultant policy search problem are compact. Based on these properties, we show that Goldstein's subgradient method and its implementable variants can be guaranteed to stay in the nonconvex feasible set and eventually find the global optimal solution of the $\mathcal{H}_\infty$ state-feedback synthesis problem. Our work builds a new connection between nonconvex nonsmooth optimization theory and robust control, leading to an interesting global convergence result for direct policy search on optimal $\mathcal{H}_\infty$ synthesis.

As virtualization trend is moving towards "client virtualization", wireless virtualization remains to be one of the technology gaps that haven't been addressed satisfactorily. Today's approaches are mainly developed for wired network, and are not suitable for virtualizing wireless network interface due to the fundamental differences between wireless and wired LAN devices that we will elaborate in this paper. We propose a wireless LAN virtualization approach named virtual WiFi that addresses the technology gap. With our proposed solution, the full wireless LAN functionalities are supported inside virtual machines; each virtual machine can establish its own connection with self-supplied credentials; and multiple separate wireless LAN connections are supported through one physical wireless LAN network interface. We designed and implemented a prototype for our proposed virtual WiFi approach, and conducted detailed performance study. Our results show that with conventional virtualization overhead mitigation mechanisms, our proposed approach can support fully functional wireless functions inside VM, and achieve close to native performance of Wireless LAN with moderately increased CPU utilization.

High-speed mobility has been a central issue to telecommunication in both theory and practice. Each generation of technology presents unique design and engineering challenges in products and deployments. It is especially critical to a class of upcoming 4G (4th-Generation) mobile broadband systems because of broader spectrum usage, aggressive modulation schemes, and sophisticated adaptation mechanisms. This paper presents a field study of the practical issues in supporting high vehicular speeds in WiMAX (IEEE 802.16e), and demonstrates solution guidelines for production WiMAX equipments to enable automobile usages at speeds up to 150 miles-per-hour (240 kph).

Multi-radio devices equipped with both Bluetooth and 4G broadband wireless technologies such as WiMAX or LTE are expected in the market soon. The spectrum available for 4G networks include the 2.3 GHz and 2.5 GHz frequency bands, both at close proximity to the 2.4 GHz ISM band where Bluetooth radio operates in. Several common usage cases require the simultaneous operations of co-located Bluetooth and 4G radios. However, State of Art filter/antenna technologies are unable to support co-located 4G transmissions and Bluetooth receptions at the same time. In this paper, we discuss a simple time domain coexistence solution that has low complexity and is easy to implement. Experiment results using off-the-shelf Bluetooth radios show that the proposed solution can well support common coexistence usage cases.

This paper addresses the design challenge of interference mitigation in the emerging high density (HD) wireless LAN. It is proposed to differentiate interference according to theirenergyandtimingrelative to desired signal, and measure packet error rate (PER) locally at transmitter for each type of interference. Then,self-adaptationalgorithms are designed to adjust a) clear channel assessment (CCA) threshold, aka physical carrier sensing threshold, to leverage spatial reuse for achieving higher aggregate throughput, and b) transmit power (TP) to compensate location difference among links, and prevent individual links from starving. Compared to an end-to-end (E2E) feedback loop, ours has negligible complexity and zero over-the-air overhead. Extensive OPNET simulations are used to compare the performance of our solutions against the legacy and the ideal.

We present algorithms to suppress the asynchronous co-channel interference (CCI) in MIMO OFDM systems, which is becoming the dominant limiting factor in the performance of the emerging high-density WLANs. The key challenge is that the cyclic prefix of the interference signal does not line up with that of the intended signal due to asynchronous transmission in WLAN. Therefore, the orthogonality among the different tones of the interference signal is destroyed and conventional frequency domain minimum mean square error (MMSE) cancelation techniques that measure the interference channel response for each tone can not work effectively. To suppress the asynchronous interference, we design an efficient estimator to measure the interference spatial covariance matrix using Cholesky decomposition and low-pass smoothing. Both a MMSE and a maximum a posteriori (MAP) receiver are derived based on the estimated interference statistics. Simulation results demonstrate the effectiveness of our solution.

In this paper, we study the emerging heterogeneous multi-radio network, focusing on interference issues due to simultaneous operation of multiple radios. A comprehensive review is given on the sources of coexistence interference, including transmitter noise, receiver blocking, and inter-modulation. We compare various coexistence techniques, and relevant standardization efforts. Then, we propose a media independent coexistence service (MICE) layer, and suggest that the performance of a multi-radio system can be further improved if necessary support and modification is added to individual wireless technology, e.g. air interface / wireless protocols. Specifically, we show that the proposed coexistence-aware TXOP adaptation achieves up to 50% improvement of the transmission efficiency in a WiFi/Bluetooth dual-radio device, where the 802.15.2 PTA (packet traffic arbitration) technique is used for coexistence.

Abstract — Fast proliferation of 802.11 devices has led to the emergence of High-Density WLAN where 100s-1000s clients being serviced by 10s-100s of APs in a multi-cell environment. Scalable network capacity becomes a primary design challenge. In this paper, we investigate the impact of hidden/exposed terminal and the lack of capture effect on today’s 802.11 devices on network throughput. We argue that the CSMA/CA parameters in 802.11 MAC protocol need to be adaptive to the changing network topology, traffic and interference. We propose a combination of Receiving-Sensitivity adaptation to reduce stronger-last collision and a Clear-Channel-Assessment adaptation to balance the hidden/exposed terminal problems. We take a hardwarecentric approach to modify the 802.11 MAC protocol and prototype adaptation algorithms on Intel Centrino TM laptops. We demonstrate through testbed measurement that Receiving-Signal-Strength-Indicator and Packet-Error-Rate are effective are readily available metric on today’s 802.11 hardware for the two adaptation algorithms. Our results show that the adaptation algorithms are able to react to the changing network environment and quickly converge to the new value setting. Up to 300 % throughput increase has been observed with the adaptation algorithms.

Spatial reuse in a wireless network can allow multiple communications to proceed simultaneously, hence proportionally improve the overall network throughput. To maximize spatial reuse, the MAC protocol must enable simultaneous transmitters to maintain the minimal separation distance to avoid interference. This paper demonstrates that physical carrier sensing enhanced with tunable sensing threshold is effective at avoiding interference in 802.11 mesh networks without requiring the use of virtual carrier sensing through RTS/CTS. We present analytical model that demonstrates how to derive the optimal sensing threshold given reception power, data rate and network topology. Simulation results are shown for large-scale 802.11b networks to demonstrate that physical carrier sensing with the optimally tuned threshold improves network throughput by maximizing the potential of spatial reuse. In the case of a regular chain topology of 90 nodes, with tuned physical carrier sensing, the end-to-end throughput approaches 90% of the theoretical upper bound that assumes a perfect MAC protocol. Hence, without modifying the 802.11 MAC protocol, our enhanced physical carrier sensing mechanism effectively maximizes the potential of improving network throughput with spatial reuse.

Abstract Spatial reuse in a mesh network can allow multiple communications to proceed simultaneously, hence proportionally improve the overall network throughput. To maximize spatial reuse, the MAC protocol must enable simultaneous transmitters to maintain the minimal separation distance that is sufficient to avoid interference. This paper demonstrates that physical carrier sensing enhanced with a tunable sensing threshold is effective at avoiding interference in 802.11 mesh networks without requiring the use of virtual carrier sensing. We present an analytical model for deriving the optimal sensing threshold given network topology, reception power and data rate. A distributed adaptive scheme is also presented to dynamically adjust the physical carrier sensing threshold based on periodic estimation of channel conditions in the network. Simulation results are shown for large‐scale 802.11b and 802.11a networks to validate both the analytical model and the adaptation scheme. It is demonstrated that the enhanced physical carrier sensing mechanism effectively improves network throughput by maximizing the potential of spatial reuse. With dynamically tuned physical carrier sensing, the end to end throughput approaches 90% of the predicted theoretical upper‐bound assuming a perfect MAC protocol, for a regular chain topology of 90 nodes. Copyright © 2004 John Wiley & Sons, Ltd.

No abstract available.