We present two comprehensive benchmarks to evaluate the performance of language models in coding assistance tasks, covering code writing, debugging, code review, and conceptual understanding. Our main contribution includes two curated datasets: StackEval, a large-scale benchmark derived from Stack Overflow questions, and StackUnseen, a dynamic benchmark featuring the most recent Stack Overflow content. These benchmarks offer novel insights into the capabilities and limitations of LLMs, particularly in handling new and emerging content. Additionally, we assess LLMs' proficiency as judges for coding tasks using a curated, human-annotated dataset, exploring their evaluation capabilities and potential biases, including whether they favor their own generated solutions. Our findings underscore the potential of these benchmarks to advance LLM development and application in coding assistance. To ensure reproducibility, we publicly share our datasets and evaluation code at https://github.com/ProsusAI/stack-eval.

The 60 GHz technology has a great potential to provide wireless communication at multi-gigabit rates in future home networks. To maintain the network connectivity with 60 GHz links, which are highly susceptible to propagation and penetration losses, is a major challenge. The quality and the robustness of the 60 GHz links can be improved by employing relay nodes in the network. In this paper, the contribution of relaying to the connectivity and the quality of the 60 GHz radio links is studied by modeling three indoor scenarios. It is analytically and through simulations shown that having a relay node in a 60 GHz network decreases the average freespace path loss 33% in the worst case scenario. The effects of relay device position and the obstacle density on the improvement of the average received signal level are investigated with a verified 3D ray tracing tool. A comparative simulation study on the performance of different relay configurations under various network conditions is conducted. The results yield that even a single relay device positioned at the height of other nodes can improve 50% of the links in considerable levels in a 60 GHz indoor network. It is also shown that additional relay nodes do not contribute to 60 GHz indoor connectivity significantly, if there are two properly positioned relay devices in a network which is moderately populated.

The 60 GHz technology has an immense potential to provide wireless communication at multi-gigabit rates. At 60 GHz frequency band, severe attenuation of the signals can significantly degrade communication performance. To cope with the attenuation problem, directional antennas with high directive gains can be utilized. When network nodes use directional antennas, the neighbor discovery process becomes more complicated and time consuming. To reduce the neighbor discovery time, we propose a smart neighbor scanning algorithm in this paper. It is observed that the proposed strategy discovers 70% of the links 81% faster and 90% of the links 15% faster than random scanning strategy.

In this paper, we investigate the robustness of the 60 GHz connectivity in typical indoor environments by analyzing the outage probability. We define three realistic indoor scenarios which may host the 60 GHz networks in the future and perform simulations with a verified 3D ray tracing tool on them. In the first set of simulations, we show the impact of access unit position on the connectivity. In the next step, we show that increasing the obstacle density linearly decreases the outage probability of the 60 GHz network and this decrease is found to be sharper for certain positions of the access point. In the last step, we demonstrate the direct effect of reflective surface availability on the connectivity. A 60 GHz indoor network relying solely on the reflections in the absence of line-of-sight path is very vulnerable to outages even in moderately populated indoor environments.

We present a novel protocol EraMobile (epidemic-based reliable and adaptive multicast for mobile ad hoc networks) and its performance analysis. EraMobile's target is group applications requiring high-level of reliability and the protocol aims to provide fully reliable multicast data delivery with minimal network overhead even in the adverse network conditions. EraMobile utilizes an epidemic-based method in multicast operation to cope with dynamic and unpredictable topology changes arising from the mobility. Our epidemic mechanism does not require the maintenance of any tree- or mesh-like structure for multicasting. It also needs neither having global or partial view of the network nor having information of neighboring nodes and group members. Besides, it substantially minimizes the overhead incurred by eliminating redundant data transmissions. Another distinguishing feature of EraMobile is its capability of adapting to varying node densities in order to provide reliable data delivery in both sparse networks, where the network connectivity is prone to interruptions, and dense networks, where congestion is likely to occur. We study the performance of EraMobile through comparative and extensive simulations on ns-2 network simulator. EraMobile is shown to achieve fully reliable multicasting for most of the scenarios investigated.

We aspire to define a reliable and scalable multicast protocol for ad hoc networks. Our target is to utilize epidemic techniques in the design of this protocol. As the first step, we developed an information dissemination method called SFG and presented its initial performance results compared to flooding in this poster paper. The most important contribution of this work is offering a method which requires the nodes neither to be aware of network topology nor to have neighborhood knowledge to disseminate information.