Multi-Robot Systems (MRS) are increasingly de-ployed for hazardous tasks in urban environments. Among many tasks, search and rescue remains challenging as it deals with exploration in an unknown indoor constrained environ-ment. For example, without global knowledge of the map of a building floor, it is not advantageous to choose one path over another at a corridor junction. Also, if the assigned frontiers are far from the robot, backtracking along a corridor will cost more than moving forward. Since exploration along corridors is similar to solving a maze, this paper examines classical maze-solving algorithms that are known to be computationally fast and lightweight, such as the Right Hand Rule (RHR), Random Mouse (RM), and more. The authors have identified two gaps that need to be addressed before these algorithms can be applied to physical MRS. Firstly, these algorithms are not designed for the cooperation of multiple agents in exploration. Secondly, they are often applied to only a low-fidelity simulation environment, which requires some work to make these algorithms transferable to work in the commonly used occupancy grid map environment. In this paper, the authors introduced RACE, a fast and lightweight collective urban exploration and search algorithm based on a modified and condensed version of the Ant Colony Optimization (ACO) algorithm. The proposed solution is successfully verified in a low-fidelity simulation, evaluated against other exploration and search algorithms like RHR and RM. An innovative approach of RACE Simulation to Physical implementation is presented and a physical system evaluation is performed to evaluate RACE against a Rapidly-Exploring Random Tree algorithm. Finally, the proposed solution is further verified with a physical experiment, in which a quadrupedal robot is assigned to explore part of a floor of SUTD, spanning approximately$(55m \times 40m)$. RACE also showed potential in handling challenging closed-loop and dead-end environments.