Massive gatherings require hosting authorities to plan and deploy extensive arrangements to facilitate attendees while considering crowd safety the core objective. The task becomes more challenging when crowd is diverse in terms of language, culture, ethics and age-group. Hajj is an annual event where more than 2 million pilgrims from all over the world, gather in Makkah Al-Mukarramah to perform a series of spatio-temporal rituals. To plan and execute the event, the management needs to understand the behavior of both individuals and groups of pilgrims belonging to different nationalities. In this paper, we proposed a big data framework that collects mobile sensory data from all pilgrims by providing them location-based context-aware services through a mobile application. We utilize the collected data to study pedestrians' flow rate, mobility patterns for individuals and groups and their behaviour and preferences during movement. The extracted information can be analyzed to propose solutions to crowd related challenges, including mobility schedules, crowd modelling and simulation and emergency evacuation plans.

Crowdsourcing offers great opportunities to recognise user context and prescribe relevant services for both offline and real-time activities. In this work, we present a zoning model that leverages spatio-temporal dimensions and then employs different contexts to recommend necessary customised services. The context model takes into consideration three context sets: fully restricted, fully unrestricted and semi-restricted with respect to both spatial and temporal dimensions. As a proof of concept, we apply this zoning model in a scenario where a very large crowd get together to perform spatio-temporal activities. The user context of the heterogeneous crowd is captured using the carried smartphones, i.e. via crowdsourcing. Depending on the context sets and zone, the system can recommend a set of services to each user. The system has been deployed since 2014 to support the spatio-temporal activities of a very large crowd. We present our implementation details and the user feedback, which is very encouraging.

Analyzing and visualizing large datasets generated by real-time spatio-temporal activities (e.g. vehicle mobility or large crowd movement) are a very challenging task. Recursive delays both at middleware and front end applications limit the of usefulness of the real-time analysis. In this paper, we present a framework “Spatial-Crowd” that first handles spatial-temporal data acquisition and processing by scaling up the middleware components and its infrastructure. Then, it enables filtering, fixing, enriching and summarising the acquired dataset, readily available for client interfaces which usually are not scalable or built to manage such large datasets. This framework follows published subscriber model and allows users to subscribe to aggregated data streams instead of requesting data in real time. The framework is tested with data generated by a very large simulated dataset and performance showed a significant data reduction on the client side to enhance data visualisation.

Recommending an optimized path for a large crowd poses a unique challenge to existing routing algorithms due to the interactions between users and the dynamic changes over road networks. Industries, researchers and end users show an enormous interest in crowdsourced data comprising social networks and user-generated content to remain updated with their concerns. In this paper, we present a data collection framework that helps users to find optimized routes in a dynamic environment. We have developed a data collection framework to collect dynamic road conditions via a set of location-based services to support a very large Hajj crowd by capturing their locations using smartphones. We also collect geotagged social network data that provides more details about road conditions. The system leverages geotagged crowdsourced information to identify constraints such as accidents, congestions, and roadblocks. Moreover, by continuously collecting real-time geotagged data of moving users, the system can also find the flow of traffic and road conditions. We propose a spatial grid index to compute the optimized path, and to identify the affected users within impact zones. The plan is to test the whole application and back-end server during Hajj 2016, where over three million pilgrims from all over the world gather to perform their rituals.

Crowdsourced multimedia data poses several challenges when it is collected, stored, indexed, retrieved, and visualized. Examples of crowd source multimedia data are social sensors, vehicle sensors, physical sensors, human sensors, etc. Analyzing such multimodal and diversified crowdsourced data provides very rich understanding about the need of individuals within a crowd. Such understanding makes it possible to tailor services to individuals' needs, also called context-aware services. In this paper, we propose a spatial multimedia big data framework that can collect multimedia data from 1) a very large crowd equipped with multi-sensory smartphones, 2) vehicles, and 3) social networks. A set of multimedia services are offered to users to support their spatio-temporal activities. These include but not limited to 1) simple user interfaces to utilize multimedia services for instant guidance, 2) navigation to points of interests (POI), and 3) efficient and cost effective intra-city rides to users. The big data framework is designed to handle a very large number of multimedia spatio-temporal queries in real-time. The system is a pilot project and will be deployed during the event of Hajj 2015 when over three million pilgrims from all over the world will visit Makkah, Saudi Arabia to perform their Hajj rituals.

Social networks, such as Facebook, Instagram, and Twitter, provide intuitive ways to share a variety of information including geotagged multimedia data within users' communities of interest (COI) or publicly in real-time. Real-time geotagged multimedia data can provide semantics to conventional spatial queries in order to enrich the user navigation experience. In this paper, we introduce a mechanism to process spatio-temporal queries by leveraging geotagged multimedia data such as images, audio, video, and text, in order to add semantics to the conventional queries. Our framework collects, stores, and spatially tags multimedia data shared by users through social networks or through our developed mobile application. The system then uses such data in order to enhance the conventional routing services by resolving existing usability issues and by providing semantics to the routes in terms of enriched points of interest while taking dynamic road conditions into account. A proof of concept of the system will be demonstrated with the following spatio-temporal queries on road networks: 1) multimedia-enhanced shortest path queries; 2) multimedia-enhanced k-nearest neighbor queries; and 3) multimedia-enhanced range queries. Finally, a novel technique for finding lost individuals using geotagged multimedia data is also introduced. The results are tailor-made to the users' smartphone bandwidth and resolution requirements

Traditional routing algorithms for calculating the fastest or shortest path become ineffective or difficult to use when both source and destination are dynamic or unknown. To solve the problem, we propose a novel semantic routing system that leverages geo-tagged rich crowdsourced multimedia information such as images, audio, video and text to add semantics to the conventional routing. Our proposed system includes a Semantic Multimedia Routing Algorithm (SMRA) that uses an indexed spatial big data environment to answer multimedia spatio-temporal queries in real-time. The results are customized to the users' smartphone bandwidth and resolution requirements. The system has been designed to be able to handle a very large number of multimedia spatio-temporal requests at any given moment. A proof of concept of the system will be demonstrated through two scenarios. These are 1) multimedia enhanced routing and 2) finding lost individuals in a large crowd using multimedia. We plan to test the system's performance and usability during Hajj 2015, where over four million pilgrims from all over the world gather to perform their rituals.

Logging on to a system using a conventional keyboard may not be feasible in certain environments, such as, in a surgical operation theatre or in an industrial manufacturing facility. We have developed a multi-sensory gesture based login system that allows a user to access secure information using body gestures. The system can be configured to use different types of gestures according to the type of sensors available to the user. We have proposed a simple scheme to represent all alphanumeric characters required for password entry as gestures within the multi-sensory environment. Our scheme is scalable enough to support sensors that detect a large number of gestures to those that can only accept a few. This allows the system to be used in a variety of situations such as usage by disabled persons with limited ability to perform gestures. We are in the midst of deploying our developed system in a clinical environment.

Traditional routing algorithms for calculating the fastest or shortest path become ineffective or difficult to use when both source and destination are dynamic or unknown. To solve the problem, we propose a novel semantic routing system that leverages geo-tagged rich crowdsourced multimedia information such as images, audio, video and text to add semantics to the conventional routing. Our proposed system includes a Semantic Multimedia Routing Algorithm (SMRA) that uses an indexed spatial big data environment to answer multimedia spatio-temporal queries in real-time. The results are customized to the users' smartphone bandwidth and resolution requirements. The system has been designed to be able to handle a very large number of multimedia spatio-temporal requests at any given moment. A proof of concept of the system will be demonstrated through two scenarios. These are 1) multimedia enhanced routing and 2) finding lost individuals in a large crowd using multimedia. We plan to test the system's performance and usability during Hajj 2015, where over four million pilgrims from all over the world gather to perform their rituals.

This paper presents a system that enables efficient and scalable real-time user and vehicle discovery using textual, audio and video mechanisms. The system allows users to group together for shared intra-city transportation with the aid of multimedia that helps individuals to 1) find community of common interest (CoCI), 2) locate individual users in a large crowd and 3) locate vehicles for mobility in an efficient and cost effective manner. The system is a pilot project and will be deployed during Hajj 2015 when over three million pilgrims from all over the world visit Makkah, Saudi Arabia.

We propose a context aware framework that offers a set of cloud-based services to support a very large Hajj and Umrah crowd by capturing their contexts using smartphones. The proposed framework captures the individual's context, provides a set of adapted services, and allows being in touch with a subset of one's community of interest. We leverage the spatiotemporal sensory data captured by our framework to define users' contexts for optimized services. Our proposed framework is also envisioned to assist the Hajj and Umrah authorities to (1) improve Hajj & Umrah documentation, (2) improve Hajj organization through better understanding of pilgrims' (individual and crowd) spatial and temporal behavior and needs, and (3) protect pilgrims' environment through environmental monitoring. In particular, the developed methods, techniques, and algorithms will support the pilgrimage quality of experience. We have tested our system through end-user subjects and due to apply for the upcoming Hajj events. We present our implementation details and the general impression of end users about our system.