Telecom Data Mining

In this thesis project at the University of Pisa, I addressed the challenge of dynamic population estimation using telecommunication Call Detail Record (CDR) data as a valuable proxy for human mobility. CDRs are inherently noisy and contain numerous oscillations. In my thesis project, endeavored to identify these noise patterns, known as stationary-state noises, and eliminate them to render the data suitable for human mobility analysis.

Previous methodologies suggested density-based clustering of CDR locations and simply increasing the neighborhood distance if needed. Consequently, if a handoff occurs between two towers within the same cluster, it is deemed a spurious movement and categorized as a stop. I understood that this approach is not suitable for large areas covered by strong towers.

My approach involved employing a graph pattern-based methodology to detect hand-off situations within a user’s CDR data timeline. I selected a couple of noise patterns. Then, utilizing sequences of tower identifiers and connection times, I constructed mobility graphs and identified frequent noise patterns within them. These discovered patterns are then interpreted as stay locations within user trajectories.

Subsequently, stay locations serve as indicators of human presence and dynamic population across various spatial areas and temporal intervals. Additionally, I complemented my research with a visualization dashboard tailored for mobile phone data analysis, accommodating diverse filtering options across multiple data dimensions, and facilitating the application of different filters across multiple dimensions.

The primary focus of my project was to dynamically estimate the population of census units. Furthermore, the refined data can be utilized to identify various events such as large gatherings, evacuations, and for transportation planning purposes.