Two provably consistent divide-and-conquer clustering algorithms for large networks
Proceedings of the National Academy of Sciences of the United States of America
In this article, we advance divide-and-conquer strategies for solving the community detection problem in networks. We propose two algorithms that perform clustering on several small subgraphs and finally patch the results into a single clustering. The main advantage of these algorithms is that they significantly bring down the computational cost of traditional algorithms, including spectral clustering, semidefinite programs, modularitybased methods, likelihood-based methods, etc., without losing accuracy, and even improving accuracy at times. These algorithms are also, by nature, parallelizable. Since most traditional algorithms are accurate, and the corresponding optimization problems are much simpler in small problems, our divide-and-conquer methods provide an omnibus recipe for scaling traditional algorithms up to large networks. We prove the consistency of these algorithms under various subgraph selection procedures and perform extensive simulations and real-data analysis to understand the advantages of the divide-and-conquer approach in various ttings.
Mukherjee, Soumendu Sundar; Sarkar, Purnamrita; and Bickel, Peter J., "Two provably consistent divide-and-conquer clustering algorithms for large networks" (2021). Journal Articles. 1716.
Open Access, Green