1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Statistics and Its Application
  4. Volume 11, 2024
  5. Article

Annual Review of Statistics and Its Application

Volume 11, 2024

Distributed Computing and Inference for Big Data

Abstract

Data are distributed across different sites due to computing facility limitations or data privacy considerations. Conventional centralized methods—those in which all datasets are stored and processed in a central computing facility—are not applicable in practice. Therefore, it has become necessary to develop distributed learning approaches that have good inference or predictive accuracy while remaining free of individual data or obeying policies and regulations to protect privacy. In this article, we introduce the basic idea of distributed learning and conduct a selected review on various distributed learning methods, which are categorized by their statistical accuracy, computational efficiency, heterogeneity, and privacy. This categorization can help evaluate newly proposed methods from different aspects. Moreover, we provide up-to-date descriptions of the existing theoretical results that cover statistical equivalency and computational efficiency under different statistical learning frameworks. Finally, we provide existing software implementations and benchmark datasets, and we discuss future research opportunities.

Keyword(s): communication efficiency, distributed learning, federated learning, heterogeneity, statistical equivalence
Loading

Article metrics loading...

/content/journals/10.1146/annurev-statistics-040522-021241
2024-04-22
2024-05-07
Loading full text...

Full text loading...

/deliver/fulltext/statistics/11/1/annurev-statistics-040522-021241.html?itemId=/content/journals/10.1146/annurev-statistics-040522-021241&mimeType=html&fmt=ahah

Literature Cited

  1. Arjevani Y, Shamir O. 2015.. Communication complexity of distributed convex learning and optimization. . In NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems, Vol. 1, ed. C Cortes, DD Lee, M Sugiyama, R Garnett , pp. 1756–64. Cambridge, MA:: MIT Press
     [Google Scholar]
  2. Bagdasaryan E, Veit A, Hua Y, Estrin D, Shmatikov V. 2020.. How to backdoor federated learning. . Proc. Mach. Learn. Res. 108::2938–48
     [Google Scholar]
  3. Banerjee M, Durot C, Sen B. 2019.. Divide and conquer in nonstandard problems and the super-efficiency phenomenon. . Ann. Stat. 47:(2):720–57
     [Crossref] [Google Scholar]
  4. Battey H, Fan J, Liu H, Lu J, Zhu Z. 2018.. Distributed testing and estimation under sparse high dimensional models. . Ann. Stat. 46:(3):1352–82
     [Crossref] [Google Scholar]
  5. Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan HB, et al. 2017.. Practical secure aggregation for privacy-preserving machine learning. . In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1175–91. New York:: ACM
     [Google Scholar]
  6. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. 2021.. Introduction to Meta-Analysis. New York:: John Wiley & Sons
  7. Boyd S. 2010.. Distributed optimization and statistical learning via the alternating direction method of multipliers. . Found. Trends® Mach. Learn. 3:(1):1–122
     [Crossref] [Google Scholar]
  8. Briggs C, Fan Z, Andras P. 2020.. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. . In 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–9. Piscataway, NJ:: IEEE
     [Google Scholar]
  9. Cai TT, Wei H. 2022.. Distributed nonparametric function estimation: optimal rate of convergence and cost of adaptation. . Ann. Stat. 50:(2):698–725
     [Crossref] [Google Scholar]
  10. Caldas S, Duddu SMK, Wu P, Li T, Konečny` J, et al. 2018.. LEAF: a benchmark for federated settings. . arXiv:1812.01097 [cs.LG]
  11. Castro M, Liskov B. 1999.. Practical Byzantine fault tolerance. . In 3rd Symposium on Operating Systems Design and Implementation (OSDI 99), pp. 173–86. Berkeley, CA:: USENIX
     [Google Scholar]
  12. Chang C, Bu Z, Long Q. 2023.. CEDAR: communication efficient distributed analysis for regressions. . Biometrics 79:(3):235769
     [Crossref] [Google Scholar]
  13. Chen SX, Peng L. 2021.. Distributed statistical inference for massive data. . Ann. Stat. 49:(5):2851–69
     [Google Scholar]
  14. Chen X, Liu C, Li B, Lu K, Song D. 2017.. Targeted backdoor attacks on deep learning systems using data poisoning. . arXiv:1712.05526 [cs.CR]
  15. Chen X, Liu W, Mao X, Yang Z. 2020.. Distributed high-dimensional regression under a quantile loss function. . J. Mach. Learn. Res. 21:(182):1–43
     [Google Scholar]
  16. Chen X, Liu W, Zhang Y. 2022.. First-order Newton-type estimator for distributed estimation and inference. . J. Am. Stat. Assoc. 117:(540):1858–74
     [Crossref] [Google Scholar]
  17. Chen X, Xie M. 2014.. A split-and-conquer approach for analysis of extraordinarily large data. . Stat. Sin. 24::1655–84
     [Google Scholar]
  18. Corinzia L, Beuret A, Buhmann JM. 2021.. Variational federated multi-task learning. . arXiv:1906.06268 [cs.LG]
  19. Dean J, Ghemawat S. 2008.. MapReduce: simplified data processing on large clusters. . Commun. ACM 51:(1):107–13
     [Crossref] [Google Scholar]
  20. Dinh CT, Tran NH, Nguyen TD. 2020.. Personalized federated learning with Moreau envelopes. . In NIPS'20: Proceedings of the 34th International Conference on Neural Information Processing Systems, ed. H Larochelle, M Ranzato, R Hadsell, MF Balcan, H Lin , pp. 21394–405. Red Hook, NY:: Curran
     [Google Scholar]
  21. Dittrich J, Richter S, Schuh S. 2013.. Efficient OR Hadoop: Why not both?. Datenbank-Spektrum 13:(1):17–22
     [Crossref] [Google Scholar]
  22. Dong J, Roth A, Su WJ. 2022.. Gaussian differential privacy. . J. R. Stat. Soc. Ser. B 84:(1):3–37
     [Crossref] [Google Scholar]
  23. du Terrail JO, Ayed SS, Cyffers E, Grimberg F, He C, et al. 2022.. FLamby: datasets and benchmarks for cross-silo federated learning in realistic healthcare settings. . arXiv:2210.04620 [cs.LG]
  24. Dwork C, McSherry F, Nissim K, Smith A. 2006.. Calibrating noise to sensitivity in private data analysis. . In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4–7, 2006, pp. 265–84. New York:: Springer
     [Google Scholar]
  25. Dwork C, Roth A. 2013.. The algorithmic foundations of differential privacy. . Found. Trends® Theor. Comput. Sci. 9:(3–4):211–407
     [Crossref] [Google Scholar]
  26. Fallah A, Mokhtari A, Ozdaglar A. 2020a.. On the convergence theory of gradient-based model-agnostic meta-learning algorithms. . Proc. Mach. Learn. Res. 108::1082–92
     [Google Scholar]
  27. Fallah A, Mokhtari A, Ozdaglar A. 2020b.. Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. . In NIPS'20: Proceedings of the 34th International Conference on Neural Information Processing Systems, ed. H Larochelle, M Ranzato, R Hadsell, MF Balcan, H Lin , pp. 3557–68. Red Hook, NY:: Curran
     [Google Scholar]
  28. Fan J, Guo Y, Wang K. 2021.. Communication-efficient accurate statistical estimation. . J. Am. Stat. Assoc. 118:(543):1000–10
     [Google Scholar]
  29. Fan J, Wang D, Wang K, Zhu Z. 2019.. Distributed estimation of principal eigenspaces. . Ann. Stat. 47:(6):3009–31
     [Crossref] [Google Scholar]
  30. Fernando N, Loke SW, Rahayu W. 2013.. Mobile cloud computing: a survey. . Future Gener. Comput. Syst. 29:(1):84–106
     [Crossref] [Google Scholar]
  31. Forero PA, Cano A, Giannakis GB. 2010.. Consensus-based distributed support vector machines. . J. Mach. Learn. Res. 11:(55):1663–707
     [Google Scholar]
  32. Gao Y, Liu W, Wang H, Wang X, Yan Y, Zhang R. 2022.. A review of distributed statistical inference. . Stat. Theory Relat. Fields 6:(2):89–99
     [Crossref] [Google Scholar]
  33. Geyer RC, Klein T, Nabi M. 2017.. Differentially private federated learning: a client level perspective. . arXiv:1712.07557 [cs.CR]
  34. Ghosh A, Chung J, Yin D, Ramchandran K. 2022.. An efficient framework for clustered federated learning. . IEEE Trans. Inf. Theory 68:(12):8076–91
     [Crossref] [Google Scholar]
  35. Gupta V, Ghosh A, Derezinski M, Khanna R, Ramchandran K, Mahoney M. 2021.. LocalNewton: reducing communication bottleneck for distributed learning. . arXiv:2105.07320 [cs.DC]
  36. Hammoud M, Rehman MS, Sakr MF. 2012.. Center-of-gravity reduce task scheduling to lower MapReduce network traffic. . In 2012 IEEE Fifth International Conference on Cloud Computing, pp. 49–58. Piscataway, NJ:: IEEE
     [Google Scholar]
  37. He C, Balasubramanian K, Ceyani E, Yang C, Xie H, et al. 2021.. FedGraphNN: a federated learning system and benchmark for graph neural networks. . arXiv:2104.07145 [cs.LG]
  38. He C, Li S, So J, Zeng X, Zhang M, et al. 2020.. FedML: a research library and benchmark for federated machine learning. . arXiv:2007.13518 [cs.LG]
  39. Horowitz E, Zorat A. 1983.. Divide-and-conquer for parallel processing. . IEEE Trans. Comput. C-32:(6):582–85
     [Crossref] [Google Scholar]
  40. Huang C, Huo X. 2019.. A distributed one-step estimator. . Math. Program. 174::41–76
     [Crossref] [Google Scholar]
  41. Huang Y, Chu L, Zhou Z, Wang L, Liu J, et al. 2021.. Personalized cross-silo federated learning on non-IID data. . Proc. AAAI Conf. Artif. Intell. 35:(9):7865–73
     [Google Scholar]
  42. Jiao S, He C, Dou Y, Tang H. 2012.. Molecular dynamics simulation: implementation and optimization based on Hadoop. . In 2012 8th International Conference on Natural Computation, pp. 1203–7. Piscataway, NJ:: IEEE
     [Google Scholar]
  43. Jordan MI, Lee JD, Yang Y. 2019.. Communication-efficient distributed statistical inference. . J. Am. Stat. Assoc. 114:(526):668–81
     [Crossref] [Google Scholar]
  44. Karimireddy SP, Kale S, Mohri M, Reddi S, Stich S, Suresh AT. 2020.. SCAFFOLD: stochastic controlled averaging for federated learning. . Proc. Mach. Learn. Res. 119::5132–43
     [Google Scholar]
  45. Khodak M, Balcan MFF, Talwalkar AS. 2019.. Adaptive gradient-based meta-learning methods. . In 33rd Conference on Neural Information Processing Systems (NeurIPS2019), ed. H Wallach, H Larochelle, A Beygelzimer, F d'Alché-Buc, E Fox, R Garnett , pp. 5917–28. Red Hook, NY:: Curran
     [Google Scholar]
  46. Kleiner A, Talwalkar A, Sarkar P, Jordan MI. 2014.. A scalable bootstrap for massive data. . J. R. Stat. Soc. Ser. B 76:(4):795–816
     [Crossref] [Google Scholar]
  47. Lee JD, Lin Q, Ma T, Yang T. 2017a.. Distributed stochastic variance reduced gradient methods by sampling extra data with replacement. . J. Mach. Learn. Res. 18:(122):1–43
     [Google Scholar]
  48. Lee JD, Liu Q, Sun Y, Taylor JE. 2017b.. Communication-efficient sparse regression. . J. Mach. Learn. Res. 18:(1):115–44
     [Google Scholar]
  49. Li Q, He B, Song D. 2021.. Model-contrastive federated learning. . In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10708–17. Piscataway, NJ:: IEEE
     [Google Scholar]
  50. Li Q, Wen Z, Wu Z, Hu S, Wang N, et al. 2023.. A survey on federated learning systems: vision, hype and reality for data privacy and protection. . IEEE Trans. Knowledge Data Eng. 35::3347–66
     [Crossref] [Google Scholar]
  51. Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V. 2020a.. Federated optimization in heterogeneous networks. . In Proceedings of Machine Learning and Systems 2 (MLSys 2020), ed. I Dhillon, D Papailiopoulos, V Sze , pp. 429–50. Indio, CA:: Syst. Mach. Learn. Found.
     [Google Scholar]
  52. Li T, Sanjabi M, Beirami A, Smith V. 2020b.. Fair resource allocation in federated learning. . In Proceedings of the Eighth International Conference on Learning Representations. N.p.:: ICLR
     [Google Scholar]
  53. Lian H, Fan Z. 2018.. Divide-and-conquer for debiased l1-norm support vector machine in ultra-high dimensions. . J. Mach. Learn. Res. 18:(182):1–26
     [Google Scholar]
  54. Lin DY, Zeng D. 2010.. On the relative efficiency of using summary statistics versus individual-level data in meta-analysis. . Biometrika 97:(2):321–32
     [Crossref] [Google Scholar]
  55. Lin N, Xi R. 2011.. Aggregated estimating equation estimation. . Stat. Interface 4:(1):73–83
     [Crossref] [Google Scholar]
  56. Liu D, Liu RY, Xie M. 2015.. Multivariate meta-analysis of heterogeneous studies using only summary statistics: efficiency and robustness. . J. Am. Stat. Assoc. 110:(509):326–40
     [Crossref] [Google Scholar]
  57. Liu Q, Ihler AT. 2014.. Distributed estimation, information loss and exponential families. . In Advances in Neural Information Processing Systems 27 (NIPS 2014), ed. Z Ghahramani, M Welling, C Cortes, N Lawrence, K Weinberger , pp. 1098–106. Red Hook, NY:: Curran
     [Google Scholar]
  58. Liu Y, Kang Y, Xing C, Chen T, Yang Q. 2020.. A secure federated transfer learning framework. . IEEE Intell. Syst. 35:(4):70–82
     [Crossref] [Google Scholar]
  59. Long G, Xie M, Shen T, Zhou T, Wang X, et al. 2023.. Multi-center federated learning: clients clustering for better personalization. . World Wide Web 26:(1):481–500
     [Crossref] [Google Scholar]
  60. Lv S, Lian H. 2017.. Debiased distributed learning for sparse partial linear models in high dimensions. . arXiv:1708.05487 [stat.ML]
  61. Lyu L, Xu X, Wang Q. 2020.. Collaborative fairness in federated learning. . arXiv:2008.12161 [cs.LG]
  62. Ma Y, Yu D, Wang H. 2019.. PaddlePaddle: an open-source deep learning platform from industrial practice. . J. Front. Comput. Sci. Technol. 13:(1):11–23
     [Google Scholar]
  63. McMahan B, Moore E, Ramage D, Hampson S, Aguera y Arcas B. 2017.. Communication-efficient learning of deep networks from decentralized data. . Proc. Mach. Learn. Res. 54::1273–82
     [Google Scholar]
  64. Menabrea L. 1843.. Sketch of the Analytical Engine Invented by Charles Babbage., transl. A Lovelace. London:: R. & J.E. Taylor
  65. Minsker S. 2019.. Distributed statistical estimation and rates of convergence in normal approximation. . Electron. J. Stat. 13:(2):5213–52
     [Crossref] [Google Scholar]
  66. Mohri M, Sivek G, Suresh AT. 2019.. Agnostic federated learning. . Proc. Mach. Learn. Res. 97::4615–25
     [Google Scholar]
  67. Nedic A, Ozdaglar A. 2009.. Distributed subgradient methods for multi-agent optimization. . IEEE Trans. Autom. Control 54:(1):48–61
     [Crossref] [Google Scholar]
  68. Nehme RV, Lim HS, Bertino E, Rundensteiner EA. 2009.. StreamShield: a stream-centric approach towards security and privacy in data stream environments. . In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, SIGMOD '09, pp. 1027–30. New York:: ACM
     [Google Scholar]
  69. Newman D, Asuncion A, Smyth P, Welling M. 2009.. Distributed algorithms for topic models. . J. Mach. Learn. Res. 10:(62):1801–28
     [Google Scholar]
  70. Noghabi SA, Paramasivam K, Pan Y, Ramesh N, Bringhurst J, et al. 2017.. Samza: stateful scalable stream processing at LinkedIn. . In Proceedings of the VLDB Endowment, Vol. 10, ed. P Boncz, K Salem , pp. 1634–45. N.p.:: VLDB Endow.
     [Google Scholar]
  71. Passino K. 2002.. Biomimicry of bacterial foraging for distributed optimization and control. . IEEE Control Syst. Mag. 22:(3):52–67
     [Crossref] [Google Scholar]
  72. Phong LT, Aono Y, Hayashi T, Wang L, Moriai S. 2018.. Privacy-preserving deep learning via additively homomorphic encryption. . IEEE Trans. Inform. Forensics Secur. 13:(5):1333–45
     [Crossref] [Google Scholar]
  73. Reisizadeh A, Farnia F, Pedarsani R, Jadbabaie A. 2020.. Robust federated learning: the case of affine distribution shifts. . In NIPS'20: Proceedings of the 34th International Conference on Neural Information Processing Systems, ed. H Larochelle, M Ranzato, R Hadsell, M Balcan, H Lin , pp. 21554–65. Red Hook, NY:: Curran
     [Google Scholar]
  74. Rosenblatt JD, Nadler B. 2016.. On the optimality of averaging in distributed statistical learning. . Inform. Inference J. IMA 5:(4):379–404
     [Crossref] [Google Scholar]
  75. Ryffel T, Trask A, Dahl M, Wagner B, Mancuso J, et al. 2018.. A generic framework for privacy preserving deep learning. . arXiv:1811.04017 [cs.LG]
  76. Shamir O, Srebro N, Zhang T. 2014.. Communication-efficient distributed optimization using an approximate Newton-type method. . Proc. Mach. Learn. Res. 32::1000–8
     [Google Scholar]
  77. Shang Z, Cheng G. 2017.. Computational limits of a distributed algorithm for smoothing spline. . J. Mach. Learn. Res. 18:(108):1–37
     [Google Scholar]
  78. Shen J, Liu RY, Xie M. 2020.. iFusion: individualized fusion learning. . J. Am. Stat. Assoc. 115:(531):1251–67
     [Crossref] [Google Scholar]
  79. Shi C, Lu W, Song R. 2018.. A massive data framework for M-estimators with cubic-rate. . J. Am. Stat. Assoc. 113:(524):1698–709
     [Crossref] [Google Scholar]
  80. Shoham N, Avidor T, Keren A, Israel N, Benditkis D, et al. 2019.. Overcoming forgetting in federated learning on non-IID data. . arXiv:1910.07796 [cs.LG]
  81. Shokri R, Stronati M, Song C, Shmatikov V. 2017.. Membership inference attacks against machine learning models. . In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18. Piscataway, NJ:: IEEE
     [Google Scholar]
  82. Singh K, Xie M, Strawderman WE. 2005.. Combining information from independent sources through confidence distributions. . Ann. Stat. 33::159–83
     [Crossref] [Google Scholar]
  83. Smith V, Chiang CK, Sanjabi M, Talwalkar A. 2017.. Federated multi-task learning. . In NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, ed. U von Luxburg, I Guyon, S Bengio, H Wallach, R Fergus , pp. 4427–37. Red Hook, NY:: Curran
     [Google Scholar]
  84. Song Q, Liang F, et al. 2015.. A split-and-merge Bayesian variable selection approach for ultrahigh dimensional regression. . J. R. Stat. Soc. Ser. B 77:(5):947–72
     [Crossref] [Google Scholar]
  85. Szabó B, Van Zanten H, et al. 2019.. An asymptotic analysis of distributed nonparametric methods. . J. Mach. Learn. Res. 20:(87):1–30
     [Google Scholar]
  86. Tan X, Chang CCH, Zhou L, Tang L. 2022.. A tree-based model averaging approach for personalized treatment effect estimation from heterogeneous data sources. . Proc. Mach. Learn. Res. 162::21013–36
     [Google Scholar]
  87. Tang L, Zhou L, Song PXK. 2020.. Distributed simultaneous inference in generalized linear models via confidence distribution. . J. Multivariate Anal. 176::104567
     [Crossref] [Google Scholar]
  88. Taylor RC. 2010.. An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics. . BMC Bioinformatics 11:(S12):S1
     [Crossref] [Google Scholar]
  89. Toh S, Rasmussen-Torvik LJ, Harmata EE, Pardee R, Saizan R, et al. 2017.. The national Patient-Centered Clinical Research Network (PCORnet) bariatric study cohort: rationale, methods, and baseline characteristics. . JMIR Res. Protoc. 6:(12):e8323
     [Crossref] [Google Scholar]
  90. Toshniwal A, Taneja S, Shukla A, Ramasamy K, Patel JM, et al. 2014.. Storm@twitter. . In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD '14, pp. 147–56. New York:: ACM
     [Google Scholar]
  91. van de Geer S, Bühlmann P, Ritov Y, Dezeure R. 2014.. On asymptotically optimal confidence regions and tests for high-dimensional models. . Ann. Stat. 42:(3):1166–202
     [Crossref] [Google Scholar]
  92. Volgushev S, Chao SK, Cheng G. 2019.. Distributed inference for quantile regression processes. . Ann. Stat. 47:(3):1634–62
     [Crossref] [Google Scholar]
  93. Wang J, Kolar M, Srebro N, Zhang T. 2017.. Efficient distributed learning with sparsity. . Proc. Mach. Learn. Res. 70::3636–45
     [Google Scholar]
  94. Wang S, Roosta-Khorasani F, Xu P, Mahoney MW. 2018.. GIANT: globally improved approximate Newton method for distributed optimization. . In NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Systems, ed. S Bengio, HM Wallach, H Larochelle, K Grauman, N Cesa-Bianchi , pp. 2338–48. Red Hook, NY:: Curran
     [Google Scholar]
  95. Wang X, Yang Z, Chen X, Liu W. 2019.. Distributed inference for linear support vector machine. . J. Mach. Learn. Res. 20:(113):1–41
     [Google Scholar]
  96. White T. 2015.. Hadoop: The Definitive Guide. Beijing:: O'Reilly. , 4th ed..
  97. Xie M, Singh K. 2013.. Confidence distribution, the frequentist distribution estimator of a parameter: a review. . Int. Stat. Rev. 81:(1):3–39
     [Crossref] [Google Scholar]
  98. Xu C, Zhang Y, Li R, Wu X. 2016.. On the feasibility of distributed kernel regression for big data. . IEEE Trans. Knowledge Data Eng. 28:(11):3041–52
     [Crossref] [Google Scholar]
  99. Yang Q, Liu Y, Chen T, Tong Y. 2019.. Federated machine learning: concept and applications. . ACM Trans. Intel. Syst. Technol. 10:(2):12
     [Crossref] [Google Scholar]
  100. Yao L, Chu Z, Li S, Li Y, Gao J, Zhang A. 2021.. A survey on causal inference. . ACM Trans. Knowledge Discov. Data 15:(5):74
     [Crossref] [Google Scholar]
  101. Yao X, Sun L. 2020.. Continual local training for better initialization of federated models. . In 2020 IEEE International Conference on Image Processing (ICIP), pp. 1736–40. Piscataway, NJ:: IEEE
     [Google Scholar]
  102. Yu T, Bagdasaryan E, Shmatikov V. 2022.. Salvaging federated learning by local adaptation. . arXiv:2002.04758 [cs.LG]
  103. Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I. 2012.. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. . In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, artic. 2. Berkeley, CA:: USENIX
     [Google Scholar]
  104. Zeng D, Lin D. 2015.. On random-effects meta-analysis. . Biometrika 102:(2):281–94
     [Crossref] [Google Scholar]
  105. Zhang CH, Zhang SS. 2014.. Confidence intervals for low dimensional parameters in high dimensional linear models. . J. R. Stat. Soc. Ser. B 76:(1):217–42
     [Crossref] [Google Scholar]
  106. Zhang M, Sapra K, Fidler S, Yeung S, Alvarez JM. 2020.. Personalized federated learning with first order model optimization. . arXiv:2012.08565 [cs.LG]
  107. Zhang Y, Duchi JC, Wainwright MJ. 2013.. Communication-efficient algorithms for statistical optimization. . J. Mach. Learn. Res. 14:(1):3321–63
     [Google Scholar]
  108. Zhang Y, Lin X. 2015.. DiSCO: distributed optimization for self-concordant empirical loss. . Proc. Mach. Learn. Res. 37::362–70
     [Google Scholar]
  109. Zhao T, Cheng G, Liu H. 2016.. A partially linear framework for massive heterogeneous data. . Ann. Stat. 44:(4):1400–37
     [Crossref] [Google Scholar]
  110. Zhou L, She X, Song PXK. 2022.. Distributed empirical likelihood approach to integrating unbalanced datasets. . Stat. Sin. 33::2209–31
     [Google Scholar]
  111. Zhou L, Song PXK. 2017.. Scalable and efficient statistical inference with estimating functions in the MapReduce paradigm for big data. . arXiv:1709.04389 [stat.ME]
  112. Zhou S, Li GY. 2021.. Communication-efficient ADMM-based federated learning. . arXiv:2110.15318 [cs.LG]
  113. Zhu H, Jin Y. 2020.. Multi-objective evolutionary federated learning. . IEEE Trans. Neural Netw. Learn. Syst. 31:(4):1310–22
     [Crossref] [Google Scholar]
/content/journals/10.1146/annurev-statistics-040522-021241
Loading
Distributed Computing and Inference for Big Data
Annual Review of Statistics and Its Application 11, 533 (2024); https://doi.org/10.1146/annurev-statistics-040522-021241
/content/journals/10.1146/annurev-statistics-040522-021241
/content/journals/10.1146/annurev-statistics-040522-021241
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error