How do you design statistics that can be conducted over many nodes? Many algorithms factorise nicely over nodes. I might list some here.
If you wish to solve this with heterogeneous, untrustworthy or ad hoc nodes, as opposed to a nice orderly campus HPC cluster, then perhaps it would be better to think of this as swarm sensing.
Placeholder; I have nothing to say about this right now, although I should mention that message-passing algorithms based on variational inference and graphical models are one possible avenue. The most interesting to me is probably Gaussian belief propagation.
References
Acemoglu, Daron, Victor Chernozhukov, and Muhamet Yildiz. 2006. βLearning and Disagreement in an Uncertain World.β Working Paper 12648. National Bureau of Economic Research.
Battey, Heather, Jianqing Fan, Han Liu, Junwei Lu, and Ziwei Zhu. 2015. βDistributed Estimation and Inference with Statistical Guarantees.β arXiv:1509.05457 [Math, Stat], September.
Bianchi, P., and J. Jakubowicz. 2013. βConvergence of a Multi-Agent Projected Stochastic Gradient Algorithm for Non-Convex Optimization.β IEEE Transactions on Automatic Control 58 (2): 391β405.
Bieniawski, Stefan, and David H. Wolpert. 2004. βAdaptive, Distributed Control of Constrained Multi-Agent Systems.β In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 3, 4:1230β31. IEEE Computer Society.
Bottou, LΓ©on, Frank E. Curtis, and Jorge Nocedal. 2016. βOptimization Methods for Large-Scale Machine Learning.β arXiv:1606.04838 [Cs, Math, Stat], June.
Boyd, Stephen. 2010. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Vol. 3. Now Publishers Inc.
Calderhead, Ben. 2014. βA General Construction for Parallelizing MetropolisβHastings Algorithms.β Proceedings of the National Academy of Sciences 111 (49): 17408β13.
Christ, Maximilian, Andreas W. Kempa-Liehr, and Michael Feindt. 2016. βDistributed and Parallel Time Series Feature Extraction for Industrial Big Data Applications.β arXiv:1610.07717 [Cs], October.
Gaines, Brian R., and Hua Zhou. 2016. βAlgorithms for Fitting the Constrained Lasso.β arXiv:1611.01511 [Stat], October.
Heinze, Christina, Brian McWilliams, and Nicolai Meinshausen. 2016. βDUAL-LOCO: Distributing Statistical Estimation Using Random Projections.β In, 875β83.
Heinze, Christina, Brian McWilliams, Nicolai Meinshausen, and Gabriel Krummenacher. 2014. βLOCO: Distributing Ridge Regression with Random Projections.β arXiv:1406.3469 [Stat], June.
Jaggi, Martin, Virginia Smith, Martin Takac, Jonathan Terhorst, Sanjay Krishnan, Thomas Hofmann, and Michael I Jordan. 2014. βCommunication-Efficient Distributed Dual Coordinate Ascent.β In Advances in Neural Information Processing Systems 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, 3068β76. Curran Associates, Inc.
Jain, Prateek, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, and Aaron Sidford. 2016. βParallelizing Stochastic Approximation Through Mini-Batching and Tail-Averaging.β arXiv:1610.03774 [Cs, Stat], October.
Krummenacher, Gabriel, Brian McWilliams, Yannic Kilcher, Joachim M Buhmann, and Nicolai Meinshausen. 2016. βScalable Adaptive Stochastic Optimization Using Random Projections.β In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 1750β58. Curran Associates, Inc.
Ma, Chenxin, Jakub KoneΔnα»³, Martin Jaggi, Virginia Smith, Michael I. Jordan, Peter RichtΓ‘rik, and Martin TakΓ‘Δ. 2015. βDistributed Optimization with Arbitrary Local Solvers.β arXiv Preprint arXiv:1512.04039.
Ma, Chenxin, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter RichtΓ‘rik, and Martin TakΓ‘Δ. 2015. βAdding Vs.Β Averaging in Distributed Primal-Dual Optimization.β arXiv:1502.03508 [Cs], February.
Mann, Richard P., and Dirk Helbing. 2016. βMinorities Report: Optimal Incentives for Collective Intelligence.β arXiv:1611.03899 [Cs, Math, Stat], November.
Mateos, G., J. A. Bazerque, and G. B. Giannakis. 2010. βDistributed Sparse Linear Regression.β IEEE Transactions on Signal Processing 58 (10): 5262β76.
McLachlan, Geoffrey J, and T Krishnan. 2008. The EM algorithm and extensions. Hoboken, N.J.: Wiley-Interscience.
Nathan, Alexandros, and Diego Klabjan. 2016. βOptimization for Large-Scale Machine Learning with Distributed Features and Observations.β arXiv:1610.10060 [Cs, Stat], October.
Shalev-Shwartz, Shai, and Tong Zhang. 2013. βStochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization.β Journal of Machine Learning Research 14 (Feb): 567β99.
Shamir, Ohad, Nathan Srebro, and Tong Zhang. 2014. βCommunication-Efficient Distributed Optimization Using an Approximate Newton-Type Method.β In ICML, 32:1000β1008.
Smith, Virginia, Simone Forte, Michael I. Jordan, and Martin Jaggi. 2015. βL1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework.β arXiv:1512.04011 [Cs], December.
Thanei, Gian-Andrea, Christina Heinze, and Nicolai Meinshausen. 2017. βRandom Projections For Large-Scale Regression.β arXiv:1701.05325 [Math, Stat], January.
Trofimov, Ilya, and Alexander Genkin. 2015. βDistributed Coordinate Descent for L1-Regularized Logistic Regression.β In Analysis of Images, Social Networks and Texts, edited by Mikhail Yu Khachay, Natalia Konstantinova, Alexander Panchenko, Dmitry I. Ignatov, and Valeri G. Labunets, 243β54. Communications in Computer and Information Science 542. Springer International Publishing.
βββ. 2016. βDistributed Coordinate Descent for Generalized Linear Models with Regularization.β arXiv:1611.02101 [Cs, Stat], November.
Yang, Tianbao. 2013. βTrading Computation for Communication: Distributed Stochastic Dual Coordinate Ascent.β In Advances in Neural Information Processing Systems, 629β37.
No comments yet. Why not leave one?