ABSTRACT
Statistical machine learning often uses probabilistic models and algorithms, such as Markov Chain Monte Carlo (MCMC), to solve a wide range of problems. Probabilistic computations, often considered too slow on conventional processors, can be accelerated with specialized hardware by exploiting parallelism and optimizing the design using various approximation techniques. Current methodologies for evaluating correctness of probabilistic accelerators are often incomplete, mostly focusing only on end-point result quality ("accuracy"). It is important for hardware designers and domain experts to look beyond end-point "accuracy" and be aware of how hardware optimizations impact statistical properties.
This work takes a first step toward defining metrics and a methodology for quantitatively evaluating correctness of probabilistic accelerators. We propose three pillars of statistical robustness: 1) sampling quality, 2) convergence diagnostic, and 3) goodness of fit. We apply our framework to a representative MCMC accelerator and surface design issues that cannot be exposed using only application end-point result quality. We demonstrate the benefits of this framework to guide design space exploration in a case study showing that statistical robustness comparable to floating-point software can be achieved with limited precision, avoiding floating-point hardware overheads.
- Tarek Ould Bachir, Mohamad Sawan, and Jean-Jules Brault. 2008. A new hardware architecture for sampling the exponential distribution. In Electrical and Computer Engineering, 2008. CCECE 2008. Canadian Conference on. IEEE, 001393?001396. https://doi.org/10.1109/CCECE. 2008.4564770 Google Scholar
Cross Ref
- Simon Baker, Daniel Scharstein, J. P. Lewis, Stefan Roth, Michael J. Black, and Richard Szeliski. 2011. A Database and Evaluation Methodology for Optical Flow. International Journal of Computer Vision 92, 1 ( 01 Mar 2011 ), 1?31. https: //doi.org/10.1007/s11263-010-0390-2 Google Scholar
- Rajeev Balasubramonian, Andrew B. Kahng, Naveen Muralimanohar, Ali Shafiee, and Vaishnav Srinivas. 2017. CACTI 7: New Tools for Interconnect Exploration in Innovative Of-Chip Memories. ACM Trans. Archit. Code Optim. 14, 2, Article 14 ( June 2017 ), 25 pages. https://doi.org/10.1145/3085572 Google Scholar
- Subho S. Banerjee, Zbigniew T. Kalbarczyk, and Ravishankar K. Iyer. 2019. AcMC 2 : Accelerating Markov Chain Monte Carlo Algorithms for Probabilistic Models. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (Providence, RI, USA) ( ASPLOS '19). ACM, New York, NY, USA, 515?528. https://doi.org/10.1145/ 3297858.3304019 Google Scholar
- Aubrey Barnard. 2019. Causal Discovery of Adverse Drug Events in Observational Data. Ph.D. Dissertation. University of Wisconsin?Madison.Google Scholar
- Stephen T. Barnard. 1989. Stochastic stereo matching over scale. International Journal of Computer Vision 3, 1 ( 01 May 1989 ), 17?32. https://doi.org/10.1007/ BF00054836 Google Scholar
- Michael Betancourt. 2017. A conceptual introduction to Hamiltonian Monte Carlo. arXiv preprint ( 2017 ). arXiv: 1701. 02434Google Scholar
- Stephen P. Brooks and Andrew Gelman. 1998. General Methods for Monitoring Convergence of Iterative Simulations. Journal of Computational and Graphical Statistics 7, 4 ( 1998 ), 434?455. https://doi.org/10.1080/10618600. 1998.10474787 Google Scholar
- Ruizhe Cai, Ao Ren, Ning Liu, Caiwen Ding, Luhao Wang, Xuehai Qian, Massoud Pedram, and Yanzhi Wang. 2018. VIBNN: Hardware Acceleration of Bayesian Neural Networks (ASPLOS '18). Association for Computing Machinery, New York, NY, USA, 476?488. https://doi.org/10.1145/3173162.3173212 Google Scholar
- Lakshmi N. Chakrapani, Bilge E.S. Akgul, Suresh Cheemalavagu, Pinar Korkmaz, Krishna V. Palem, and Balasubramanian Seshasayee. 2006. Ultra-Eficient (Embedded) SOC Architectures based on Probabilistic CMOS (PCMOS) Technology. In Proceedings of the Design Automation Test in Europe Conference, Vol. 1. 1?6. https://doi.org/10.1109/DATE. 2006.243978 Google Scholar
- Jerry Chee and Panos Toulis. 2018. Convergence diagnostics for stochastic gradient descent with constant learning rate. In Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 84 ), Amos Storkey and Fernando Perez-Cruz (Eds.). PMLR, 1476?1485. http://proceedings.mlr.press/v84/chee18a.htmlGoogle Scholar
- Wenjun Cheng, Luyao Ma, Tiejun Yang, Jiali Liang, and Yan Zhang. 2016. Joint lung CT image segmentation: a hierarchical Bayesian approach. PloS one 11, 9 ( 2016 ). https://doi.org/10.1371/journal.pone.0162211 Google Scholar
- Timothy H. Click, Aibing Liu, and George A. Kaminski. 2011. Quality of random number generators significantly afects results of Monte Carlo simulations for organic and biological systems. Journal of computational chemistry 32, 3 ( 2011 ), 513?524. https://doi.org/10.1002/jcc.21638 Google Scholar
- Paul D. Coddington. 1994. Analysis of random number generators using Monte Carlo simulation. International Journal of Modern Physics C 5, 03 ( 1994 ), 547?560. https://doi.org/10.1142/S0129183194000726 Google Scholar
- Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. 2015. BinaryConnect: Training Deep Neural Networks with binary weights during propagations. In Advances in Neural Information Processing Systems 28. Curran Associates, Inc., 3123?3131. http://papers.nips.cc/paper/5647-binaryconnect-training-deepneural-networks-with-binary-weights-during-propagations.pdfGoogle Scholar
- Mary Kathryn Cowles and Bradley P. Carlin. 1996. Markov chain Monte Carlo convergence diagnostics: a comparative review. J. Amer. Statist. Assoc. 91, 434 ( 1996 ), 883?904. https://doi.org/10.1080/01621459. 1996.10476956 Google Scholar
- Keivan Dabiri, Mehrdad Malekmohammadi, Ali Sheikholeslami, and Hirotaka Tamura. 2020. Replica Exchange MCMC Hardware With Automatic Temperature Selection and Parallel Trial. IEEE Transactions on Parallel and Distributed Systems 31, 7 ( 2020 ), 1681?1692. https://doi.org/10.1109/TPDS. 2020.2972359 Google Scholar
- Eva Darulova. 2014. Programming with numerical uncertainties. Ph.D. Dissertation. EPFL. https://doi.org/10.5075/epfl-thesis-6343 Google Scholar
- Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting on association for computational linguistics. Association for Computational Linguistics, 363?370. https: //doi.org/10.3115/1219840.1219885 Google Scholar
- James M. Flegal, Murali Haran, and Galin L. Jones. 2008. Markov chain Monte Carlo: Can we trust the third significant figure? Statist. Sci. ( 2008 ), 250?260. https://doi.org/10.1214/08-STS257 Google Scholar
- Linton C. Freeman. 1965. Elementary Applied Statistics: For Students in Behavioral Science. Wiley.Google Scholar
- Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning. 1050?1059. http://proceedings.mlr.press/v48/gal16.htmlGoogle Scholar
- Rong Ge, Holden Lee, and Andrej Risteski. 2018. Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition. arXiv preprint ( 2018 ). arXiv: 1812.00793Google Scholar
- Andrew Gelman and Donald B. Rubin. 1992. Inference from iterative simulation using multiple sequences. Statistical science 7, 4 ( 1992 ), 457?472. https://doi.org/ 10.1214/ss/1177011136 Google Scholar
- Stuart Geman and Donald Geman. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on pattern analysis and machine intelligence 6 ( 1984 ), 721?741. https://doi.org/10.1109/ TPAMI. 1984.4767596 Google Scholar
- Sinong Geng, Zhaobin Kuang, Jie Liu, Stephen Wright, and David Page. 2018. Stochastic learning for sparse discrete markov random fields with controlled gradient approximation error. In Conference on Uncertainty in Artificial Intelligence, Vol. 2018. NIH Public Access, 156. https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC6292514Google Scholar
- Zoubin Ghahramani. 2015. Probabilistic machine learning and artificial intelligence. Nature 521, 7553 ( 2015 ), 452?459. https://doi.org/10.1038/nature14541 Google Scholar
- Lei Gong and James M. Flegal. 2016. A practical sequential stopping rule for highdimensional Markov chain Monte Carlo. Journal of Computational and Graphical Statistics 25, 3 ( 2016 ), 684?700. https://doi.org/10.1080/10618600. 2015.1044092 Google Scholar
- Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. Journal of Machine Learning Research 13, Mar ( 2012 ), 723?773. https://www.jmlr.org/papers/volume13/ gretton12a/gretton12a.pdfGoogle Scholar
- Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan. 2015. Deep Learning with Limited Numerical Precision. arXiv preprint ( 2015 ). arXiv: 1502. 02551Google Scholar
- Allan Haldane and Ronald M. Levy. 2020. Mi3-GPU: MCMC-based inverse ising inference on GPUs for protein covariation analysis. Computer Physics Communications ( 2020 ), 107312. https://doi.org/10.1016/j.cpc. 2020.107312 Google Scholar
- Ghassan Hamra, Richard MacLehose, and David Richardson. 2013. Markov chain Monte Carlo: an introduction for epidemiologists. International journal of epidemiology 42, 2 ( 2013 ), 627?634. https://doi.org/10.1093/ije/dyt043 Google Scholar
- Marcel Häselich, Simon Eggert, and Dietrich Paulus. 2012. Parallelized energy minimization for real-time Markov random field terrain classification in natural environments. In 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 1823?1828. https://doi.org/10.1109/ROBIO. 2012.6491233 Google Scholar
- Jonathan C. Hedstrom, Chung Him Yuen, Rong-Rong Chen, and Behrouz FarhangBoroujeny. 2017. Achieving near MAP performance with an excited Markov chain Monte Carlo MIMO detector. IEEE Transactions on Wireless Communications 16, 12 ( 2017 ), 7718?7732. https://doi.org/10.1109/TWC. 2017.2750667 Google Scholar
- Gregory Herschlag, Han Sung Kang, Justin Luo, Christy Vaughn Graves, Sachet Bangia, Robert Ravier, and Jonathan C. Mattingly. 2018. Quantifying gerrymandering in north carolina. arXiv preprint ( 2018 ). arXiv: 1801.03783Google Scholar
- Jordan L. Holi and Jenq-Neng Hwang. 1993. Finite precision error analysis of neural network hardware implementations. IEEE Trans. Comput. 42, 3 ( 1993 ), 281?290. https://doi.org/10.1109/12.210171 Google Scholar
- Eyke Hüllermeier and Willem Waegeman. 2019. Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods. arXiv preprint ( 2019 ). arXiv: 1910.09457Google Scholar
- Skand Hurkat and José F Martínez. 2019. VIP: A Versatile Inference Processor. In 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 345?358. https://doi.org/10.1109/HPCA. 2019.00049 Google Scholar
- Intel ®. 2019. Floating-Point IP Cores User Guide. https://www.intel.com/ content/www/us/en/programmable/documentation/eis1410764818924.htmlGoogle Scholar
- Intel ®. 2019. Intel® Quartus® Prime Software Suite. https://www.intel.com/ content/www/us/en/software/programmable/quartus-prime/overview.htmlGoogle Scholar
- James E. Johndrow, Jonathan C. Mattingly, Sayan Mukherjee, and David Dunson. 2015. Optimal approximating Markov chains for Bayesian inference. arXiv preprint ( 2015 ). arXiv: 1508. 03387Google Scholar
- Robert E. Kass, Bradley P. Carlin, Andrew Gelman, and Radford M. Neal. 1998. Markov chain Monte Carlo in practice: a roundtable discussion. The American Statistician 52, 2 ( 1998 ), 93?100. https://doi.org/10.2307/2685466 Google Scholar
- Alex Kendall and Yarin Gal. 2017. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) ( NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 5580?5590.Google Scholar
- Osama U. Khan and David D. Wentzlof. 2016. Hardware Accelerator for Probabilistic Inference in 65-nm CMOS. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 24, 3 ( 2016 ), 837?845. https://doi.org/10.1109/ TVLSI. 2015.2420663 Google Scholar
- Paresh Kharya. 2020. NVIDIA Blogs: TensorFloat-32 Accelerates AI Training HPC upto 20x. https://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precisionformat/Google Scholar
- Leslie Kish. 1965. Survey sampling. New York: John Wiley & Sons.Google Scholar
- Glenn G. Ko, Yuji Chai, Rob A. Rutenbar, David Brooks, and Gu-Yeon Wei. 2019. Accelerating Bayesian Inference on Structured Graphs Using Parallel Gibbs Sampling. In 2019 29th International Conference on Field Programmable Logic and Applications (FPL). 159?165. https://doi.org/10.1109/FPL. 2019.00033 Google Scholar
- Ranganath Krishnan and Omesh Tickoo. 2020. Improving model calibration with accuracy versus uncertainty optimization. arXiv preprint ( 2020 ). arXiv: 2012.07923Google Scholar
- Yongchan Kwon, Joong-Ho Won, Beom Joon Kim, and Myunghee Cho Paik. 2020. Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation. Computational Statistics & Data Analysis 142 ( 2020 ), 106816. https://doi.org/10.1016/j.csda. 2019.106816 Google Scholar
- Jianhua Lin. 1991. Divergence measures based on the Shannon entropy. IEEE Transactions on Information theory 37, 1 ( 1991 ), 145?151. https://doi.org/10.1109/ 18.61115 Google Scholar
- Mingjie Lin, Ilia Lebedev, and John Wawrzynek. 2010. High-throughput Bayesian Computing Machine with Reconfigurable Hardware. In Proceedings of the 18th Annual ACM/SIGDA International Symposium on Field Programmable Gate Arrays (Monterey, California, USA) ( FPGA '10). ACM, New York, NY, USA, 73?82. https: //doi.org/10.1145/1723112.1723127 Google Scholar
- Michael D. Linderman, Matthew Ho, David L. Dill, Teresa H. Meng, and Garry P. Nolan. 2010. Towards Program Optimization Through Automated Analysis of Numerical Precision. In Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization (Toronto, Ontario, Canada) ( CGO '10). ACM, New York, NY, USA, 230?237. https://doi.org/10.1145/1772954.1772987 Google Scholar
- Shuanglong Liu, Grigorios Mingas, and Christos-Savvas Bouganis. 2015. An exact MCMC accelerator under custom precision regimes. In 2015 International Conference on Field Programmable Technology (FPT). IEEE, 120?127. https:// doi.org/10.1109/FPT. 2015.7393138 Google Scholar
- Divya Mahajan, Jongse Park, Emmanuel Amaro, Hardik Sharma, Amir Yazdanbakhsh, Joon Kyung Kim, and Hadi Esmaeilzadeh. 2016. Tabla: A unified template-based framework for accelerating statistical machine learning. In High Performance Computer Architecture (HPCA), 2016 IEEE International Symposium on. IEEE, 14?26. https://doi.org/10.1109/HPCA. 2016.7446050 Google Scholar
- Divya Mahajan, Amir Yazdanbakhsh, Jongse Park, Bradley Thwaites, and Hadi Esmaeilzadeh. 2016. Towards Statistical Guarantees in Controlling Quality Tradeofs for Approximate Acceleration. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA). 66?77. https://doi.org/10.1109/ ISCA. 2016.16 Google Scholar
- Andrey Malinin and Mark Gales. 2018. Predictive Uncertainty Estimation via Prior Networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montréal, Canada) ( NIPS'18). Curran Associates Inc., Red Hook, NY, USA, 7047?7058.Google Scholar
- Vikash Mansinghka and Eric Jonas. 2014. Building fast Bayesian computing machines out of intentionally stochastic, digital parts. arXiv preprint ( 2014 ). arXiv: 1402. 4914Google Scholar
- Luca Martino, Víctor Elvira, David Luengo, Jukka Corander, and Francisco Louzada. 2016. Orthogonal parallel MCMC methods for sampling and optimization. Digital Signal Processing 58 ( 2016 ), 64?84. https://doi.org/10.1016/ j.dsp. 2016. 07.013 Google Scholar
- Mayler Martins, Jody Maick Matos, Renato P. Ribas, André Reis, Guilherme Schlinker, Lucio Rech, and Jens Michelsen. 2015. Open Cell Library in 15Nm FreePDK Technology. In Proceedings of the 2015 Symposium on International Symposium on Physical Design (Monterey, California, USA) ( ISPD '15). ACM, New York, NY, USA, 171?178. https://doi.org/10.1145/2717764.2717783 Google Scholar
- Patrick McClure, Nao Rho, John A. Lee, Jakub R. Kaczmarzyk, Charles Y. Zheng, Satrajit S. Ghosh, Dylan M. Nielson, Adam G. Thomas, Peter Bandettini, and Francisco Pereira. 2019. Knowing What You Know in Brain Segmentation Using Bayesian Deep Neural Networks. Frontiers in Neuroinformatics 13 ( 2019 ), 67. https://doi.org/10.3389/fninf. 2019.00067 Google Scholar
- Grigorios Mingas, Leonardo Bottolo, and Christos-Savvas Bouganis. 2017. Particle MCMC algorithms and architectures for accelerating inference in state-space models. International Journal of Approximate Reasoning 83 ( 2017 ), 413?433. https://doi.org/10.1016/j.ijar. 2016. 10.011 Google Scholar
- Radford M. Neal. 2011. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo 2, 11 ( 2011 ), 2. arXiv: 1206.1901Google Scholar
- Jongse Park, Emmanuel Amaro, Divya Mahajan, Bradley Thwaites, and Hadi Esmaeilzadeh. 2016. AxGames: Towards Crowdsourcing Quality Target Determination in Approximate Computing. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems (Atlanta, Georgia, USA) ( ASPLOS '16). Association for Computing Machinery, New York, NY, USA, 623?636. https://doi.org/10.1145/2872362.2872376 Google Scholar
- Georg Ch Pflug. 1992. Gradient estimates for the performance of Markov chains and discrete event processes. Annals of Operations Research 39, 1 ( 1992 ), 173?194. https://doi.org/10.1007/BF02060941 Google Scholar
- Janis Postels, Francesco Ferroni, Huseyin Coskun, Nassir Navab, and Federico Tombari. 2019. Sampling-free epistemic uncertainty estimation using approximated variance propagation. In Proceedings of the IEEE International Conference on Computer Vision. 2931?2940. https://doi.org/10.1109/ICCV. 2019.00302 Google Scholar
- Christian Robert and George Casella. 2013. Monte Carlo statistical methods. Springer Science & Business Media. https://doi.org/10.1007/978-1-4757-4145-2 Google Scholar
- Bita Rouhani, Daniel Lo, Ritchie Zhao, Ming Liu, Jeremy Fowers, Kalin Ovtcharov, Anna Vinogradsky, Sarah Massengill, Lita Yang, Ray Bittner, Alessandro Forin, Haishan Zhu, Taesik Na, Prerak Patel, Shuai Che, Lok Chand Koppaka, Xia Song, Subhojit Som, Kaustav Das, Saurabh Tiwary, Steve Reinhardt, Sitaram Lanka, Eric Chung, and Doug Burger. 2020. Pushing the Limits of Narrow Precision Inferencing at Cloud Scale with Microsoft Floating Point. In NeurIPS 2020. ACM. https://www.microsoft.com/en-us/research/publication/pushing-the-limits-ofnarrow-precision-inferencing-at-cloud-scale-with-microsoft-floating-point/Google Scholar
- Feras A. Saad, Cameron E. Freer, Nathanael L. Ackerman, and Vikash K. Mansinghka. 2019. A Family of Exact Goodness-of-Fit Tests for High-Dimensional Discrete Distributions. In The 22nd International Conference on Artificial Intelligence and Statistics. 1640?1649. http://proceedings.mlr.press/v89/saad19a.htmlGoogle Scholar
- Charbel Sakr, Yongjune Kim, and Naresh Shanbhag. 2017. Analytical Guarantees on Numerical Precision of Deep Neural Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70 ). PMLR, International Convention Centre, Sydney, Australia, 3007?3016. http://proceedings.mlr.press/v70/sakr17a.htmlGoogle Scholar
- Daniel Scharstein and Richard Szeliski. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International journal of computer vision 47, 1-3 ( 2002 ), 7?42. https://doi.org/10.1023/A:1014573219977 Google Scholar
- Friederike Schmid and Nigel B. Wilding. 1996. Errors in Monte Carlo simulations using shift register random number generators. International Journal of Modern Physics C 6, 06 ( 1996 ), 781?787. https://doi.org/10.1142/S0129183195000642 Google Scholar
- Claude E. Shannon. 1948. A mathematical theory of communication. The Bell system technical journal 27, 3 ( 1948 ), 379?423. https://doi.org/10.1002/j.1538-7305. 1948.tb01338.x Google Scholar
- Priyesh Shukla, Ahish Shylendra, Theja Tulabandhula, and Amit R. Trivedi. 2020. MC2RAM: Markov Chain Monte Carlo Sampling in SRAM for Fast Bayesian Inference. In 2020 IEEE International Symposium on Circuits and Systems (ISCAS). 1?5. https://doi.org/10.1109/ISCAS45731. 2020.9180701 Google Scholar
- David B. Thomas and Wayne Luk. 2009. Using FPGA resources for direct generation of multivariate gaussian random numbers. In Field-Programmable Technology, 2009. FPT 2009. International Conference on. IEEE, 344?347. https: //doi.org/10.1109/FPT. 2009.5377680 Google Scholar
- Madeleine B. Thompson. 2010. A comparison of methods for computing autocorrelation time. arXiv preprint ( 2010 ). arXiv: 1011. 0175Google Scholar
- Tijmen Tieleman. 2008. Training restricted Boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference on Machine learning. 1064?1071. https://doi.org/10.1145/1390156.1390290 Google Scholar
- Seth D. Tribble. 2007. Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences. Ph.D. Dissertation. Stanford University.Google Scholar
- Kush R. Varshney and Homa Alemzadeh. 2017. On the safety of machine learning: Cyber-physical systems, decision sciences, and data products. Big data 5, 3 ( 2017 ), 246?255. https://doi.org/10.1089/big. 2016.0051 Google Scholar
- Dootika Vats, James M. Flegal, and Galin L. Jones. 2015. Multivariate Output Analysis for Markov chain Monte Carlo. arXiv preprint ( 2015 ). arXiv: 1512. 07713Google Scholar
- Dootika Vats and Christina Knudson. 2018. Revisiting the Gelman-Rubin Diagnostic. arXiv preprint ( 2018 ). arXiv: 1812.09384Google Scholar
- Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao, and Jonah Gabry. 2015. Pareto smoothed importance sampling. arXiv preprint ( 2015 ). arXiv: 1507. 02646Google Scholar
- Shibo Wang and Pankaj Kanwar. 2019. BFloat16: the secret to high performance on cloud TPUs. Google Cloud Blog ( 2019 ). https: //cloud.google.com/blog/products/ai-machine-learning/bfloat16-the-secret-tohigh-performance-on-cloud-tpusGoogle Scholar
- Siyang Wang, Xiangyu Zhang, Yuxuan Li, Ramin Bashizade, Song Yang, Chris Dwyer, and Alvin R. Lebeck. 2016. Accelerating Markov Random Field Inference Using Molecular Optical Gibbs Sampling Units. In Proceedings of the 43rd International Symposium on Computer Architecture (Seoul, Republic of Korea) (ISCA '16). IEEE Press, Piscataway, NJ, USA, 558?569. https://doi.org/10.1109/ISCA. 2016.55 Google Scholar
- Yu Emma Wang, Yuhao Zhu, Glenn G. Ko, Brandon Reagen, Gu-Yeon Wei, and David Brooks. 2019. Demystifying Bayesian Inference Workloads. In 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 177?189. https://doi.org/10.1109/ISPASS. 2019.00031 Google Scholar
- Max Welling and Yee Whye Teh. 2011. Bayesian Learning via Stochastic Gradient Langevin Dynamics. In Proceedings of the 28th International Conference on International Conference on Machine Learning (Bellevue, Washington, USA) ( ICML'11). Omnipress, Madison, WI, USA, 681?688.Google Scholar
- Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman. 2018. Yes, but Did It Work?: Evaluating Variational Inference. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80 ), Jennifer Dy and Andreas Krause (Eds.). PMLR, Stockholmsmässan, Stockholm Sweden, 5581?5590. http://proceedings.mlr.press/v80/yao18a.htmlGoogle Scholar
- Xiangyu Zhang, Ramin Bashizade, Craig LaBoda, Chris Dwyer, and Alvin R. Lebeck. 2018. Architecting a stochastic computing unit with molecular optical devices. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA). IEEE, 301?314. https://doi.org/10.1109/ISCA. 2018.00034 Google Scholar
- Stephanie Zierke and Jason D. Bakos. 2010. FPGA acceleration of the phylogenetic likelihood function for Bayesian MCMC inference methods. BMC bioinformatics 11, 1 ( 2010 ), 1?12. https://doi.org/10.1186/ 1471-2105-11-184 Google Scholar
Index Terms
- Statistical robustness of Markov chain Monte Carlo accelerators
Recommendations
Analyzing Markov chain Monte Carlo output
AbstractMarkov chain Monte Carlo (MCMC) is a sampling‐based method for estimating features of probability distributions. MCMC methods produce a serially correlated, yet representative, sample from the desired distribution. As such it can be difficult to ...
Visual representation of the multivariate correlation structure in a Markov chain. Output analysis that accounts for this multivariate structure is able to more accurately summarize variability in the simulation process. We review the output analysis ...
Optimal Markov chain Monte Carlo sampling
This article is a review article on the optimal Markov chain Monte Carlo MCMC sampling. The focus is on homogeneous Markov chains. This article first reviews the problem of finding the optimal transition matrix, which is defined to minimize the ...
Segmentation Based on Improved Markov Chain Monte Carlo
ICECC '12: Proceedings of the 2012 International Conference on Electronics, Communications and ControlTo overcome the drawback of Simulated Annealing (SA), a novel algorithm Using Population based Markov chain Monte Carlo (Pop-MCMC) is proposed for segmentation. It takes less time from the initial state to the state that chains are coupling with Pop-...
Comments