Copyright © 2006 The Institute of Electronics, Information and Communication Engineers
Regular Section -- Papers -- Biocybernetics, Neurocomputing |
Naive Mean Field Approximation for Sourlas Error Correcting Code
1 The author is with Nara Women's University, Nara-shi, 630–8506 Japan. E-mail: takata{at}ics.nara-wu.ac.jp, 2 The author is with Yamaguchi University, Ube-shi, 755–8611 Japan. E-mail: shouno{at}yamaguchi-u.ac.jp, 3 The author is with The University of Tokyo, Kashiwa-shi, 277–8561 Japan. E-mail: okada{at}k.u-tokyo.ac.jp
Solving the error correcting code is an important goal with regard to communication theory. To reveal the error correcting code characteristics, several researchers have applied a statistical-mechanical approach to this problem. In our research, we have treated the error correcting code as a Bayes inference framework. Carrying out the inference in practice, we have applied the NMF (naive mean field) approximation to the MPM (maximizer of the posterior marginals) inference, which is a kind of Bayes inference. In the field of artificial neural networks, this approximation is used to reduce computational cost through the substitution of stochastic binary units with the deterministic continuous value units. However, few reports have quantitatively described the performance of this approximation. Therefore, we have analyzed the approximation performance from a theoretical viewpoint, and have compared our results with the computer simulation.
Key Words: naive mean field approximation, MPM inference, error correcting code, analog neural network
Manuscript received September 27, 2005. Manuscript revised January 23, 2006.
References
[1] N. Sourlas, "Spin glasses, error-correcting codes and finite-temperature decoding," Europhys. Lett., vol.25, no.3, pp.159–164, 1994.
[2] D.C. Mattis, "Solvable spin systems with random interactions," Phys. Lett. A, vol.56A, no.5, pp.421–422, April 1976.
[3] C.E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol.27, pp.379–423, 623–656, 1948.
[4] Y. Kabashima and D. Saad, "Belief propagation vs. TAP for decoding corrupted messages," EuroPhys. Lett., vol.44, no.5, pp.668–674, 1999.
[5] P. Ruján, "Finite temperature error-correcting codes," Phys. Rev. Lett., vol.70, no.19, pp.2968–2971, 1993.
[6] H. Nishimori and K.Y.M. Wong, "Statistical mechanics of image restoration and error-correcting codes," Phys. Rev. E., vol.60, no.1, pp.132–144, 1999.
[7] A.J. Bray, H. Sompolinsky, and C. Yu, "On the 'naive' mean-field equations for spin glasses," J. Phys. C, Solid State Phys., vol.19, pp.6389–6406, 1986.
[8] J.J. Hopfield and D.W. Tank, "Computing with neural circuits: A model," Science, vol.233, pp.625–633, 1986.
[9] H. Shouno, K. Wada, and M. Okada, "Naive mean field approximation for image restoration," J. Phys. Soc. Jpn., vol.71, no.10, pp.2406–2413, Oct. 2002.
[10] S. Kirkpatrick, C. Gelatt, and M. Vecchi, "Optimization by simulated annealing," Science, vol.220, no.4589, pp.671–680, 1983.
[11] D.J.C. Mackay, Information Theory, Inference and Learning Algorithm, Cambridge University Press, 2003.
[12] J. Inoue and K. Tanaka, "Dynamics of maximum marginal likelihood hyper-parameter estimation in image restoration: Gradient descent vs. EM algorithm," Phys. Rev. E, vol.65, no.1, 016125, 2002.
[13] J. Inoue and K. Tanaka, "Mean field theory of EM algorithm for Bayesian grey scale image restoration," J. Phys. A, Math. Gen., vol.36, pp.10997–11010, 2003.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||