Perceptrones

Otros algoritmos de entrenamiento para redes neuronales simples

Adaline: Regla LMS (mínimos cuadrados)

• Widrow, B. & Hoff, M. E. (1960): "Adaptive switching circuits." Record of IRE Eastern Electronic Show & Convention (WESCON), Vol. 4, pp. 96–104. URL http://www-isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf
• Widrow, B. & Lehr, M. A. (1990): "30 years of adaptive neural networks: Perceptron, Madaline, and backpropagation." Proceedings of the IEEE, 78(9):1415–1442. DOI https://doi.org/10.1109/5.58323

Reglas de Ho-Kashyap

• Ho, Y. C. & Kashyap, R. L. (1965): "An algorithm for linear inequalities and its applications." IEEE Transactions of Electronic Computers, 14:683–688. DOI https://doi.org/10.1109/PGEC.1965.264207
• Hassoun, M. H. & Song, J. (1992): "Adaptive Ho-Kashyap rules for perceptron training." IEEE Transactions on Neural Networks, 3(1):51–61. DOI https://doi.org/10.1109/72.105417

Thermal perceptron (enfriamiento simulado)

• Frean, M. (1992): "A thermal perceptron learning rule." Neural Computation, 4(6):946–957. DOI http://dx.doi.org/10.1162/neco.1992.4.6.946

P-Delta Rule: Regla delta paralela (aproximador universal)

• Auer, P., Burgsteiner, H., & Maass, W. (2008): "A learning rule for very simple universal approximators consisting of a single layer of perceptrons." Neural Networks, 21, 786–795. DOI https://doi.org/10.1016/j.neunet.2007.12.036
• Fernandez-Delgado, M., Ribeiro, J., Cernadas, E. & Ameneiro, S. B. (2011): "Direct parallel perceptrons (DPPs): Fast analytical calculation of the parallel perceptrons weights with margin control for classification tasks." IEEE Transactions on Neural Networks, 22(11), 1837–1848. DOI https://doi.org/10.1109/TNN.2011.2169086

Optimización convexa

• Castillo, E., Fontenla-Romero, O., Alonso-Betanzos, A. & Guijarro-Berdinas, B. (2002): "A global optimum approach for one-layer neural networks." Neural Computation, 14(6):1429–1449. DOI https://doi.org/10.1162/089976602753713007
• Fontenla-Romero, O., Guijarro-Berdinas, B., Perez-Sanchez, B. & Alonso-Betanzos, A. (2010): "A new convex objective function for the supervised learning of single-layer neural networks." Pattern Recognition, 43(5), 1984–1992. DOI https://doi.org/10.1016/j.patcog.2009.11.024

Gradiente descendente con restricciones [constrained steepest descent]

• Perantonis, S. J. & Virvilis, V. (2000): "Efficient perceptron learning using constrained steepest descent." Neural Networks, 13(3):351–364. DOI https://doi.org/10.1016/S0893-6080(00)00016-2

Optimización no-suave [non-smooth optimization]

• Eitzinger, C. & Plach, H. (2003): "A new approach to perceptron training." IEEE Transactions on Neural Networks, 14(1):216–221. DOI https://doi.org/10.1109/TNN.2002.806631

Gradientes conjugados

• Nagaraja, G. & Bose, R. P. J. C. (2006): "Adaptive conjugate gradient algorithm for perceptron training." Neurocomputing, 69, 368–386. DOI https://doi.org/10.1016/j.neucom.2005.03.007
• Diene, O. & Bhaya, A. (2009): "Perceptron training algorithms designed using discrete-time control Liapunov functions." Neurocomputing, 72, 3131–3137. DOI https://doi.org/10.1016/j.neucom.2009.03.007

Más variantes del perceptrón

Spiking perceptron (biológicamente plausible)

• Rowcliffe, P., Feng, J. & Buxton, H. (2006): "Spiking perceptrons." IEEE Transactions on Neural Networks, 17(3):803–807. DOI https://doi.org/10.1109/TNN.2006.873274

Sign-constrained perceptron (¿biológicamente plausible?)

• Amit, D. J.,Wong, K. Y. M. & Campbell, C. (1989): "Perceptron learning with sign-constrained weights." Journal of Physics A: Mathematical and General, 22, 2039–2045. DOI https://doi.org/10.1088/0305-4470/22/12/009
• Legenstein, R. & Maass, W. (2008): "On the classification capability of sign-constrained perceptrons." Neural Computation, 20, 288–309. DOI http://dx.doi.org/10.1162/neco.2008.20.1.288

Shifted perceptron

• Cesa-Bianchi, N. & Gentile, C. (2006): "Tracking the best hyperplane with a simple budget perceptron." In COLT'06 Proceedings of the 19th Annual Conference on Learning Theory, pp. 483-498. Pittsburgh, PA, June 22 - 25, 2006. DOI https://doi.org/10.1007/11776420_36
• Cavallanti, G., Cesa-Bianchi, N. & Gentile, C. (2007): "Tracking the best hyperplane with a simple budget perceptron." Machine Learning, 69, 143–167. DOI https://doi.org/10.1007/s10994-007-5003-0

ROMMA

• Li, Y. & Long, P. (2002): "The relaxed online maximum margin algorithm." Machine Learning, 46, 361–387. DOI https://doi.org/10.1023/A:1012435301888

NORMA

• Kivinen, J., Smola, A. J., & Williamson, R. C. (2004): "Online learning with kernels." IEEE Transactions on Signal Processing, 52(8), 2165–2176. DOI https://doi.org/10.1109/TSP.2004.830991

Perceptrón pasivo-agresivo

• Crammer, K., Dekel, O., Shalev-Shwartz, S. & Singer, Y. (2006): "Online passive aggressive algorithms." Journal of Machine Learning Research, 7, 551–585. URL http://jmlr.csail.mit.edu/papers/volume7/crammer06a/crammer06a.pdf

Ballseptron

• Shalev-Shwartz, S. & Singer, Y. (2005): "A new perspective on an old perceptron algorithm." In COLT'05 Proceedings of the 16th Annual Conference on Computational Learning Theory, pp. 264–278. DOI https://doi.org/10.1007/11503415_18

Perceptrones tolerantes al ruido

• Khardon, R. & Wachman, G. (2007): "Noise tolerant variants of the perceptron algorithm." Journal of Machine Learning Research, 8, 227–248. URL http://www.jmlr.org/papers/volume8/khardon07a/khardon07a.pdf

Más sobre propiedades y limitaciones del perceptrón

• Muselli, M. (1997): "On convergence properties of pocket algorithm." IEEE Transactions on Neural Networks, 8(3):623–629. DOI https://doi.org/10.1109/72.572101
• Ho, C. Y.-F., Ling, B. W.-K., Lam, H.-K. & Nasir, M. H. U. (2008): "Global convergence and limit cycle behavior of weights of perceptron." IEEE Transactions on Neural Networks, 19(6):938–947. DOI https://doi.org/10.1109/TNN.2007.914187