Reconocimiento de patrones en FPGA para aplicaciones aeroespaciales

Autores/as

DOI:

https://doi.org/10.33448/rsd-v10i12.19181

Palabras clave:

Satélites inteligentes; Nanosatélites; Inteligencia artificial en hardware; Visión artificial; Aprendizaje automática; Morfología matemática; Reconocimiento de patrones; Inteligencia artificial en tiempo real; Aplicaciones aeroespaciales

Resumen

El presente trabajo presenta una técnica de reconocimiento de patrones en tiempo real basada en Morfología Matemática-MM implementada en FPGA (Field Programmable Gate Array). La estrategia para la efectividad de este enfoque tiene que ver con las ventajas del paradigma de aprendizaje automática aplicada al modelo de correspondencia con la invariancia traslacional de operadores elementales da MM. El artículo muestra que las composiciones de operadores elementales simples de morfología matemática basadas en ELUT (tablas de consulta elementales) son adecuadas para integrarse en dispositivos FPGA. Este artículo también muestra técnicas de desarrollo de sistemas de reconocimiento de patrones, desde el modelado matemático de operadores morfológicos hasta la implementación del dispositivo electrónico utilizando el software System Generator. En general, las operaciones para el procesamiento de imágenes en FPGAs se implementan a un bajo nivel de abstracción de los lenguajes de descripción del hardware-HDL. Esto crea una gran complejidad en la implementación de operaciones en imágenes a nivel de píxeles. Sin embargo, este trabajo presenta un dispositivo reconfigurable de reconocimiento de patrones implementado directamente en FPGA a partir de simulación de modelado matemático en el software Matlab/Simulink/System Generator. Esta estrategia ha reducido la complejidad del desarrollo de hardware. El dispositivo será útil principalmente cuando se aplique en tareas de teledetección para misiones aeroespaciales utilizando sensores pasivos o activos.

Biografía del autor/a

Francisco de Assis Tavares Ferreira da Silva, Instituto Nacional de Pesquisas Espaciais

Francisco A. Tavares F. da Silva received the B.Sc. in electrical and electronic engineering from Federal University of Campina Grande, Campina Grande-RN, Brazil, in 1986, the M.Sc. in electronic and computer engineering from Aeronautics Institute of Technology, São José dos Campos-SP, Brazil, in 1993 and the D.Sc. degree from National Institute for Space Research (INPE) São José dos Campos-SP, Brazil, in 1998. Since 1986 he has worked at INPE and currently conducts research in digital signal processing applied to pattern recognition and telecommunications.

Magno Prudêncio de Almeida Filho, Federal University of Ceara

Magno Prudêncio de Almeida Filho received the B.Sc. degree in telecommunication engineering from the University of Fortaleza (UNIFOR), Fortaleza-CE, Brazil, in 2008. M.Sc. degree in electrical engineering from Federal University of Ceará (UFC), Fortaleza-CE, Brazil, in 2016 and D.Sc. degree in electrical engineering from Federal University of Ceará (UFC), Fortaleza-CE, Brazil in 2020. From 2011 to 2015, through a partnership between the National Counsel of Technological and Scientific Development (CNPq) and the National Institute for Space Research (INPE) he performed research at INPE in digital communications, signal processing for satellite communication, digital image processing, pattern recognition and RADAR signal processing. His main research interest include communications theory, space communications, digital signal processing, digital image processing, artificial neural networks, FPGA, embedded systems, control theory, model-based controllers and control of dead-time systems.

Antonio Macilio Pereira de Lucena, National Institute for Space Research

Antonio Macilio Pereira de Lucena received the B.Sc. degree in electronics engineering from Technological Institute of Aeronautics (ITA), São José dos Campos-SP, Brazil, in 1980, the M.Sc. degree in space telecommunications and electronics from National Institute for Space Research (INPE), São José dos Campos-SP, Brazil, in 1986, and the D.Sc. degree in teleinformatics engineering from Federal University of Ceara (UFC), Fortaleza-CE, Brazil, in 2006.
He is with INPE since 1983 where he has been involved in various projects in the areas of satellite communications, electronics, and radio-astronomy. Since 2007, he is also professor at University of Fortaleza (UNIFOR), Fortaleza-CE, Brazil. His present research interests include modulations, space communications, signal processing, and communication theory.

Alexandre Guirland Nowosad, National Institute for Space Research

Alexandre Guirland Nowosad received the B. Sc. degree in electronics engineering from Federal University of Rio de Janeiro (UFRJ), Rio de Janeiro-RJ, Brazi, in 1987, the M.Sc. degree in Electrical Engineering from NYU, USA, in 1988, and the D.Sc. degree from National Institute for Space Research (INPE) São José dos Campos-SP, Brazil, in 2001.
Since 1988 he has worked at INPE in signal processing applications in meteorology, environmental science and space communications.

Citas

Akil, M., & Zahirazami, S. (1998). Multi-FPGA Processor for Gray Scale Morphology, European Association for Signal Processing, Eusipco-98, IX European Signal Processing Conference, Island of Rhodes, Greece, 8-11 September.

Arechiga, A. P., Michaels, A. J., & Black, J. T. (2018). "Onboard Image Processing for Small Satellites," NAECON 2018 - IEEE National Aerospace and Electronics Conference, 234-240, 10.1109/NAECON.2018.8556744.

Astua, C., Crespo, J., & Barber, R. (2014). Detecting Objects for Indoor Monitoring and Surveillance for Mobile Robots, Emerging Security Technologies (EST), in Fifth International Conference, 124-129.

Ban, Y., Zhang, P., Nascetti, A., Bevington, A. R., & Wulder M. A. (2020). Near Real-Time Wildfire Progression Monitoring with Sentinel-1 SAR Time Series and Deep Learning, Nature Scientific. 10(1322).

Banon, G. J. F., (1995). Characterization of translation invariant elementary morphological operators between gray-level images, Research report, INPE-5616-RPQ/671, INPE.

Banon, G. J. F., & Barrera, J. (1998). Bases da morfologia matemática para análise de imagens binárias, (2nd ed.), MCT/INPE, INPE-6779-RPQ/682.

Banon, G. J. F. (2000). Formal introduction to digital image processing. Technical Report: INPE-7682-PUD/43, São José dos Campos, Brasil., 2000. dpi.inpe.br/ banon/2001/01.11.16.04.

Banon, G. J. F., & Faria, S. D. (1997). Morphological approach for template matching, SIBGRAPI'97, IEEE Computer Society, 171-178.

Bekker, D. L., Werne, T. A., Wilson, T. O., Pingree, P.J., Dontchev, K., Heywood, M., Ramos, R., Freyberg, B., Saca, F., Gilchrist, B., Gallimore, A., & Cutler, J. (2010). A CubeSat design to validate the Virtex5 FPGA for spaceborne image processing, Aerospace Conference, 2010 IEEE, pp.1-9.

Belmonte L. M, Morales R., & Fernández-Caballero A. (2019). Computer Vision in Autonomous Unmanned Aerial Vehicles—A Systematic Mapping Study. Applied Sciences. 2019, 9(15):3196. https://doi.org/10.3390/app9153196

Beul, M., Krombach, N., Zhong, Y., Droeschel, D., Nieuwenhuisen, M., & Behnke, S. (2015). A high-performance MAV for autonomous navigation in complex 3D environments. In Unmanned Aircraft Systems (ICUAS), International Conference on, Denver, USA, 1241-1250. IEEE.

Blake, G., Deslinski, R.G., & Mudge, T. (2009). A survey of multicore processors: A review of their common attributes, IEEE Signal Processing Magazine, vol. 26, no. 6, pp. 26–37.

Brugger, C., Dal'Aqua, L., Varela, J., De Schryver, C., When, N., Klein, M., & Siegrist, M. (2015). A Quantitative Cross-Architecture Study of Morphological Image Processing on CPUs, GPUs, and FPGAs. In Proceedings of the IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), April 2015.

Castelli, T., Sharghi, A., Harper, D., Tremeau, A., & Shah, M. (2016). Autonomous navigation for low-altitude UAVs in urban areas. arXiv preprint arXiv:1602.08141.

Chien, S., Sherwood, R., Tran, D., Cichy, B., Rabideau, G., Castano, R., Davies, A., Lee, R., M, D., Frye, S. Trout, B. Hengemihle, J., Shulman, S., Ungar, S., & Brakke, T. (2004). The EO-1 autonomous science agent.

Chien, S., Bue, B., Castillo-Rogez, J., Gharibian, D., Knight, R., Schaffer, S., Thompson, D., & Wagstaff, K. (2014). Agile Science: Using Onboard Autonomy for Primitive Bodies and Deep Space Exploration.

Chowdhary, G., Johnson, E. N., Magree, D., Wu, A., & Shein, A. (2013). GPS‐denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft. Journal of Field Robotics, 30(3), 415-438.

D'AMORE, R. (2005). VHDL: Descrição e Síntese de Circuitos Digitais. Editora LTC, 276 p.

Dawood, A., Visser, S., & Williams, J. (2002). Reconfigurable FPGAs for real time image processing in space. In Digital Signal Processing. IEEE 14th International Conference on, 2. 845–848.

Dinelli, G., Meoni, G., Rapuano, E., Pacini, T., & Fanucci, L. (2020). MEM-OPT: A Scheduling and Data Re-use System to Optimize On-chip Memory Usage for CNNs On-board FPGAs. IEEE J. Emerg. Sel. Top. Circuits Syst. 2020, 10, 335–347.

Downton, A., & Crookes, D. (1998). Parallel Architectures for Image Processing. Electronics & Communication Engineering Journal, 10, 139-151.

Felipe, I., Dohm, J. M., Baker, V. R., Doggett, T., Davies, A. G., Castano, R., Chien, S., Cichy, B., Greeley, R., Sherwood, R., Tran, D., & Rabideau, G. (2006). Flood detection and monitoring with the Autonomous Sciencecraft Experiment onboard EO-1: Remote Sensing of Environment, 101(4), 463-481.

Filho, A., M. P., Silva, F. A. T. F., & Braga, A. P. S. (2014). Machine learning and adaptive morphological operators. In: XI Encontro Nacional de Inteligência Artificial e Computacional (ENIAC 2014). São Carlos, Brasil. Annals of XI Encontro Nacional de Inteligência Artificial e Computacional.

Filho, A., M. P., Silva, F. A. T. F., & Braga, A. P. S. (2015). Operador morfológico adaptativo de casamento de padrões – proposta e aplicação na análise de imagens de satélite. In: Simpósio Brasileiro de Automação Inteligente (SBAI 2015). Natal, Brasil.

Franchi, G., Fehri, A., & Yao, A. (2020). Deep morphological networks, Pattern Recognition, Volume 102.

Fuchs, T., Bue, B., Rogez, J. C., Wagstaff, K., & Thompson, D. (2014). Autonomous Onboard Surface Feature Detection for Flyby Missions, International Symposium on Artificial Intelligence, Robotics and Automation in Space (iSAIRAS), June 2014.

Furano, G., Meoni, G., Dunne, A., Moloney, D., Ferlet-Cavrois, V., Tavoularis, A., Byrne, J., Buckley, L., Psarakis, M., Voss, K. O., et al. (2020). Towards the Use of Artificial Intelligence on the Edge in Space Systems: Challenges and Opportunities. IEEE Aerosp. Electron. Syst. Mag. 2020, 35, 44–56.

Giuffrida, G., Diana, L., de Gioia, F., Benelli, G., Meoni, G., Donati, M., & Fanucci, L. (2020). CloudScout: A Deep Neural Network for On-Board Cloud Detection on Hyperspectral Images. Remote. Sens. 2020, 12, 2205.

Guo, K., Sui, L., Qiu, J., Yu, J., Wang, J., Yao, S., Han, S., Wang, Y., & Yang, H. (2018). Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 2018, 37, 35–47

Hagiwara, H., Asami, K., & Komori, M. (2011). FPGA Implementation of Image Processing for Real-time Robot Vision System. Proc. of International Conference on Convergence and Hybrid Information Technology, pp.134-141, Daejeon, Korea, September.

Hao, R., Wang, X., Zhang, Liu, J. J., Du, X., & Liu, L. (2019). Automatic Detection of Fungi in Microscopic Leucorrhea Images Based on Convolutional Neural Network and Morphological Method. IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) (2019): 2491-2494.

Heijmans, H. J. A. M. (1991). Theoretical Aspects of Gray-Level Morphology. IEEE Transactions on Pattern Analysis and Machine Intelligence, v.13, pp. 568-582.

Hentati, R., Hentati, M., Aoudni, Y., & Abid, M. (2014). The implementation of basic morphological operations on FPGA using partial reconfiguration. IEEE IPAS'14: INTERNATIONAL IMAGE PROCESSING APPLICATIONS AND SYSTEMS CONFERENCE 2014.

Johnston, C. T., Gribbon, K. T., & Bailey, D. G. (2004). Implementing Image Processing Algorithms on FPGAs. In: Proc. Electronics New Zealand Conference, Palmeston North, New Zealand, pp. 118-123.

Jouni, M., Mura, M. D., & Comon, P. (2020). Hyperspectral Image Classification Based on Mathematical Morphology and Tensor Decomposition, Mathematical Morphology - Theory and Applications, 4(1), 1-30.

Kalomiros, J. A., & Lygouras, J. (2008). Design and evaluation of a hardware/software FPGA based system for fast image processing, Microprocessors and Microsystems, pp. 95-106.

Khosravi, M., & Schafer, R. W. (1996). Template matching based on a grayscale hit-or-miss transform. Image Processing, IEEE Transactions on, 5(6), pp. 1060-1066.

Lange, S., Sünderhauf, N., & Protzel, P. (2008). Autonomous Landing for a Multirotor UAV Using Vision. In Workshop Proceedings of SIMPAR 2008 Intl. Conf. on Simulation, Modeling and Programming for Autonomous Robts, Venice, Italy, 3-4, 482-491.

Matlab-Simulink, (2015). http://www.mathworks.com.

Mellouli, D., Hamdani, T. M., Sanchez-Medina, J. J., Ayed, M. B., & Alimi, A. (2019). Morphological Convolutional Neural Network Architecture for Digit Recognition. IEEE Transactions on Neural Networks and Learning Systems, 30, 2876-2885.

Nagel, G. W., de Novo, E. M. L. M., & Kampel, M. (2020). Nanosatellites applied to optical Earth observation: A review.Rev. Ambient. Água 2020, 15.

Nogueira, K. J. Chanussot, Mura, M. D., & Santos, J. A. D. (2021). "An Introduction to Deep Morphological Networks," in IEEE Access, 9, 114308-114324, 2021, 10.1109/ACCESS.2021.3104405.

Pell, O., Mencer, O., Tsoi, K.H., & Luk, W. (2013). Maximum performance computing with dataflow engines, in High-Performance Computing Using FPGAs, ed: Springer, pp. 747-774.

Ramos, R., A., Sampedro, C., Carrio, A., Bavle, H., Fernández, R. A. S., Miloševic, Z., & Campoy, P. (2016). A monocular pose estimation strategy for uav autonomous navigation in gnss-denied environments. In International micro air vechicle competition and conference 2016, Beijing, PR of China, pp. 22-27.

Rapuano, E., Meoni, G., Pacini, T., Dinelli, G., Furano, G., Giuffrida, G., & Fanucci, L. (2021). An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study. Remote Sens. 2021, 13, 1518. https://doi.org/10.3390/rs13081518.

Rempel, E. L., & Silva, F. A. T. F. (2001). Reconhecimento de Padrões Invariante a Rotação Utilizando uma Rede Neural Morfológica Não Supervisionada. In: V Brazilian Conference on Neural Networks, 2001, Rio de Janeiro. Proceedings of V Brazilian Conference on Neural Networks, 2001. 109-112.

Shen, Y., Zhong, X., & Shih, F. (2019). Deep Morphological Neural Networks. ArXiv, abs/1909.01532.

Silva, F. A.T. F. (1998). Rede Morfológica Não Supervisionada-RMNS. Thesis (Graduate Program in Applied Computing-CAP of the National Institute for Space Research-INPE), INPE-8759-TDI/800, Instituto Nacional de Pesquisas Espaciais-INPE, São José dos Campos, 1998. http://gjfb0520.sid.inpe.br/c ol/dpi.inpe.br/banon/1998/07.30.19.40/doc/tese.pdf

Silva, F. A. T. F. & Banon, G. J. F. (1999). Rede Morfológica Não Supervisionada., Proceedings of the IV Brazilian Conference on neural Networks. ITA- São José dos Campos, pp. 400-405.

Silva, F. A. T. F., & Silva, L. A. (2004). Detecção de múltiplos padrões usando redes neurais morfológicas. Anais do Congresso Brasileiro de Automática, A8-Sistemas Inteligentes, S0801, 1-6, RS, Brasil.

Silva, F. A. T. F., & Lucena, A. M. P. (2005a). Satélites inteligentes aplicados ao monitoramento ambiental em tempo real. 1-6. 10.21528/CBRN2005-068.

Silva, F. A. T. F., & Lucena, A. M. P. (2005b). Processamento Inteligente de Sinais aplicado ao Monitoramento Ambiental em Tempo Real. Anais XII Simpósio Brasileiro de Sensoriamento Remoto, Goiânia, Brasil, 16-21 abril 2005, INPE,. 3341-3348.

Silva, F. A. T. F. (2005). Neurônios morfológicos: Uma introdução às células on-off artificiais. Anais do VII CONGRESSO BRASILEIRO DE REDES NEURAIS – CBRN, São José dos Campos, Brasil, 2005, 1-6. https://sbic.org.br/eventos/cbrn_2005/CBRN2005_041/

Silva, F. A. T. F. (2006). Operadores morfológicos adaptativos. In: Congresso de Matemática e suas Aplicações, Foz do Iguaçu, Anais do Congresso de Matemática e suas Aplicações, pp.1-2.

Silva, F. A. T. F., Filho, P. M., Moreira, N. A., Rios, C. S. N., Oliveira, P. D. L., Camurca, P. J., & Lucena, A. M. P. (2015). Modelagem matemátia em microeletrônica reconfigurável: Estudo de caso sobre modulares BPSK. Relatório de Pesquisa INPE, sid.inpe.br/mtc-m21b/2015/05.28.17.26-RPQ, 2015. http://mtc-m21b.sid.inpe.br/col/sid.inpe.br/mtc-m21b/2015/05.28.17.26/doc/publicacao.pdf?languagebutton=pt-BR

Souza, O., Cortez, P. C., & Silva, F. A. T. F. (2012). Grayscale images and RGB video: compression by morphological neural network. In: Artificial Neural Networks in Pattern Recognition. Springer Berlin Heidelberg, 213-224.

Souza, O., Cortez, P. C., & Silva, F. A. T. F. (2013). Artificial Neural Networks for Compression of Gray Scale Images: A Benchmark. In: X Encontro Nacional de Inteligência Artificial e Computacional (ENIAC 2013). Fortaleza, Brasil. Anais of X Encontro Nacional de Inteligência Artificial e Computacional.

Thompson, D. R, Rogez, J. C., Chien, S., Doyle, R., Estlin, T., & McLaren, D. (2012). Agile science operations: A new approach for primitive bodies exploration. In Proceedings of SpaceOps, Stockholm, Sweden, June 2012.

Zakerhaghighi, M. R., & Naji, H. R. (2013). Whale algorithm for image processing, a hardware implementation, Machine Vision and Image Processing (MVIP), in 8th Iranian Conference, 355-359.

Zhou, G., & Kafatos, M. (2002). Future intelligent Earth observing satellites. Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings.

Xilinx-RTL, (2011). Technology Schematic Viewers Tutorial, UG685, v13.1 (v2011).

Xilinx-SG, (2014). Vivado Design Suite: Model-Based DSP Design using System Generator, UG897 (v2014.1).

XILINX-KC705, (2014). Hardware User Guide. v.1.6. USA: XILINX, 2014. 110 p. <http://www.Xilinx.com/support/documentation/boards_a nd_ki ts/kc705/ug810_KC705_Eval_Bd.pdf>.

Xilinx-RTK, (2021). Radiation Tolerant Kintex UltraScale XQRKU060 FPGA. https://www.xilinx.com/support/documentation/data_sheets/ds8 82-xqr-kintex-ultrascale.pdf.

Descargas

Publicado

14/09/2021

Cómo citar

SILVA, F. de A. T. F. da .; ALMEIDA FILHO, M. P. de .; LUCENA, A. M. P. de; NOWOSAD, A. G. . Reconocimiento de patrones en FPGA para aplicaciones aeroespaciales. Research, Society and Development, [S. l.], v. 10, n. 12, p. e83101219181, 2021. DOI: 10.33448/rsd-v10i12.19181. Disponível em: https://rsdjournal.org/index.php/rsd/article/view/19181. Acesso em: 30 jun. 2024.

Número

Vol. 10 Núm. 12

Sección

Ingenierías

Licencia

Derechos de autor 2021 Francisco de Assis Tavares Ferreira da Silva; Magno Prudêncio de Almeida Filho; Antonio Macilio Pereira de Lucena; Alexandre Guirland Nowosad

Creative Commons License

Esta obra está bajo una licencia internacional Creative Commons Atribución 4.0.

Los autores que publican en esta revista concuerdan con los siguientes términos:

1) Los autores mantienen los derechos de autor y conceden a la revista el derecho de primera publicación, con el trabajo simultáneamente licenciado bajo la Licencia Creative Commons Attribution que permite el compartir el trabajo con reconocimiento de la autoría y publicación inicial en esta revista.

2) Los autores tienen autorización para asumir contratos adicionales por separado, para distribución no exclusiva de la versión del trabajo publicada en esta revista (por ejemplo, publicar en repositorio institucional o como capítulo de libro), con reconocimiento de autoría y publicación inicial en esta revista.

3) Los autores tienen permiso y son estimulados a publicar y distribuir su trabajo en línea (por ejemplo, en repositorios institucionales o en su página personal) a cualquier punto antes o durante el proceso editorial, ya que esto puede generar cambios productivos, así como aumentar el impacto y la cita del trabajo publicado.