SPIKING REPRESENTATIONS COMPARISON FOR LOCALIZATION AND NAVIGATION IN THE KEYFRAME MAP
Abstract
The task of navigating a mobile robotic platform in a known environment has been efficiently solved for a long time and using a flat passability map, which is built using lidar. Nevertheless, situations regularly arise when, for one reason or another, the platform is not equipped with lidar or other active navigation tools. At the same time, a camera is usually installed on the robotic platform, designed for visual monitoring of the situation by the operator, which can also be used for navigation when moving the robot in a known environment. There are well-known examples of navigation algorithms based on the use of sequences of keyframes, for example, visual SLAM. At the same time, various variants of video images (blurred, masked, etc.) are considered as keyframes. In this paper, a cognitive (non-metric, non-spatial) map of keyframes representing a spiking representation of the observed images is considered as a base for navigation. The possibility of using neuromorphic information control elements developed at the RTC to compare the current spiking representation with all spiking representations of a key sequence is analyzed. It is shown that by such a comparison, the keyframe closest to the current one can be determined, and parameters for the shift of spiking representations can also be selected, which is an analog of localization and navigation for a cognitive map. The description of a software tool for emulating the construction of a map and moving in it for experimental testing of the proposed algorithms is given. Data collection and experimental evaluation of the quality of localization and navigation algorithms have been performed. To do this, we have collected several keyframe maps with different patterns of movement between frames. When determining the position of the frame in the map, the quality was from 70 to 98%, when determining the direction of displacement between frames, the accuracy was from 94 to 97%. The results obtained are assessed as sufficient to solve the tasks assigned to the algorithm.
References
1. Fox D., Burgard W., Thrun S. The dynamic window approach to collision avoidance, IEEE Robotics & Automation Magazine, 1997, Vol. 4, No. 1, pp. 23-33.
2. Brock O., Khatib O. High-speed navigation using the global dynamic window approach, Proceedings 1999 ieee international conference on robotics and automation (Cat. No. 99CH36288C). IEEE, 1999, Vol. 1, pp. 341-346.
3. Pan S., Xie Z., Jiang Y. Sweeping robot based on laser SLAM, Procedia Computer Science, 2022,
Vol. 199, pp. 1205-1212.
4. Hicks II R.W., Hall E.L. Survey of robot lawn mowers, Intelligent Robots and Computer Vision XIX: Algorithms, Techniques, and Active Vision. SPIE, 2000, Vol. 4197, pp. 262-269.
5. Kiril'chenko A.A., Platonov A.K., Sokolov S.M. Teoreticheskie aspekty organizatsii interpretiruyushchey navigatsii mobil'nogo robota [Theoretical Aspects of mobile robot interpreting navigation], Preprint In-ta prikl. matem. im. MV Keldysha RAN [Preprint of M.V. Keldysch RSA institute of applied math.], 2002, No. 5, pp. 40.
6. Orchard G. et al. Converting static image datasets to spiking neuromorphic datasets using saccades, Frontiers in neuroscience, 2015, Vol. 9, pp. 437.
7. Fomin I., Korsakov A., Arkhipov A. Comparison of Key Frames of Video Sequences Using the Izhikevich Spiking Neuron, 2023 International Ural Conference on Electrical Power Engineering (UralCon). IEEE, 2023, pp. 594-598.
8. Ivanova V.V., Berkman D.A., Korsakov A.M. Metod umen'sheniya razmernosti prostranstva priznakov pri reshenii zadachi klassifikatsii [A method for reducing the dimension of the feature space when solv-ing a classification problem], Perspektivnye sistemy i zadachi upravleniya [Perspective systems and tasks of control], 2023, pp. 591-596.
9. Korsakov A.M., Stepanov D.N., Smirnova E.Yu. Algoritm opredeleniya orientatsii transportnogo sredstva na perekrestke po dannym sistemy tekhnicheskogo zreniya [An algorithm for determining the orientation of a vehicle at an intersection based on data from the vision system], Intellektual'nye sistemy, upravlenie i mekhatronika – 2016 [Intellectual systems, control and mechatronics-2016], 2016, pp. 396-399.
10. Ivanova V.V., Demcheva A.A., Korsakov A.M. Neyromorfnyy mekhanizm upravleniya zadaniem po rezul'tatam analiza situatsii [Neuromorphic mechanism of task control using situation analysis], Ekstrem-al'naya robototekhnika [Extreme robotics], 2024, Vol. 35, No. 1, pp. 308-3015.
11. Taketomi T., Uchiyama H., Ikeda S. Visual SLAM algorithms: A survey from 2010 to 2016, IPSJ transactions on computer vision and applications, 2017, Vol. 9, pp. 1-11.
12. Davison A.J. et al. MonoSLAM: Real-time single camera SLAM, IEEE transactions on pattern analysis and machine intelligence, 2007, Vol. 29, No. 6, pp. 1052-1067.
13. Klein G., Murray D. Parallel tracking and mapping for small AR workspaces, 2007 6th IEEE and ACM international symposium on mixed and augmented reality. IEEE, 2007, pp. 225-234.
14. Mur-Artal R., Montiel J.M.M., Tardos J.D. ORB-SLAM: a versatile and accurate monocular SLAM system, IEEE transactions on robotics, 2015, Vol. 31, No. 5, pp. 1147-1163.
15. Mur-Artal R., Tardós J.D. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras, IEEE transactions on robotics, 2017, Vol. 33, No. 5, pp. 1255-1262.
16. Rublee E. et al. ORB: An efficient alternative to SIFT or SURF, 2011 International conference on com-puter vision. Ieee, 2011, pp. 2564-2571.
17. Engel J., Schöps T., Cremers D. LSD-SLAM: Large-scale direct monocular SLAM, European confer-ence on computer vision. Cham: Springer International Publishing, 2014, pp. 834-849.
18. Engel J., Stückler J., Cremers D. Large-scale direct SLAM with stereo cameras, 2015 IEEE/RSJ interna-tional conference on intelligent robots and systems (IROS). IEEE, 2015, pp. 1935-1942.
19. Smirnova E., Stepanov D., Goryunov V. A technique of natural visual landmarks detection and descrip-tion for mobile robot cognitive navigation, Annals of 26th DAAAM International Symposium on Intelli-gent Manufacturing and Automation, DAAAM, 2015.
20. Kalal Z., Mikolajczyk K., Matas J. Tracking-learning-detection, IEEE transactions on pattern analysis and machine intelligence, 2011, Vol. 34, No. 7, pp. 1409-1422.
21. Bakhshiev A.V., Korban P.A., Kirpan' N.A. Programmnyy kompleks opredeleniya prostranstvennoy orientatsii ob"ektov po televizionnomu izobrazheniyu v zadache kosmicheskoy stykovki [A software package for determining the spatial orientation of objects based on a television image in the task of space docking], Extreme Robotics [Extreme Robotics], 2013, Vol. 1, No. 1, pp. 288-293.
22. Korsakov A.M. et al. Determination of an Unmanned Mobile Object Orientation by Natural Landmarks, AIST (Supplement), 2016, pp. 91-101.
23. Kirpan' N.A., Stepanov D.N. Programma modelirovaniya stseny s repernymi markerami i issledovaniya metodov opredeleniya polozheniya ob"ektov [A program for modeling a scene with reference markers and researching methods for determining the position of objects], 2016 registration No. 2016612470).
24. Bakhshiev A., Demcheva A., Stankevich L. CSNM: the compartmental spiking neuron model for devel-oping neuromorphic information processing systems, Advances in Neural Computation, Machine Learning, and Cognitive Research V: Selected Papers from the XXIII International Conference on Neu-roinformatics, October 18-22, 2021, Moscow, Russia. Springer International Publishing, 2022, pp. 327-333.
25. Boiko A., Bakhshiev A., Korsakov A. The Hardware Implementation of the Compartmental Spiking Neu-ron Model (CSNM) Based on Single Supply Operational Amplifiers, International Conference on Neu-roinformatics. Cham: Springer Nature Switzerland, 2024, pp. 48-57.
26. Kirillov A. et al. Segment anything, Proceedings of the IEEE/CVF International Conference on Comput-er Vision, 2023, pp. 4015-4026.
27. Yang Z., Yang Y. Decoupling features in hierarchical propagation for video object segmentation, Advanc-es in Neural Information Processing Systems, 2022, Vol. 35, pp. 36324-36336.
28. He K. et al. Deep residual learning for image recognition, Proceedings of the IEEE conference on com-puter vision and pattern recognition, 2016, pp. 770-778.
29. Liu Z. et al. A convnet for the 2020s, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11976-11986.
30. Fomin I. et al. Investigation of a Spike Segment Neuron in the Offline Multi-Object Tracking Task with Embeddings Constructed by a Convolutional Network, International Conference on Neuroinformatics. Cham: Springer Nature Switzerland, 2023, pp. 346-354.
31. Ren S. et al. Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE transactions on pattern analysis and machine intelligence, 2016, Vol. 39, No. 6, pp. 1137-1149.
32. He K. et al. Mask r-cnn, Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
33. Varghese R., Sambath M. YOLOv8: A Novel Object Detection Algorithm with En-hanced Performance and Robustness, 2024 International Conference on Advances in Data Engineering and Intelligent Com-puting Systems (ADICS). IEEE, 2024, pp. 1-6.
34. Wang A. et al. Yolov10: Real-time end-to-end object detection, arXiv preprint arXiv:2405.14458, 2024.
35. Alhasanat M.N. et al. RetinaNet-based Approach for Object Detection and Distance Estimation in an Image, International Journal on Communications Antenna and Propagation (IRECAP), 2021, Vol. 11, No. 1, pp. 1-9.
36. Duan K. et al. CenterNet++ for object detection, IEEE transactions on pattern analysis and machine intelligence. – 2023.
37. Chandarana P., Ou J., Zand R. An adaptive sampling and edge detection approach for encoding static images for spiking neural networks, 2021 12th International Green and Sustainable Computing Confer-ence (IGSC). IEEE, 2021, pp. 1-8.
