Virtual Cart System with Hand Gesture Recognition

Abstract

In the contemporary world filled with e-commerce, the user experience is the most crucial aspect of any business dealing. During shopping online, users miss the good old relationship and atmosphere that real shops provide. The current interfaces are way off from being intuitive and immersive; these selected shortcomings will be highlighted and addressed within this project by developing some Virtual Cart system with the use of state-of-the-art technologies like Computer Vision, Real-Time Communication, and Automated Transaction Processing. The Virtual Cart system is developed to change the purchase nature via the web with virtual shopping environments where a shopper communicates using natural hand gestures. The system can accurately identify products when a user points their gesture towards a product, and it allows an intuitive selection process. It is based on the state-of-the-art object detection algorithm in YOLOv4. For instance, a user selects an item in the virtual shelf and drags it under a clearly marked green line of the display. This is then validated post, which the system auto-replies with an automated message through the WhatsApp API to the user about the total due amount and the payment link such that the checkout process is made very simple. Integration of the real-time channel allows for immediate push of instructions for making a payment, immediately improving the process in terms of efficiency and user satisfaction. This can not only take the frustration out of online checkout but also makes accessible and usable a widely used messaging platform. This report details the design, implementation, and evaluation of a Virtual Cart system. It is underlined by Python for programming, Docker containers to achieve scalable and isolated environments, while integrating RabbitMQ for reliable message transfer. The project approach will undertake the project methodology, with an overview from concept to deployment; it points out the iterative development process and stage evaluation processes with challenges encountered. The fact that the Virtual Cart system moves online shopping into a level of user interaction and satisfaction significantly higher than anything realized up to now cannot be understated. This offers a much more natural and efficient model of interactions with online shopping convenience, bridging the tactile character of physical stores. This project lays out the foundation for e-commerce innovation in the future where advanced technologies will further enhance personalization and ease of shopping.