In recent years, the application of artificial intelligence and machine learning to sports has created new opportunities for real-time data analytics. Football clubs, in particular, are seeking advanced tools to track player performance, optimize strategies, and make data-driven decisions. The explosion of convolutional neural networks (CNNs) has led to significant advancements in object detection, with architectures such as YOLOv3 (You Only Look Once, version 3) now capable of detecting multiple objects in real-time.
Our study focuses on the development of a software-based solution for tracking football players in real-time, utilizing video footage. The aim is to replace the reliance on specialized hardware, providing coaches and managers with essential insights into player movements, tactics, and in-game statistics. The result is a practical tool that leverages machine learning algorithms to detect and track players with high precision, even in the challenging environment of a dynamic football field.
The field of object detection has rapidly evolved, driven by breakthroughs in deep learning. Convolutional Neural Networks (CNNs) have played a pivotal role in improving object detection accuracy. Leading technologies in this space include Single Shot Multibox Detector (SSD), RetinaNet, and Region-based Fully Convolutional Networks (R-FCN). Among these, YOLOv3 has gained prominence due to its balance of speed and accuracy, making it particularly suitable for real-time applications.
In the sports domain, traditional tracking systems often rely on hardware like wearable devices or complex camera setups. These systems, though accurate, come with limitations, such as high costs and complex infrastructure. By contrast, software-based solutions leveraging machine learning offer a scalable and less intrusive alternative. Our study builds on the advancements in CNNs, aiming to enhance football analytics with minimal reliance on specialized equipment.
Study Details
The primary goal of this study was to create a reliable, software-based solution for tracking football players during matches and training sessions. The system needed to provide accurate real-time tracking without the use of specialized hardware. By leveraging machine learning techniques, specifically deep learning, our goal was to detect and track players across dynamic and cluttered environments like football fields. Additionally, we aimed to process videos efficiently, making the system suitable for post-game analysis and strategy development.
We divided the problem into multiple stages, each representing a component of the tracking pipeline. By breaking the task into these steps, we addressed the challenges associated with each. The methodology focused on four core areas: field recognition, player detection, player tracking, and data analysis.
The first step in the tracking process involved isolating the football field from other elements in the video. Using image processing techniques, we applied color limits and morphological operations to isolate the green of the field from spectators and other elements. By filtering based on hue, we were able to remove irrelevant sections of the video, focusing solely on the playing area. Morphological operations like erosion and closing further refined the field detection, removing noise and irrelevant details.
We employed the YOLOv3 model for player detection. This model was chosen due to its proven ability to detect multiple objects in real time with high accuracy. The challenge in this step was not only detecting players but distinguishing between players, referees, and other objects, such as the ball. YOLOv3 was configured to focus on detecting football players as the primary class of interest. We tuned the model by adjusting the resolution to find the optimal trade-off between speed and detection accuracy. Based on our experiments, the YOLOv3-608 configuration provided the best balance, detecting up to 98% of players at a reasonable processing speed.
The next step was to ensure continuous tracking across frames, which involved identifying players between frames and tracking their movements. While YOLOv3 detected players in each frame, it didn’t inherently provide a method for tracking these detections across time. To solve this, we developed a tracking algorithm that associated bounding boxes across consecutive frames. This algorithm predicted the likely position of each player in the next frame based on their speed and direction. A key metric for this association was the Intersection over Union (IoU) score, which calculated the overlap between bounding boxes in consecutive frames. In cases where YOLOv3 missed a player, the support algorithm re-detected lost players by expanding the search area around their last known location.
One of the core challenges we encountered was processing speed. While YOLOv3-608 produced accurate results, it was still slower than real-time processing, with an average of 250ms per frame on a mid-range GPU. To improve the frame rate, we explored using YOLOv3-Tiny, a more lightweight, in-house trained, version of YOLOv3. However, YOLOv3-Tiny's accuracy was insufficient, especially for smaller or overlapping players, making it unsuitable as a primary method. Instead, we combined YOLOv3-Tiny with our support algorithm to quickly re-detect missed players in localized areas, significantly reducing processing times without sacrificing accuracy.
Findings
By integrating YOLOv3 with our tracking algorithms, we achieved an overall detection accuracy of over 95% in most scenarios. The tracking system performed well even in challenging conditions, such as when players overlapped or the camera moved. Below are some key findings from the study:
- Precision and Frame Rate: YOLOv3-608 provided the best trade-off between detection precision and processing time, achieving a detection rate of up to 98.2% of players in each frame. However, processing times remained around 250ms per frame, leading to a frame rate of approximately 4 frames per second (FPS). Though this is below the desired real-time threshold, further hardware optimization could bridge this gap.
- Challenges with Player Occlusion: One of the primary challenges was handling cases where players overlapped or temporarily disappeared from view due to occlusion. While YOLOv3 could detect players accurately, it struggled when players were in close proximity. Our support algorithm mitigated this by attempting to track players based on their previous positions, re-identifying them when they reappeared in view.
- Support Algorithm Success: The support algorithm we developed significantly improved the system's ability to recover lost detections. By expanding the search area and using a faster, less accurate model like YOLOv3-Tiny, the system re-detected players 95% of the time within a 15ms window. This hybrid approach allowed us to maintain high accuracy without overly increasing processing time.
The ability to track players in real-time opens up many opportunities for football clubs and analysts. This system provides coaches with valuable insights into player positioning, movement patterns, and overall performance. By eliminating the need for specialized hardware, the cost barrier is lowered, making advanced player analytics accessible to clubs of all sizes. The system can also be used for post-game analysis, allowing teams to refine their strategies based on player behavior.
Future work will focus on player identification and fine-tuning the algorithms to handle more complex game scenarios, such as detecting individual player actions (e.g., passing, shooting) and achieving full real-time performance with upgraded hardware.
