Fantasy-Premier-League
NOTICE
This weekly updates for the repository have been stopped at the end of the 2024-25 season. NO weekly updates will be posted to this repository after the 2024-25 season.
Instead, there will be 3 major updates posted to this repository:
- One at the start of the season
- One at the end of the January transfer window
- One at the end of the season
Description
A FPL library that gets all the basic stats for each player, gw-specific data for each player and season history of each player
How to CIte this dataset?
BibTeX:
@misc{anand2016fantasypremierleague,
title = {{FPL Historical Dataset}},
author = {Anand, Vaastav},
year = {2022},
howpublished = {Retrieved August 2022 from \url{https://github.com/vaastav/Fantasy-Premier-League/}}
}
Acknowledgement
- rin-hairie for adding master team lists and merge scripts
- ergest for adding merged_gw.csv files for 2016-17 and 2017-18 seasons
- BDooley11 for providing top managers script
- speeder1987 for providing 2018/19 fixtures.csv file
- ravgeetdhillon for github actions automation for data update
- kz4killua for fixing GW37 data for the 21-22 season
- SaintJuniper for id-dictionary update for the 21-22 season
FAQ
Data Structure
The data folder contains the data from past seasons as well as the current season. It is structured as follows:
- season/cleaned_players.csv : The overview stats for the season
- season/gws/gw_number.csv : GW-specific stats for the particular season
- season/gws/merged_gws.csv : GW-by-GW stats for each player in a single file
- season/players/player_name/gws.csv : GW-by-GW stats for that specific player
- season/players/player_name/history.csv : Prior seasons history stats for that specific player.
Accessing the Data Directly in Python
You can access data files within this repository programmatically using Python and the pandas library. Below is an example using the data/2023-24/gws/merged_gw.csv file. Similar methods can be applied to other data files in the repository. Note this is using the raw URL for direct file access, bypassing the GitHub UI.
import pandas as pd
# URL of the CSV file (example)
url = "https://raw.githubusercontent.com/vaastav/Fantasy-Premier-League/master/data/2023-24/gws/merged_gw.csv"
# Read the CSV file into a pandas DataFrame
df = pd.read_csv(url)
Data Dictionary
For a comprehensive explanation of all variables and columns in the dataset, see the DATA_DICTIONARY.md file.
Player Position Data
In players_raw.csv, element_type is the field that corresponds to the position.
1 = GK
2 = DEF
3 = MID
4 = FWD
Errata
- GW35 expected points data is wrong (all values are 0).
xP column — potential lookahead for ML models
The xP column in gws/merged_gw.csv is sourced from the FPL bootstrap-static API’s ep_this field. The scraper runs after each gameweek ends, so if FPL updates ep_this post-match, the scraped value will contain information that was not available to managers before the deadline. FPL’s update cadence for ep_this is not documented, so the exact behaviour is uncertain.
Empirical comparisons suggest the scraped xP values diverge from live pre-match ep_this:
- Live API
ep_this vs form correlation (fetched pre-deadline): ~0.98
- Scraped
xP vs form correlation (this dataset): ~0.75
xP rolling-3 vs same-GW total_points correlation: ~0.40 (unusually high for a genuinely pre-match feature)
If you are training ML models on this dataset: treat xP as potentially post-match. Either apply shift(1) within each element group, or exclude the column entirely. Using it unshifted as a feature to predict same-GW total_points has been observed to cause severe lookahead bias.
See this analysis for the full investigation.
Contributing
- If you feel like there is some data that is missing which you would like to see, then please feel free to create a PR or create an issue highlighting what is missing and what you would like to be added
- If you have access to old data (pre-2016) then please feel free to create Pull Requests adding the data to the repo or create an issue with links to old data and I will add them myself.
Using
If you use data from here for your website or blog posts, then I would humbly request that you please add a link back to this repo as the data source (and I would in turn add a link to your post/site as a notable usage of this repo).
Downloading Your Team Data
You can download the data for your team by executing the following steps:
python teams_scraper.py <team_id>
#Eg: python teams_scraper.py 4582
This will create a new folder called “team_<team_id>_data18-19” with individual files of all the important data
Notable Usages of this Repository
评论