A toolbox to fairly evaluate automatic sleep staging algorithms
SLEEPYLAND is a Python-based web application developed for the automated FAIR sleep staging analysis of large sleep datasets. Utilizing state-of-the-art machine learning and deep learning models, SLEEPYLAND mainly simplifies sleep stage classification, empowering researchers and clinicians to analyse sleep recordings.
If you use SLEEPYLAND in your research or applications, please cite the following paper:
Rossi AD, Metaldi M, Bechny M, Filchenko I, Meer JV, Schmidt MH, Bassetti CLA, Tzovara A, Faraci FD, Fiorillo L.
SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models.
npj Digital Medicine. 2025 Dec 16. doi:10.1038/s41746-025-02237-2.
Available at: https://www.nature.com/articles/s41746-025-02237-2
Download & Run SLEEPYLAND
To get started, choose your operating system below. Just follow the step-by-step guideline. This is the recommended method as it allows you to visualize the installation process directly in the shell.
| Linux |
Mac |
Windows |
 |
 |
 |
Step 1: Download linux_setup.sh or linux_rocky_setup.sh |
Step 1: Download macos_setup.sh |
Step 1: Download win_setup.ps1 |
Step 2: Download docker-compose.yml |
Step 2: Download docker-compose.yml |
Step 2: Download docker-compose.yml |
Step 3: Place the .sh/.yml files in the same folder. |
Step 3: Place the .sh/.yml files in the same folder. |
Step 3: Place the .ps1/.yml files in the same folder. |
| Step 4: Open a terminal, navigate to the folder, and run: |
Step 4: Open a terminal, navigate to the folder, and run: |
Step 4: Open PowerShell as Administrator, navigate to the folder, and run: |
chmod +x linux_setup.sh && ./linux_setup.sh |
chmod +x macos_setup.sh && ./macos_setup.sh |
./win_setup.ps1 |
❗GOOD TO KNOW❗
Running .sh and .ps1 scripts will also update SLEEPYLAND, automatically pulling the latest version of the shared Docker images.
Windows Users:
If you encounter issues running the .ps1 script, you may need to change the PowerShell execution policy. To do this, open PowerShell as Administrator and run:
Set-ExecutionPolicy Unrestricted
Confirm the change if prompted. This will allow the script to execute properly.
❗WARNING❗
SLEEPYLAND WebApp does not open automatically:
Ensure that all containers are running.
You can check the status of your Docker containers by executing this command in your terminal:
docker ps
Look for a container named gui to confirm the gui service is up.
Access SLEEPYLAND via your browser.
Open your web browser and navigate to:
http://localhost:8887
This will open the SLEEPYLAND GUI where you can perform various tasks interactively.
❗WARNING❗
How to stop SLEEPYLAND:
SLEEPYLAND runs in the background, even if you close the web window.
To completely stop the application, execute the following command in the terminal from the folder containing docker-compose.yml:
docker compose -f docker-compose.yml -p sleepyland stop
❗WARNING❗
Please be aware that SLEEPYLAND attempts to pull/install external software dependencies during its setup process.
Important Notice:
If you are operating on an institutional machine (e.g., hospital or university systems), this process is likely to fail due to permission restrictions. To avoid this issue, ensure you have the necessary administrative permissions or consult your IT department before proceeding with the installation.
Key Features
| Local Processing & Privacy |
Fair Model Training |
| All analyses run locally on your device, ensuring complete data privacy. Your data never leaves your laptop. |
The included deep learning sleep staging models have been fairly trained and validated on consistent public datasets, ensuring robust and reliable performance for accurate sleep analysis and stage classification. |
All the exposed sleep staging models run locally on your CPU (GPU support is not available). As a result, some models, such as the deepresnet and sleeptransformermodel, may perform slowly when predicting or evaluating multiple combinations of EEG and EOG channels.
For instructions on navigating the UI and making the most of the analysis tools, check out this Quickstart Tutorial Video in our YouTube playlist.
List of Models for Sleep Staging
| Model |
Description |
Developed By |
References |
Available |
| YASA |
A lightweight, open-source package for sleep staging, offering rapid sleep analysis using EEG/EOG/EMG signals. |
Raphael Vallat, et al |
Vallat R, Walker MP. An open-source, high-performance tool for automated sleep staging. Elife. 2021. |
✅ |
| U-Sleep |
A robust deep learning model designed to handle various PSG setups and mixed-cohort datasets for automated sleep staging. |
Perslev, et al |
Perslev M, Darkner S, et al. U-Sleep: resilient high-frequency sleep staging. NPJ Digital Medicine. 2021. |
✅ |
| DeepResNet |
A deep residual network model for high-frequency sleep stage classification using PSG data. |
Olesen, et al |
Olesen AN, Jennum PJ, et al. Automatic sleep stage classification with deep residual networks in a mixed-cohort setting. Sleep. 2021. |
✅ |
| SleepTransformer |
A transformer-based model that provides sleep staging with interpretability and uncertainty quantification. |
Phan, et al |
Phan H, Mikkelsen K, et al. Sleeptransformer: Automatic sleep staging with interpretability and uncertainty quantification. IEEE Transactions on Biomedical Engineering. 2022. |
✅ |
The YASA model has not been retrained from scratch on NSRR datasets within the SLEEPYLAND.
All the other models have been implemented to ensure consistency and fairness in their evaluation on the NSRR datasets.
All the models are available in a single-channel configuration (e.g., EEG or EOG) and in the multi-channel configuration (e.g., EEG+EOG).
Need additional help? Click the Quickstart Tutorial button for detailed step-by-step instructions.
Technical Section
Datasets
NSRR Datasets Included for Model Train/Valid/Test
| Dataset |
Subject Count |
PSG Count |
Age (µ ± σ) |
Sex %M |
BMI (µ ± σ) |
Sleep Disorders |
Annotation Format |
Included |
| ABC |
49 |
132 |
48.8 ± 9.9 |
57.1 |
38.9 ± 3.0 |
OSA |
.xml |
✅ |
| APOE |
712 |
712 |
45.7 ± 13.6 |
59.1 |
27.2 ± 6.5 |
SDB |
.STA |
✅ |
| APPLES |
1094 |
1094 |
50.1 ± 12.9 |
63.2 |
32.1 ± 7.8 |
OSA |
.annot |
✅ |
| CCSHS |
515 |
515 |
17.7 ± 0.4 |
50.5 |
25.1 ± 5.9 |
SRMD |
.xml |
✅ |
| CFS |
144 |
730 |
41.4 ± 19.3 |
44.8 |
32.4 ± 9.5 |
OSA |
.xml |
✅ |
| CHAT |
1232 |
1638 |
7.0 ± 1.4 |
48.3 |
19.0 ± 4.9 |
OSA |
.xml |
✅ |
| HOMEPAP |
246 |
246 |
46.5 ± 11.9 |
60.3 |
37.2 ± 8.9 |
OSA |
.xml |
✅ |
| MESA |
2056 |
2056 |
69.9 ± 9.2 |
46.4 |
28.7 ± 5.6 |
SDB |
.xml |
✅ |
| MNC_CNC |
78 |
78 |
28.5 ± 16.9 |
51.3 |
23.2 ± 11.5 |
NT1, NT2, IH |
.xml |
✅ |
| MNC_DHC |
83 |
83 |
33.4 ± 14.8 |
50.0 |
24.8 ± 4.9 |
- |
.xml |
✅ |
| MNC_SSC |
767 |
767 |
45.4 ± 13.8 |
59.4 |
23.9 ± 6.5 |
- |
.xml |
✅ |
| MROS |
2905 |
3930 |
76.4 ± 5.5 |
100 |
27.2 ± 3.9 |
SDB |
.xml |
✅ |
| MSP |
105 |
105 |
26.8 ± 5.9 |
0 |
42.4 ± 6.6 |
SDB |
.annot |
✅ |
| NCHSDB |
3651 |
3950 |
8.8 ± 5.9 |
56.3 |
22.7 ± 9.9 |
- |
.tsv |
✅ |
| SHHS |
5797 |
8444 |
63.1 ± 11.2 |
47.6 |
28.2 ± 5.1 |
OSA, SDB |
.xml |
✅ |
| SOF |
453 |
453 |
82.9 ± 3.2 |
0 |
27.7 ± 4.6 |
SDB |
.xml |
✅ |
| WSC |
1123 |
2569 |
56.4 ± 8.1 |
54.1 |
31.7 ± 7.1 |
SDB |
.txt |
✅ |
Open Access Datasets Included for Model Testing
| Dataset |
Subject Count |
PSG Count |
Age (µ ± σ) |
Sex %M |
BMI (µ ± σ) |
Sleep Disorders |
Annotation Format |
Included |
| DCSM |
255 |
255 |
- |
- |
- |
- |
.ids |
✔️ |
| DOD-H |
25 |
25 |
- |
- |
- |
- |
.h5 |
✔️ |
| DOD-O |
55 |
55 |
- |
- |
- |
- |
.h5 |
✔️ |
| PHYS |
994 |
994 |
55.2 ± 14.3 |
67 |
- |
SDB |
.ids |
✔️ |
| SEDF_SC |
78 |
153 |
58.8 ± 22.0 |
47 |
- |
- |
.edf |
✔️ |
| SEDF_ST |
22 |
44 |
40.2 ± 17.7 |
32 |
- |
- |
.edf |
✔️ |
Note: All the deep learning-based models for sleep staging have been trained, validated, and tested on in-domain (ID) training datasets - datasets included marked with a ✅.
We assess the generalizability of the multiple models on the largest out-of-domain set of data - datasets included marked with a ✔️.
Architecture
Getting Started with SLEEPYLAND
Follow these steps to kickstart your sleep staging data analysis journey on supported Datasets.
The user has three main options to perform the sleep staging via the UI:
🟢 NSRR Data: Benchmark and compare different sleep staging models using NSRR recordings. This option allows you to evaluate the performance of various models on standardized datasets.
🟢 MyEDF Data: Analyze/Predict your own recordings (user-provided .edf files) while ensuring complete data privacy and security.
🟢 NSRR Data
Upload/Download Data
-
Upload
When uploading NSRR data, ensure your dataset follows the hierarchical structure below for SLEEPYLAND to recognize and organize files correctly:
Example structure for a dataset named learn:
**learn**
├── `learn-nsrr01.edf`
├── `learn-nsrr01.xml`
├── `learn-nsrr02.edf`
├── `learn-nsrr02.xml`
├── `learn-nsrr03.edf`
└── `learn-nsrr03.xml`
-
Download Data
- Insert a token obtained via the NSRR data request platform.
- Select the desired dataset among the ones supported in SLEEPYLAND.
- Choose the specific NSSR recording/s.
Specify Channels
- Single-channel configuration: Choose either EEG or EOG (set the other to “None”).
- Multi-channel configuration: Choose both EEG and EOG set of channels (e.g., EEG:
C4-M1, EOG: E1-M2).
Note: Specify also the name of the folder where outcomes will be saved (the folder will be created under the shared volume output).
Models for Sleep Staging
Select a set of models for sleep staging:
- Feature-based models
- Deep learning-based models.
For details, see the List of Models for Sleep Staging section.
Run Evaluations
Once configurations are set, choose the Evaluate option (since supported NSRR data includes annotation files).
Explore Results
Explore results using:
- UI Options for interactive visualization.
- Notebook Option for advanced analysis using Python and Jupyter Notebooks.
🟢 MyEDF Data
Upload Data
- Upload a
.edf file and optionally an annotation file.
Specify Channels
- Channel type options:
- EEG only
- EOG only
- EEG and EOG (multi-channel).
- If mastoid channels are present, select the re-reference option.
Note: Specify also the name of the folder where outcomes will be saved (the folder will be created under the shared volume output).
Models for Sleep Staging
Select a set of models for sleep staging:
- Feature-based models
- Deep learning-based models.
For details, see the List of Models for Sleep Staging section.
Run Predictions/Evaluations
- Choose Predict if only
.edf files were uploaded.
- Choose Evaluate if an annotation file is also provided.
Explore Results
Explore results using:
- UI Options for interactive visualization.
- Notebook Option for advanced analysis using Python and Jupyter Notebooks.
评论