ミライ
Minimalist Async Evaluation Framework for R
→ Event-driven core with microsecond messaging
→ Scale from laptop to HPC and cloud — add or remove compute on the fly
→ Built for production — bounded queues, cancellation, distributed
tracing
Installation
install.packages("mirai")
Quick Start
library(mirai)
# Launch 6 background R processes (daemons) to run tasks on
daemons(6)
# mirai() starts an async task and returns immediately
m <- mirai({ Sys.sleep(1); mean(rnorm(1e6)) })
# Task still running, but the console is free
unresolved(m)
#> [1] TRUE
# Async map: [] collects results; options for progress bar and flatmap
mirai_map(1:9, \(x) { Sys.sleep(0.5); x^2 })[.progress, .flat]
#> [1] 1 4 9 16 25 36 49 64 81
# m[] waits for and returns the result
m[]
#> [1] 0.0005714758
# Shut down all daemons
daemons(0)
Architecture
mirai() sends tasks to daemons — persistent R worker processes. The
host listens at a URL; daemons dial in and pull work via an in-process
dispatcher thread that handles scheduling, cancellation, and bounded
queues. Add or remove daemons at any time, and direct tasks to different
compute profiles (CPU pool, GPU pool, remote cluster) from the same
session.
Round-trip latency stays in the microseconds:
daemons(1)
bench::mark(mirai("hello world")[])
#> # A tibble: 1 × 6
#> expression min median `itr/sec` mem_alloc `gc/sec`
#> <bch:expr> <bch:tm> <bch:tm> <dbl> <bch:byt> <dbl>
#> 1 "mirai(\"hello world\")[]" 64.3µs 94.2µs 10167. 9.68KB 4.04
daemons(0)
Deploy
| Where |
Setup |
| Local machine |
daemons(n) |
| SSH (direct or tunnelled) |
ssh_config() |
| HPC scheduler — Slurm, SGE, Torque/PBS, LSF |
cluster_config() |
| HTTP API — Posit Workbench, custom |
http_config() |
| Anywhere else |
remote_config() |
daemons(
n = 6,
url = host_url(tls = TRUE),
remote = cluster_config(options = "#SBATCH --mem=10G")
)
See the reference
vignette for the
full deployment guide.
What’s inside
- Async
—
mirai(), mirai_map(), everywhere(), race_mirai(),
try_mirai()
- Collection
—
m[], collect_mirai(), call_mirai(), .flat, .progress,
.stop
- Promises —
as.promise() for mirai and mirai_map; event-driven Shiny
ExtendedTask
- Cancellation &
timeouts
—
stop_mirai(), .timeout, .stop
- Backpressure
—
daemons(memory = …) capacity, peak watermark via
status()$memory, non-blocking try_mirai()
- Serialization
—
serial_config() for torch, Arrow, polars, ADBC; mori::share()
for local shared memory
- Reproducibility
— L’Ecuyer-CMRG streams;
daemons(seed = …) for deterministic
parallel RNG
- Observability
—
info(), status(), OpenTelemetry spans via otel
- Compute
profiles
— independent daemon pools,
with_daemons(), local_daemons()
- R parallel
cluster —
parallel::makeCluster(type = "MIRAI") (R ≥ 4.5)
Across the R stack
mirai has become the shared async layer for the R ecosystem. It’s the
recommended
async backend for Shiny and the only one for plumber2, the engine behind
purrr::in_parallel() and targets pipelines through crew, and is
the first official alternative communications
backend
for base R’s parallel package.
Acknowledgements
Will Landau for being instrumental in
shaping development of the package, from initiating the original request
for persistent daemons, through to orchestrating robustness testing for
the high performance computing requirements of crew and targets.
Joe Cheng for integrating the ‘promises’
method to work seamlessly within Shiny, and prototyping event-driven
promises.
Luke Tierney of R Core, for discussion
on L’Ecuyer-CMRG streams to ensure statistical independence in parallel
processing, and reviewing mirai’s implementation as the first
‘alternative communications backend for R’.
Travers Ching for a novel idea in
extending the original custom serialization support in the package.
Hadley Wickham, Henrik
Bengtsson, Daniel
Falbel, and Kirill
Müller for many deep insights and
discussions.
Links
mirai |
nanonext | CRAN HPC Task
View
AI coding agents: the r-lib agent skill from the
posit-dev-skills plugin
provides mirai-specific guidance.
–
Please note that this project is released with a Contributor Code of
Conduct. By participating
in this project you agree to abide by its terms.
评论