sparklyr: R Interface to Apache Spark

R interface to Apache Spark, a fast and general engine for big data processing, see <>. This package supports connecting to local and remote Apache Spark clusters, provides a 'dplyr' compatible back-end, and provides an interface to Spark's built-in machine learning algorithms.

Version: 1.7.2
Depends: R (≥ 3.2)
Imports: assertthat, base64enc, blob, config (≥ 0.2), DBI (≥ 0.6-1), dbplyr (≥ 1.1.0), digest, dplyr (≥ 0.7.2), ellipsis (≥ 0.1.0), forge, generics, globals, glue, httr (≥ 1.2.1), jsonlite (≥ 1.4), lifecycle, methods, openssl (≥ 0.8), purrr, r2d3, rappdirs, rlang (≥ 0.1.4), rprojroot, rstudioapi (≥ 0.10), stringr, tibble, tidyr (≥ 0.3.0), tidyselect, uuid, vctrs, withr, xml2
Suggests: arrow (≥ 0.14.0), broom, diffobj, foreach, ggplot2, iterators, janeaustenr, Lahman, mlbench, nnet, nycflights13, R6, RCurl, reshape2, shiny (≥ 1.0.1), testthat
Published: 2021-09-16
Author: Javier Luraschi [aut], Kevin Kuo ORCID iD [aut], Kevin Ushey [aut], JJ Allaire [aut], Samuel Macedo [ctb], Hossein Falaki [aut], Lu Wang [aut], Andy Zhang [aut], Yitao Li ORCID iD [aut, cre], Jozef Hajnala [ctb], Maciej Szymkiewicz ORCID iD [ctb], Wil Davis [ctb], RStudio [cph], The Apache Software Foundation [aut, cph]
Maintainer: Yitao Li <yitao at>
License: Apache License 2.0 | file LICENSE
NeedsCompilation: no
SystemRequirements: Spark: 1.6.x, 2.x, or 3.x
Materials: README NEWS
In views: Databases, ModelDeployment
CRAN checks: sparklyr results


Reference manual: sparklyr.pdf


Package source: sparklyr_1.7.2.tar.gz
Windows binaries: r-devel:, r-release:, r-oldrel:
macOS binaries: r-release (arm64): sparklyr_1.7.2.tgz, r-release (x86_64): sparklyr_1.7.2.tgz, r-oldrel: sparklyr_1.7.2.tgz
Old sources: sparklyr archive

Reverse dependencies:

Reverse imports: apache.sedona, catalog, geospark, graphframes, mleap, rsparkling, shinyML, spark.sas7bdat, sparkavro, sparkbq, sparkhail, sparklyr.flint, sparklyr.nested, sparktf, sparkwarc, sparkxgb, variantspark
Reverse suggests: corrr, mlflow, modeltime, parsnip, pointblank, resourcer, vip


Please use the canonical form to link to this page.