- Pip install supports Mac and Linux platforms.
- You need to install Java >= JDK8 before runing BigDL, which is required by PySpark 2.2.0.
- Pip install only supports local mode. Might support cluster mode in the future. For those who want to use BigDL in cluster mode, try to install without pip.
- We've tested this package with Python 2.7, Python 3.5 and Python 3.6. Only these three Python versions are supported for now.
- Note that Python 3.6 is only compatible with Spark 1.6.4, 2.0.3, 2.1.1 and 2.2.0. See this issue for more discussion.
Install BigDL release via pip (we tested this on pip 9.0.1)
You might need to add
sudoif without permission for the installation.
pysparkwill be automatically installed first before installing BigDL if it hasn't been detected locally.
pip install --upgrade pip pip install BigDL==0.11.0.dev0 # for Python 2.7 pip3 install BigDL==0.11.0.dev0 # for Python 3.5 and Python 3.6