- Pip install supports Mac and Linux platforms.
- You need to install Java >= JDK8 before runing BigDL, which is required by PySpark.
- Pip install only supports local mode. Might support cluster mode in the future. For those who want to use BigDL in cluster mode, try to install without pip.
- We've tested this package with Python 3.5 and Python 3.6. Support for Python 2.7 has been removed due to its end of life.
- Note that Python 3.6 is only compatible with Spark 1.6.4, 2.0.3, 2.1.1 and >=2.2.0. See this issue for more discussion.
Install BigDL from pip
Install BigDL release via pip (we tested this on pip 9.0.1)
You might need to add
sudoif without permission for the installation.
pyspark==2.4.3and their dependencies will be automatically installed if they haven't been detected in the current Python environment.
- Note that installing BigDL from pip will automatically install
pyspark==2.4.3. You are highly recommended to unset SPARK_HOME to avoid possible conflicts.
pip3 install --upgrade pip pip3 install BigDL