From pip
NOTES
- Pip install supports Mac and Linux platforms.
- You need to install Java >= JDK8 before runing BigDL, which is required by PySpark.
- Pip install only supports local mode. Might support cluster mode in the future. For those who want to use BigDL in cluster mode, try to install without pip.
- We've tested this package with Python 3.5 and Python 3.6. Support for Python 2.7 has been removed due to its end of life.
- Note that Python 3.6 is only compatible with Spark 1.6.4, 2.0.3, 2.1.1 and >=2.2.0. See this issue for more discussion.
Install BigDL from pip
Install BigDL release via pip (we tested this on pip 9.0.1)
Remark:
-
You might need to add
sudo
if without permission for the installation. -
numpy>=1.7
,six>=1.10.0
,pyspark==2.4.3
and their dependencies will be automatically installed if they haven't been detected in the current Python environment. - Note that installing BigDL from pip will automatically install
pyspark==2.4.3
. You are highly recommended to unset SPARK_HOME to avoid possible conflicts.
pip3 install --upgrade pip
pip3 install BigDL