Jupyter: Difference between revisions

From FlowerHouseWiki
(Undo revision 892 by Tropaion (talk))
Tag: Undo
 
(17 intermediate revisions by the same user not shown)
Line 95: Line 95:
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
apt install sagemath
apt install sagemath
</syntaxhighlight>
=== MatLab ===
==== Install MatLab ====
<p>Mount iso from USB via Proxmox to LXC</p>
<syntaxhighlight lang="console">
mount -o loop,ro /media/sda/MatLab/Matlab910R2021a_Lin64.iso /rpool/data/subvol-111-disk-0/mnt
</syntaxhighlight>
<p>Create installation procedure file and add this</p>
<syntaxhighlight lang="console">
nano /input.txt
</syntaxhighlight>
<p>Start installation with:</p>
<syntaxhighlight lang="console" line>
cd /mnt
./install -inputFile /input.txt
</syntaxhighlight>
<p>Check log if installation was successful</p>
<syntaxhighlight lang="console">
nano ./InstallMatLab.log
</syntaxhighlight>
<p>Delete log</p>
<syntaxhighlight lang="console">
rm ./InstallMatLab.log
</syntaxhighlight>
<p>Unmount ISO in ProxMox</p>
<syntaxhighlight lang="console">
umount /rpool/data/subvol-111-disk-0/mnt
</syntaxhighlight>
<p>Open .bashrc</p>
<syntaxhighlight lang="console">
nano /etc/bash.bashrc
</syntaxhighlight>
<p>Add to end of file</p>
<syntaxhighlight lang="console">
export MATLAB_ROOT_DIR="/usr/local/Polyspace/R2021a"
</syntaxhighlight>
<p>Apply changes by executing the command</p>
<syntaxhighlight lang="console">
source /etc/bash.bashrc
</syntaxhighlight>
<p>Check if if works:</p>
<syntaxhighlight lang="console">
echo $MATLAB_ROOT_DIR
</syntaxhighlight>
==== Install Python-3.8 ====
<p>Download Python-3.8</p>
<syntaxhighlight lang="console" line>
wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tar.xz
tar -xf Python-3.8.12.tar.xz
rm Python-3.8.12.tar.xz
mv Python-3.8.12 /opt/Python-3.8.12
</syntaxhighlight>
<p>Install requirements</p>
<syntaxhighlight lang="console">
apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev -y
</syntaxhighlight>
<p>Configure</p>
<syntaxhighlight lang="console"line>
cd /opt/Python-3.8.12
./configure --enable-optimizations --enable-shared
</syntaxhighlight>
<p>Compile Python-3.8, x = CPU Cores to use</p>
<syntaxhighlight lang="console">
make -j x
</syntaxhighlight>
<p>Install Python</p>
<syntaxhighlight lang="console">
make altinstall
</syntaxhighlight>
<p>Delete compile files</p>
<syntaxhighlight lang="console">
cd ..
rm -r Python-3.8.12
</syntaxhighlight>
<p>Configure dynamic linker</p>
<syntaxhighlight lang="console">
ldconfig /usr/local/lib
</syntaxhighlight>
==== Install Kernel ====
<p>Enter environment folder</p>
<syntaxhighlight lang="console">
cd /home/venv
</syntaxhighlight>
<p>Create environment with Python-3.8</p>
<syntaxhighlight lang="console">
python3.8 -m venv MatLab
</syntaxhighlight>
<p>Activate environment</p>
<syntaxhighlight lang="console">
source MatLab/bin/activate
</syntaxhighlight>
<p>Install MatLab Kernel</p>
<syntaxhighlight lang="console" line>
pip install matlab_kernel
python -m matlab_kernel install
</syntaxhighlight>
<p>Install MatLab Python API</p>
<syntaxhighlight lang="console" line>
cd $MATLAB_ROOT_DIR/extern/engines/python
python setup.py install
</syntaxhighlight>
</syntaxhighlight>


Line 107: Line 229:
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
bash Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
</syntaxhighlight>
<p>Add Environment variable</p>
<syntaxhighlight lang="console">
echo "export PATH=$PATH:~/miniconda3/bin">> ~/.bashrc
</syntaxhighlight>
<p>Reload shell</p>
<syntaxhighlight lang="console">
source ~/.bashrc
</syntaxhighlight>
</syntaxhighlight>


Line 142: Line 274:


=== AI ===
=== AI ===
==== Create environment ====
<p>Install venv</p>
<syntaxhighlight lang="console">
apt install python3-venv
</syntaxhighlight>
<p>Create folder where environment should be located at</p>
<syntaxhighlight lang="console" line>
mkdir /home/venv
cd /home/venv
</syntaxhighlight>
<p>Create Keras-Environment</p>
<syntaxhighlight lang="console">
python3 -m venv Keras
</syntaxhighlight>
<p>Activate environment</p>
<syntaxhighlight lang="console" line>
cd /home/venv
source Keras/bin/activate
</syntaxhighlight>
<p>Install iPython Kernel</p>
<syntaxhighlight lang="console">
pip install ipykernel matplotlib
</syntaxhighlight>
<p>Add virtual environment to jupyter</p>
<syntaxhighlight lang="console">
python -m ipykernel install --name=Keras
</syntaxhighlight>
<p>Deactivate with:</p>
<syntaxhighlight lang="console">
deactivate
</syntaxhighlight>
==== Tensorflow ====
==== Tensorflow ====
<p>Install Tensorflow with pip</p>
<p>Activate environment</p>
<syntaxhighlight lang="console" line>
cd /home/venv
source Keras/bin/activate
</syntaxhighlight>
 
<p>Install tensorflow</p>
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
pip3 install --upgrade tensorflow requests
pip install tensorflow
</syntaxhighlight>
</syntaxhighlight>


<p>Check if installation works</p>
<p>Verify tensorflow installation:</p>
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
python3 -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
pip show tensorflow
</syntaxhighlight>
</syntaxhighlight>


<p>As server has no GPU, disable it with environment variable:</p>
<p>Update tensorflow with</p>
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
nano /etc/environment
pip install --upgrade tensorflow
</syntaxhighlight>
</syntaxhighlight>


<p>Add following line:</p>
<p>Deactivate with:</p>
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
CUDA_VISIBLE_DEVICES="-1"
deactivate
</syntaxhighlight>
</syntaxhighlight>


==== Keras ====
==== Keras ====
<p>Activate environment</p>
<syntaxhighlight lang="console" line>
cd /home/venv
source Keras/bin/activate
</syntaxhighlight>
<p>Install Keras</p>
<p>Install Keras</p>
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
apt install python3-keras
pip install keras
</syntaxhighlight>
 
<p>Verify Keras installation:</p>
<syntaxhighlight lang="console">
pip show keras
</syntaxhighlight>
 
<p>Update Keras with</p>
<syntaxhighlight lang="console">
pip install --upgrade keras
</syntaxhighlight>
 
<p>Deactivate with:</p>
<syntaxhighlight lang="console">
deactivate
</syntaxhighlight>
</syntaxhighlight>


Latest revision as of 17:46, 15 July 2022

Juypter.png

Network

IP: 192.168.88.18
MAC: 6A:1B:42:9D:D6:4B
Domain: jupyter.flowerhouse.at

System

OS: Debian Bullseye
RAM: 4096MB
Cores: 4
Privileged: No

The Jupyter-LXC is reachable under 192.168.88.18 which is located in the ServerVLAN.

The subdomain is jupyter.flowerhouse.at which is handled by the ReverseProxy.

Basic Setup

Installation

Install required packages

apt install python3-pip npm

Install JupyterHub, responsible for managing multiple users

python3 -m pip install jupyterhub

Install Proxy needed by JupyterHub

npm install -g configurable-http-proxy

Install webinterface and notebook

python3 -m pip install jupyterlab notebook

Configuration

Create config file

Create folder for config

mkdir /etc/jupyterhub

Generate config file

jupyterhub --generate-config -f /etc/jupyterhub/jupyterhub_config.py

Create systemd service

Create service file

nano /etc/systemd/system/jupyter.service

Add this content to file

[Unit]
Description=Jupyterhub
After=syslog.target network.target

[Service]
User=root
Environment="PATH=/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/anaconda3/bin"
ExecStart=jupyterhub -f /etc/jupyterhub/jupyterhub_config.py

[Install]
WantedBy=multi-user.target

Reload systemd

systemctl daemon-reload

Start Jupyter and check if its running

systemctl start jupyter
systemctl status jupyter

Start Jupyter at boot

systemctl enable jupyter

Create/Add user

Create new user

adduser [username]

Extensions/Kernels

SageMath

Install SageMath Kernel

apt install sagemath

MatLab

Install MatLab

Mount iso from USB via Proxmox to LXC

mount -o loop,ro /media/sda/MatLab/Matlab910R2021a_Lin64.iso /rpool/data/subvol-111-disk-0/mnt

Create installation procedure file and add this

nano /input.txt

Start installation with:

cd /mnt
./install -inputFile /input.txt

Check log if installation was successful

nano ./InstallMatLab.log

Delete log

rm ./InstallMatLab.log

Unmount ISO in ProxMox

umount /rpool/data/subvol-111-disk-0/mnt

Open .bashrc

nano /etc/bash.bashrc

Add to end of file

export MATLAB_ROOT_DIR="/usr/local/Polyspace/R2021a"

Apply changes by executing the command

source /etc/bash.bashrc

Check if if works:

echo $MATLAB_ROOT_DIR

Install Python-3.8

Download Python-3.8

wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tar.xz
tar -xf Python-3.8.12.tar.xz
rm Python-3.8.12.tar.xz
mv Python-3.8.12 /opt/Python-3.8.12

Install requirements

apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev -y

Configure

cd /opt/Python-3.8.12
./configure --enable-optimizations --enable-shared

Compile Python-3.8, x = CPU Cores to use

make -j x

Install Python

make altinstall

Delete compile files

cd ..
rm -r Python-3.8.12

Configure dynamic linker

ldconfig /usr/local/lib

Install Kernel

Enter environment folder

cd /home/venv

Create environment with Python-3.8

python3.8 -m venv MatLab

Activate environment

source MatLab/bin/activate

Install MatLab Kernel

pip install matlab_kernel
python -m matlab_kernel install

Install MatLab Python API

cd $MATLAB_ROOT_DIR/extern/engines/python
python setup.py install

C++

Install Miniconda

Download install script

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Run install script and reboot

bash Miniconda3-latest-Linux-x86_64.sh

Add Environment variable

echo "export PATH=$PATH:~/miniconda3/bin">> ~/.bashrc

Reload shell

source ~/.bashrc

Update conda

conda update --all

Install Xeus-Cling

Install xeus-cling

conda install xeus-cling -c conda-forge

Add kernel

Add C++11 kernel

jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp11

Add C++14 kernel

jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp14

Add C++17 kernel

jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp17

Restart Jupyter

systemctl restart jupyter

AI

Create environment

Install venv

apt install python3-venv

Create folder where environment should be located at

mkdir /home/venv
cd /home/venv

Create Keras-Environment

python3 -m venv Keras

Activate environment

cd /home/venv
source Keras/bin/activate

Install iPython Kernel

pip install ipykernel matplotlib

Add virtual environment to jupyter

python -m ipykernel install --name=Keras

Deactivate with:

deactivate

Tensorflow

Activate environment

cd /home/venv
source Keras/bin/activate

Install tensorflow

pip install tensorflow

Verify tensorflow installation:

pip show tensorflow

Update tensorflow with

pip install --upgrade tensorflow

Deactivate with:

deactivate

Keras

Activate environment

cd /home/venv
source Keras/bin/activate

Install Keras

pip install keras

Verify Keras installation:

pip show keras

Update Keras with

pip install --upgrade keras

Deactivate with:

deactivate

Sources

Retrieved from "https://wiki.flowerhouse.at/index.php?title=Jupyter&oldid=894"