Jupyter: Difference between revisions

From FlowerHouseWiki
(Created page with "{{ContainerInfo |title = Juypter-LXC |image = Reverse proxy.png |MAC = 6A:1B:42:9D:D6:4B |IP = 192.168.88.18 |Privileged = No |OS = Debian Bullseye |RAM = 4096MB |Cores = 4 }}...")
 
(Undo revision 892 by Tropaion (talk))
Tag: Undo
 
(38 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{ContainerInfo
{{ContainerInfo
|title = Juypter-LXC
|image = Juypter.png
|image = Reverse proxy.png
|Domain = [https://jupyter.flowerhouse.at jupyter.flowerhouse.at]
|MAC = 6A:1B:42:9D:D6:4B
|MAC = 6A:1B:42:9D:D6:4B
|IP = 192.168.88.18
|IP = 192.168.88.18
Line 9: Line 9:
|Cores = 4
|Cores = 4
}}
}}
<p> is reachable under <syntaxhighlight lang="Bash" inline>192.168.88.3</syntaxhighlight> which is located in the ServerVLAN.</p>
 
<p>Every incoming packages from outside are forwarded to this IP.</p>
<p>The Jupyter-LXC is reachable under <syntaxhighlight lang="Bash" inline>192.168.88.18</syntaxhighlight> which is located in the ServerVLAN.</p>
<p>The ReverseProxy also forces outside connections to use HTTPS/SSL and will provide a SSL-Certificate.</p>
<p>The subdomain is [https://jupyter.flowerhouse.at jupyter.flowerhouse.at] which is handled by the [[ReverseProxy]].</p>
__TOC__
__TOC__
== Basic Setup ==
== Basic Setup ==
=== Nginx Proxy Manager ===
=== Installation ===
<p>Install required packages</p>
<syntaxhighlight lang="console">
apt install python3-pip npm
</syntaxhighlight>
 
<p>Install JupyterHub, responsible for managing multiple users</p>
<syntaxhighlight lang="console">
python3 -m pip install jupyterhub
</syntaxhighlight>
 
<p>Install Proxy needed by JupyterHub</p>
<syntaxhighlight lang="console">
npm install -g configurable-http-proxy
</syntaxhighlight>
 
<p>Install webinterface and notebook</p>
<syntaxhighlight lang="console">
python3 -m pip install jupyterlab notebook
</syntaxhighlight>
 
== Configuration ==
=== Create config file ===
<p>Create folder for config</p>
<syntaxhighlight lang="console">
mkdir /etc/jupyterhub
</syntaxhighlight>
 
<p>Generate config file</p>
<syntaxhighlight lang="console">
jupyterhub --generate-config -f /etc/jupyterhub/jupyterhub_config.py
</syntaxhighlight>
 
=== Create systemd service ===
<p>Create service file</p>
<syntaxhighlight lang="console">
nano /etc/systemd/system/jupyter.service
</syntaxhighlight>
 
<p>Add this content to file</p>
<syntaxhighlight lang="console" line>
[Unit]
Description=Jupyterhub
After=syslog.target network.target
 
[Service]
User=root
Environment="PATH=/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/anaconda3/bin"
ExecStart=jupyterhub -f /etc/jupyterhub/jupyterhub_config.py
 
[Install]
WantedBy=multi-user.target
</syntaxhighlight>
 
<p>Reload systemd</p>
<syntaxhighlight lang="console">
systemctl daemon-reload
</syntaxhighlight>
 
<p>Start Jupyter and check if its running</p>
<syntaxhighlight lang="console" line>
systemctl start jupyter
systemctl status jupyter
</syntaxhighlight>
 
<p>Start Jupyter at boot</p>
<syntaxhighlight lang="console" line>
systemctl enable jupyter
</syntaxhighlight>
 
=== Create/Add user ===
<p>Create new user</p>
<syntaxhighlight lang="console">
adduser [username]
</syntaxhighlight>
 
== Extensions/Kernels ==
=== SageMath ===
<p>Install SageMath Kernel</p>
<syntaxhighlight lang="console">
apt install sagemath
</syntaxhighlight>
 
=== MatLab ===
==== Install MatLab ====
<p>Mount iso from USB via Proxmox to LXC</p>
<syntaxhighlight lang="console">
mount -o loop,ro /media/sda/MatLab/Matlab910R2021a_Lin64.iso /rpool/data/subvol-111-disk-0/mnt
</syntaxhighlight>
 
<p>Create installation procedure file and add this</p>
<syntaxhighlight lang="console">
nano /input.txt
</syntaxhighlight>
 
<p>Start installation with:</p>
<syntaxhighlight lang="console" line>
cd /mnt
./install -inputFile /input.txt
</syntaxhighlight>
 
<p>Check log if installation was successful</p>
<syntaxhighlight lang="console">
nano ./InstallMatLab.log
</syntaxhighlight>
 
<p>Delete log</p>
<syntaxhighlight lang="console">
rm ./InstallMatLab.log
</syntaxhighlight>
 
<p>Unmount ISO in ProxMox</p>
<syntaxhighlight lang="console">
umount /rpool/data/subvol-111-disk-0/mnt
</syntaxhighlight>
 
<p>Open .bashrc</p>
<syntaxhighlight lang="console">
nano /etc/bash.bashrc
</syntaxhighlight>
 
<p>Add to end of file</p>
<syntaxhighlight lang="console">
export MATLAB_ROOT_DIR="/usr/local/Polyspace/R2021a"
</syntaxhighlight>
 
<p>Apply changes by executing the command</p>
<syntaxhighlight lang="console">
source /etc/bash.bashrc
</syntaxhighlight>
 
<p>Check if if works:</p>
<syntaxhighlight lang="console">
echo $MATLAB_ROOT_DIR
</syntaxhighlight>
 
==== Install Python-3.8 ====
<p>Download Python-3.8</p>
<syntaxhighlight lang="console" line>
wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tar.xz
tar -xf Python-3.8.12.tar.xz
rm Python-3.8.12.tar.xz
mv Python-3.8.12 /opt/Python-3.8.12
</syntaxhighlight>
 
<p>Install requirements</p>
<syntaxhighlight lang="console">
apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev -y
</syntaxhighlight>
 
<p>Configure</p>
<syntaxhighlight lang="console"line>
cd /opt/Python-3.8.12
./configure --enable-optimizations --enable-shared
</syntaxhighlight>
 
<p>Compile Python-3.8, x = CPU Cores to use</p>
<syntaxhighlight lang="console">
make -j x
</syntaxhighlight>
 
<p>Install Python</p>
<syntaxhighlight lang="console">
make altinstall
</syntaxhighlight>
 
<p>Delete compile files</p>
<syntaxhighlight lang="console">
cd ..
rm -r Python-3.8.12
</syntaxhighlight>
 
<p>Configure dynamic linker</p>
<syntaxhighlight lang="console">
ldconfig /usr/local/lib
</syntaxhighlight>
 
==== Install Kernel ====
<p>Enter environment folder</p>
<syntaxhighlight lang="console">
cd /home/venv
</syntaxhighlight>
 
<p>Create environment with Python-3.8</p>
<syntaxhighlight lang="console">
python3.8 -m venv MatLab
</syntaxhighlight>
 
<p>Activate environment</p>
<syntaxhighlight lang="console">
source MatLab/bin/activate
</syntaxhighlight>
 
<p>Install MatLab Kernel</p>
<syntaxhighlight lang="console" line>
pip install matlab_kernel
python -m matlab_kernel install
</syntaxhighlight>
 
<p>Install MatLab Python API</p>
<syntaxhighlight lang="console" line>
cd $MATLAB_ROOT_DIR/extern/engines/python
python setup.py install
</syntaxhighlight>
 
=== C++ ===
==== Install Miniconda ====
<p>Download install script</p>
<syntaxhighlight lang="console">
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
</syntaxhighlight>
 
<p>Run install script and reboot</p>
<syntaxhighlight lang="console">
bash Miniconda3-latest-Linux-x86_64.sh
</syntaxhighlight>
 
<p>Add Environment variable</p>
<syntaxhighlight lang="console">
echo "export PATH=$PATH:~/miniconda3/bin">> ~/.bashrc
</syntaxhighlight>
 
<p>Reload shell</p>
<syntaxhighlight lang="console">
source ~/.bashrc
</syntaxhighlight>
 
<p>Update conda</p>
<syntaxhighlight lang="console">
conda update --all
</syntaxhighlight>
 
==== Install Xeus-Cling ====
<p>Install xeus-cling</p>
<syntaxhighlight lang="console">
conda install xeus-cling -c conda-forge
</syntaxhighlight>
 
==== Add kernel ====
<p>Add C++11 kernel</p>
<syntaxhighlight lang="console">
jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp11
</syntaxhighlight>
 
<p>Add C++14 kernel</p>
<syntaxhighlight lang="console">
jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp14
</syntaxhighlight>
 
<p>Add C++17 kernel</p>
<syntaxhighlight lang="console">
jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp17
</syntaxhighlight>
 
<p>Restart Jupyter</p>
<syntaxhighlight lang="console">
systemctl restart jupyter
</syntaxhighlight>
 
=== AI ===
==== Create environment ====
<p>Install venv</p>
<syntaxhighlight lang="console">
apt install python3-venv
</syntaxhighlight>
 
<p>Create folder where environment should be located at</p>
<syntaxhighlight lang="console" line>
mkdir /home/venv
cd /home/venv
</syntaxhighlight>
 
<p>Create Keras-Environment</p>
<syntaxhighlight lang="console">
python3 -m venv Keras
</syntaxhighlight>
 
<p>Activate environment</p>
<syntaxhighlight lang="console" line>
cd /home/venv
source Keras/bin/activate
</syntaxhighlight>
 
<p>Install iPython Kernel</p>
<syntaxhighlight lang="console">
pip install ipykernel matplotlib
</syntaxhighlight>
 
<p>Add virtual environment to jupyter</p>
<syntaxhighlight lang="console">
python -m ipykernel install --name=Keras
</syntaxhighlight>
 
<p>Deactivate with:</p>
<syntaxhighlight lang="console">
deactivate
</syntaxhighlight>
 
==== Tensorflow ====
<p>Activate environment</p>
<syntaxhighlight lang="console" line>
cd /home/venv
source Keras/bin/activate
</syntaxhighlight>
 
<p>Install tensorflow</p>
<syntaxhighlight lang="console">
pip install tensorflow
</syntaxhighlight>
 
<p>Verify tensorflow installation:</p>
<syntaxhighlight lang="console">
pip show tensorflow
</syntaxhighlight>
 
<p>Update tensorflow with</p>
<syntaxhighlight lang="console">
pip install --upgrade tensorflow
</syntaxhighlight>
 
<p>Deactivate with:</p>
<syntaxhighlight lang="console">
deactivate
</syntaxhighlight>
 
==== Keras ====
<p>Activate environment</p>
<syntaxhighlight lang="console" line>
cd /home/venv
source Keras/bin/activate
</syntaxhighlight>
 
<p>Install Keras</p>
<syntaxhighlight lang="console">
pip install keras
</syntaxhighlight>
 
<p>Verify Keras installation:</p>
<syntaxhighlight lang="console">
pip show keras
</syntaxhighlight>
 
<p>Update Keras with</p>
<syntaxhighlight lang="console">
pip install --upgrade keras
</syntaxhighlight>
 
<p>Deactivate with:</p>
<syntaxhighlight lang="console">
deactivate
</syntaxhighlight>
 
== Sources ==
* [https://github.com/jupyterhub/jupyterhub Jupyterhub]
* [https://web.mat.upc.edu/joaquim.puig/post/installing-sagemath-kernel-for-jupyter/ SageMath on Jupyterhub]
* [https://queirozf.com/entries/jupyter-kernels-how-to-add-change-remove Manage Jupyterhub Kernels]
* [https://jupyterhub.readthedocs.io/en/latest/getting-started/authenticators-users-basics.html Basic User Management]

Latest revision as of 17:46, 15 July 2022

Juypter.png

Network

IP: 192.168.88.18
MAC: 6A:1B:42:9D:D6:4B
Domain: jupyter.flowerhouse.at

System

OS: Debian Bullseye
RAM: 4096MB
Cores: 4
Privileged: No

The Jupyter-LXC is reachable under 192.168.88.18 which is located in the ServerVLAN.

The subdomain is jupyter.flowerhouse.at which is handled by the ReverseProxy.

Basic Setup

Installation

Install required packages

apt install python3-pip npm

Install JupyterHub, responsible for managing multiple users

python3 -m pip install jupyterhub

Install Proxy needed by JupyterHub

npm install -g configurable-http-proxy

Install webinterface and notebook

python3 -m pip install jupyterlab notebook

Configuration

Create config file

Create folder for config

mkdir /etc/jupyterhub

Generate config file

jupyterhub --generate-config -f /etc/jupyterhub/jupyterhub_config.py

Create systemd service

Create service file

nano /etc/systemd/system/jupyter.service

Add this content to file

[Unit]
Description=Jupyterhub
After=syslog.target network.target

[Service]
User=root
Environment="PATH=/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/anaconda3/bin"
ExecStart=jupyterhub -f /etc/jupyterhub/jupyterhub_config.py

[Install]
WantedBy=multi-user.target

Reload systemd

systemctl daemon-reload

Start Jupyter and check if its running

systemctl start jupyter
systemctl status jupyter

Start Jupyter at boot

systemctl enable jupyter

Create/Add user

Create new user

adduser [username]

Extensions/Kernels

SageMath

Install SageMath Kernel

apt install sagemath

MatLab

Install MatLab

Mount iso from USB via Proxmox to LXC

mount -o loop,ro /media/sda/MatLab/Matlab910R2021a_Lin64.iso /rpool/data/subvol-111-disk-0/mnt

Create installation procedure file and add this

nano /input.txt

Start installation with:

cd /mnt
./install -inputFile /input.txt

Check log if installation was successful

nano ./InstallMatLab.log

Delete log

rm ./InstallMatLab.log

Unmount ISO in ProxMox

umount /rpool/data/subvol-111-disk-0/mnt

Open .bashrc

nano /etc/bash.bashrc

Add to end of file

export MATLAB_ROOT_DIR="/usr/local/Polyspace/R2021a"

Apply changes by executing the command

source /etc/bash.bashrc

Check if if works:

echo $MATLAB_ROOT_DIR

Install Python-3.8

Download Python-3.8

wget https://www.python.org/ftp/python/3.8.12/Python-3.8.12.tar.xz
tar -xf Python-3.8.12.tar.xz
rm Python-3.8.12.tar.xz
mv Python-3.8.12 /opt/Python-3.8.12

Install requirements

apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl libbz2-dev -y

Configure

cd /opt/Python-3.8.12
./configure --enable-optimizations --enable-shared

Compile Python-3.8, x = CPU Cores to use

make -j x

Install Python

make altinstall

Delete compile files

cd ..
rm -r Python-3.8.12

Configure dynamic linker

ldconfig /usr/local/lib

Install Kernel

Enter environment folder

cd /home/venv

Create environment with Python-3.8

python3.8 -m venv MatLab

Activate environment

source MatLab/bin/activate

Install MatLab Kernel

pip install matlab_kernel
python -m matlab_kernel install

Install MatLab Python API

cd $MATLAB_ROOT_DIR/extern/engines/python
python setup.py install

C++

Install Miniconda

Download install script

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Run install script and reboot

bash Miniconda3-latest-Linux-x86_64.sh

Add Environment variable

echo "export PATH=$PATH:~/miniconda3/bin">> ~/.bashrc

Reload shell

source ~/.bashrc

Update conda

conda update --all

Install Xeus-Cling

Install xeus-cling

conda install xeus-cling -c conda-forge

Add kernel

Add C++11 kernel

jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp11

Add C++14 kernel

jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp14

Add C++17 kernel

jupyter kernelspec install ~/miniconda3/share/jupyter/kernels/xcpp17

Restart Jupyter

systemctl restart jupyter

AI

Create environment

Install venv

apt install python3-venv

Create folder where environment should be located at

mkdir /home/venv
cd /home/venv

Create Keras-Environment

python3 -m venv Keras

Activate environment

cd /home/venv
source Keras/bin/activate

Install iPython Kernel

pip install ipykernel matplotlib

Add virtual environment to jupyter

python -m ipykernel install --name=Keras

Deactivate with:

deactivate

Tensorflow

Activate environment

cd /home/venv
source Keras/bin/activate

Install tensorflow

pip install tensorflow

Verify tensorflow installation:

pip show tensorflow

Update tensorflow with

pip install --upgrade tensorflow

Deactivate with:

deactivate

Keras

Activate environment

cd /home/venv
source Keras/bin/activate

Install Keras

pip install keras

Verify Keras installation:

pip show keras

Update Keras with

pip install --upgrade keras

Deactivate with:

deactivate

Sources

Retrieved from "https://wiki.flowerhouse.at/index.php?title=Jupyter&oldid=894"