wirawan0/WP-hpc-tools
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Files
4a1b5d0a69cd9fe5334d44ec69315325f46a7a4f
WP-hpc-tools/lmod/ipython/startup
History
Wirawan Purwanto 4a1b5d0a69 * Saved developmental notebooks to devise/test the "lmod_python_fix.py". 
These were done on Wahab with legacy (lmod-based) Python suite.

  Sigs:
    # -rw-r--r-- 1 wpurwant users 56588 2020-06-09 17:35 debug_ipython_start_1.ipynb
    # -rw-r--r-- 1 wpurwant users 20395 2020-06-09 17:34 debug_ipython_start_2.ipynb
    # -rw-r--r-- 1 wpurwant users 21965 2020-06-09 19:20 debug_ipython_start_3.ipynb
    # -rw-r--r-- 1 wpurwant users 10717 2020-06-09 19:23 debug_ipython_start_4_vanilla.ipynb
    # -rw-r--r-- 1 wpurwant users 37041 2020-06-09 19:59 debug_ipython_start_5.ipynb
    # b061fbc819806bc18d48287a9654bcc5  debug_ipython_start_1.ipynb
    # bc262fc0d3cd290da1f46a242a6fcccb  debug_ipython_start_2.ipynb
    # 5f289c909e967096712b3196a19979b7  debug_ipython_start_3.ipynb
    # b5f0cd9251e325b0c7e0222ba4cfc037  debug_ipython_start_4_vanilla.ipynb
    # abc440a885da0872d5bc215adfc3edd7  debug_ipython_start_5.ipynb
2022-02-22 10:44:05 -05:00
..
testbeds
* Saved developmental notebooks to devise/test the "lmod_python_fix.py".
2022-02-22 10:44:05 -05:00
000-odurc-lmod.py.orig
* Saved the original lmod_python.py and
2022-02-22 10:11:47 -05:00
lmod_python_fix.py
* Third attempt to fix lmod "module" for ipython ("fix3")
2022-02-22 10:28:14 -05:00
lmod_python.py.orig
* Saved the original lmod_python.py and
2022-02-22 10:11:47 -05:00
README.txt
* Saved README.txt of the lmod_python fix (notes written to Minhao, 2020-06-09).
2022-02-22 10:41:11 -05:00

README.txt 

Date: 2020-06-09

Purpose: troubleshoot & fix the ipython startup file
which caused issue.

The original setup (combined `000-odurc-lmod.py` and `lmod_python.py`)
have several issues:

* On Turing (vanilla ipython), could not find the module command.

* On Wahab (vanilla ipython), invoking the "module" command resulted
  in an error due to unset `LMOD_MANUAL_LD_LIBRARY_PATH` env var.

* On Wahab/OOD, the "module" command messed up all the symlinks in the
  specially prepared library dir (LMOD_MANUAL_LD_LIBRARY_PATH).
  As a result, all symlinks become circular links.

I fixed these files with the following improvements:

1) Allow the setup to work on both Wahab and Turing. (The new
   `lmod_python.py` needs to be installed somewhere on Turing;
   I made up the location to be
   `/cm/shared/applications/lmod/lmod/scripts/init`.

2) Fix the indefinite lengthening of `sys.path` as a result of
   "module" command invocation: now only the *changes* are updated
   onto `sys.path`. Still not perfect (see notes in the Python code)
   but should work ok for most cases.

3) As a result of fix #2 above, when we invoke `module("unload",
   A_MODULE)`, the paths associated with `A_MODULE` will also be removed.

4) Fix the "circular links" problem. If `LMOD_MANUAL_LD_LIBRARY_PATH`
   is not specified, it quietly ignores the symlink update process.

See if you like these changes---I suggest implementing them soon.

I found that the existing `000-odurc-lmod.py` is always overwritten
whenever I launch Jupyter on Wahab OOD. It was due to `before.sh` of
the OOD Jupyter notebook setup ... grrr. I know you assume that users
won't mess with this file (but I did, while doing all this testing).
I'll make my own workaround.