Useful links

• Smilei doc: https://smileipic.github.io/Smilei
• Tutorials: https://smileipic.github.io/tutorials

General commands

• Directories
  • Show current directory: pwd
  • List files in current directory: ls
  • Change directory: cd path/to/other/folder
  • Go to your home directory: cd ~
  • Make a new folder: mkdir new_folder
  • Remove a folder: rm -r my_folder
  • Copy a folder: cp -r my_folder new_folder
• Files
  • Remove a file: rm my_file
  • Copy a file: cp my_file new_file
  • Move a file: mv my_file other_folder/new_name
• Text editors
  • emacs some_file &
  • vim some_file
  • nano some_file
• Get a file from the web: wget some_URL

Running Smilei on the RUCHE cluster

• Connect to the cluster

  ssh -Y mylogin@ruche.mesocentre.universite-paris-saclay.fr

• Commands to manage jobs:
  • Submit a job: sbatch submission_script.sh
  • Check job status: squeue -u username
  • Cancel a job: scancel job_id
• Get Smilei

  git clone https://github.com/SmileiPIC/Smilei.git --depth 1

• Prepare the post-processing tool happi

  cd ~/Smilei
  source /gpfs/workdir/labotm/miniconda3/smilei_env.sh
  make happi

• Compile Smilei:
  • Copy the following in a new file compile.sh in the Smilei folder.
  • Start the compilation with the command sbatch compile.sh

  #!/bin/bash
  #SBATCH --job-name=smilei
  #SBATCH --nodes=1
  #SBATCH --output=smilei.out
  #SBATCH --time=00:05:00
  #SBATCH --ntasks=40
  ##SBATCH --partition=cpu_short
  
  source /gpfs/workdir/labotm/miniconda3/smilei_env.sh
  
  set -x
  cd ${SLURM_SUBMIT_DIR}
  
  make -j 40 machine=ruche

• Run smilei_test:

  ~/Smilei/smilei_test input.py

  Do not forget to replace input.py by your input file!
• Run Smilei without parallelization:

  export OMP_NUM_THREADS=4
  ~/Smilei/smilei input.py

• Launch a parallel simulation:
  • Copy the following in a new file run.sh in the folder of your input file.
  • Modify values according to the needs of the simulation:
    • Replace my_input_file.py by the name of your input file
    • ntasks = the number of MPI processes
    • cpus-per-task = the number of threads per MPI process
    • OMP_SCHEDULE can be set to static or dynamic
  • Start the simulation, in the folder of your input file, with sbatch run.sh
  • The result will appear in the folder $WORKDIR/simulation_results

  #!/bin/bash
  #SBATCH --job-name=mpi_openmp
  #SBATCH --output=smilei.out
  #SBATCH --time=00:05:00
  #SBATCH --ntasks=1           # number of MPI processes
  #SBATCH --cpus-per-task=16   # number of threads per MPI process
  
  INPUT_FILE=my_input_file.py
  
  source /gpfs/workdir/labotm/miniconda3/smilei_env.sh
  set -x
  cd ${SLURM_SUBMIT_DIR}
  
  # Dynamic scheduling for patch-spec loop
  export OMP_SCHEDULE=dynamic
  # number of OpenMP threads per MPI process
  export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
  # Binding OpenMP Threads of each MPI process on cores
  export OMP_PLACES=cores
  
  # execution
  F=$WORKDIR/simulation_results
  rm -fr $F
  mkdir $F
  cp $INPUT_FILE $F/input.py
  cd $F
  srun ~/Smilei/smilei input.py
  

• Start Python & happi for post-processing If you haven't done so already, load the environment:

  source /gpfs/workdir/labotm/miniconda3/smilei_env.sh

  Then you may launch: ipython in which you import happi

Using Paraview on RUCHE

• Connect to the visualization system in your web browser

  https://ruche.mesocentre.universite-paris-saclay.fr:8443

  Enter your RUCHE credentials
• On the left menu, click on the first "Linux" item Wait for the session to start, then click on the screen that has appeared.
• Open a terminal Right-click on the desktop > Open terminal
• Load & launch Paraview

  paraview/5.8.0/gcc-9.2.0
  paraview &