============================= Basic Linux for new HPC users ============================= .. contents:: :local: -------------- Working with HPC resources requires a basic understanding of Linux systems. Some of our users are new to using Linux-based systems and have asked for introductory materials. Here is a collection for the basic command-line operations needed to get started with the cluster. Using commands in the shell --------------------------- The **shell** is the program from which the user controls everything in a text-interface. When you login to the cluster remotely, you are already in the shell window of the system. If you login to your own system, you are probably on a graphical screen. From there, search for terminal or Ctrl+Alt+T to enter the shell terminal. In the shell, you can start typing commands to perform some action. The default shell on the cluster is **bash**. Useful Shell commands --------------------- Upon login we are greeted with the shell :: ssh user@sunrise.fysikum.su.se Last login: Fri Aug 8 10:14:59 2017 from example.com user@sol-login:~> _ Bash: Files and directories =========================== - Command **pwd** tells me where I am. After login I am in the “home” directory :: pwd /cfs/home/user - I can change the directory with **cd** :: cd tmp/talks /tmp/talks pwd /cfs/home/user/tmp/talks - I can go one level up with **cd ..** - I can return to my HOME folder with **cd** - List the contents with **ls -l** :: user@sol-login:~/tmp/talks ls -l total 237 drwx------ 3 user hpc-users 2048 Aug 17 15:21 img -rw------- 1 user hpc-users 18084 Aug 17 15:21 env.html -rw------- 1 user hpc-users 222051 Aug 17 15:22 remark-latest.min.js Bash: creating directories and files ==================================== - We create a new directory called *results* and change to it :: mkdir results cd results - Creating and editing files - Textfiles can be edited on your local computer and then transferred - Textfiles can also be edited locally using text editor like **nano**/**emacs**/**vim** Copying, moving renaming and deleting ===================================== :: # copy file cp draft.txt backup.txt # recursively copy directory cp -r results backup # move/rename file mv draft.txt draft_2.txt # move/rename directory mv results backup # move directory one level up mv results .. # remove file rm draft.txt # remove directory and all its contents rm -r results File permissions with chmod =========================== In Linux systems all files have a user, a group and a set of privileges which determines what resources a user can access. Every file has three different kind of access: read(r), write(w) and execute(x), as well as three different kind of permissions depending on if the person is the owner(u=user) of the file, in the same group(g) or someone else(o=other). :: chmod g+w file Adds(+) write(w) permissions for group(g) to the *file*. There is another way to set the permissions by using numbers. Assume that each permission equals the number listed below: +----------------------+-----------------------------------------------------+ | Number | Type | +======================+=====================================================+ | 0 | no permissions | +----------------------+-----------------------------------------------------+ | 1 | execute | +----------------------+-----------------------------------------------------+ | 2 | write | +----------------------+-----------------------------------------------------+ | 4 | read | +----------------------+-----------------------------------------------------+ :: chmod 753 file Gives the user the read, write and execute permission(4+2+1), whereas users in the same group get read and execute permissions (4+1) while others get write and execute permissions (2+1). Bash: history and tab completion ================================ - history preserve commands used :: history 689 cd .. 691 cd Documents/ 692 cp -r introduction /cfs/home/user/Documents/Presentations 693 ssh sunrise.fysikum.su.se 694 cd introduction/ 695 ls -l 696 pwd 697 history - If I want to repeat... :: !696 pwd ~/Documents/introduction - Use also the **TAB** key for completion - **CTRL/R** to search for previous commands - Arrows up/down to scroll for earlier commands Bash: finding things ==================== - Extract lines which contain an expression with **grep** :: # extract all lines that contain searchme grep searchme draft.txt - If you do not know what a UNIX command does, examine it with **man** :: man [command] - Find files with **find** :: find ~ | grep lostfile.txt - We can pipe commands and filter results with \| :: grep energy results.out | sort | uniq Bash: Redirecting output ======================== - Print content of a file to screen :: cat test.out - Redirect output to a file :: cat test.out > myfile.txt - Append output to a file :: cat test.out >> myfile.txt Bash: Writing shell scripts =========================== :: #!/bin/bash # here we loop over all files that end with *.out for file in *.out; do echo $file cat $file done We make the script executable and then run it :: # Make it executable chmod u+x my_script # run it ./my_script Arguments to script can be passed by using **$** ================================================ File example :: #!/bin/bash echo "Arg 1 = $1, Arg 2 = $2, Args = $*" :: ./myscript Hello, World! Arg 1 = Hello,, Arg2 = World!, Args = Hello, World! where ``$1`` to ``$n`` denote the first to *n*-th argument. To starting executing such scripts, you would need to start with a text-editor. Choosing a text-editor is a matter of personal choice, the most popular ones being Vim and Emacs. But there are a lot more new and interesting ones. Open your favorite text-editor and copy-paste the file example above and save with file as