Linux Shell and Bash

Bash Tutorial

A brief overview of Bash, on your way to becoming a Linux expert. When a computer boots up, a kernel (MacOS, Windows, Linux) is started. This kernel provides a shell, or terminal, that allows user to interact with a most basic set of commands. Typically, the casual user will not interact with the shell/terminal as a Desktop User Interface is started by the computer boot up process. To activate a shell directly, users will run a “terminal” through the Desktop. VS Code provides ability to activate "terminal" while in the IDE.

Variable Prerequisites

Setup bash shell dependency variables for this page. Variables are one of the first aspects of programming. Variables have "name" and a "value".

Hack Note: Change variables to match your student project.

Define variable

The following code cell defines 3 variables and assigns each a value. There are some extra commands, called a HERE document, that write these variables to a file. This is so we can use these variables over and over below. When you run this script, it sets these variables with their respective values: project_dir is set to the path to the vscode directory in your home directory. Project is set to the combination of project_dir and the string teacher, which represents a directory path. Project_repo is set to the GitHub repository URL. These variables can be used later in your script or in subsequent commands and scripts to reference the specified values.

#!/bin/bash

# Set project directory
project_dir="$HOME/vscode"  # Change 'vscode' to your desired directory name

# Define project path by combining project_dir and 'teacher'
project="$project_dir/teacher"

# Specify the GitHub repository URL
project_repo="https://github.com/nighthawkcoders/teacher.git"

Output the value of a variable

The following code cell outputs the value of the variables, using the echo command. For visual understanding in the output, each echo command provide a title before the $variable. When you run this script, it will include the variables defined in /tmp/variables.sh into your current shell session, and then it will display the values of those variables. This allows you to reuse and access those variables in your current shell environment.

%%script bash

# Extract saved variables
source /tmp/variables.sh

# Output shown title and value variables
echo "Project dir: $project_dir"
echo "Project: $project"
echo "Repo: $project_repo"

Project Setup and Analysis with Bash Scripts

The bash scripts that follow automate what was done in the setup procedures. The purpose of this is to show that many of the commands we performed can be added to a script, then performed automatically.

Pull Code

Pull code from GitHub to your machine. This is a bash script, a sequence of commands, that will create a project directory and add the “project” from GitHub to the vscode directory. There is conditional logic to make sure that clone only happen if it does not (!) exist. Here are some key elements in this code…

cd command (change directory), remember this from terminal session
if statements (conditional statement, called selection statement by College Board), code inside only happens if condition is met

Explanation of the Code:

The provided Bash script begins by sourcing external variables from the file /tmp/variables.sh. This step allows it to access and utilize the values you’ve previously defined.

Afterward, the script changes its current working directory to the user’s home directory (cd ~). This step is essential to ensure the subsequent operations are performed relative to the home directory.

The script then employs conditional statements to manage the creation of a project directory ($project_dir) and the cloning of a Git repository ($project_repo) if they do not already exist.

First, it checks if the project directory ($project_dir) exists. If it doesn’t exist, the script creates it using the mkdir -p command, ensuring that any necessary parent directories are created as well.

Next, it checks for the existence of the project directory ($project). If the project directory does not exist, the script proceeds to clone the Git repository specified by $project_repo into the project directory. This is accomplished by navigating to the project_dir, performing the Git clone operation, and returning to the home directory.

Finally, the script provides status messages to indicate whether the project directory and repository were created or already existed.

In summary, this script automates the process of setting up a project directory and cloning a Git repository, ensuring that the necessary directories are created and the repository is cloned only when they are not already present.

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Using conditional statement to create a project directory and project"

cd ~    # start in home directory

# Conditional block to make a project directory
if [ ! -d $project_dir ]
then 
    echo "Directory $project_dir does not exists... makinng directory $project_dir"
    mkdir -p $project_dir
fi
echo "Directory $project_dir exists." 

# Conditional block to git clone a project from project_repo
if [ ! -d $project ]
then
    echo "Directory $project does not exists... cloning $project_repo"
    cd $project_dir
    git clone $project_repo
    cd ~
fi
echo "Directory $project exists." 

Look at files Github project

All computers contain files and directories. The clone brought more files from cloud to your machine. Review the bash shell script, observe the commands that show and interact with files and directories. These were used during setup.

“ls” lists computer files in Unix and Unix-like operating systems
“cd” offers way to navigate and change working directory
“pwd” print working directory
“echo” used to display line of text/string that are passed as an argument

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
pwd

echo ""
echo "list top level or root of files with project pulled from github"
ls

Look at file list with hidden and long attributes

Most linux commands have options to enhance behavior. The enhanced listing below shows permission bits, owner of file, size and date.

ls reference

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
pwd

echo ""
echo "list all files in long format"
ls -al   # all files -a (hidden) in -l long listing

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Look for posts"
export posts=$project/_posts  # _posts inside project
cd $posts  # this should exist per fastpages
pwd  # present working directory
ls -l  # list posts

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Look for notebooks"
export notebooks=$project/_notebooks  # _notebooks is inside project
cd $notebooks   # this should exist per fastpages
pwd  # present working directory
ls -l  # list notebooks

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Look for images in notebooks, print working directory, list files"
cd $notebooks/images  # this should exist per fastpages
pwd
ls -l

Look inside a Markdown File

“cat” reads data from the file and gives its content as output

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Navigate to project, then navigate to area wwhere files were cloned"

cd $project
echo "show the contents of README.md"
echo ""

cat README.md  # show contents of file, in this case markdown
echo ""
echo "end of README.md"

Env, Git and GitHub

Env(ironment) is used to capture things like path to Code or Home directory. Git and GitHub is NOT Only used to exchange code between individuals, it is often used to exchange code through servers, in our case deployment for Website. All tools we use have a behind the scenes relationships with the system they run on (MacOS, Windows, Linus) or a relationship with servers which they are connected to (ie GitHub). There is an “env” command in bash. There are environment files and setting files (.git/config) for Git. They both use a key/value concept.

“env” show setting for your shell
“git clone” sets up a director of files
“cd $project” allows user to move inside that directory of files
“.git” is a hidden directory that is used by git to establish relationship between machine and the git server on GitHub.

%%script bash

# This command has no dependencies

echo "Show the shell environment variables, key on left of equal value on right"
echo ""

env

%%script bash

# Extract saved variables
source /tmp/variables.sh

cd $project

echo ""
echo "show the secrets of .git"
cd .git
ls -l

echo ""
echo "look at config file"
cat config

Advanced Student Request - Make a file in Bash

This example was requested by a student (Jun Lim, CSA). The request was to make jupyer file using bash, I adapted the request to markdown. This type of thought will have great extrapolation to coding and possibilities of using List, Arrays, or APIs to build user interfaces. JavaScript is a language where building HTML is very common.

To get more interesting output from terminal, this will require using something like mdless (https://github.com/ttscoff/mdless). This enables see markdown in rendered format.

On Desktop Install PKG from MacPorts

In Terminal on MacOS

Install ncurses

gem install mdless

Output of the example is much nicer in “jupyter”

Explanation of Code:

The Bash script provided operates as follows: It changes the working directory to /tmp. Defines a file variable and removes the file if it exists. Appends Markdown content to file using tee and echo. Creates an array actions and formats its elements. Iterates through actions, appends them to file, and adds a section break. Displays file information (details, word count) and attempts to render the Markdown content using mdless. Removes the temporary file. This script illustrates the creation and manipulation of a Markdown file with formatting, content addition, and rendering attempts using mdless.

%%script bash

# This example has error in VSCode, it run best on Jupyter
cd /tmp

file="sample.md"
if [ -f "$file" ]; then
    rm $file
fi

tee -a $file >/dev/null <<EOF
# Show Generated Markdown
This introductory paragraph and this line and the title above are generated using tee with the standard input (<<) redirection operator.
- This bulleted element is still part of the tee body.
EOF

echo "- This bulleted element and lines below are generated using echo with standard output (>>) redirection operator." >> $file
echo "- The list definition, as is, is using space to seperate lines.  Thus the use of commas and hyphens in output." >> $file
actions=("ls,list-directory" "cd,change-directory" "pwd,present-working-directory" "if-then-fi,test-condition" "env,bash-environment-variables" "cat,view-file-contents" "tee,write-to-output" "echo,display-content-of-string" "echo_text_>\$file,write-content-to-file" "echo_text_>>\$file,append-content-to-file")
for action in ${actions[@]}; do  # for loop is very similar to other language, though [@], semi-colon, do are new
  action=${action//-/ }  # convert dash to space
  action=${action//,/: } # convert comma to colon
  action=${action//_text_/ \"sample text\" } # convert _text_ to sample text, note escape character \ to avoid "" having meaning
  echo "    - ${action//-/ }" >> $file  # echo is redirected to file with >>
done

echo ""
echo "File listing and status"
ls -l $file # list file
wc $file   # show words
mdless $file  # this requires installation, but renders markown from terminal

rm $file  # clean up termporary file

Hack Preparation.

Review Tool Setup Procedures and think about some thing you could verify through a Shell notebook.

Come up with your own student view of this procedure to show your tools are installed. It is best that you keep the few things you understand, add things later as you start to understand them.

Name and create blog notes on some Linux commands you will use frequently.

Is there anything we use to verify tools we installed? Review versions?

How would you update a repository? Use the git command line?

Linux Shell and Bash • 31 min read

Description