Hands-on #2: The UNIX Time-Sharing System

March 12, 2015

Complete the following hands-on assignment. Do the activities described, and submit your solutions for following questions to ftp://535603986:public@public.sjtu.edu.cn/upload/hands-on-2 Please submit a doc or a pdf with title like 513037xxxx.pdf. You are free to use either Chinese or English. Due time is 2015-3-24 23:59.

This assignment is longer than the DNS one. Before attempting this hands-on, you should read The Unix Time-Sharing System. You might also find Section 2.5 of the book useful for questions 8-17.

With any problems, contact Zheng Yang(tomsun.0.7@gmail.com).

Warmup

Log into a Linux machine (because we can hardly get a UNIX machine these days) and get access to the command prompt in terminal. Running command with your favorite shell (like bash, zsh, etc.) and you'wll get prompt like:

tom@tom-pc $

We'll start off with an extremely simple example that most of you are probably familiar with already:

tom@tom-pc $ cd /bin
tom@tom-pc $ ls -1 | more

Here, we are first changing into the /bin directory, which contains many of the executable commands for the system. The command ls -1 gives us a listing of all the files in the current directory with one file per line. (Note that -1 is the numeral "one", not the letter "L".) We then pipe the output from ls to the command more, which displays the results one page at a time. (Press the space bar to show the next page.) You can refer to the manual pages for ls and moreto see more details and options for each command. Manual pages let you read information about various commands on UNIX systems; to use them,run

tom@tom-pc $ man command

where command is the command you are interested in. If you are unfamiliar with manual pages, you may want to try running

tom@tom-pc $ man man

for information on the man command itself. Keep in mind that the manual pages for basic commands vary from system to system (much as the commands themselves do).

Now, try this:

tom@tom-pc $ cd /bin
tom@tom-pc $ ls -1 | grep p | more

This runs the same ls -1 command, but only lists the executable files which happen to contain the letter "p" somewhere in their names.

The point here is to observe that you can chain together multiple commands using the pipe character (|), and the output from each command will be passed to the input of the next, in left-to-right order. This allows you to treat any command that uses standard input and output as a primitive from which you can build more complex and useful commands.

Building Blocks

Now, we'd like you to figure out on your own how to solve some problems by chaining different commands together.

If you aren't already familiar with these commands, you may want to briefly skim through their man pages to familiarize yourself with what they do. You will probably need to use some of the options for the different commands in order to solve these problems.

Here are the commands you may find useful:

cat
fmt
grep
head
ls
ps
sort
tail
top
wc
yes

Pipe questions

For each of the outputs listed below, find one sequence of commands connected by pipes that produces the output. For each problem, turn in the command sequence that you used to generate the requested output. (Do NOT turn in the output itself.)

Question 1: A listing of all processes that you are currently running on the Athena machine you are using, sorted by the command name in alphabetical order (i.e. a process running acroread should be listed before a process runningzwgc). The output should consist only of the processes you are running, and nothing else (i.e. if you are running 6 processes, the output should only have 6 lines).

Question 2: The number of words in the file /usr/share/dict/words (*) which do not contain any of the letters a, e, i, o, or u.

[* Note: On some Unix/Linux systems, the dictionary has the filename /usr/dict/words]

Question 3: A 5x6 matrix of entries of alternating 1's and 0's. It should look like this:

Question 4: A "long" listing of the smallest 5 files in the /etc directory whose name contains the string ".conf", sorted by increasing file size.

Now we'd like to explore something slightly different, having to do with file redirection as discussed in section 6.2-6.4 of the paper. The authors explain that the following two commands are functionally equivalent (except that you have to remove the temp file afterwards in the second case):

tom@tom-pc $ ls | head -1
tom@tom-pc $ ls > temp; head -1 < temp

Question 5: Try the above commands in a few different directories. What happens if you try both of the commands in the /etc directory on your Linux distribution? How else can the second command not produce the same output as the first? Can you think of any negative side effects that the second construction might cause for the user? You might want to think about the commands you used to solve the first four questions and consider their behavior.

The UNIX paper authors explain in section 6.3 that a user can type two commands together in parenthesis separated by a semicolon, and redirect the output to a file. The file will then contain the concatenation of the two commands. The example from the paper is roughly:

tom@tom-pc $ (date ; ls) > temp1 &

Note that this example uses the & operator to run the process asynchronously. That is, the shell will accept and run another command without waiting for the first to finish. The authors also mention that one can use the & operator multiple times on one line. For example, we can do almost the same thing:

tom@tom-pc $ (date & ls) > temp2 &

See if you can figure out for yourself what exactly the difference is between using ";" and "&" in the examples above.

Let's explore this for ourselves. First, we will write a very simple variation on the "yes" program that you encountered earlier on in this assignment. To do so, we will use the "command file" functionality described in section 6.4 of the paper. Most people call these command files "shell scripts", since they are essentially simple scripts that are executed by the shell.

First, start up a copy of emacs(*) editing a new file called "myyes".

(*) Of course you can use any editors you like to open this file.

tom@tom-pc $ emacs myyes

Now, enter the following lines into your file:

#!/bin/sh

echo y
sleep 1
echo n

Save the file and exit emacs. If you don't already know what the echo and sleep commands do, look them up in the man pages. Lastly, make the file executable by running the following command:

tom@tom-pc $ chmod a+rx myyes

Now let's try running the following two commands:

tom@tom-pc $ (./myyes ; ./myyes) > temp3
tom@tom-pc $ (./myyes & ./myyes) > temp4

Note: Some fast multicore machines may occasionally give unexpected answers, where some output is lost - originally unexpected even to the TA. If this occurs for you on your home machine, you may want to think briefly about why this occurs.

Question 6: Compare the two temp files. Based on your understanding of file I/O in UNIX, what is going on here, and why? Is this different from what you would expect? (If there is more than one difference between the two files, it is the ordering of the letters y and n that we are interested in).

Question 7: The paper describes the Unix system call interface in some detail. In particular, the read and write system calls do not take the offset as an argument. Why did the Unix designers not include the offset as an argument to read and write? How would an application write to a specific offset in a file?

Looking Around

When you log in, the system sets your current working directory to your home directory, your personal name space where you can create your own files, directories and links. You can view the contents of your current working directory with the ls command (see above). Use the pwd command to learn the absolute path of your current working directory. This will tell you where you are in the directory name space even if you move around in the directories.

tom@tom-pc $ pwd

The output of pwd reveals where the Athena administrators store your home directory. For example, /home/tom tells us that the your home directory is stored as a user in the Athena name space.

The stat program reports detailed information about a file including its inode number, link count, file type and other metadata. To use it, type stat followed by a file name at the command prompt. Run stat on your home directory:

tom@tom-pc $ stat .

Creating Directories and Files

Now create a directory named cse-handson2 in your home directory using the mkdir command. You can learn more about the mkdir command with man mkdir.

tom@tom-pc $ mkdir cse-handson2

Use ls to verify that the new directory exists. Now change your current working directory to your new cse-handson2directory using the cd command and verify that your working directory has changed using pwd.

tom@tom-pc $ cd cse-handson2
tom@tom-pc $ pwd

View the contents of your new directory using ls -a -l. ls normally hides the directories "." and "..", but the -a option forces it to show them.

tom@tom-pc $ ls -a -l

Question 8: Change to the '.' entry in your new directory. What happens to your working directory? Next, change to the '..' entry. What happens to your working directory?

Question 9: Describe a scenario where you might need to use the '.' directory.

Change your current directory back to your new cse-handson2 directory and stat the current directory; note the link count. Now create a couple files in your new directory using the touch command and stat the directory again.

tom@tom-pc $ stat .
tom@tom-pc $ touch foo bar
tom@tom-pc $ ls
tom@tom-pc $ stat .

Question 10: What has changed in the stat output and why has it changed?

Now create a subdirectory baz in cse-handson2 and stat the directory once more.

tom@tom-pc $ mkdir baz
tom@tom-pc $ stat .

Question 11: What has changed in the stat output this time and why has it changed? Why does the link count only change when you create a new directory?

Creating Links

The ln command can create both hard links and soft (symbolic) links. Read the man page for more information.

tom@tom-pc $ man ln

First stat your file foo and read the output information. Then create a hard-link named foo-lnk and stat both foo andfoo-lnk.

tom@tom-pc $ stat foo
tom@tom-pc $ ln foo foo-lnk
tom@tom-pc $ stat foo
tom@tom-pc $ stat foo-lnk

Note that everything about foo and foo-lnk is identical except for their names. If you modify foo you will see the modifications in foo-lnk.

tom@tom-pc $ echo Hello >> foo
tom@tom-pc $ cat foo-lnk

Now create a symbolic link to foo and note that the symbolic link differs from the original file in several ways. Creating the symbolic link does not increase the link count of foo and the symbolic link does not share an inode with foo.

tom@tom-pc $ stat foo
tom@tom-pc $ ln -s foo foo-slnk
tom@tom-pc $ stat foo
tom@tom-pc $ stat foo-slnk

Question 12: One reason for supporting symbolic links is to allow linking from one disk to another disk, but you can also create a symbolic link to a file on the same disk. Name one advantage and one disadvantage of using symbolic links on the same disk.

Now change your shell to sh with the command:

tom@tom-pc $ sh

Then let's prepare our environment for some funny things:

tom@tom-pc $ mkdir -p a/b
tom@tom-pc $ touch a/b/c
tom@tom-pc $ mkdir d && ln -s ../a/b d/e
tom@tom-pc $ cd d/e
tom@tom-pc $ ls

We'll get c, which is under a/b directory. Then, let's get ".." involved:

tom@tom-pc $ ls ..
tom@tom-pc $ cd ../b

Question 13: What happened? Why?

Now you can switch back to your original shell by exit:

tom@tom-pc $ exit

If you repeat the above command sequence, you might get different results, which are dependent on implementation of your shells.

Question 14: How would you change the file system to make this command (cd d/e; cd ../b) actually change to your current working directory (a/b or equally d/e)?

The Search Path

Now, download this to your home directory (commonly known as ~) and change directory to decompressed folder:

tom@tom-pc $ tar -zxf ~/toys.tar.gz
tom@tom-pc $ cd ~/toys

The UNIX shell has a configuration variable named PATH that tells the shell where to look in the file system for programs we type on the command line. You can see your PATH variable with this command:

tom@tom-pc $ echo $PATH

You can configure your shell to search the current working directory by adding '.' to the PATH using these commands:

tom@tom-pc $ export OLDPATH=$PATH
tom@tom-pc $ export PATH=.:$PATH

OK, since everything is well-prepared, we'll do something interesting:

tom@tom-pc $ demo

Oh no, something terrible just happened! Just kidding, the demo program did not actually do anything. Let's drink a cup of coffee to calm down and verify that nothing happened with ls.

tom@tom-pc $ ls -l /

Question 15: What happened to ls? Why isn't it listing files like it did before? (Hint: set your path back to its original state: export PATH=$OLDPATH)

Usually, it is a bad idea of have '.' in your PATH, because it is easy to run the wrong programs by accident. Instead, you can use '.' explicitly to run programs in your working directory like this:

tom@tom-pc $ ./demo

System calls

In this final question we take a quick peek the systems calls that a program issues to the operating systems using the program strace. Run the following command:

tom@tom-pc $ strace ls

strace (you may need to install it from repository of your Linux distributions, e.g.: apt-get install strace) shows all the system calls that ls makes. Look for one of the write systems calls, and answer the following question:

Question 16: What does 1 represent in the first argument of write?

You may have to consult section 3.6 of the Unix paper to answer the question.

For Fun (not required)

Question 17: Chapter 2.5 describes how (in UNIX) a file's inode maps to the file's data blocks using direct pointers and indirect blocks. An alternative strategy, as used in FAT, stores a file's data as a linked list of FAT entries, each corresponding to a block (cluster). Name one advantage and one disadvantage of the linked list strategy.

Question 18: Some shells like bash try to make '..' always work properly, namely, cd d/e; cd ../b will place you in your current directory. Does bash always get this behavior correct?