Advanced Unix commands interview question

Debugging tools

Debugging a running process

$ strace
Traces the system calls used by program 

It is used to monitor and tamper with interactions between processes and the Linux kernel, which include system calls, signal deliveries, and changes of process state

Attach to processes and begins tracing

Lets you know what’s happening even if there is no debugger or source code and Useful in a live environment

Memory used by a process

$ pmap

pmap reports the memory map of a process

Amount of memory used by the process

Details of the map are also available in the /proc 
Report memory map of a process

Stack trace of a running process

$pstack

prints the stack trace of the running process – quite useful along with strace.

Attaches to the active process given on the command line

Command displays a stack trace for each process.The output is read bottom up.

The pstack command must be run by the owner of the process or by root. You can use the pstack command to determine where a process is hung. The only option that is allowed with this command is the process ID of the process that you want to check.

Symbol names in an object or exe

$nm

Used to  examine binary file retrieve information on symbol names inside an object file or executable file. 

Helps in resolving problems due to name conflicts

If there are poorly defined headers tracking down the offending module becomes easier using nm. 
nm is used to examine binary files and to display the contents of those files, or meta information stored in them, specifically the symbol table

Data-network packet analyzer
$tcpdump
utility that allows you to capture and analyze network traffic going through your system
It allows the user to display TCP/IP and other packets being transmitted or received over a network to which the computer is attached

Display memory usage
$free
command shows the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.


System monitoring Tools
UNIX box needs monitoring how a system is running on a regular basis

Disk Spaces
The df command running on its own & reports current disk space usage.
Option k in the below is for output in kilobytes
$ df -k

Disk spaces for housing the current directory
$ cd /app/addvl/   &&  df  -k

The du command can be used to report the spaces used by the particular directory or file
$ du -k

Open files and Processes opening the files
$ lsof 
Reports a list of all open files and processes that opened them
Open files in the system include Disk files ,Pipes,Network sockets and Devices opened by all processes.

System Performances

The ps commad can be used to observe CPU usage & it reports only the current system state. 
$ps –efx
Report a snapshot of the current processes
use the –o option with ps and get many details such as virtual memory, cpu usage, current state and lot more

The top command can be used for monitoring system‘s dynamic behaviour
 $ top
Displays system summary and details, status of individual tasks (memory, cpu, pid and more)
The top line prints the current time, uptime since the last reboot, users logged in, and the load average. 

The netstat command displays network related data structures.
 $ netstat –i

Netstat gets the information about the status of network connections
Displays the kernel routing tables

Retrieve information on Network interfaces

The ping command can be used to detect whether the system is running or not.
$ ping camgdsd1
Ping uses ICMP(Internet Control Message Protocol) to send an ICMP echo message to the specified host if that host is available then it sends ICMP reply message.


System Activity Reporter (sar) can be used to used for automated system information gathering
$ sar
Collect, report, or save system activity information
Displays the CPU activity and Memory usage can be monitored

Disk I/O activity

Displays the currently active and inactive network interfaces.

$ifconfig

Allows to configure network interfaces (bring interfaces up or down)

Supports a variety of address families and hardware types - IPv4 and IPv6

Reports terminal and disk  I/O  activity and  CPU utilization 
$iostat

Reports the CPU statistics and Input/Output statistics for devices and partitions
Can be used with sar and vmstat 
CPU utilization gives the percentage of idle time with/with out any outstanding I/Os requests (iowait, idle).
Blocks read and written is given for device utilization 
Used to identify performance issues with storage devices, including local disks, or remote disks accessed over network file systems such as NFS

system monitor tool used to collect and show operating system storage input and output statistics

Reports virtual memory statistics of   process, virtual memory, disk, trap, and CPU activity 
$vmstat
It collects and reports data about the system’s memory, swap, and processor resource utilization in real time. It can be used to determine the root cause of performance and issues related to memory use.

System status and User status

The rup command reports the immediate overview of the system
$ rup

The uptime command can be used for load average for a single machine
$ uptime

The w command displays current system activity & user status like who is logged in & what they are doing
$ w
Also
$ w –W

The rusers command displays local network & user details like who is logged on & where 
 $ rusers

The finger displays more information about the user
$ finger

Command to find the number of files owned by each user
$ ls -l |  awk  '{print $3}' |  uniq -c  | sort  -nr


How to find text in Binaries
To read the ASCII text that is inside a binary file we can use strings to extract all the human-readable ASCII text. 

1) Find occurrences of libc in ls
$ strings /bin/ls | grep -i libc 

2) List all ASCII text in ls
$ cat /bin/ls | strings 

3) List all ASCII text in ls
$ strings /bin/ls 

How to keep Files Safe from Accidental Overwriting
Tell the shell to be more careful, as follows
set -o noclobber
If you decide you do not want to be so careful, then turn the option off:
set +o noclobber

The noclobber option tells bash not to overwrite any existing files when you redirect output
$ set +o noclobber
$ echo something > file.txt
$ echo some more > file.txt
$ set -o noclobber
$ echo something > file.txt

bash: file.txt: cannot overwrite existing file

How to find or search for files in Linux?
1)The locate program can find a file based solely on its name
$locate

• The locate program performs a rapid database search of pathnames and yields every name that matches a given substring.
• locate will search its database of pathnames and output any that contain the string tar
• Note: To update the database, execute the updatedb program manually by becoming the root user

[linux~]$ locate tar
/usr/bin/tar

2)The find program searches a given directory (and its sub directories ) for files based on various attributes
$find

To print files that have the file extension .tar  use the command:
find ~ -type f -name '*.tar' -print

3)The whereis command is useful because it not only finds commands, it also finds man pages and configuration files associated with a command. The which command is useful when you’re looking for the actual location of an executable file in your PATH
$whereis

$ whereis man
man: /usr/bin/man /usr/share/man/man1/man1.gz

Explain archiving and Backup tools used in Linux
1) gzip -- Compress or expand files
The gzip program is used to compress one or more files. When executed it replaces the original file with a compressed version of the original
[linuxbox ~]$ ls -l /etc > foo.txt
[linuxbox ~]$ ls -l foo.*
-rw-r--r-- 1 root root 5678 2020-02-14 07:15 foo.txt
[@linuxbox ~]$ gzip foo.txt
The gunzip program, which uncompresses gzip files
[@linuxbox ~]$ gunzip foo.gz

2) bzip2 -- A block sorting file compressor
The bzip2 program is similar to gzip but uses a different compression algorithm that achieves higher levels of compression at the cost of compression speed
[@linuxbox ~]$ ls -l /etc > foo.txt
[@linuxbox ~]$ ls -l foo.txt
-rw-r--r-- 1 root root 5678 2020-01-17 13:51 foo.txt
[@linuxbox ~]$ bzip2 foo.txt
[@linuxbox ~]$ ls -l foo.txt.bz2
-rw-r--r-- 1 root root 27672 2020-01-17 11:31 foo.txt.bz2
[@linuxbox ~]$ bunzip2 foo.txt.bz2

Archiving is the process of gathering up many files and bundling them together into a single large file

1) tar -- Tape archiving utility
A tool for making backup tapes.A tar archive can consist of a group of separate files, one or more directory hierarchies, or a mixture of both. 
[@linuxbox ~]$ tar cf test.tar test

2) zip -- Package and compress files
The zip program is both a compression tool and an archiver. The file format used by the program is familiar to Windows users, as it reads and writes .zip files.
[@linuxbox ~]$ zip -r test.zip test
Extracting the contents of a zip file is straightforward when using the unzip program.
[@linuxbox ~]$ cd foo
[@linuxbox foo]$ unzip ../test.zip

umask: Set Default Permissions
The umask command controls the default permissions given to a file when it is created. It uses octal notation to express a mask of bits to be removed from a file’s mode attributes

[@linuxbox ~]$ rm -f test.txt
[@linuxbox ~]$ umask
0002
[@linuxbox ~]$ > test.txt
[@linuxbox ~]$ ls -l test.txt
-rw-rw-r-- 1 root root 0 2020-03-06 14:53 test.txt

Script for defaulting value to the variable
#!/bin/sh
column=${1:-1}
awk '{print $'$column'}'
(OR)
#!/bin/sh
awk '{print $c}' c=${1:-1}

Command to print the name of each file along with its size:
$ ls -l | awk '$1 !~ /total/ { printf "%-32s %s\n",$9,$5 ; }'| sort –A

Command that counts the number of blank lines in a file:
$ awk ' /^ *$/ { x=x+1 ; print x ; }' 1.sh  | wc -l
(OR)
$ grep  -c ^$ q.sh

Command to count the number lines in a file:
$ awk ' { x=x+1 ; print x ; }' test.sh  | wc –l
(OR)
$ awk '{  print NR ;}' test.sh  | wc -l

Command to print each user's username and home directory
$ awk  -F: '{ printf "%-32s %s\n",$1,$6 ;}'  /etc/passwd

Command to print the size of the file
$ ls -l filename | awk '{print $5}'

Command to print each user's username and home directory
$ awk  -F: '{ printf "%-32s %s\n",$1,$6 ;}'  /etc/passwd

Command for removing only files 
$ ls -l|awk '$1!~/^drwx/{print $9}'|xargs rm

Command for Removing Directories(Be careful when trying this out in your home directory)
$ ls -l|awk '$1~/^d.*x/{print $9}'|xargs rm -r

tr : tr copies the standard input to the standard output with substitution or deletion of selected characters

Script for removing colon
remove_colon () { echo "$@" | tr ':' ' ' ; }
(OR)
$ echo Satish.Vanahalli | tr ':' ' '

Script to set filenames to lowercase
for FILE in *
do
     mv -i "$FILE" ´echo "$FILE" | tr '[A-Z]' '[a-z]'´ 2> /dev/null
done

Command to set file contents to lowercase
$ cat zxcvbnm.sh |  tr '[A-Z]' '[a-z]' > zaq.sh

Command for listing only Directories
$ ls  -ltr  |  grep  ^d

Command for listing only Files
$ ls  -ltr  | grep  ^-

Command for displaying the Directory contents
$ ls –R *

Command to comment the first line on checking the existence of the  word
$ sed  -e 's/^srishti/#srishti/g' krishna.sh

Command to find the Software installed in UNIX box
$ swlist -l fileset

Copying a Directory
Copying a directory  with  its content needs a little trick from  copying a file
$ cp –pr  Originaldirectory Newdirectory
p is for preserving the permission & attribute .  r is recursive .It copies the directory & its contents.

Command for converting Column into Row
$ sed -e :a -e '$!N; s/\n//; ta'  satish.pld  > satish1.pld

Useful Tips

Crontab 
#1: If you inadvertently enter the crontab command with no argument(s), do not attempt to get out with Control-d. This removes all entries in your crontab file. Instead, exit with Control-c

#2: By default cron jobs sends a email to the user account executing the cronjob. If this is not needed put the following command At the end of the cron job line .
>/dev/null 2>&1

Mail
All the mails from the UNIX box is logged in log file & available in
$/var/adm/syslog /mail.log

Syntax
Shell script syntax can be checked by the below & produces no output when there is no syntax error in the script
$ /bin/sh –n test.sh

Comment
The #!/bin/sh must be the first line of a shell script in order for sh to be used to run the script. If this appears on any other line, it is treated as a comment and ignored by all shells.

Awk :  An expression which is awkward to manipulate through normal {regexp} facilities, for example, one containing a {newline}.

Initialization Script
• .profile, the Bourne shell ( sh) initialization script
• .kshrc, the Korn shell ( ksh) initialization script
• .cshrc, the C shell ( csh) initialization script

Unix Script running from another User

Script may be available in User A’s directory but if it has to run from User B (for example killing its own session) then following would be helpful .

Login as User A
$ chmod UserB filename

Login as User B
chmod u=rwxs,g=rxs,o=rx filename

Now running the script from anyuser will do the desired result

Shorthand Notations & Wildcards 
. Current directory
.. Parent directory
~ Your home directory
~user home directory of user
* Any number of characters (not '.')    Ex: *.c is all files ending in '.c'
? Any single character ( not '.')