Unix commands for software developers

Unix is very powerful, and you can achieve a lot with less amount of code. Most production systems run in a Unix environment, and it is very useful to know your commands to solve technical issues. Some handy Unix commands looping, recursive search, control characters,  and crontab. Q. How will you go about concatenating the contents in a number of text/sql files to a single file? The account_sqls.patch file contains the following entries to a number of SQL files to be executed sequentially. ./tables/account_summary.sql ./tables/account_detail.sql ./tables/account_history.sql ./tables/account_link.sql How will you combine the SQL contents in the above 4 files into one?

 

A. Firstly, cd to the folder where the account_sqls.patch file is, and then type the following command on a shell command line.

for sql in $(sed -n 1,4p account_sqls.patch); do cat $sql >> accounts_combined.sql; done

1, 4p                                                                    // print lines 1 to 4 from account_sqls.patch
for sql in $(sed -n 1,4p account_sqls.patch);  // for each line read
do cat $sql >> bulk_data_changes.sql; // append the contents of each file to a different file accounts_combined.sql.

Note: This is handy for combining contents from a number of different files.

Q. How will you recursively search for presence of a particular text under multiple folders unix? For example, search for text "CashEnum" within a java project?

A. Use the find and grep commands. The find will look for the files and the grep will filter out the files that has the text "CashEnum".

find myproject -type f -exec grep "CashEnum" {} /dev/null ;

• -type f --> find files. 
• the addition of /dev/null is so that grep will consistently print the file name when it finds a match.

The above command will search all file extensions, including .jar. You could restrict it to look for only java files as shown below

find myproject -type f -name *.java -exec grep "CashEnum" {} /dev/null ;

-name *.java --> looks for only java files.

Q.How will you go about executing multiple maven commands on unix? For example, build two separate projects one after another, for example -- project1 and project2. The project2 to should only build if project1 successfully builds.

A. Use the && control operator to combine two commands so that the second is run only if the first command returns a zero exit status. In other words, if the first command runs successfully, the second command runs. If the first command fails, the second command does not run at all. For example:

$ mvn --file project1/pom.xml clean install -Dmaven.test.skip && mvn --file project2/pom.xml clean install -Dmaven.test.skip

Similarly, the || control operator separates two commands and runs the second command only if the first command returns a non-zero exit status, that is if it fails. In other words, if the first command is successful, the second command does not run. This operator is often used when testing for whether a given directory exists and, if not, it creates one.

cd project1/src/main || mkdir -p project1/src/main

the -p command allows you to create any relevant parent directories as well. The && and || control characters can be used together.

Q. How will you check using unix commands that a given class file exists within a jar file?
A. Let us assume that there are a number of jar files sitting under the myjars/lib folder. The following script "search-class-file.sh" sits under myjars.

#!/bin/bash

pattern=$1
shift

for jar in $(find $* -type f -name *.jar)
do
  match=`jar -tvf $jar | grep $pattern`
  if [ ! -z "$match" ]
  then
    echo "Found in: $jar"
    echo "$match"
  fi
done

The above script can be executed as shown below

$ ./search-class-file.sh Rebalance.class ./lib

$0 --> will be ./search-class-file.sh
$1 --> will be Rebalance.class
After shift, $* will be $1 which is ./lib
-z is a string function
if [ ! -z "$match" ] --> means if $match is not zero length

The output will look like

$ ./search-class-file.sh Rebalance.class ./lib
Found in: ./lib/myapp.jar
  7007 Wed Jul 11 16:44:16 EST 2012 com/myapp/mortgage/MortgageRebalance.class
  5956 Wed Jul 11 16:44:16 EST 2012 com/myapp/bulk/BulkRebalance.class
 15880 Wed Jul 11 16:44:16 EST 2012 com/myapp/model/ModelRebalance.class

similar results can be accomplished with either

$ for i in ./lib/*.jar; do jar -tvf "$i" | grep  -i Rebalance.class; done

or

$ find ./lib -name *.jar -o -name *.war -type f | xargs -i bash -c "jar -tvf {}| grep Rebalance.class {}"

Q. How will you go about listing all the users cron jobs?
A. If you are a root user, you can list the cron jobs for all the users as shown below:

for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -l $user; done

Where the /etc/passwd file looks something like

root:x:0:1:Super-User:/:/usr/bin/ksh
daemon:x:1:1::/:
bin:x:2:2::/usr/bin:
......

If you need to just list your scheduled jobs then try

crontab -l | more

The ouput will look something like:

# PUWPD001 - Loads insurance data from ins system into online DB
# PUWPD002 - Loads mortgage data from mort system into online DB

# ----------------------------------------------------------------------------------------
0 5 * * 0,1,2,3,4,5,6   . /usr/my_insurance.ksh
10 5 * * 0,1,2,3,4,5,6  . /usr/my_mortgage.ksh

Related Posts by Categories