September 2007 – IndianGNU.org

How to find ALL host within Network (LAN)

By Arun Bagul | September 30, 2007 | Comments 0 Comment

Introduction-

There are many different to find out the number of hosts (machine) within network specially LAN.

Method(1)– Simplest way to use ping command with -b option… and specify broadcast address. Use ifconfig command to find the broadcast address….

root@arunbagul:/home/arun# ping -b 192.168.0.255
WARNING: pinging broadcast address
64 bytes from 192.168.0.100: icmp_seq=1 ttl=64 time=0.512 ms
64 bytes from 192.168.0.190: icmp_seq=1 ttl=64 time=0.531 ms
…
64 bytes from 192.168.0.232: icmp_seq=1 ttl=64 time=0.557 ms
…….

** If the number of Host within network is less (class C network), then you can below script (ping command)…

root@arunbagul:~# cat arun.sh

#!/bin/bash
for ip in $(seq 1 254);
do
ping -c 1 192.168.0.${ip} >/dev/null
if [ $? -eq 0 ]; then
echo “192.168.0.$ip is UP”
fi
done
echo “done”

root@arunbagul:~#

root@arunbagul:~# sh arun.sh
192.168.0.1 is UP
192.168.0.2 is UP
…
192.168.0.13 is UP
192.168.0.14 is UP
192.168.0.15 is UP
…..
…….
done

root@arunbagul:~#

Method(2) fping command –

fping send ICMP ECHO_REQUEST packets to network host. fping uses the Internet Control Message Protocol (ICMP) echo request to determine if a target host is responding. fping differs from ping in that you can specify any number of targets on the command line, or specify a file containing the lists of targets to ping. Instead of sending to one target until it times out or replies, fping will send out a ping packet and move on to the next target in a round-robin fashion.

In the default mode, if a target replies, it is noted and removed from the list of targets to check; if a target does not respond within a certain
time limit and/or retry limit it is designated as unreachable. fping also supports sending a specified number of pings to a target, or looping indefinitely (as in ping ).

Options-

-a Show systems that are alive.

-A Display targets by address rather than DNS name.

-bn Number of bytes of ping data to send.

-c Number of request packets to send to each target.

-d Use DNS to lookup address of return ping packet. This allows you to give fping a list of IP addresses as input and print hostnames in the
output.

-e Show elapsed (round-trip) time of packets.

-g Generate a target list from a supplied IP netmask, or a starting and ending IP

-l Loop sending packets to each target indefinitely.

root@arunbagul:~# fping -g 192.168.0.0 192.168.0.255

ICMP Host Unreachable from 192.168.0.1 for ICMP Echo sent to 192.168.0.2
ICMP Host Unreachable from 192.168.0.1 for ICMP Echo sent to 192.168.0.3
ICMP Host Unreachable from 192.168.0.1 for ICMP Echo sent to 192.168.0.4
ICMP Host Unreachable from 192.168.0.1 for ICMP Echo sent to 192.168.0.5
ICMP Host Unreachable from 192.168.0.1 for ICMP Echo sent to 192.168.0.6
…

root@arunbagul:~#

root@arunbagul:~# fping -g 192.168.0.0/24

root@arunbagul:~#

Method(3) nmap command –

Nmap (“Network Mapper“) is an open source tool for network exploration and security auditing. It was designed to rapidly scan large networks, although it works fine against single hosts. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what
services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. While Nmap is commonly used for security audits, many systems and network
administrators find it useful for routine tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime.

Options –

-sL (List Scan)
The list scan is a degenerate form of host discovery that simply lists each host of the network(s) specified, without sending any packets
to the target hosts. By default, Nmap still does reverse-DNS resolution on the hosts to learn their names.

-sP (Ping Scan)
This option tells Nmap to only perform a ping scan (host discovery), This is one step more intrusive than the list scan.

-P0 (No ping)
This option skips the Nmap discovery stage altogether. Normally, Nmap uses this stage to determine active machines for heavier scanning. By
default, Nmap only performs heavy probing such as port scans, version detection, or OS detection against hosts that are found to be up.

-PS [portlist] (TCP SYN Ping)
This option sends an empty TCP packet with the SYN flag set. The default destination port is 80. A comma separated list of ports can even be
specified (e.g. -PS22,23,25,80,113,1050,35000), in which case probes will be attempted against each port in parallel.

-PU [portlist] (UDP Ping)
Another host discovery option is the UDP ping, which sends an empty (unless –data-length is specified) UDP packet to the given ports. The
portlist takes the same format as with the previously discussed -PS and -PA options

-n (No DNS resolution)
Tells Nmap to never do reverse DNS resolution on the active IP addresses it finds

-R (DNS resolution for all targets)
Tells Nmap to always do reverse DNS resolution on the target IP addresses. Normally reverse DNS is only performed against responsive (online) hosts.

root@arunbagul:~# nmap -sP 192.168.0.0/24

Starting Nmap 4.20 ( http://insecure.org ) at 2008-04-06 10:52 IST
Host arunbagul.laptop (192.168.0.1) appears to be up.
Host shekar.com (192.168.0.2) appears to be up.
Host tsantosh.desktop (192.168.0.3) appears to be up.
…
…….

Nmap finished: 256 IP addresses (1 host up) scanned in 5.670 seconds
root@arunbagul:~#

Method(4) NAST (Network Analyzer Sniffer Tool) –

Nast is a packet sniffer and a LAN analyzer based on Libnet and Libpcap. It can sniff in normal mode or in promiscuous mode. It dumps the headers of packets and the payload in ascii or ascii-hex format. You can apply a filter. The sniffed data can be saved in a separated file.

Options-

-i, –interface
Select the Interface, if not specified will be auto-detected.

-p, –promisc
Disable promiscuous mode on NIC.

-d, –ascii-data
Print data in ascii format.

-x, –ascii-hex-data
Print data in ascii-hex format.

-f, –filter <“filter”>
Apply <“filter”> to sniffer (see “FILTER SYNTAX” section below for syntax)

–ld <filename>
Log captured data to <filename> (only payload). Use -l to log all packet instead, useful with -B

-m, –host-list
Map the LAN by performing a series of ARP request to sequential subnet IP addresses.

-s, –tcp-stream
Follow a TCP/IP connection printing all data in payload. You must specify the IP addresses of the ends.

-g, –find-gateway
Try to find possible Internet-gateways.

-r, –reset-connection
Destroy an established connection. You must specify the IP addresses of the ends and at least one port.(?)

-S, –port-scanner
Performs a half-open port scanning on the selected host. It tries also to determine some firewall (just iptables) rules.

-M, –multi-port-scanner
Same as above (-S) but done on all hosts of the lan.

root@arunbagul:~# nast
Nast V. 0.2.0
Sniffing on:

– Device: eth0
– MAC address: XX:1X:3X:XX:0X:XX
– IP address: 192.168.XX.XX
– Netmask: 255.255.255.0
– Promisc mode: Set
– Filter: None
– Logging: None

—[ ARP ]———————————————————–
XX:XX:XX:0X:C5:FA -> XX:XX:FF:00:XX:XX
Type: ARP request: Who has 192.XX.XX.? Tell XX.XX.XX.XX
Hardware size: 6 – Protocol size: 4
Packet Number: 1

—[ UDP ]———————————————————–
192.168.X.X:1985(unknown) -> X.X.X.X:1985(unknown)
Version: 4 Total Lenght: 48 TTL: 1
Packet Number: 2

Packets Received: 4
Packets Dropped by kernel: 0
root@arunbagul:~#

root@arunbagul:~# nast -m
Nast V. 0.2.0
Mapping the Lan for 255.255.254.0 subnet … please wait

MAC address Ip address (hostname)
===========================================================
XX:XX:XX:XX:XX:XX 192.168.X.X (mygateway.com)
XX:XX:XX:XX:XX:XX 192.168.X.X (ravi.desktop)
….
……

root@arunbagul:~#

Method(5) nbtscan command –

nbtscan is a program for scanning networks for NetBIOS name information. It sends NetBIOS status query to each address in supplied range and lists received information in human readable form. For each responded host it lists IP address, NetBIOS computer name, logged-in user name and MAC address.

Options-

-v Verbose output. Print all names received from each host.

-d Dump packets. Print whole packet contents. Cannot be used with -v, -s or -h options.

-e Format output in /etc/hosts format.

-t timeout
Wait timeout seconds for response. Default 1.

-m retransmits
Number of retransmits. Default 0.

-f filename
Take IP addresses to scan from file “filename”

scan_range
What to scan. Can either be single IP or range of addresses in one of two forms: X.X.X.X/24 or X.X.X.X-24

root@arunbagul:~# nbtscan 192.18.0.0/24
Doing NBT name scan for addresses from 192.18.0.0/24

IP address NetBIOS Name Server User MAC address
——————————————————————————
192.168.0.X MINDTOOKPUG-DX <server> XXX XX:XX:XX:XX:XX:XX
192.168.0.X MUCHTHESE-DV <server> <unknown> XX:XX:XX:XX:XX:XX
…
……

root@arunbagul:~#

Thank you,
Arun Bagul

How to optimize PHP code (tips)

By Santhosh Tirunahari | September 16, 2007 | Comments 0 Comment

Introduction –

1. If a method can be static, declare it static. Speed improvement is by a factor of 4.

2. echo is faster than print.

3. Use echo’s multiple parameters instead of string concatenation.

4. Set the max value for your for-loops before and not in the loop.

5. Unset your variables to free memory, especially large arrays.

6. Avoid magic like __get, __set, __autoload

7. require_once() is expensive

8. Use full paths in includes and requires, less time spent on resolving the OS paths.

9. If you need to find out the time when the script started executing, $_SERVER[’REQUEST_TIME’] is preferred to time()

10. See if you can use strncasecmp, strpbrk and stripos instead of regex

11. str_replace is faster than preg_replace, but strtr is faster than str_replace by a factor of 4.

12. If the function, such as string replacement function, accepts both arrays and single characters as arguments, and if your argument list is not too long, consider writing a few redundant replacement statements, passing one character at a time, instead of one line of code that accepts arrays as search and replace arguments.

13. It’s better to use select statements than multi if, else if, statements.

14. Error suppression with @ is very slow.

15. Turn on apache’s mod_deflate

16. Close your database connections when you’re done with them

17. $row[’id’] is 7 times faster than $row[id]

18. Error messages are expensive

19. Do not use functions inside of for loop, such as for ($x=0; $x <>

20. Incrementing a local variable in a method is the fastest. Nearly the same as calling a local variable in a function.

21. Incrementing a global variable is 2 times slow than a local var.

22. Incrementing an object property (eg. $this->prop++) is 3 times slower than a local variable.

23. Incrementing an undefined local variable is 9-10 times slower than a pre-initialized one.

24. Just declaring a global variable without using it in a function also slows things down (by about the same amount as incrementing a local var). PHP probably does a check to see if the global exists.

25. Method invocation appears to be independent of the number of methods defined in the class because I added 10 more methods to the test class (before and after the test method) with no change in performance.

26. Methods in derived classes run faster than ones defined in the base class.

27. A function call with one parameter and an empty function body takes about the same time as doing 7-8 $localvar++ operations. A similar method call is of course about 15 $localvar++ operations.

28. Surrounding your string by ‘ instead of ” will make things interpret a little faster since php looks for variables inside “…” but not inside ‘…’. Of course you can only do this when you don’t need to have variables in the string.

29. When echoing strings it’s faster to separate them by comma instead of dot. Note: This only works with echo, which is a function that can take several strings as arguments.

30. A PHP script will be served at least 2-10 times slower than a static HTML page by Apache. Try to use more static HTML pages and fewer scripts.

31. Your PHP scripts are recompiled every time unless the scripts are cached. Install a PHP caching product to typically increase performance by 25-100% by removing compile times.

32. Cache as much as possible. Use memcached – memcached is a high-performance memory object caching system intended to speed up dynamic web applications by alleviating database load. OP code caches are useful so that your script does not have to be compiled on every request

33. When working with strings and you need to check that the string is either of a certain length you’d understandably would want to use the strlen() function. This function is pretty quick since it’s operation does not perform any calculation but merely return the already known length of a string available in the zval structure (internal C struct used to store variables in PHP). However because strlen() is a function it is still somewhat slow because the function call requires several operations such as lowercase & hashtable lookup followed by the execution of said function. In some instance you can improve the speed of your code by using an isset() trick.

Example –

if (strlen($foo) < 5) { echo “Foo is too short”; }

vs.

if (!isset($foo{5})) { echo “Foo is too short”; }

** Calling isset() happens to be faster then strlen() because unlike strlen(), isset() is a language construct and not a function meaning that it’s execution does not require function lookups and lowercase. This means you have virtually no overhead on top of the actual code that determines the string’s length.

34. When incrementing or decrementing the value of the variable $i++ happens to be a tad slower then ++$i. This is something PHP specific and does not apply to other languages, so don’t go modifying your C or Java code thinking it’ll suddenly become faster, it won’t. ++$i happens to be faster in PHP because instead of 4 opcodes used for $i++ you only need 3. Post incrementation actually causes in the creation of a temporary var that is then incremented. While pre-incrementation increases the original value directly. This is one of the optimization that opcode optimized like Zend’s PHP optimizer. It is a still a good idea to keep in mind since not all opcode optimizers perform this optimization and there are plenty of ISPs and servers running without an opcode optimizer.

35. Not everything has to be OOP, often it is too much overhead, each method and object call consumes a lot of memory.

36. Do not implement every data structure as a class, arrays are useful, too

37. Don’t split methods too much, think, which code you will really re-use

38. You can always split the code of a method later, when needed

39. Make use of the countless predefined functions

40. If you have very time consuming functions in your code, consider writing them as C extensions

41. Profile your code. A profiler shows you, which parts of your code consumes how many time. The Xdebug debugger already contains a profiler. Profiling shows you the bottlenecks in overview.

42. mod_gzip which is available as an Apache module compresses your data on the fly and can reduce the data to transfer up to 80%.

Thanks,

Santhosh T.

PHP – IP address to Country

By Santhosh Tirunahari | September 15, 2007 | Comments 0 Comment

Introduction –

This is not exactly a tutorial, but a small trick, to access ip-to-country.wehbosting.info demo as a web-service. ip-to-country.wehbosting.info does provide a nice CSV format of transferring IPs to country. Pretty amazing. But the problem is you need to have a good DB support at your end to use it

For example –

<?php

$ip = $_GET[‘ip’];

function get_country($ip)

{

$f = fsockopen(‘ip-to-country.webhosting.info’, 80);

if (!$f)

{

return false;

}

$postdata = “ip_address=”.urlencode($ip).”&submit=”.urlencode(‘Find Country’);

$request = ”;

$request .= “POST /node/view/36 HTTP/1.1\r\n”;

$request .= “Host: ip-to-country.webhosting.info\r\n”;

$request .= “User-Agent: Its me again\r\n”;

$request .= “Content-Length: “.strlen($postdata).”\r\n”;

$request .= “Content-Type: application/x-www-form-urlencoded\r\n”;

$request .= “\r\n”;

$request .= “$postdata\r\n”;

fwrite($f, $request);

$response = ”;

while (!feof($f))

{

$response .= fgets($f, 128);

}

$pos1 = strpos ( $response , ‘</from>’);

$pos2 = strpos ( $response , ‘<br><br><img’ , $pos1 );

$parse_from = substr( $response, $pos1+21, ($pos2-$pos1) );

$pattern = “/<b>([^\/]*)<\/b>/si”;

preg_match_all($pattern, $parse_from, $matches);

return $matches[1][1];

}

echo (get_country($ip));

Thank you,

Santhosh Tirumahari

crontab – How it works?

By Arun Bagul | September 10, 2007 | Comments 1 comment

Introduction – cron is a daemon to execute scheduled commands (Vixie Cron). cron daemon has following options…

cron [-f] [-l] [-L loglevel]

Options
-f Stay in foreground mode, don’t daemonize.

-l Enable LSB compliant names for /etc/cron.d files

-L loglevel

Sets the loglevel for cron. The standard logging level (1) will
log the start of all the cron jobs. A higher loglevel (2) will
cause cron to log also the end of all cronjobs, which can be
useful to audit the behaviour of tasks run by cron. Logging
will be disabled if the loglevel is set to zero (0).

There are two type of cron job (crontab) –

1) User based cron job (crontab) –
2) System wide cron job (crontab) –

1] User based cron job (crontab) –

cron daemon searches its spool directory “/var/spool/cron/crontabs” for crontab files. After scheduling cron job by any user, the cron daemon will create file with that user name in it’s spool directory to store cron job for that user. Then cron daemon will load all cron jobs which are scheduled for each user into memory by reading contab file for each user from this spool directory. Please note that crontabs in this directory should not be accessed directly the crontab command should be used to access and update them.

root@arunbagul:~# ls -l /var/spool/cron/crontabs/
total 8
-rw——- 1 arun crontab 265 2008-03-31 19:22 arun
-rw——- 1 root crontab 317 2008-03-31 19:57 root
root@arunbagul:~#

root@arunbagul:~# ls -dl /var/spool/cron/crontabs/
drwx-wx–T 2 root crontab 4096 2008-03-31 19:57 /var/spool/cron/crontabs/
root@arunbagul:~#

root@arunbagul:~# grep crontab /etc/group
crontab:x:111:
root@arunbagul:~#

** root is the owner and crontab is the group owner of spool directory of cron daemon ie “/var/spool/cron/crontabs/”. The crontab files in spool directory is owned by respective user and group owner is “crontab”. please note the permission of spool directory and crontab files. As sticky bit is assigned on spool directory. only user can delete the crontab file(of course root also!).

For example – As shown below the crontab for user “arun” are stored in file “arun” in spool directory ie “/var/spool/cron/crontabs/arun” file.

root@arunbagul:~# cat /var/spool/cron/crontabs/arun

* 0 * * * /home/arun/update.sh

root@arunbagul:~#

How to start/stop/restart the cron daemon –

root@arunbagul:~# /etc/init.d/cron
* Usage: /etc/init.d/cron {start|stop|restart|reload|force-reload}
root@arunbagul:~#

root@arunbagul:~# /etc/init.d/cron start
* Starting periodic command scheduler crond [ OK ]
root@arunbagul:~#

root@arunbagul:~# /etc/init.d/cron restart
* Restarting periodic command scheduler crond [ OK ]
root@arunbagul:~#

root@arunbagul:~# /etc/init.d/cron stop
* Stopping periodic command scheduler crond [ OK ]
root@arunbagul:~#

NOTE – No need to reload the cron daemon after modifying the crontab file by any user.

2] System wide cron job (crontab) –

System wide setting for crontab is defined in “/etc/crontab” file and in “/etc/cron.d/” directory. The intended purpose of this feature is to allow packages that require control of their scheduling. The packages can add there crontab files to /etc/cron.d directory and such files should be named after
the package that supplies them. For example the crontab “/etc/cron.d/php5” is set by php5 package. see below… But this is not the case alway! You can use this features for customized scheduling!!

root@arunbagul:~# ls -l /etc/cron.d/
total 8
-rw-r–r– 1 root root 244 2007-03-05 12:08 anacron
-rw-r–r– 1 root root 456 2007-10-05 05:06 php5
root@arunbagul:~#

root@arunbagul:~# cat /etc/cron.d/php5
# /etc/cron.d/php5: crontab fragment for php5

# Look for and purge old sessions every 30 minutes
09,39 * * * * root [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -r -0 rm
root@arunbagul:~#

root@arunbagul:~# cat /etc/crontab
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user command
17 * * * * root cd / && run-parts –report /etc/cron.hourly
25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.weekly )
52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )
#
root@arunbagul:~#

a) Hourly crontab(job) –

Setting for hourly crontabs is defined /etc/crontab file . Let’s see what is there in above /etc/crontab file. First and second line in /etc/crontab file defines SHELL and executable PATH variables respectively and third line is comments.

Now we will see fourth line…

17 * * * * root cd / && run-parts –report /etc/cron.hourly

cron daemon will run the command “cd / && run-parts –report /etc/cron.hourly” at 17th minute of every hour as “root” owner. Lets see what this command is doing. “&&” operator means that if first command is successful then only execute the second command (see example below) ie “&&” is logical AND. The first command is “cd /” which will set pwd as /. Once this command is successful, next command “run-parts –report /etc/cron.hourly” will be executed..

root@arunbagul:~# ls -l && pwd
total 48
drwxr-xr-x 2 root root 4096 2006-02-25 05:19 Desktop
drwxr-xr-x 2 root root 4096 2006-02-25 05:19 Documents
drwxr-xr-x 2 root root 4096 2006-02-25 05:19 Music

/root

root@arunbagul:~#

root@arunbagul:~# ls -M && pwd
ls: invalid option — M
Try `ls –help’ for more information.
root@arunbagul:~#

command (1) run-parts – is the command which runs all the executable files , script or programs in given directory!.

–test print the names of the scripts which would be run, but don’t actually run them.

–list print the names of the all matching files (not limited to executables), but don’t actually run them. This option cannot be used with –test.

-v, –verbose
print the name of each script to stderr before running.

–report
similar to –verbose, but only prints the name of scripts which produce output. The script’s name is printed to whichever of stdout or
stderr the script first produces output on.

–reverse
reverse the scripts’ execution order.

–exit-on-error
exit as soon as a script returns with a non-zero exit code.

Now we will see what this command is doing …

root@arunbagul:~# run-parts –list /etc/cron.hourly/
/etc/cron.hourly/arun
/etc/cron.hourly/autocommit
/etc/cron.hourly/backup
/etc/cron.hourly/download
/etc/cron.hourly/ftp
root@arunbagul:~#

so all above script will be run by crontab hourly. –list option will list the script which would be run by crontab hourly and –report option will run the all scripts within given directory…

* This is how hourly schedule jobs are executed/run by cron tab.

b) Daily, Weekly and Hourly crontabs –

Setting for daily, weekly and monthly crontabs is defined in /etc/crontab file. Now we will see 5th, 6th and 7th line of /etc/crontab file…

25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.weekly )
52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )

* crontab (cron daemon) will execute/run the script “test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.daily )” at 6:25 AM on every day, as this is daily job.

*cron will run this script “test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.weekly )” at 6:47 AM on every sunday as this is weekly job.

* cron will run this script “test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )” at 6:52 AM on 1st day of every month as this is monthly job.

Now we will see what is this command doing – “||” operator is act as logical OR, means if first script is successful then second script would not be executed or run. and vice versa. let’s see one example here….

root@arunbagul:~# ll || pwd
total 48
drwxr-xr-x 2 root root 4096 2008-04-03 23:23 Desktop
drwxr-xr-x 2 root root 4096 2006-02-25 05:19 Documents
-rwxr-x— 1 root root 266 2008-02-15 09:48 myscript.sh

root@arunbagul:~#

root@arunbagul:~# ll -M || pwd
ls: invalid option — M
Try `ls –help’ for more information.

/root

root@arunbagul:~#

** Now we will check first part of command ie “test -x /usr/sbin/anacron“. ‘test‘ is the command which is used to check ‘file’ types and compare values. Here “-x” options check whether file exist or not and is file executalble (executable permission) or not?.. see below

root@arunbagul:~# test -x /usr/sbin/anacron
root@arunbagul:~#

root@arunbagul:~# echo $?
0
root@arunbagul:~#

So “/usr/sbin/anacron” file exist and it is executable that’s why command is successful so second part of command ie “( cd / && run-parts –report /etc/cron.weekly )” will NOT be executed as first part of command is successfully executed!

root@arunbagul:~# run-parts –list /etc/cron.weekly
/etc/cron.weekly/0anacron
/etc/cron.weekly/man-db
/etc/cron.weekly/popularity-contest
/etc/cron.weekly/sysklogd
root@arunbagul:~#

root@arunbagul:~# run-parts –test /etc/cron.weekly
/etc/cron.weekly/0anacron
/etc/cron.weekly/man-db
/etc/cron.weekly/popularity-contest
/etc/cron.weekly/sysklogd
root@arunbagul:~#

command (2) anacron – runs commands periodically. Anacron can be used to execute commands periodically, with a frequency specified in days. Unlike cron, it does not assume that the machine is running continuously. Hence, it can be used on machines that aren’t running 24 hours a day and to control daily, weekly, and monthly jobs that are usually controlled by cron.

When executed, Anacron reads a list of jobs from a configuration file, normally “/etc/anacrontab“. This file contains the list of jobs that Anacron controls. Each job entry specifies a period in days, a delay in minutes, a unique job identifier, and a shell command. For each job, Anacron checks whether this job has been executed in the last n days, where n is the period specified for that job. If not, Anacron runs the job’s shell command, after waiting for the number of minutes specified as the delay parameter. After the command exits, Anacron records the date in a special timestamp file for that job, so it can know when to execute it again. Only the date is used for the time calculations. The hour is not used. When there are no more jobs to be run, Anacron exits. Anacron only considers jobs whose identifier, as specified in the anacrontab matches any of the job command-line arguments. The job arguments can be shell wildcard patterns (be sure to protect them from your shell with adequate quoting). Specifying no job arguments, is equivalent to specifying “*”.

** “/var/spool/anacron” is the spool directory used by Anacron for storing timestamp files.

-f Force execution of the jobs, ignoring the timestamps.

-u Only update the timestamps of the jobs, to the current date, but don’t run anything.

-s Serialize execution of jobs. Anacron will not start a new job before the previous one finished.

-n Run jobs now. Ignore the delay specifications in the /etc/anacrontab file. This options implies -s.

-d Don’t fork to the background. In this mode, Anacron will output informational messages to standard error, as well as to syslog. The out‐
put of jobs is mailed as usual.

-q Suppress messages to standard error. Only applicable with -d.

** Check what is there in /etc/anacrontab file –

root@arunbagul:~# cat /etc/anacrontab
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# These replace cron’s entries
1 5 cron.daily nice run-parts –report /etc/cron.daily
7 10 cron.weekly nice run-parts –report /etc/cron.weekly
@monthly 15 cron.monthly nice run-parts –report /etc/cron.monthly

root@arunbagul:~#

The format of “/etc/anacrontab” file is ..

Field No Field
——– —————-
(1) Period in days
(2) Delay in minutes
(3) Unique job identifier
(4) Shell command

* Daily – The period is 1 day , delay is 5 min and command is “nice run-parts –report /etc/cron.daily”.
* Weekly – The period is 7 days , 10 min delay time and command is “nice run-parts –report /etc/cron.weekly”
* Monthly – The period is one month and 15 min as delay time

Please read the man page of ‘nice’ command and we know what this “run-parts” command is doing…

root@arunbagul:~# run-parts –list /etc/cron.weekly
/etc/cron.weekly/0anacron
/etc/cron.weekly/man-db
/etc/cron.weekly/popularity-contest
/etc/cron.weekly/sysklogd
root@arunbagul:~#

root@arunbagul:~# run-parts –test /etc/cron.weekly
/etc/cron.weekly/0anacron
/etc/cron.weekly/man-db
/etc/cron.weekly/popularity-contest
/etc/cron.weekly/sysklogd
root@arunbagul:~#

root@arunbagul:~# run-parts –list /etc/cron.monthly/
/etc/cron.monthly//0anacron
/etc/cron.monthly//proftpd
/etc/cron.monthly//scrollkeeper
/etc/cron.monthly//standard
root@arunbagul:~#

root@arunbagul:~# run-parts –test /etc/cron.monthly/
/etc/cron.monthly//0anacron
/etc/cron.monthly//proftpd
/etc/cron.monthly//scrollkeeper
/etc/cron.monthly//standard
root@arunbagul:~#

** Check what is there in spool directory “/var/spool/anacron/” –

root@arunbagul:~# cd /var/spool/anacron/
root@arunbagul:/var/spool/anacron# ll
total 12
-rw——- 1 root root 9 2008-04-03 23:35 cron.daily
-rw——- 1 root root 9 2008-03-13 14:32 cron.monthly
-rw——- 1 root root 9 2008-04-04 00:19 cron.weekly
root@arunbagul:/var/spool/anacron#

root@arunbagul:/var/spool/anacron# cat cron.daily
20080403
root@arunbagul:/var/spool/anacron#

root@arunbagul:/var/spool/anacron# cat cron.monthly
20080313
root@arunbagul:/var/spool/anacron#

root@arunbagul:/var/spool/anacron# cat cron.weekly
20080404
root@arunbagul:/var/spool/anacron#

Summary of article –

Scheduling in GNU/Linux or Unix is achieved by crontab (cron daemon) with the help of anacron and run-parts commands!

Thank you,

Arun Bagul

How to schedule job in Linux/Unix – cron , crontab

By Arun Bagul | September 6, 2007 | Comments 0 Comment

Introduction – cron is a daemon to execute scheduled commands on Linux and UNIX operating system.There are two type of cron job (crontab) –

1) User based cron job (crontab)
2) System wide cron job (crontab)

1] User based cron job (crontab) –

‘crontab’ command is used to schedule user based cron jobs. Each user has their own crontab and cron jobs will be executed as the user who owns the crontab.

Cron daemon automatically sets several environment variables like SHELL is set to /bin/sh, and LOGNAME or USER and HOME are set from the /etc/passwd line of the crontab’s owner. PATH is set to “/usr/bin:/bin”. But user can change HOME, SHELL, and PATH in his the crontab; LOGNAME or USER is the user that the job is running from, and may not be changed….

In addition cron daemon will also check system/environment variable MAILTO for sending mail as a result of running commands. If MAILTO is defined (and non-empty) then mail is sent to the user. If MAILTO is defined but empty (MAILTO=””), no mail will be sent. Otherwise mail is sent to the owner of the crontab.

root@arunbagul:~# echo $MAILTO

root@arunbagul:~#

* How to set MAILTO variable-

arun@arunbagul:~$ crontab -l

MAILTO=”arunbagul@indiangnu.org,indiangnu@yahoo.com”
PATH=”/bin:/sbin/:/usr/bin”
SHELL=/bin/bash

arun@arunbagul:~$

The system/environment EDITOR also used by cron daemon. this variable defaine the deafult editor which is used by crontab for editing the crontab file or cron job scheduling…

arun@arunbagul:~$ echo $EDITOR

arun@arunbagul:~$

arun@arunbagul:~$ export EDITOR=pico
arun@arunbagul:~$ echo $EDITOR
pico
arun@arunbagul:~$

If you want to change the default editor for any user then add this line “export EDITOR=pico” to profile files of that user(.bashrc , .bash_profile or .profile). Cron also supports the pam_env module, and loads the environment specified by /etc/security/pam_env.conf. But PAM setting do NOT override the settings described above or any settings in the crontab file itself.

root@arunbagul:~# crontab
crontab: usage error: file name must be specified for replace
usage: crontab [-u user] file
crontab [-u user] { -e | -l | -r }
(default operation is replace, per 1003.2)
-e (edit user’s crontab)
-l (list user’s crontab)
-r (delete user’s crontab)
root@arunbagul:~#

1) How to list crontab or cron job –

To list the crontab/cron job use below command..

arun@arunbagul:~$ crontab -l

* * * * * /home/arun/mycron.sh
arun@arunbagul:~$

root@arunbagul:~# crontab -l -u arun

* * * * * /home/arun/mycron.sh
root@arunbagul:~#

** -u <user name> options is used to list crontab for particular user…

2) How to schedule (edit) crontab or cron job –

** -e -u <user name> option is used to edit the cron tab file for user mentioned with -u options

root@arunbagul:~# crontab -u arun -e
No modification made
root@arunbagul:~# crontab -u arun -e
crontab: installing new crontab
root@arunbagul:~#

arun@arunbagul:~$ crontab -u arun -e
crontab: installing new crontab
arun@arunbagul:~$

3) How to delete user’s cron job –

** -r -u <user name> option is used to delete the crontab for user mentioned with -u option

arun@arunbagul:~$ crontab -u arun -l

1 * * * * /home/arun/update.sh 2>&1 /home/arun/update.log
* * * * * /home/arun/mycron.sh

arun@arunbagul:~$

arun@arunbagul:~$ crontab -u arun -r
arun@arunbagul:~$

arun@arunbagul:~$ crontab -u arun -l
no crontab for arun
arun@arunbagul:~$

4) Format of crontab file –

Active line in a crontab will be either an environment setting or a cron command. An environment setting is of the form,
name = value

where the spaces around the equal-sign (=) are optional.

crontab file has six columns/fields and each line is consider as one cron job/command (row), except environment setting line whose format is like “name = value”. The first five columns/fields are time and date column/fields by which user can schedule the job. The six column/field is command. The fields may be separated by spaces or tabs. The time and date fields are….

Field No Field Allowed values
————- ————- —————————-
(1) Minute 0-59
(2) Hour 0-23
(3) Day of Month 1-31
(4) Month 1-12 (or names, see below)
(5) Day of Week 0-7 (0 or 7 is Sunday, or use names)

* Names can also be used for the “Month” and “Day of Week” fields. Use the first three letters of the particular day or month (case doesn’t matter).

Months Name used in cronjob – jan, feb ,mar, apr, may, jun, jul, aug, sep, oct, nov and dec.
Day of the Week – sun, mon, tue, wed, thu, fri and sat

Field (6) – The “sixth field” (the rest of the line) specifies the command to be run. The entire command portion of the line, up to a newline or character, will be executed by /bin/sh or by the shell specified in the SHELL variable of the crontab file. Percent-signs (%) in the command, unless
escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input. There is no way to split a single command line onto multiple lines, like the shell’s trailing “\”.

Note:- If both fields ‘Day of Month‘ and ‘Day of week‘ are specified (i.e. they are not *), then the command will be run when either field matches the current time.

For example…

arun@arunbagul:~$ crontab -l -u arun

30 6 1,7,14,21,28 * 5 /home/arun/weekly.sh > /home/arun/weekly.log

arun@arunbagul:~$

* The above cron job will be executed at 6:30 am on the 1st, 7th,14th, 21st and 28th day of each month (and)plus every Friday.

5) Different types of filed format –

* cron daemon examines cron entries once every minute.
* A field may be an asterisk (*), which always stands for ‘first-last‘.
* Ranges of numbers are allowed. Ranges are two numbers separated with a hyphen. The specified range is inclusive. For example, 7-11 for an ‘hours’ entry specifies execution at hours 7, 8, 9, 10 and 11.

* Lists are allowed. A list is a set of numbers (or ranges) separated by commas. For examples: “1,2,5,9”, “0-4,8-12”.

* Step values can be used with ranges. Following a range with “/<number>” specifies skips of the number’s value through the range.
For example, “0-23/2” can be used in the hours field to specify command execution every other hour. Steps are also permitted after an asterisk(*), so if you want to say “every three hours“, just use “*/3“.

* Ranges or lists of names are not allowed.

** Instead of the first five fields, one of eight special strings may be used….

String Meaning
————- ——————-
@reboot Run once, at startup.
@yearly Run once a year, “0 0 1 1 *”.
@annually (same as @yearly)
@monthly Run once a month, “0 0 1 * *”.
@weekly Run once a week, “0 0 * * 0”.
@daily Run once a day, “0 0 * * *”.
@midnight (same as @daily)
@hourly Run once an hour, “0 * * * *”.

6) Examples of crontab –

root@arunbagul:~# crontab -l

#run command at midnight, every day
0 0 * * * /root/bin/daily.job >> /tmp/daily.log 2>&1

# run at 3:40pm on the first of every month
40 14 1 * * /root/bin/monthly.log > /dev/null 2> /var/log/monthly.error

# Send birth day mail to your friend
1 0 17 jan * mail -s “Hi!! Ravi,” Wish you%HAPPY BIRTH DAY!!%%your friend%%Arun%

root@arunbagul:~#

2] System wide cron job (crontab) –

System wide setting for cron is defined in file /etc/crontab and /etc/cron.d directory. This file is slightly different from crotab(user based) file. This file has seven field/column. First five fields are same as crontab(user based) file. 6th field is the User Name, who is the owner of cron job scheduled to run and 7th field is actually 6th field of crontab(user based).

root@arunbagul:~# cat /etc/crontab
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user command
17 * * * *     root cd / && run-parts –report /etc/cron.hourly
25 6 * *   *    root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.weekly )
52   6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )
#

root@arunbagul:~#

** Unlike any other crontab you don’t have to run the ‘crontab’ command to modify the “/etc/crontab” file and files in /etc/cron.d directory. These files also have user name fields, that none of the other crontabs do.

For example…

root@arunbagul:/etc/cron.d# pwd
/etc/cron.d
root@arunbagul:/etc/cron.d# ls
anacron php5
root@arunbagul:/etc/cron.d# cat anacron
# /etc/cron.d/anacron: crontab entries for the anacron package

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

30 7 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null
root@arunbagul:/etc/cron.d#

Now check the “/etc/crontab” file. There are several cron jobs entry…..

a) Hourly cron job –

In /etc/crontab the first cron job entry is for hourly process. All the files (crontab files) in directory /etc/cron.hourly/ will be
kick off at 17min of every hour.
For example

root@arunbagul:/etc/cron.hourly# pwd
/etc/cron.hourly
root@arunbagul:/etc/cron.hourly# ls
mail_filter_from_arun.sh
root@arunbagul:/etc/cron.hourly#

b) Daily cron job –

The 2nd cron entry in /etc/crontab file is for daily job scheduling. All the files (crontab files) in directory /etc/cron.daily/ will be
kick off at 06:25am daily. For example…

root@arunbagul:/etc/cron.daily# pwd
/etc/cron.daily
root@arunbagul:/etc/cron.daily# ls
0anacron apport aptitude exim4-base find.notslocate.dpkg-new man-db quota standard
apache2 apt bsdmainutils find.notslocate logrotate perlindex slocate sysklogd

root@arunbagul:/etc/cron.daily#
root@arunbagul:/etc/cron.daily# cat logrotate
#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf
root@arunbagul:/etc/cron.daily#

c) Weekly cron job –

The 3rd entry in /etc/crontab file is for weekly job scheduling. All the files (crontab files) in directory /etc/cron.weekly/ will be
kick off at 06:47am every sunday. For example…

root@arunbagul:/etc/cron.weekly# pwd
/etc/cron.weekly
root@arunbagul:/etc/cron.weekly# ls
0anacron man-db popularity-contest sysklogd
root@arunbagul:/etc/cron.weekly#

root@arunbagul:/etc/cron.weekly# cat 0anacron
#!/bin/sh
#
# anacron’s cron script
#
# This script updates anacron time stamps. It is called through run-parts
# either by anacron itself or by cron.
#
# The script is called “0anacron” to assure that it will be executed
# _before_ all other scripts.

test -x /usr/sbin/anacron || exit 0
anacron -u cron.weekly
root@arunbagul:/etc/cron.weekly#

d) Monthly cron job –

The 4rd entry in /etc/crontab file is for monthly job scheduling. All the files (crontab files) in directory /etc/cron.monthly/ will be
kick off at 06:52am on 1st Day of every month. For example….

root@arunbagul:/etc/cron.monthly# pwd
/etc/cron.monthly
root@arunbagul:/etc/cron.monthly# ls
0anacron proftpd scrollkeeper standard
root@arunbagul:/etc/cron.monthly#

root@arunbagul:/etc/cron.monthly# cat proftpd
#!/bin/sh
#
# cron script to rotate the proftpd server logfile, based on the
# wu-ftpd script by Peter Tobias <tobias@et-inf.fho-emden.de>.

[ -x /usr/sbin/ftpstats ] || exit 0

cd /var/log/proftpd
savelog -q -u root -g adm -m 640 -c 12 /var/log/proftpd/xferreport
ftpstats -a -r -l 2 -d 2>/dev/null >/var/log/proftpd/xferreport
savelog -q -u root -g adm -m 640 -c 7 /var/log/proftpd/xferlog
savelog -q -u root -g adm -m 640 -c 7 /var/log/proftpd/proftpd.log
savelog -q -u root -g adm -m 640 -c 7 /var/log/proftpd/controls.log
root@arunbagul:/etc/cron.monthly#

** Now It’s clear to all how to schedule job in Linux/Unix operating system by using cron daemon.

Thank you,
Arun Bagul