FTP server – IndianGNU.org

Parse FTP server log and send mail for every uploaded files

By Arun Bagul | July 13, 2012 | Comments 0 Comment

Introduction –

One of my friend was looking for script to parse FTP server log after every 10mins and send mail for every file uploaded by user
with Username, Uploaded File Name, Date and Size. Please prefer the follow perl script to this task….

Download ftp file from here – http://www.indiangnu.org/wp-content/uploads/2012/ftp_logparser-pl.txt

1] Script –

arunb@mylaptop:~$ cat perl-prog/ftp_logparser.pl

#!/usr/bin/perl

use strict;
use warnings;

no warnings 'uninitialized';

my $argument = $#ARGV + 1 ;
my $mylog_file = $ARGV[0];

if ( -f $mylog_file ) {
  print "\nChecking FTP log file for Upload files ~ $mylog_file";
  open(FILE, "<$mylog_file") or die("\nFile not found : $! ");

  print "\nContent of file is....\n";
  while() {
	chomp($_);
	## Fri Jul  6 14:15:25 2012 [pid 27841] [vivek] OK UPLOAD: Client "172.30.1.76"," /squid-2 7-Template-v23.docx", 594488 bytes
	#print "\nLine=> $_";
	if ($_ !~ /^\s*$/) {
	  #print "\nLine=> $_";
	  if ($_ =~ m/(.*)\s+\[pid.*\]\s+\[(.*)\]\s+OK\s+UPLOAD:\s+Client\s+"(\d+.\d+.\d+.\d+)","(.*)",\s+(\d+)\sbytes.*/) {
	     my ($mydate, $myser,$ipaddr, $file_upload, $file_size) = ($1,$2,$3,$4,$5);
	     print "\n'$mydate' - '$myser' - '$ipaddr' - '$file_upload' - '$file_size'";
	     ## send mail
	     my $CONTACTEMAIL = "abc\@xyz.com";
	     open(MAIL, "|/usr/sbin/sendmail -t");
	     print MAIL "To: $CONTACTEMAIL\n";
	     print MAIL "From: ftp\@abc.com\n";
	     print MAIL "Subject: File Uploaded- $file_upload";
	     #print MAIL "\nContent-type: text/html\n\n";
	     print MAIL "\nContent-type: text/plain\n\n";
	     print MAIL "\nFile Name: $file_upload";
	     print MAIL "\nFile Size: $file_size bytes";
	     print MAIL "\nUser: $myser";
	     print MAIL "\nIPaddr: $ipaddr";
	     print MAIL "\nDate: $mydate";
	     close(MAIL);
	  }
	}
  }
  close(FILE);

  print "\nDone";
} else {
   print "\n * Usage: $0  \n";
}

#end
print "\n";
arunb@mylaptop:~$

2] How to Use –

arunb@mylaptop:~$ ./perl-prog/ftp_logparser.pl

* Usage: ./perl-prog/ftp_logparser.pl

arunb@mylaptop:~$ ./perl-prog/ftp_logparser.pl /var/log/ftp.log

Thank you,
Arun Bagul

Secure FTP using rssh or scponly with chroot

By Arun Bagul | August 18, 2010 | Comments 0 Comment

Introduction –

In this article we will setup SFTP using rssh with chroot ….

A] How to configure rssh + chroot for SFTP

Step 1) Install RSSH Shell ~

* Ubuntu –

root@me:/home/arunb# apt-get install scponly rssh coreutils17

* Redhat/CentOS

[root@arun.me chroot]# yum install rssh
[root@arun.me chroot]# rpm -qa | grep rssh
rssh-2.3.2-1.2.el5.rf
[root@arun.me chroot]#

Step 2) Your Chroot Setting –

[root@arun.me ~]# cat /etc/rssh.conf
# This is the default rssh config file

logfacility = LOG_USER
###### arun ########
allowsftp
# set the default umask
umask = 022
chrootpath =”/home/chroot”
###### end ########

Step 3) Create user ~

[root@arun.me ~]# mkdir /home/chroot/
[root@arun.me ~]# useradd -d /home/chroot/home/sftp_test -s /usr/bin/rssh sftp_test
[root@arun.me ~]# passwd sftp_test

* Now add following line in “/etc/rssh.conf” file…

user =”sftp_test:022:00010:/home/chroot/sftp_test”

*** [root@arun.me ~]# tail /etc/passwd

sftp_test:x:503:503::/home/chroot/home/sftp_test:/usr/bin/rssh

Step 4) Chroot allow to log via syslogd ~

add following entry in file “/etc/sysconfig/syslog” and restart syslogd

SYSLOGD_OPTIONS=”-m 0 -a /home/chroot/dev/log -a /dev/log”

root@arun.me ~]# /etc/init.d/syslog restart

Step 5) Now setup Chroot ENV –

[root@arun.me ~]# /var/src/arun_rssh_mkchroot.sh /home/chroot/

[root@arun.me ~]# cd /home/chroot/

[root@arun.me chroot]# mknod –mode=600 dev/console c 5 1
[root@arun.me chroot]# mknod –mode=666 dev/null c 1 3

[root@arun.me chroot]# /var/src/arun_shared_lib.sh usr/libexec/openssh/sftp-server

[root@arun.me chroot]# cp /lib/libnss_files.so.2 lib/libnss_files.so.2
[root@arun.me chroot]# cp /lib64/libnss_files.so.2 lib64/libnss_files.so.2

Step 6) Test now ~

arunb@me:~$ sftp sftp_test@192.168.0.1
Connecting to 192.168.0.1…
sftp_test@192.168.0.1’s password:
sftp> pwd
Remote working directory: /home/sftp_test
sftp> ls
arun manoj mayur ravi
sftp>

sftp> ls /
/dev /etc /home /lib /lib64 /usr
sftp>

Step 7) Process for New user ~

a) [root@arun.me chroot]# useradd -d /home/chroot/home/sftp_tmp -s /usr/bin/rssh sftp_tmp
[root@arun.me chroot]# passwd sftp_tmp

b) Add line in file “/etc/rssh.conf”

user =”sftp_tmp:022:00010:/home/chroot”

c) Copy user entry from /etc/{passwd,group,shadow} files TO CHROOT {passwd,group,shadow} files ~

NOTE ~ be careful…

[root@arun.me chroot]# grep sftp_tmp /etc/passwd >> /home/chroot/etc/passwd
[root@arun.me chroot]# grep sftp_tmp /etc/group >> /home/chroot/etc/group
[root@arun.me chroot]# grep sftp_tmp /etc/shadow >> /home/chroot/etc/shadow

[root@arun.me ~]# cat /home/chroot/etc/{passwd,group,shadow}

sftp_tmp:x:504:504::/home/chroot/home/sftp_tmp:/usr/bin/rssh
bin:x:1:bin,daemon
daemon:x:2:bin,daemon
sftp_test:x:503:
sftp_tmp:x:504:
bin:*:14797:0:99999:7:::
daemon:*:14797:0:99999:7:::
sftp_test:$1$Ei5oj.yu$P5FDHHI1POxIIv5562BIm/:14798:0:99999:7:::
sftp_tmp:$1$wZ6Qk3R/$ANRx5MkBA91pjzE/Dr3vK.:14798:0:99999:7:::
[root@arun.me chroot]#

Step 8) Test it now from other linux host

bagul@me:~$ sftp sftp_tmp@192.168.0.1
Connecting to 192.168.0.1…
sftp_tmp@192.168.0.1’s password:
sftp> pwd
Remote working directory: /home/sftp_tmp
sftp>
sftp> ls
sftp> ls
1 2 3 a b c
sftp> pwd
Remote working directory: /home/sftp_tmp
sftp>
sftp>
sftp> ls /
/dev /etc /home /lib /lib64 /usr
sftp> cd /etc
sftp> pwd
Remote working directory: /etc
sftp> ls
group ld.so.cache ld.so.conf localtime nsswitch.conf passwd shadow
sftp>
sftp> get 1
Fetching /home/sftp_tmp/1 to 1
/home/sftp_tmp/1 100% 14 0.0KB/s 00:00
sftp> mkdir arun
sftp> ls
1 2 3 a arun b c
sftp> version
SFTP protocol version 3
sftp> rm c
Removing /home/sftp_tmp/c
sftp>

Step 9) What is required for CHROOT ~

[root@arun.me chroot]# pwd
/home/chroot
[root@arun.me chroot]# ll
drwxr-xr-x 2 root root 4096 Jul 8 07:50 dev
drwxr-xr-x 2 root root 4096 Jul 8 07:50 etc
drwxr-xr-x 4 root root 4096 Jul 8 07:51 home
drwxr-xr-x 2 root root 4096 Jul 8 07:05 lib
drwxr-xr-x 2 root root 4096 Jul 8 07:28 lib64
drwxr-xr-x 5 root root 4096 Jul 8 07:05 usr
[root@arun.me chroot]# ll -ld /home/
drwxr-xr-x 6 root root 4096 Jul 8 08:00 /home/
[root@arun.me chroot]# ll dev/
crw——- 1 root root 5, 1 Jul 8 07:13 console
srw-rw-rw- 1 root root 0 Jul 8 07:50 log
crw-rw-rw- 1 root root 1, 3 Jul 8 07:14 null
[root@arun.me chroot]# ll etc/
-rw-r–r– 1 root root 74 Jul 8 07:55 group
-rw-r–r– 1 root root 81321 Jul 8 07:05 ld.so.cache
-rw-r–r– 1 root root 28 Jul 8 07:05 ld.so.conf
-rw-r–r– 1 root root 3519 Jul 8 07:16 localtime
-rw-r–r– 1 root root 1696 Jul 8 07:16 nsswitch.conf
-rw-r–r– 1 root root 192 Jul 8 07:55 passwd
-r——– 1 root root 180 Jul 8 07:55 shadow
[root@arun.me chroot]# ll lib
-rwxr-xr-x 1 root root 46680 Jul 8 07:28 libnss_files-2.5.so
lrwxrwxrwx 1 root root 19 Jul 8 07:05 libnss_files.so.2 -> libnss_files-2.5.so
[root@arun.me chroot]# ll lib64
-rwxr-xr-x 1 root root 139416 Jul 8 07:10 ld-linux-x86-64.so.2
-rwxr-xr-x 1 root root 10000 Jul 8 07:18 libcom_err.so.2
-rwxr-xr-x 1 root root 1366176 Jul 8 07:18 libcrypto.so.6
-rwxr-xr-x 1 root root 48600 Jul 8 07:18 libcrypt.so.1
-rwxr-xr-x 1 root root 1717800 Jul 8 07:18 libc.so.6
-rwxr-xr-x 1 root root 23360 Jul 8 07:18 libdl.so.2
-rwxr-xr-x 1 root root 9472 Jul 8 07:18 libkeyutils.so.1
-rwxr-xr-x 1 root root 114352 Jul 8 07:18 libnsl.so.1
-rwxr-xr-x 1 root root 53880 Jul 8 07:28 libnss_files.so.2
-rwxr-xr-x 1 root root 145824 Jul 8 07:18 libpthread.so.0
-rwxr-xr-x 1 root root 92736 Jul 8 07:18 libresolv.so.2
-rwxr-xr-x 1 root root 95464 Jul 8 07:18 libselinux.so.1
-rwxr-xr-x 1 root root 247496 Jul 8 07:18 libsepol.so.1
-rwxr-xr-x 1 root root 18152 Jul 8 07:18 libutil.so.1
[root@arun.me chroot]# ll usr/
drwxr-xr-x 2 root root 4096 Jul 8 07:05 bin
drwxr-xr-x 2 root root 4096 Jul 8 07:05 lib64
drwxr-xr-x 3 root root 4096 Jul 8 07:05 libexec
[root@arun.me chroot]# ll usr/bin
-rwxr-xr-x 1 root root 33265 Jul 8 07:05 rssh
-rwxr-xr-x 1 root root 53384 Jul 8 07:05 scp
[root@arun.me chroot]# ll usr/lib64
-rwxr-xr-x 1 root root 190976 Jul 8 07:18 libgssapi_krb5.so.2
-rwxr-xr-x 1 root root 153464 Jul 8 07:18 libk5crypto.so.3
-rwxr-xr-x 1 root root 613896 Jul 8 07:18 libkrb5.so.3
-rwxr-xr-x 1 root root 35728 Jul 8 07:18 libkrb5support.so.0
-rwxr-xr-x 1 root root 229272 Jul 8 07:18 libnspr4.so
-rwxr-xr-x 1 root root 1221496 Jul 8 07:18 libnss3.so
-rwxr-xr-x 1 root root 119696 Jul 8 07:18 libnssutil3.so
-rwxr-xr-x 1 root root 17736 Jul 8 07:18 libplc4.so
-rwxr-xr-x 1 root root 13800 Jul 8 07:18 libplds4.so
-rwxr-xr-x 1 root root 85608 Jul 8 07:18 libz.so.1
[root@arun.me chroot]# ll usr/libexec
drwxr-xr-x 2 root root 4096 Jul 8 07:05 openssh
-rwsr-xr-x 1 root root 69892 Jul 8 07:05 rssh_chroot_helper
[root@arun.me chroot]# ll usr/libexec/openssh
total 56
-rwxr-xr-x 1 root root 53080 Jul 8 07:05 sftp-server
[root@arun.me chroot]#

Thank you,
Arun Bagul

How to configure FTP server on Redhat & debian based system

By Arun Bagul | January 15, 2008 | Comments 0 Comment

Introduction – FTP is File Transfer Protocol, the protocol for exchanging files over the Internet. FTP uses the Internet’s TCP/IP protocols to enable data transfer

What is anonymous FTP?

Anonymous FTP is a privilege granted by the organization that owns the computer to which you are connecting by using FTP without any restriction. FTP runs exclusively over TCP. FTP servers by default listen on port 21 for incoming connections from FTP clients

Type of FTP connection?

1) In active mode, the FTP client opens a random port (> 1023), sends the FTP server the random port number on which it is listening over the control stream and waits for a connection from the FTP server. When the FTP server initiates the data connection to the FTP client it binds the source port to port 20 on the FTP server.

2) In passive mode, the FTP server opens a random port (> 1023), sends the FTP client the server’s IP address to connect to and the port on which it is listening (a 16 bit value broken into a high and low byte, like explained before) over the control stream and waits for a connection from the FTP client. In this case the FTP client binds the source port of the connection to a random port greater than 1023.

3) In extended passive mode, the FTP server operates exactly the same as passive mode, however it only transmits the port number (not broken into high and low bytes) and the client is to assume that it connects to the same IP address that was originally connected to

List of FTP Server –

The ProFTP , vsFTP and Pure-FTP are the widely used FTP servers.

1] How to Setup FTP server on Redhat/Fedora –

For this post I am using vsFTP server on Fedora..

[root@indianGNU.org vsftpd]# pwd
/etc/vsftpd
[root@indianGNU.org vsftpd]# ls
chroot_list ftpusers user_list vsftpd.conf vsftpd_conf_migrate.sh
[root@indianGNU.org vsftpd]#

Configuration file of vsFTP server –

[root@indianGNU.org vsftpd]# cat vsftpd.conf
# Example config file /etc/vsftpd/vsftpd.conf
#
##anonymous disabled
#anonymous_enable=YES
#
# allow local users to log in.
local_enable=YES
#
# enable any write operation form of FTP.
write_enable=YES
#
# Default umask for local users is 077.
local_umask=022
#
# messages given to remote users when they go into a certain directory.
dirmessage_enable=YES
#
# Activate logging of uploads/downloads.
xferlog_enable=YES
#
# Make sure PORT transfer connections originate from port 20 (ftp-data).
connect_from_port_20=YES
#
# If you want, you can have your log file in standard ftpd xferlog format
xferlog_std_format=YES
#
#chroot_list_file=/etc/vsftpd/chroot_list
chroot_local_user=YES
userlist_deny=NO
pam_service_name=vsftpd
userlist_enable=YES
#enable for standalone mode
listen=YES
tcp_wrappers=YES
[root@indianGNU.org vsftpd]#

2] Add FTP users who will use FTP service –

[root@indianGNU.org vsftpd]# tail /etc/passwd
…..

…..

reply:x:553:553::/home/reply:/bin/bash
arun:x:544:544::/var/www/indianGNU.org/html:/sbin/nologin
[root@indianGNU.org vsftpd]#

3] Add user in “/etc/vsftpd/user_list file” to login via FTP –

[root@indianGNU.org vsftpd]# cat /etc/vsftpd/user_list
# vsftpd userlist
# If userlist_deny=NO, only allow users in this file
# If userlist_deny=YES (default), never allow users in this file, and
# do not even prompt for a password.
# Note that the default vsftpd pam config also checks /etc/vsftpd/ftpusers
# for users that are denied.
arun
myftp
ftpuser
[root@indianGNU.org vsftpd]#
4] Now restart your FTP service and try to login —

[root@indianGNU.org vsftpd]# ftp localhost
Connected to localhost.
220 (vsFTPd 2.0.4)
530 Please login with USER and PASS.
530 Please login with USER and PASS.
KERBEROS_V4 rejected as an authentication type
Name (localhost:root): arun
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> pwd
257 “/”
ftp> ls
227 Entering Passive Mode (127,0,0,1,213,215)
150 Here comes the directory listing.
-rwxrwxrwx 1 0 0 66 Jan 15 12:10 index.php
-rwxrwxrwx 1 0 0 21 Jan 08 13:32 myinfo.php
drwxrwxrwx 5 0 0 4096 Jan 09 04:32 sendmail
drwxr-xr-x 5 0 0 4096 Jan 10 07:13 test.txt

226 Directory send OK.
ftp> bye
221 Goodbye.
[root@indianGNU.org vsftpd]#

Thank you,

Arun