Category: General information

General information

Parse FTP server log and send mail for every uploaded files

Parse FTP server log and send mail for every uploaded files

Introduction –

One of my friend was looking for script to parse FTP server log after every 10mins and send mail for every file uploaded by user
with Username, Uploaded File Name, Date and Size.  Please prefer the follow perl script to this task….

Download ftp file from here – http://www.indiangnu.org/wp-content/uploads/2012/ftp_logparser-pl.txt

1] Script –

arunb@mylaptop:~$ cat perl-prog/ftp_logparser.pl

#!/usr/bin/perl

use strict;
use warnings;

no warnings 'uninitialized';

my $argument = $#ARGV + 1 ;
my $mylog_file = $ARGV[0];

if ( -f $mylog_file ) {
  print "\nChecking FTP log file for Upload files ~ $mylog_file";
  open(FILE, "<$mylog_file") or die("\nFile not found : $! ");

  print "\nContent of file is....\n";
  while() {
	chomp($_);
	## Fri Jul  6 14:15:25 2012 [pid 27841] [vivek] OK UPLOAD: Client "172.30.1.76"," /squid-2 7-Template-v23.docx", 594488 bytes
	#print "\nLine=> $_";
	if ($_ !~ /^\s*$/) {
	  #print "\nLine=> $_";
	  if ($_ =~ m/(.*)\s+\[pid.*\]\s+\[(.*)\]\s+OK\s+UPLOAD:\s+Client\s+"(\d+.\d+.\d+.\d+)","(.*)",\s+(\d+)\sbytes.*/) {
	     my ($mydate, $myser,$ipaddr, $file_upload, $file_size) = ($1,$2,$3,$4,$5);
	     print "\n'$mydate' - '$myser' - '$ipaddr' - '$file_upload' - '$file_size'";
	     ## send mail
	     my $CONTACTEMAIL = "abc\@xyz.com";
	     open(MAIL, "|/usr/sbin/sendmail -t");
	     print MAIL "To: $CONTACTEMAIL\n";
	     print MAIL "From: ftp\@abc.com\n";
	     print MAIL "Subject: File Uploaded- $file_upload";
	     #print MAIL "\nContent-type: text/html\n\n";
	     print MAIL "\nContent-type: text/plain\n\n";
	     print MAIL "\nFile Name: $file_upload";
	     print MAIL "\nFile Size: $file_size bytes";
	     print MAIL "\nUser: $myser";
	     print MAIL "\nIPaddr: $ipaddr";
	     print MAIL "\nDate: $mydate";
	     close(MAIL);
	  }
	}
  }
  close(FILE);

  print "\nDone";
} else {
   print "\n * Usage: $0  \n";
}

#end
print "\n";
arunb@mylaptop:~$

2] How to Use –

arunb@mylaptop:~$ ./perl-prog/ftp_logparser.pl

* Usage: ./perl-prog/ftp_logparser.pl

arunb@mylaptop:~$ ./perl-prog/ftp_logparser.pl /var/log/ftp.log

Thank you,
Arun Bagul

Karesansui – Xen and kernel-based Virtual Machine (KVM) Manager

Karesansui – Xen and kernel-based Virtual Machine (KVM) Manager

Introduction –

Karesansui is the best web based kernel-based Virtual Machine (KVM) and Xen Manager.
Also one of the leading Japanese open source project.

Karesansui has Simple, easy web-based interface. Easy installation. Saves initial cost to use. Free for all.
Supports Xen/Kernel-based Virtual Machine(KVM) hypervisor. Other hypervisors/virtualization support on future plan.


Please refer the project URL for more information-  http://karesansui-project.info/

* How to install –

Please go through the steps as mentioned here- http://karesansui-project.info/wiki/1/En_tutorial

Thank You,
Arun Bagul

Top 5 Open Source Software Testing QA tools

Top 5 Open Source Software Testing QA tools

Introduction –
Software testing is the process of testing software product’s quality, risk and operability. Testing also helps to find bugs and fix them during product development.

1) Selenium (http://seleniumhq.org)

Selenium is a portable software testing framework for web applications. It supports number of popular programming languages, including C#, Java, Groovy, Perl, PHP, Python and Ruby for writing test cases. The tests can then be run against most modern web browsers. Selenium deploys on Windows, Linux, and Macintosh platforms.

2) Sikuli (http://sikuli.org)

Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API’s support. You can programmatically control a web page, a Windows/Linux/Mac OS X desktop application, or even an iphone or android application running in a simulator or via VNC.

3) Watir (http://watir.com)

Watir is an open-source (BSD) family of Ruby libraries for automating web browsers. It supports your app no matter what technology it is developed in. Watir drives browsers the same way people do.  It clicks links, fills in forms, presses buttons. Watir also checks results, such as whether expected text appears on the page. It allows you to write simple and flexible tests that are easy to read and maintain.

4) Unit Testing Tools –

Unit testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable
part of an application. In procedural programming a unit may be an individual function or procedure. In object-oriented programming a unit is usually an interface, such as a class.

** PHP
PHPUnit (http://phpunit.sourceforge.net/) ~ PHPUnit comes as PEAR package. It is Unit testing framework for PHP based on the “JUnit” framework for Java.
Amock (http://www.amock.org/) ~ Amock is a mock object library written in PHP 5, inspired by EasyMock.

** PERL
HTTP::Recorder  ~ is Browser-independent recorder that records interactions with web sites and produces scripts for automated playback. Recorder produces WWW::Mechanize scripts by default, but provides functionality to use your own custom logger. We have used ‘WWW::Mechanize’ to automate website load time testing using showslow (http://www.showslow.com/) and yslow firefox extension.

** Python
PyUnit ( http://pyunit.sourceforge.net/ ) ~ A unit testing framework for Python based on JUnit and XUnit, the de-facto standard frameworks for Java and SmallTalk respectively.
Achoo (http://web.quuxo.com/products/achoo/) ~ Achoo is a fluent interface for unit testing Python objects. Achoo makes it easy to make assertions about the properties and behaviours of Python objects.

** SQL
–  SQLUnit (http://sqlunit.sourceforge.net/) ~  SQLUnit is a regression and unit testing harness for testing database stored procedures.
An SQLUnit test suite would be written as an XML file. The SQLUnit harness, which is written in Java, uses the JUnit unit testing framework to
convert the XML test specifications to JDBC calls and compare the results generated from the calls with the specified results.

5)  iMacros (http://www.iopus.com/iMacros) –
iMacros is an extension for the Mozilla Firefox, Google Chrome, and Internet Explorer web browsers which adds record and replay functionality similar to that found in web testing and form filler software.
The macros can be combined and controlled via JavaScript.

Reference:- http://www.opensourcetesting.org/

Thank you,
Arun Bagul

How to take Live-Backup of VMware(Esxi Server) VM host

How to take Live-Backup of VMware(Esxi Server) VM host

Introduction
ghettoVCB is a simple shell script which you can use to take backup of LIVE/ONLINE backup of Vm hosts/machines/servers on VMware ESXi-4.x Servers. It is a alternate solution of VCB (Vmware Comprehension Backup)
We need to copy this script on VMware ESXi servers. Please enable SSH on ESXi server and then copy this tool/script on the server.

Step 1) Download and Copy ghettoVCB Tool on Physical host of VM like host01 etc at following location

[root@localhost]# wget -c http://communities.vmware.com/servlet/JiveServlet/download/8760-71-63631/ghettoVCB.tar.gz
/var/src/ghettoVCB
[root@localhost]# tar xvfz ghettoVCB.tar.gz
[root@localhost]# ls ghettoVCB
arunb_vm_to_backup  ghettoVCB.conf  ghettoVCB.sh
[root@localhost]#

** I am copying ghettoVCB to host01 which is my VMware ESXi Server

[root@localhost]# scp -r   ghettoVCB host01:/vmfs/volumes/datastore1/

Step 2) Then Login to  Physical host of VM  say host01

[root@localhost]# ssh host01

~ # ls /vmfs/volumes/datastore1/ghettoVCB
arunb_vm_to_backup          arunb_vm_to_backup-temlate  ghettoVCB.conf              ghettoVCB.sh
~ #

Step 3) Please add VM you want to BACKUP (Live Backup)

* Open file in vi (for example I wanted to take backup of host01vm8)

~ # vi /vmfs/volumes/4e1e2374-4417303c-c9e5-00259037a1e1/ghettoVCB/arunb_vm_to_backup

~ # cat /vmfs/volumes/4e1e2374-4417303c-c9e5-00259037a1e1/ghettoVCB/arunb_vm_to_backup
host01vm8
~ #

Step 4) Start Backup in DRY-RUN Mode First ~

[root@localhost]# ssh host01

~ # cd /vmfs/volumes/datastore1/ghettoVCB/
/vmfs/volumes/4e1e2374-4417303c-c9e5-00259037a1e1/ghettoVCB # ls
arunb_vm_to_backup          arunb_vm_to_backup-temlate  ghettoVCB.conf              ghettoVCB.sh

** DRY-RUN First –

/vmfs/volumes/4e1e2374-4417303c-c9e5-00259037a1e1/ghettoVCB # ./ghettoVCB.sh -g ./ghettoVCB.conf  -f arunb_vm_to_backup -d dryrun

Logging output to “/tmp/ghettoVCB-2011-04-09_06-45-38.log” …
2011-04-09 06:45:38 — info: ============================== ghettoVCB LOG START ==============================

2011-04-09 06:45:38 — info: CONFIG – USING GLOBAL GHETTOVCB CONFIGURATION FILE = ./ghettoVCB.conf
2011-04-09 06:45:38 — info: CONFIG – VERSION = 2011_05_22_1
2011-04-09 06:45:38 — info: CONFIG – GHETTOVCB_PID = 636718
2011-04-09 06:45:38 — info: CONFIG – VM_BACKUP_VOLUME = /vmfs/volumes/NFS_admin_nfs/vmware/vm_backup
2011-04-09 06:45:38 — info: CONFIG – VM_BACKUP_ROTATION_COUNT = 3

….
2011-04-09 06:45:39 — dryrun: VM_ID: 112
2011-04-09 06:45:39 — dryrun: VMX_PATH: /vmfs/volumes/datastore1/host01vm8/host01vm8.vmx
2011-04-09 06:45:39 — dryrun: VMX_DIR: /vmfs/volumes/datastore1/host01vm8
2011-04-09 06:45:39 — dryrun: VMX_CONF: host01vm8/host01vm8.vmx
2011-04-09 06:45:39 — dryrun: VMFS_VOLUME: datastore1
2011-04-09 06:45:39 — dryrun: VMDK(s):
2011-04-09 06:45:39 — dryrun:  host01vm8.vmdk 2020 GB
2011-04-09 06:45:39 — dryrun: INDEPENDENT VMDK(s):
2011-04-09 06:45:39 — dryrun: TOTAL_VM_SIZE_TO_BACKUP: 20 GB
2011-04-09 06:45:39 — dryrun: ###############################################

2011-04-09 06:45:39 — info: ###### Final status: OK, only a dryrun. ######

2011-04-09 06:45:39 — info: ============================== ghettoVCB LOG END ================================

/vmfs/volumes/4e1e2374-4417303c-c9e5-00259037a1e1/ghettoVCB #

Step 5) Start Backup in NOW  ~

** RUN BACKUP Command –

/vmfs/volumes/4e1e2374-4417303c-c9e5-00259037a1e1/ghettoVCB # ./ghettoVCB.sh -g ./ghettoVCB.conf  -f arunb_vm_to_backup

Logging output to “/tmp/ghettoVCB-2011-04-09_06-46-22.log” …
2011-04-09 06:46:22 — info: ============================== ghettoVCB LOG START ==============================

2011-04-09 06:46:22 — info: CONFIG – USING GLOBAL GHETTOVCB CONFIGURATION FILE = ./ghettoVCB.conf
2011-04-09 06:46:22 — info: CONFIG – VERSION = 2011_05_22_1
2011-04-09 06:46:22 — info: CONFIG – GHETTOVCB_PID = 636880

….
2011-04-09 06:46:22 — info: CONFIG – LOG_LEVEL = info
2011-04-09 06:46:22 — info: CONFIG – BACKUP_LOG_OUTPUT = /tmp/ghettoVCB-2011-04-09_06-46-22.log
2011-04-09 06:46:22 — info: CONFIG – VM_SNAPSHOT_MEMORY = 0
2011-04-09 06:46:22 — info: CONFIG – VM_SNAPSHOT_QUIESCE = 0
2011-04-09 06:46:22 — info: CONFIG – VMDK_FILES_TO_BACKUP = all
2011-04-09 06:46:22 — info: CONFIG – EMAIL_LOG = 0
2011-04-09 06:46:22 — info:

2011-04-09 06:46:25 — info: Initiate backup for host01vm8
2011-04-09 06:46:25 — info: Creating Snapshot “ghettoVCB-snapshot-2011-04-09” for host01vm8
Destination disk format: VMFS thin-provisioned
Cloning disk ‘/vmfs/volumes/datastore1/host01vm8/host01vm8.vmdk’…
Clone: 3% done.

++++++++++++++++++++++++++++++++++

That’s it!!

NOTE – PLEASE CHECK BACKUP PATH (where do you want to TAKE BACKUP) in “ghettoVCB.conf”

VM_BACKUP_VOLUME=/vmfs/volumes/NFS_admin_nfs/vmware/vm_backup

Please visit following urls for more information –

Blog – http://www.virtuallyghetto.com/
http://communities.vmware.com/docs/DOC-8760

Thank You
Mohit Vadhera

Steps needed to create LUNs and to make them accessible by windows ISCSI initiators.

Steps needed to create LUNs and to make them accessible by windows ISCSI initiators.

Steps needed to create LUN and to make accessible by windows ISCSI initiators.

  • mohit_netapp02*> lun setup

This setup will take you through the steps needed to create LUNs
and to make them accessible by initiators. You can type ^C (Control-C)
at any time to abort the setup and no unconfirmed changes will be made
to the system.

  • Do you want to create a LUN? [y]: y
  • Multiprotocol type of LUN (solaris/windows/hpux/aix/linux/netware/vmware/windows_gpt)[linux]: windows

A LUN path must be absolute. A LUN can only reside in a volume or
qtree root. For example, to create a LUN with name “lun0” in the
qtree root /vol/vol1/q0, specify the path as “/vol/vol1/q0/lun0”.

  • Enter LUN path: /vol/test_vol/test_lun0

A LUN can be created with or without space reservations being enabled.
Space reservation guarantees that data writes to that LUN will never
fail.

  • Do you want the LUN to be space reserved? [y]: Object action: admin.util.true

Size for a LUN is specified in bytes. You can use single-character
multiplier suffixes: b(sectors), k(KB), m(MB), g(GB) or t(TB).

  • Enter LUN size: 55g

You can add a comment string to describe the contents of the LUN.
Please type a string (without quotes), or hit ENTER if you don’t
want to supply a comment.

  • Enter comment string:

The LUN will be accessible to an initiator group. You can use an
existing group name, or supply a new name to create a new initiator
group. Enter ‘?’ to see existing initiator group names.

  • Name of initiator group [windows]: Test_windows

Type of initiator group Test_windows (FCP/iSCSI) [iSCSI]: iSCSI
An iSCSI initiator group is a collection of initiator node names.Each
node name can begin with either ‘eui.’ or ‘iqn.’ and should be in the
following formats: eui.{EUI-64 address} or iqn.yyyy-mm.{reversed domain
name}:{optional string composed of alphanumeric characters, ‘-‘, ‘.’
and ‘:’}
Eg: iqn.2001-04.com.acme:storage.tape.sys1.xyz or eui.02004567A425678D
You can separate node names by commas. Enter ‘?’ to display a list of
connected initiators. Hit ENTER when you are done adding node names to
this group.

  • Enter comma separated nodenames: ?

Initiators connected on adapter ism_sw1:
iSCSI Initiator Name                  Group
iqn.1991-05.com.microsoft:appoms2   windows
iqn.1991-05.com.microsoft:colodc1.projecty.com   windows
iqn.1991-05.com.microsoft:appoff1
Enter comma separated nodenames: iqn.1991-05.com.microsoft:appoms2
Enter comma separated nodenames:
The initiator group has an associated OS type. The following are
currently supported: solaris, windows, hpux, aix, linux, netware
or vmware.

  • OS type of initiator group “Test_windows” [windows]: windows

The LUN will be accessible to all the initiators in the
initiator group. Enter ‘?’ to display LUNs already in use
by one or more initiators in group “Test_windows”.

  • LUN ID at which initiator group “Test_windows” sees “/vol/test_vol/test_lun0” [0]: 1

LUN 1 is already mapped to by initiator iqn.1991-05.com.microsoft:appoms2
in group windows
LUN ID at which initiator group “Test_windows” sees “/vol/test_vol/test_lun0” [0]: 2
LUN Path                : /vol/test_vol/test_lun0
OS Type                 : windows
Size                    : 55.0g (59057510400)
Initiator Group         : Test_windows
Initiator Group Type    : iSCSI
Initiator Group Members : iqn.1991-05.com.microsoft:appoms2
Mapped to LUN-ID        : 2

  • Do you want to accept this configuration? [y]: y
  • Do you want to create another LUN? [n]: n
  • mohit_netapp02*> lun showq

lun: unrecognized command “showq”
The following commands are available; for more information
type “lun help <command>”
aluadb              destroy             move                set
attribute           df                  offline             setup
clone               geometry            online              share
comment             help                rescan              show
config              hist                resize              snap
config_check        map                 select              stats
create              maxsize             serial              unmap
dbedit

  • mohit_netapp02*> lun show

/vol/bod_iscsi/asm1            5g (5368709120)    (r/w, online, mapped)
/vol/mohitsnap/lun0         502.0g (539019048960)  (r/w, online, mapped)
/vol/test_vol/test_lun0     55.0g (59057510400)   (r/w, online, mapped)

 

 

Maximum LUN resize limits on NetApp

Be aware there are some limits to resize the Netapp size. Due to the geometry of the LUN at the time it was created it can only be resized to a maximum of ~10 times the original size of the LUN.

One way to see this is at the command line, for the command to be available you need to run ‘priv set diag‘ and then you will have access to the ‘lun geometry‘ command. From there you can see the LUN size as well as the maximum LUN size that is available.

 

Here is a table of some of the results:

LUN Size Maximum Resize
< 50g 502g
51-100g 1004g
101-150g 1506g
151-200g 2008g
201-251g 2510g
252-301g 3012g
302-351g 3514g
352-401g 4016g

etc..

How to Switch (change) from Unity Desktop to GNOME Desktop theme

How to Switch (change) from Unity Desktop to GNOME Desktop theme

Introduction –

For some people (like me) the new default Unity desktop in Ubuntu doesn’t look easy to opertate.  However the Ubuntu is trying to differentiate itself with a distinctive to improve user experience (Ubuntu is rocking!!). Ubuntu 11.04, is using Unity as its default desktop interface instead of the classic GNOME.

If you’re running Ubuntu 11.4 and don’t like to use Unity, you can switch back to the old classic GNOME interface as shown below.

It’s actually pretty simple  to choose between Unity and the classic GNOME user interface…

[1] During login time, Go to your logon screen, select the user you want to log in as.
– At the bottom task bar you can choose between different modes. Select “Ubuntu Classic”.
– Log in as normal.

* Another way to change theme is..

[2] Go to your (click on) “logout” button on top panel, select the “System Settings”..

– This will open “Control Center” then click on “Login Screen” setting tool.
– Where you can choose different themes. Select “Ubuntu Classic”.
NOTE- Please unlock before changing the setting ie provide your password

That’s it.

Regards,
Arun

Benchmarking for System and High Memory-CPU utilization – Part 1

Benchmarking for System and High Memory-CPU utilization – Part 1

Hello Guy’s,

Every System Admin and technical person who is working or doing any kind of High Availability stuff, He really needs bench marking to check and verify system health under development, here I am very beginner in this kind of system benchmarking to know how server/CPU/memory and network reacts when doing this and given load (purposely) on system to use or run the service or process. I will explain how I have done this in the part -2 😉

Memory Util

 

CPU Util

 

-Ravi

IndianGNU.org – Fourth Anniversary!!

IndianGNU.org – Fourth Anniversary!!

Dear All,

Today , we are celebrating fourth anniversary!! We have successfully completed four years and still growing with full speed! It was simply impossible without your support, contribution and hard work.

openlsm Project ~

This year was tough for openlsm and we will be launching Admin panel development version by end of Dec,2010. We are expecting a lot more from second milestone…

Cheers,
IndianGNU.org