Overview: 

                    Chef enables you to automate your infrastructure. It provides a command line tool called knife to help you manage your configurations. Using the knife EC2 plugin you can manage your Amazon EC2 instances with Chef. knife EC2 makes it possible to create and bootstrap Amazon EC2 instances in just one line – if you go through a few setup steps. Following are steps to setup your Chef installation and AWS configuration so that we can easily bootstrap new Amazon EC2 instances with Chef’s knife

Following are the steps need to launch AWS instance.

A. Installation and Configuration of Knife Ec2 instance

  1.  Instaiing knife-ec2 instance:

a. If you’re using ChefDK, simply install the Gem:
$ chef gem install knife-ec2

b. If you’re using bundler, simply add Chef and Knife EC2 to your Gemfile:
$ gem ‘knife-ec2′

c. If you are not using bundler, you can install the gem manually from Rubygems:
$ gem install knife-ec2

In my setup I used ChefDK.

2.  Add ruby’s gem path to PATH variable to work knife-ec2 with AWS

$  export PATH=/root/.chefdk/gem/ruby/2.1.0/bin:$PATH

 3. Add the AWS credentials of knife user to knife configuration file i.e ~/.chef/knife.rb.

——————————————————————————–

knife[:aws_access_key_id] = “user_key_ID”
knife[:aws_secret_access_key] = “User_secret_key”

———————————————————————————

 B. Prepare SSH access to Amazon EC2 Instance.
 1. Configure Amazon Security Group
As Amazon blocks all incoming traffic to EC2 instances by default. We’ll need to open the SSH(22) port for knife to access a newly created instance. Also HTTPS(443) port to communicate launched instance’s chef client with chefserver.Just login to the AWS management console and navigate to EC2 Services Compute Security Groups default group.Then add a rule for Type SSH and HTTPS with Source Anywhere and save the new inbound rule

2. Generate Key Pair in AWS Console
To enable SSH access to Amazon EC2 instances you need to create a key pair. Amazon will install the public key of that key pair on every EC2 instance. knife will use the private key of that key pair to connect to your Amazon EC2 instances. Store the downloaded private key knife.pem in “~/.ssh/knife.pem” of ec2-user.

3. Prepare SSH configuration to avoid host key mismatch errors:
Create “/home/ec2-user/.ssh/config and add below content:
_________________________________________________________
Host ec2*compute-1.amazonaws.com
StrictHostKeyChecking no
User ec2-user
IdentityFile /home/ec2-user/.ssh/knife.pem
_________________________________________________________

 C. Choose an AMI for your Amazon EC2 instances
We need to choose the right AMI for region, architecture and root storage. Note down the AMI ID (ami-XXXXXXXX) to use it with knife.

D. Create an EC2 instance using Chef knife:
Now, it’s time to use knife to fire up and configure a new Amazon EC2 instance. Execute below command to create instance.
$sudo knife ec2 server create -r “recipe[dir]” -I ami-0396cd69 -f m3.large -S knife -i /home/ec2-user/.ssh/knife.pem –ssh-user ec2-user –region us-east-1 -Z us-east-1b

Options:
-r is the run_list I want to associate with the newly created node. You can put any roles and recipes you like here
-I is the AMI ID
-f is the Amazon EC2 instance type
-S is the name you gave to the SSH key pair generated in the AWS management console
-i points to the private key file of that SSH key pair as downloaded when the key pair was created in the AWS management console
–ssh-user the official EC2 AMIs use ec2-user as the default user
–region us-east-1 If you want your instances to be deployed in any specific Amazon AWS region, add this parameter and the desired region
-Z us-east-1b is the availability zone within your region

NOTE:
If you did not give the –r i.e run list with above mentioned command, then it throws the Exception below:

     “EXCEPTIONS : NoMethodError Undefined method ‘empty?’ for nil:NilClass”

E.   Terminate instance and delete the corresponding Chef node
$ knife ec2 server delete i-XXXXXXXX –region us-east-1
$ knife node delete i-XXXXXXXX

(i-XXXXXXXX is the ID of the instance as found in the AWS management console)

Introduction-

Ravi Bhure co-founder of IndianGNU attended  2nd Check_MK Conference in Munich, Germany last October,2015 as delegate.

Ravi talked about “Migration from Solarwinds to Check_MK” in Check_MK Munich Conference.

For more information on presentation please visit https://www.linkedin.com/pulse/open-source-monitering-deadly-combo-migration-from-checkmk-burkule

Thank you,

Arun

Introduction-

Website Functional testing using Browser automation is very important and better than just monitoring url with 200 OK! Your Application or Website URL may be working fine but this doesn’t mean that all components are working fine. So testing your Web Application functionalities with screenshot is important in cloud based product and Uptime Reporting for Customers.

Few years back I wrote article “Firefox yslow and Showslow for Website testing and performance” Ref url- http://www.indiangnu.org/2012/firefox-yslow-and-showslow-for-website-testing-and-automation/

The Mechanize library is used for automating interaction with websites and available for Perl, Python and Ruby..
URL-
http://search.cpan.org/~ether/WWW-Mechanize-1.75/lib/WWW/Mechanize.pm
http://wwwsearch.sourceforge.net/mechanize/
https://pypi.python.org/pypi/mechanize/
http://mechanize.rubyforge.org/

There are several wrappers around “Mechanize” designed for functional testing of web applications: zope.testbrowser and twill

Top 5 Products for “Website Functional testing using Browser automation”…

1) Selenium Browser Automation (http://www.seleniumhq.org) -
Selenium automates browsers. Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). Selenium WebDriver is the successor to Selenium RC. Selenium WebDriver accepts commands (sent in Selenese, or via a Client API) and sends them to a browser. This is implemented through a browser-specific browser driver, which sends commands to a browser, and retrieves results. Most browser drivers actually launch and access a browser application (such as Firefox or Internet Explorer); there is also an HtmlUnit browser driver, which simulates a browser using HtmlUnit.

Selenium Grid- Selenium Grid is a server that allows tests to use web browser instances running on remote machines. With Selenium Grid, one server acts as the hub. the hub has a list of servers that provide access to browser instances (WebDriver nodes)

2) Splinter (https://splinter.readthedocs.org) -
Splinter is an open source tool for testing web applications using Python. It lets you automate browser actions, such as visiting URLs and interacting with their items. It supports multi webdrivers (chrome webdriver, firefox webdriver, phantomjs webdriver, zopetestbrowser, remote webdriver), support to iframe and alert and execute javascripts.

3) twill (http://twill.idyll.org) -
twill: a simple scripting language for Web browsing. twill is a simple language that allows users to browse the Web from a command-line interface. With twill, you can navigate through Web sites that use forms, cookies, and most standard Web features.

4) zope.testbrowser (https://pypi.python.org/pypi?:action=display&name=zope.testbrowser) -
zope.testbrowser provides an easy-to-use programmable web browser with special focus on testing. It is used in Zope, but it’s not Zope specific at all. For instance, it can be used to test or otherwise interact with any web site.

5) PAMIE  (http://pamie.sourceforge.net) -
P.A.M.I.E. – stands for Python Automated Module For I.E.  Pamie’s main use is for testing web sites by which you automate the Internet Explorer client using the Pamie scripting language. Simply create a script using the free PythonWin IDE that comes with the win32all extensions. import cPAMIE and use the Pamie Scripting Language (PSL) to write a script that simulates a user navigating a web site. It’s simple to use.

There are many product available for web site monitoring…

http://www.monitor.us/

https://ghostinspector.com/

https://www.browserstack.com

https://www.alertbot.com/

https://www.pingdom.com/

Thank you,
Arun Bagul

CMDB-
Configuration management database (CMDB) is a repository that acts as a data warehouse for information technology (IT) organizations. Its contents are intended to hold a collection of IT assets that are commonly referred to as configuration items (CI), as well as descriptive relationships between such assets. A Configuration item type (or CI Type) is the data type of the element or configuration item an enterprise wishes to store within the CMDB. At a minimum, all software, hardware, network, and storage CI Types are stored and tracked in a CMDB.

Top 5 CMDB Tools-

1) Combodo iTOP-

IT Operations Portal: a complete open source, ITIL, web based service management tool including a fully customizable CMDB, a helpdesk system and a document management tool. iTop also offers mass import tools and web services to integrate with your IT.

2) OneCMDB -

OneCMDB is a CMDB aimed at small and medium sized businesses. OneCMDB can be used as a stand-alone CMDB to keep track of software and hardware assets and their relations. Thanks to its open API:s it can also be a flexible and powerful Configuration Management engine for other Service Management software.

3) i-doit -

i-doit allows a rich amount of technical information to be filed for each element from a wall outlet to a mainframe in a structured way. Every employee can access this information easily (and in a selective way) through a web browser. Due to its modular architecture, it is possible to deploy functionality add-ons or develop extensions.

4) CMDBuild -

CMDBuild is a configurable web application to model and manage a database containing assets (CMDB stands for “Configuration and Management Data Base”) and handle related workflow operations.
The aim is to let the operators have full control of the assets used, knowing exactly composition, dislocation, functional relations and history.

5) HP UCMDB (Commercial) -

UCMDB is a CMDB software product produced by Hewlett Packard supporting ITIL Configuration Management and which features a Configuration Management Database, as well as a mechanism for the automatic discovery of IT infrastructure components, such as computers, network devices and composing relationships between them.

Thank you,
Arun

Introduction-

Cloud computing has been evolving into different technology areas such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). These technology areas are finding increasing adoption in the marketplace.

A recent report published by International Data Corporation (IDC) has indicated that the Cloud Computing market is expected to cross $70 billion in 2015. Ref- https://www.idc.com/getdoc.jsp?containerId=prUS25797415

The economic drivers for adoption of Cloud Technologies have been well understood in the market.

Lower Investments: Cloud offerings typically have very low upfront costs. All SaaS product offerings are charged on a pay per use monthly subscription basis, which implies lower cost of ownership.
No Infrastructure Costs: Cloud service offerings use cloud based infrastructures such as Amazon or Rackspace or a private cloud. Hence the customer does not have to bear the cost of the infrastructure setup with cloud services.
Outsourced Technology Expertise: With cloud offerings, customers get a ready to use product that can be immediately consumed by the end user. Customer does not need to invest in an in-house IT team having that particular technology expertise.
Simplified Management: With offerings in the cloud, the service provider deals with on-going management, maintenance and upgrades of technology. The customer can focus on his core business needs.

So Why I’m telling this story to you? As infrastructure engineer I believe that this is going to change how we do things in Traditional vs Could Infrastructure. Let us take example of “Monitoring”. Here in this blog, I’m going to talk about monitoring of Infrastructure and applications in Traditional vs Cloud. Adoption of “Cloud computing” is going to change they way we do monitoring of our servers and applications.

Monitoring as a Service (MaaS) in the Cloud! -

What does Monitoring-as-a-Service (MaaS) mean?
Monitoring-as-a-service (MaaS) is one of many cloud delivery models under anything as a service (XaaS). It is a framework that facilitates the deployment of monitoring functionalities for various other services and applications within the cloud. The most common application for MaaS is online state monitoring, which continuously tracks certain states of applications, networks, systems, instances or any element that may be deployable within the cloud. MaaS offerings consist of multiple tools and applications meant to monitor a certain aspect of an application, server, system or any other IT component. There is a need for proper data collection, especially of the performance and real-time statistics of IT components, in order to make proper and informed management possible.

IT infrastructure monitoring should an essential part of the IT Management Policy for an organization that is reliant on IT infrastructure. Proactive monitoring provides business continuity, quicker disaster recovery and easier capacity planning for all mission critical applications.

Monitoring as a Service (MaaS) in the Cloud is a concept that combines the benefits of cloud computing technology and traditional on-premise IT infrastructure monitoring solutions. MaaS is a new delivery model that is suited for organizations looking to adopt a monitoring framework quickly with minimal investments.

Traditional On-Premise Monitoring Framework-

On premise monitoring is the traditional deployment model for monitoring private networks (internal IT infrastructure). This has been a very effective model over the years and works well for organization that can afford to implement this monitoring framework. On-premise monitoring involves purchase of software tools and investing in monitoring infrastructure and skilled IT personnel.

On-Premise monitoring provides the following benefits:

In-House Monitoring Infrastructure:Customers can own the in-house infrastructure for monitoring. This implies more control over the infrastructure with regards to upgrades, maintenance and management
Higher Levels of Security:Since the monitoring infrastructure is located in-house, customer gets better security where the monitoring tool does not need to cross firewall domains and connect over the open internet.
Inherent Connectivity to Internal Assets:Monitoring infrastructure is already a part of the internal network (LAN and MPLS). Hence connecting to all the infrastructure assets is easy.
Real Time Monitoring Data:On-premise monitoring provides real-time data where alerts are generated and shown to the customer immediately. The monitoring dashboard and email servers are all within the customer premise and hence there are no delays.
Customization and Extensions:On-premise monitoring solutions can be heavily customized to meet the exact needs of a specific customer environment. This could be in the form of monitoring of custom applications or personalized dashboards and escalation matrices.
The Adoption of Cloud and Software as a Service Delivery Models

Monitoring as a Service Offering -

The monitoring as a service (MaaS) offering provides a monitoring solution based on a monitoring infrastructure in the cloud. The MaaS vendor invests in the monitoring framework including the hardware, monitoring software and specialized IT personnel on behalf of the customer. The customer just needs to pay for the service he wants to use – on a subscription model similar to any SaaS product offering.

Benefits of Monitoring as a Service (MaaS)

The following are the benefits of a monitoring as a service (MaaS) product:

    Ready to Use Monitoring Tool Login: The vendor takes care of setting up the hardware infrastructure, monitoring tool, configuration and alert settings on behalf of the customer. The customer gets a ready to use login to the monitoring dashboard that is accessible using an internet browser. A mobile client is also available for the MaaS dashboard for IT administrators.
    Inherently Available 24x7x365: Since MaaS is deployed in the cloud, the monitoring dashboard itself is available 24x7x365 that can be accessed anytime from anywhere. There are no downtimes associated with the monitoring tool.
Easy Integration with Business Processes: MaaS can generate alert based on specific business conditions. MaaS also supports multiple levels of escalation so that different user groups can get different levels of alerts.
Cloud Aware and Cloud Ready: Since MaaS is already in the cloud, MaaS works well with other cloud based products such as PaaS and SaaS. MaaS can monitor Amazon and Rackspace cloud infrastructure. MaaS can monitor any private cloud deployments that a customer might have.
Zero Maintenance Overheads: As a MaaS, customer, you don’t need to invest in a network operations centre. Neither do you need to invest an in-house team of qualified IT engineers to run the monitoring desk since the MaaS vendor is doing that on behalf of the customer.
    Price Sensitive Customers: For small and medium enterprises, MaaS provides cost effective pay per use pricing model. Customers don’t need to make any heavy investments neither in capital expenditures (capex) nor in operating expenditures (opex).

Monitoring as a service (MaaS) is an attractive choice for the following scenarios:

Cloud Based SaaS and PaaS offering Add-On: MaaS provides a better technology fit for monitoring cloud based SaaS and PaaS offerings. MaaS can be provided as an add-on product offering along with SaaS and PaaS.
Distributed Infrastructure Assets: In scenarios where the IT infrastructure assets are distributed across different locations and branch offices, MaaS is a good option since the monitoring infrastructure is centralized in the cloud and can easily monitor all distributed infrastructure assets.
Mixture of Cloud and On-Premise Infrastructure: MaaS is already in the cloud. Hence in deployments where customer has a mix of on-premise and cloud infrastructure, MaaS provides good monitoring options for the hybrid environment.
Multitenant Monitoring Requirements: For vendors offering multi-tenant functionality on their hosted services, MaaS provides a strong backend framework for monitoring the multi-tenant services and their availability.

Thank you,

Arun Bagul

Introduction-

A. P. J. Abdul Kalam (Avul Pakir Jainulabdeen Abdul Kalam) was the 11th President of India from 2002 to 2007. He was great engineer,scienctist and Leader.

I pay tribute to him and May his soul rest in peace.

ॐ असतो मा सद्गमय ।
तमसो मा ज्योतिर्गमय ।
मृत्योर्मा अमृतं गमय ।
ॐ शान्तिः शान्तिः शान्तिः ॥
Om Asato Maa Sad-Gamaya |
Tamaso Maa Jyotir-Gamaya |
Mrtyor-Maa Amrtam Gamaya |
Om Shaantih Shaantih Shaantih ||

Thank you,

Arun Bagul

Here’s an open source IaaS platform to set up an on-demand, elastic cloud computing service. It enables utility computing services by allowing cloud service providers to offer self-service compute instances, storage volumes and networking configurations, and to set up a private cloud for internal use.

Apache CloudStack is an open source, multi-tenanted cloud orchestration platform, which is used to build private, public and hybrid IaaS clouds by pooling computing resources. It manages computing, networking and storage resources. CloudStack is hypervisor agnostic; it uses multiple virtualisation platforms such as KVM, vSphere and XenServer. It supports the Amazon Web Services API, apart from its own APIs.

Features and use cases:
CloudStack supports Citrix XenServer, VMware vSphere and KVM on Ubuntu or CentOS. It can manage multiple geographically distributed data centres. The CloudStack API gives programmatic access to all managed resources and hence it is easier to create command line tools. Multi-node installation support and load balancing makes it highly available. In addition, MySQL replication is also useful for maintaining high availability.

Service providers and organisations use CloudStack to set up an elastic and on-demand IaaS. It can also be used to set up an on-premise private cloud behind the organisation’s firewall for internal purposes like gaining better control over infrastructure.

A host is a computer that provides the computing resources such as the CPU, storage, memory, networking, etc, to run the virtual machines. Each host has a hypervisor installed to manage the VMs. As CloudStack is hypervisor agnostic, multiple hypervisor-enabled servers such as a Linux KVM-enabled server, a Citrix XenServer server and an ESXi server can be used.

One or more primary storage is coupled with a cluster that stores the disk volumes for all the VMs running on hosts in that specific cluster.
A cluster consists of one or more hosts and one or more primary storage servers. In other words, a cluster can be considered as a set of XenServer servers or a set of KVM servers.

A CloudStack pod represents a single rack. A CloudStack pod consists of one or more clusters of hosts and one or more primary storage servers. Hosts in the same pod are in the same subnet.

A zone typically corresponds to a single data centre; it is permissible to have multiple zones in a data centre. Pods are contained within zones. Each zone can contain one or more pods. Zones can be public or private.

Secondary storage is shared by all the pods in the zone that stores templates, ISO images and disk volume snapshots.

 A CloudStack installation consists of a management server and the cloud infrastructure. Hosts, storage and IP addresses are managed by the management server. The minimum installation consists of one virtual machine running the CloudStack management server and another virtual machine to act as the cloud infrastructure the host running the hypervisor software.

The management server manages cloud resources, and the administrator can manage and interact with the management server by using a UI and APIs. It also managew s the assignment of guest VMs to particular hosts, the assignment of public and private IP addresses, templates and ISO images, as well as snapshots.

The CloudStack API:

 The CloudStack API supports three access roles: root admin, domain admin and user. The root admin can access all the features in addition to both virtual and physical resource management; the domain admin can access only the virtual resources that belong to the administrator’s domain, while the user can access the features that allow the management of the user’s virtual machines, storage and network.

To use the CloudStack API, knowledge of Java or PHP, HTTP GET/POST and query strings, XML or JSON, URL of the CloudStack server, and API key and secret key is necessary.

Thanks,

Pravin Babar.

This issue affects all products which use the Bash shell and parse values of environment variables. This issue is especially dangerous as there are many possible ways Bash can be called by an application. Quite often if an application executes another binary, Bash is invoked to accomplish this. Because of the pervasive use of the Bash shell, this issue is quite serious and should be treated as such.

All versions prior to those listed as updates for this issue are vulnerable to some degree.

My infected OS version is CentOS-6 and bash version 4.1.2

[root@host75 ~]# lsb_release -a
lsb_release -a
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.4 (Final)
Release: 6.4
Codename: Final


[root@host75 ~]# bash --version
bash --version
GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


Diagnostic Steps:

To test if your version of Bash is vulnerable to this issue, run the following command:

$ env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

If the output of the above command looks as follows:

vulnerable
this is a test

hmm, I got infected!

[root@host75 ~]# env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
vulnerable
this is a test

You are using a vulnerable version of Bash. The patch used to fix this issue ensures that no code is allowed after the end of a Bash function. Thus, if you run the above example with the patched version of Bash, you should get an output similar to:

$ env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'
this is a test


If your system is vulnerable, update to the most recent version of the Bash package by running the following command:

$yum update bash

This fix my bash ??????

[root@host75 ~]# yum update bash
Loaded plugins: fastestmirror, security, tmprepo
Loading mirror speeds from cached hostfile
epel/metalink | 15 kB 00:00
* base: centos.eecs.wsu.edu
* epel: mirrors.kernel.org
* extras: centos.chi.host-engine.com
* updates: mirror.raystedman.net
base | 3.7 kB 00:00
epel | 4.4 kB 00:00
epel/primary_db | 6.3 MB 00:05
extras | 3.3 kB 00:00
updates | 3.4 kB 00:00
updates/primary_db | 5.3 MB 00:04
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package bash.x86_64 0:4.1.2-14.el6 will be updated
---> Package bash.x86_64 0:4.1.2-15.el6_5.1 will be an update
--> Finished Dependency Resolution

Complete!

Test if update fixed to patch your bash

[root@host75 ~]# env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'
this is a test


How does this impact systems:
This issue affects all products which use the Bash shell and parse values of environment variables. This issue is especially dangerous as there are many possible ways Bash can be called by an application. Quite often if an application executes another binary, Bash is invoked to accomplish this. Because of the pervasive use of the Bash shell, this issue is quite serious and should be treated as such.

All versions prior to those listed as updates for this issue are vulnerable to some degree.

See the appropriate remediation article for specifics.

Functions written in Bash itself do not need to be changed, even if they are exported with “export -f”. Bash will transparently apply the appropriate naming when exporting, and reverse the process when importing function definitions.

Ref:

http://www.bbc.com/news/technology-29361794

https://www.us-cert.gov/ncas/current-activity/2014/09/24/Bourne-Again-Shell-Bash-Remote-Code-Execution-Vulnerability

https://access.redhat.com/articles/1200223

https://securityblog.redhat.com/2014/09/24/bash-specially-crafted-environment-variables-code-injection-attack/

http://lists.gnu.org/archive/html/bug-bash/2014-09/threads.html

https://rhn.redhat.com/errata/RHSA-2014-1306.html

Introduction-

Many many congratulations to ISRO (Indian Space Research Organisation) team for making Mangalyaan successful!. we are proud of you all!

Short story of Mangalyaan!-

India’s maiden mission to Mars, the Mangalyaan, has arrived in orbit after a 300-day marathon covering over 670 million kilometres.

“India will become the first Asian country to have achieved this and if it happens in the maiden attempt itself, India could become the first country in the world to have reached distant Mars on its own steam in the first attempt,” said Isro chairman K Radhakrishnan as it approached.

“We have gone beyond the boundaries of human enterprise” – Narendra Modi (PM. India)

Ref Links-  http://www.isro.org/mars/home.aspx

http://en.wikipedia.org/wiki/Mars_Orbiter_Mission

Thanks you,
Arun Bagul

This Error will Occur when your are trying to clone a VM whose tools are out of date.