Category: Azure

Packer – How packer has simplified Cloud provisioning

Packer – How packer has simplified Cloud provisioning

What is Packer  –

Packer (HashiCorp) is an open source tool for creating identical machine images for multiple platforms from a single source configuration. Packer is lightweight, creating machine images for multiple platforms in parallel. Packer does not replace configuration management like Chef or Puppet. In fact, when building images, Packer can use tools like Chef or Puppet to install software onto the image.

A machine image is a single static unit that contains a per-configured operating system and installed software which is used to quickly create new running machines. Machine image formats change for each platform. Some examples include AMIs for EC2, VMDK/VMX files for VMware, OVF exports for VirtualBox, etc.

Packer and Continuous Delivery (CD) –

Packer is lightweight, portable, and command-line driven. This makes it the perfect tool to put in the middle of your continuous delivery pipeline. As part of this pipeline, the newly created images can then be launched and tested, verifying the infrastructure changes work. If the tests pass, you can be confident that the image will work when deployed. This brings a new level of stability and testability to infrastructure changes.

Packer template –

A Packer template is a configuration file that defines the image you want to build and how to build it. Packer templates use the Hashicorp Configuration Language (HCL). Packer configuration  has different block in configuration like mentioned below…

  • Packer Block –  The packer {} block contains Packer settings, including specifying a required Packer version.
  • Source Block – Thesource {} block configures a specific builder plugin, which is then invoked by a build block. Source blocks use builders and communicators to define what kind of virtualization to use, how to launch the image you want to provision, and how to connect to it.
  • Build block – The build  { } block defines what Packer should do with the Virtual Machine or Docker container after it launches.

Builders are responsible for creating machines and generating images from them for various platforms. For example, there are separate builders for EC2, VMware, VirtualBox, etc.

Please find the supported builder list here – https://www.packer.io/docs/builders

- virtualbox-iso
- vmware-iso
- hyperv-iso
- docker
  • Provisioner block – The provisioner { } block defines what Packer should do with the image build or Docker container after it launches.

Provisioners use builtin and third-party software to install and configure the machine image after booting. Provisioners prepare the system for use, so common use cases for provisioners include:

  • installing packages
  • patching the kernel
  • creating users
  • downloading application code
  • Post-Processors – Post-processors run after the image is built by the builder and provisioned by the provisioner(s). Post-processors are optional, and they can be used to upload artifacts, re-package, or more.
  • Communicators – are the mechanism Packer uses to upload files, execute scripts, etc. with the machine being created.

Communicators are configured within the builder section. Packer currently supports three kinds of communicators:

  • none – No communicator will be used. If this is set, most provisioners also can’t be used.
  • ssh – An SSH connection will be established to the machine. This is usually the default.
  • winrm – A WinRM connection will be established.

Packer Workflow –

The Packer workflow takes a defined source (usually an operating system installation media in ISO format or an existing machine), creates a machine on the builder, executes the Provisioner’s defined tasks and generates an output. Doing this on Hyper-V, VMWare workstation and Docker may look like what is shown below.

How to use Packer –

Initialize your Packer configuration and check and validate packer template…

$ packer init .
$ packer fmt .
$ packer validate .

Build the image with the packer build command. Packer will print output like what is shown below.

$ packer build new-os-image-template.hcl

Reference –

Packer software – https://www.packer.io

Thank you,

Arun Bagul

(Personal Blog and This is based on my knowledge)

IaC ~ Terraform and Pulumi

IaC ~ Terraform and Pulumi

Terraform and Pulumi

Before reading this blog I would recommend to read this blog – https://www.indiangnu.org/2017/top-5-infrastructure-as-code-iac-software/

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

Key feature of Terraform and Pulumi are..

  1. IaC (Infrastructure as Code) for better management of
  2. Infrastructure and state management
  3. Supports all Cloud providers like Azure, GCP, AWS and virtualization infrastructure like VMware and HyperV
  4. Change Automation
  5. Help you to adapt IaaS, SaaS and PaaS

How to user Terraform –

Terraform has 3 steps . First is “terraform init” initialization step in which Terraform will check and download required providers and plugins. Second is “terraform plan” in this step Terraform will create plan of change in your infrastructure and user has choice to verify and confirm then plan and then start third step ie “terraform apply” this in step Terraform will create resources in providers infrastructure like azure, aws or gcp. Terraform will also save all objects created in Terraform stafe “.tfstate” file and we can use same configuration to add or delete or update resource in same workflow….

To Debug Terraform error we can define debug and log fails mentioned below-

export TF_LOG_PROVIDER=TRACE
export TF_LOG_CORE=TRACE
export TF_LOG_PATH=logs.txt
terraform init
terraform plan
terraform apply

For formatting and configuration validation use below command –

terraform validate
terraform fmt
 
terraform refresh

HashiCorp Configuration Language (HCL)

In Terraform we can define infrastructure required to deploy as code using HCL.

HCL is a toolkit for creating structured configuration languages that are both human- and machine-friendly, for use with command-line tools. Although intended to be generally useful, it is primarily targeted towards devops tools, servers, etc. HCL has both a native syntax, intended to be pleasant to read and write for humans, and a JSON-based variant that is easier for machines to generate and parse. HCL syntax is designed to be easily read and written by humans, and allows declarative logic to permit its use in more complex applications

What is Pulumi ?

Pulumi is a modern infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure.

Pulumi vs. Terraform

Here is a summary of the key differences between Pulumi and Terraform:

ComponentPulumiTerraform
Language SupportPython, TypeScript, JavaScript, Go, C#, F#Hashicorp Configuration Language (HCL)
State ManagementManaged through Pulumi Service by default, self-managed options availableSelf-managed by default, managed SaaS offering available
Provider SupportNative cloud providers with 100% same-day resource coverage plus Terraform-based providers for additional coverageSupport across multiple IaaS, SaaS, and PaaS providers
OSS LicenseApache License 2.0Mozilla Public License 2.0

Ref –

Pulumi  – https://www.pulumi.com

Terraform – https://www.terraform.io

HCL –  https://github.com/hashicorp/hcl

Terraform Error – https://learn.hashicorp.com/tutorials/terraform/troubleshooting-workflow

Thank you,

Arun Bagul

(Personal Blog and This is based on my knowledge)

Azure VM Inventory report

Azure VM Inventory report

Hi, This script will help you to pull Azure VM details. You can use this script for various purpose. This script makes use of AzureRM module, please install below prerequisite modules :-

Install-Module -Name AzureRM.Compute
Install-Module -Name AzureRM.Network

    # Sign into Azure Portal
    login-azurermaccount

    # Fetching subscription list
    $subscription_id = "xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"      # Enter your subscription details

    # Fetch current working directory 
    $working_directory = "c:\AzureInventory"

    new-item $working_directory -ItemType Directory -Force

function Get-AzureInventory{

# Selecting the subscription
Select-AzureRmSubscription -Subscription $subscription_id

# Fetch the Virtual Machines from the subscription
$azureVMDetails = get-azurermvm

# Fetch the NIC details from the subscription
$azureNICDetails = Get-AzureRmNetworkInterface | ?{ $_.VirtualMachine -NE $null}

# Fetch the Virtual Networks from the subscription
$azureVirtualNetworkDetails = Get-AzureRmVirtualNetwork


#-----------------Fetching Virtual Machine Details-----------------#

    $virtual_machine_object = $null
    $virtual_machine_object = @()


    # Iterating over the NIC Interfaces under the subscription
        
        foreach($azureNICDetail in $azureNICDetails){ 
        $azureVMDetail = $azureVMDetails | ? -Property Id -eq $azureNICDetail.VirtualMachine.id
        $vm_status = get-azurermvm -ResourceGroupName $azureVMDetail.resourcegroupname -name $azureVMDetail.name -Status
        $vm_tags = ($azureVMDetail.Tags.values) -join ';'
        $osversion = $azureVMDetail.StorageProfile.ImageReference.id
        $vmsize = Get-AzureRmVMSize -VMName $azureVMDetail.Name -ResourceGroupName $azureVMDetail.ResourceGroupName | ? {$_.Name -eq $azureVMDetail.HardwareProfile.VmSize}
        
        #Fetching the private IP
        #write-Host $vm.NetworkInterfaceIDs
        $private_ip_address = ($azureNICDetail.IpConfigurations | select-object -ExpandProperty PrivateIpAddress) -Join ';'
        $virturalnetwork = $azureNICDetail.IpConfigurations.subnet.Id.Split("/")[-3]
        $subnet = $azureNICDetail.IpConfigurations.subnet.Id.Split("/")[-1]
            
        #Fetching data disk names
        $data_disks = $azureVMDetail.StorageProfile.DataDisks
        $data_disk_name_list = ''

            foreach ($data_disk in $data_disks) {
            $data_disk_name_list_temp = $data_disk_name_list + "; " +$data_disk.name 
            #Trimming the first three characters which contain --> " ; "
            $data_disk_name_list = $data_disk_name_list_temp.Substring(2)
            #write-host $data_disk_name_list
            }

        #}            

            # Fetching OS Details (Managed / un-managed)

            if($azureVMDetail.StorageProfile.OsDisk.manageddisk -eq $null){
                # This is un-managed disk. It has VHD property

                $os_disk_details_unmanaged = $azureVMDetail.StorageProfile.OsDisk.Vhd.Uri
                $os_disk_details_managed = "This VM has un-managed OS Disk"

            }else{
                
                $os_disk_details_managed = $azureVMDetail.StorageProfile.OsDisk.ManagedDisk.Id
                $os_disk_details_unmanaged = "This VM has Managed OS Disk"
            }

            $virtual_machine_object_temp = new-object PSObject 
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "VMName" -Value $azureVMDetail.Name
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "ResourceGroupName" -Value $azureVMDetail.ResourceGroupName
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "Location" -Value $azureVMDetail.Location
#           $virtual_machine_object_temp | add-member -membertype NoteProperty -name "Zone" -Value $azureVMDetail.Zones
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "Tags" -Value $vm_tags
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "VMStatus" -Value $vm_status.Statuses[1].DisplayStatus
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "VMSize" -Value $azureVMDetail.HardwareProfile.VmSize
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "CPU" -Value $vmsize.NumberOfCores
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "RAM in MB" -Value $vmsize.MemoryInMB
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "OSFamily" -Value $azureVMDetail.StorageProfile.OsDisk.OsType
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "DiskCount" -Value $azureVMDetail.StorageProfile.DataDisks.Count
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "AdminUserName" -Value $azureVMDetail.OSProfile.AdminUsername
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "PrivateIP" -Value $private_ip_address
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "Vnet-Zone" -Value $virturalnetwork
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "Subnet" -Value $subnet
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "OSVersion" -Value $osversion
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "DataDiskNames" -Value $data_disk_name_list
            $virtual_machine_object_temp | add-member -membertype NoteProperty -name "ManagedOSDiskURI" -Value $os_disk_details_managed
                        


            $virtual_machine_object += $virtual_machine_object_temp

            
        }

        $virtual_machine_object | Export-Csv "$working_directory\Virtual_Machine_details_$(get-date -f yyyyMMdd).csv" -NoTypeInformation -Force

}

Get-AzureInventory