azure windows

Azure – Who de-allocated my virtual machine?

Many a time we might want to know details about certain operations performed on our Azure resources.

Once such case study would be to track how many virtual machines are being de-allocated by users, so we can make a decision on not to monitor them.

I have written a simple script that would make the tracking easy.

Download the script

 

This script will fetch information of certain Azure operation against Azure resources and create a CSV file. Specifically, this script will create a CSV file that contains a list of Azure operations that de-allocates an Azure virtual machine.

You may alter the IF condition statement to produce desired results.

Example, fetch operational logs for Azure Storage only. Or fetch operational logs for re-start VM or any operation on any Azure resource.

The CSV file will be saved in the same folder from where you run the script and will be saved as “Azure_activity_logs.csv”

 

Click here to download my PowerShell scripts for Free !!

Click here for Azure tutorial videos !!

Advertisements

Azure – Install exe files (BigFix) on Azure windows virtual machine using Azure Custom Script Extension (CSE)

What is custom script extension?

The Custom Script Extension downloads and executes scripts on Azure virtual machines. This extension is useful for post-deployment configuration, software installation, or any other configuration/management task. Scripts can be downloaded from Azure storage or GitHub, or provided to the Azure portal at extension runtime. The Custom Script extension integrates with Azure Resource Manager templates, and can also be run using the Azure CLI, PowerShell, Azure portal, or the Azure Virtual Machine REST API.

This document details on how to use Custom Script Extension using the Azure PowerShell Module against an already provisioned Azure Windows virtual machine to install BigFix client.

Pre-requisites:

Operating System

The Custom Script Extension for Windows can be run on Windows 10 Client, Windows Server 2008 R2, 2012, 2012 R2, and 2016 releases.

Script Location

The script needs to be stored in Azure Blob storage, or any other location accessible through a valid URL.

Internet Connectivity

The Custom Script Extension for Windows requires that the target virtual machine is connected to the internet.

The BigFix client files are stored in the storage account:

1

We shall be naming the extension as “bigfixinstallextension.” Make sure that an extension with the same name already does not exist.

Step 1: Get the Azure virtual machine config object

$vm = get-azurermvm -ResourceGroupName “datadog-test” -Name “dg-private-1”

Step 2: Query the Virtual Machine object for existing extensions:

$vm.Extensions

You should see an output similar to below if it does not have any custom extensions.

2

Note: any azure virtual machine will have one default extension – “MicrosoftMonitoringAgent.” This is because Azure installs “Microsoft Monitoring Agent” on every virtual machine. Make sure, the virtual machine does not have another extension with the name “ bigfixinstallextension.” If it does have, we have to remove that extension.

Below link provides an Azure Powershell cmdlet to remove the extension:

https://docs.microsoft.com/en-us/powershell/module/azurerm.compute/remove-azurermvmextension?view=azurermps-5.5.0

Once, we have confirmed that a custom extension with name “ bigfixinstallextension” does not exists, we can proceed in adding one. Below is the powershell code:

# Resource group of virtual machine

$resource_group = “datadog-test”

# location of virtual machine

$location = “East US 2”

# azure virtual machine name

$vm_name = “dg-private-1”

# storage account name where the custom script is stored

$storage_account_name = “xxxx”

# storage account key of where the custom script is stored

$storage_account_key = “xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx”

# custom script file name

$file_name = “azure_custom_script_execution_install_bigfix.ps1”

# container name where the custom script is stored

$container_name = “msifiles”

# extension name for the custom script extension

$extension_name = “bigfixinstallextension”

# azure powershell cmdlet to execute add the custom script extension and to execute the powershell file

Set-AzureRmVMCustomScriptExtension -ResourceGroupName $resource_group -Location $location -VMName $vm_name -Name $extension_name -TypeHandlerVersion “1.1” -StorageAccountName $storage_account_name -StorageAccountKey $storage_account_key -FileName $file_name -ContainerName $container_name

Output:

4

Now login to the Azure windows virtual machine to confirm if the BigFix client is installed and running:

5

The downloaded file can be found inside the virtual machine at the below file path:

C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.9\Downloads\1

Extension execution output is logged to files found under the following directory on the target virtual machine. For troubleshooting.

C:\WindowsAzure\Logs\Plugins\Microsoft.Compute.CustomScriptExtension

 

Explaining the PowerShell scriptazure_custom_script_execution_install_bigfix.ps1

This script gets executed as part of the Custom Script Execution. And it is responsible for installing the BigFix agent.

Below is the code:

# Create a directory to hold BigFix files

new-item ‘c:\bigfix’ -ItemType directory

# Copy BigFix files from Azure storage to local directory

Invoke-WebRequest -Uri https://customsc.blob.core.windows.net/msifiles/clientsettings.cfg -outfile ‘c:\bigfix\clientsettings.cfg’

Invoke-WebRequest -Uri https://customsc.blob.core.windows.net/msifiles/masthead.afxm -outfile ‘c:\bigfix\masthead.afxm’

Invoke-WebRequest -Uri https://customsc.blob.core.windows.net/msifiles/setup.exe -outfile ‘c:\bigfix\setup.exe’

# Execute the setup file

$arguments = “/S /v/qn”

$filepath = “c:\bigfix\setup.exe”

Start-Process $filepath $arguments -wait

Execution Flow:

1. Create a directory to hold big fix files.

2. Copy the three files associated with BigFix installation to the directory created in Step 1.

3. Execute the setup file in silent mode.

Click here to download my PowerShell scripts for Free !!

Click here for Azure tutorial videos !!

 

 

Azure – Configure Storage Spaces for Azure VM for increased disk performance

This blog will walk you through on how to configure Storage Spaces for Azure Virtual Machine (Windows). Finally, we get to see some IOPS benchmarks.

Each data disk (Standard Storage Account) has about 500 IOPS. In this example, we are going to create a Storage Space by attaching 4 data disks to a Standard A2 sized Azure VM. In theory, this should increase the IOPS to 2k. (500 x 4 = 2000)

 

Configuring Storage Spaces for Azure windows VM

Step 1: Attach four data disks to your virtual machine.

From the azure portal, select your virtual machine >> Click on “Disks” >> click on the “+ Add data disk” >> Fill out the details accordingly >> Save the disk.

1

Repeat this process 3 more times and we will have 4 data disks attached to our virtual machine as shown below:

4_disk_attached_azure_portal.PNG

 

Inside the VM, we can see the disks attached:

4_disk_not_initialized

 

 

Step 2: Login to the virtual machine and run the following PowerShell cmdlets. This will configure Storage Space and will create a drive for you.

 

In our example, we will configure one volume. Hence, only one storage pool. If you are implementing SQL Server or any other architecture, you may need more than one storage pool.

Create a new virtual disk using all the space available from the storage pool using a Simple configuration. The interleave is set to 256KB. We are also setting the number of columns to be equal to the number of disks in the pool

Format the disk with NTFS filesystem and a 64KB allocation unit size.

Below is a snippet of the PowerShell console after executing the above cmdlets.

create_storage_space.PNG

Finally, we can see the drive. A drive named “E” will be created with a free space of ~4TB.

e_drive_created.png

 

Benchmark Tests

Obviously, this works. However, I have run IOPS test to have a visual. You may choose any standard benchmark testing tools. To keep it simple, I have used a PowerShell script authored by Mikael Nystrom, Microsoft MVP. This script is a wrapper to the SQLIO.exe. You may download the PowerShell script and SQLIO.exe file, HERE.

 

Download the archive file to your local system and copy it to the server. Extract the contents to any folder.

 

Below is a sample script to estimate IOPS:

.\DiskPerformance.ps1 -TestFileName test.dat –TestFileSizeInGB 1 -TestFilepath F:\temp -TestMode Get-SmallIO -FastMode True -RemoveTestFile True -OutputFormat Out-GridView

Feel free to tweak the parameter values for different results.

Explaination of parameters:

-TestFileName test.dat

The name of the file, it will create the file using FSUTIL, but it checks if it exists and if it does it stops, you can override that with the –RemoveTestFile True

–TestFileSizeInGB 1

Size of the file, it has fixed values, use the TAB key to flip through them

-TestFilepath C:\VMs

The folder can also be an UNC path, it will create the folder so it does not need to exist.

-TestMode Get-SmallIO

There is too test modes Get-LargeIO or Get-SmallIO, you use Get-LargeIO to measure the transfer rate and you use Get-SmallIO to measure IOPS

-FastMode True

Fast mode true runs each test for just 10 seconds, it gives you a hint, if you don’t set it or set it to false it will run for 60 sec (it will take a break for 10 sec between each run)

-RemoveTestFile True

Removes the test file if it exists

-OutputFormat Out-GridView

Choose between Out-Gridview or Format-Table

 

IOPS for C drive on Azure VM [OS Disk]:

C_drive

 

IOPS for D drive on Azure VM [Temporary Disk]:

D_drive

 

IOPS for E drive on Azure VM [Standard data disk]:

E_drive

 

IOPS for F drive on Azure VM [Storage Spaces]:

F_drive

 

We can use this storage strategy when we have a small amount of data but the IOPS requirement is huge.

Example scenario:

You have 500GB of data, and the IOPS for that data exceeds 1K. Storing 500GB of data in one data disk will create IOPS problems since each data disk has a 500 IOPS limit. But, if we combine 4 disks and create a storage space, the IOPS will increase to ~2k [we have to consider latency etc., to have a correct figure]. Since we are using the same Standard A2 virtual machine and Azure charges for the overall data and not per disk, the pricing will be the same.