Azure

Azure – Cost Management Better Than Ever Using Cloudyn (Registration)

Microsoft’s acquisition of Cloudyn will help Azure customers manage and optimize their cloud usage. Read more about the acquisition here. A message from Sharon Wagner, CEO of Cloudyn.

About Cloudyn

Azure Cost Management by Cloudyn empowers organizations to monitor cloud spend, drive organizational accountability, and optimize cloud efficiency so they can accelerate future cloud investments with confidence.

Microsoft’s acquisition of Cloudyn will help Azure customers and partners as they face the challenges of growing their multi-cloud environments. It will enable them to gain visibility, understand and optimize cloud consumption, as well as accurately project future usage.

Microsoft will continue to support multi-cloud environments, including Azure, AWS, and GCP. Azure Cost Management by Cloudyn is available for free to customers and partners managing Azure spend. Additional premium capabilities are available at no cost through June 2018, once they will become paid features.

Let us look into how to sign up to Cloudyn if you are an Azure customer

Step 1: Login to your Azure Subscription via the Azure portal. Select the “Cost Management + Billing” blade. Then select “Cost Management” from the options on the left-hand side of the pane.

Click on “Go to Cost Management”

image_1

 

Step 2: Once you click the”Go to Cost Management” button, you will be redirected to  Cloudlyn’s page to set up your Cost Management details.

Enter your organization name and the type of Azure access you have on your Azure account. I have a personal subscription, so I have chosen as “Azure Individual Subscription Owner.”

image_2

 

Step 3: Cloudlyn Account name and Tenant ID will be automatically populated. Now, select the offer-ID from the drop-down list.

If you do not know your Offer-ID, then go back to your Azure portal. Click on “Subscriptions,” that should provide you the type of subscription that you have.

image3_1

 

Step 4: Click “Next”

image_4

 

Step 5: Click “Next”

image_5

Step 6: We are done with the registration with Cloudlyn. Cloudlyn needs about 2 hours for collecting the data.

image_6

 

If you find the content valuable, do consider sharing with your friends and colleagues.

Click here to download my PowerShell scripts for Free !!

Advertisements

Azure – PowerShell in Azure Cloud Shell

Today we are looking into PowerShell in Azure Cloud Shell. This is still in public preview as of this writing.

If you are wondering why Microsoft would introduce a PowerShell console inside the Azure Cloud Shell, then have a look at the below features:

Features

Browser-based shell experience

Cloud Shell enables access to a browser-based command-line experience built with Azure management tasks in mind. Leverage Cloud Shell to work untethered from a local machine in a way only the cloud can provide.

Choice of preferred shell experience

Azure Cloud Shell gives you the flexibility of choosing the shell experience that best suits the way you work. Linux users can opt for a Bash experience, while Windows users can opt for PowerShell.

Pre-configured Azure workstation

Cloud Shell comes pre-installed with popular command-line tools and language support so you can work faster.
View the full tooling list for Bash experience and PowerShell experience.

Automatic authentication

Cloud Shell securely authenticates automatically on each session for instant access to your resources through the Azure CLI 2.0.

Connect your Azure File storage

Cloud Shell machines are temporary and as a result, require an Azure file share to be mounted as clouddrive to persist your $Home directory. On the first launch, Cloud Shell prompts to create a resource group, storage account, and file share on your behalf. This is a one-time step and will be automatically attached for all sessions. A single file share can be mapped and will be used by both Bash and PowerShell in Cloud Shell.

Below are some conditions that we have to remember:

Cloud Shell runs on a temporary machine provided on a per-session, per-user basis
Cloud Shell times out after 20 minutes without interactive activity
Cloud Shell can only be accessed with a file share attached
Cloud Shell uses the same file share for both Bash and PowerShell
Cloud Shell is assigned one machine per user account
Permissions are set as a regular Linux user (Bash)

Now that we have some background knowledge on the PowerShell in Cloud Shell, let us dig more into the usage of it:

To access the Cloud Shell, click on the PowerShell icon in the Azure portal:

image_1

Once you click on the icon, a pane is opened at the bottom of the screen as shown below. You can choose from two options – BASH or PowerShell. Since we are interested in learning PowerShell in CloudShell, let us choose PowerShell as our desired option.

image_2

When you are starting for the first time, the Shell will configure an Azure File Storage. Cloud Shell machines are temporary and as a result, require an Azure file share to be mounted as clouddrive to persist your $Home directory. Alternatively, if you have multiple subscriptions, you will be allowed to choose your favorite subscription to work with.

image_3

Azure Authentication, Resource Group, Storage Account and File Storage are automatically created as shown below:

image_4

Testing an Azure command. Works perfectly.

image_5

If you are idle for more than 20 minutes, you will be kicked off the session, and you will have to start the session again:

image_6

Discovering the drives under PowerShell in Cloud Shell:

Now let us execute the Get-ChildItem cmdlet and see what we can find.

image_8

As we can see, running the Get-ChildItem in the current scope will list out the subscriptions that your account is associated with.

Traversing one step deeper into the directory, we can see the resources related to the subscription.

image_9

Let us get into the “StroageAccounts” directory to confirm if we get to see a list of Storage Accounts under the selected subscription:

image_10

PowerShell cmdlets to manage PowerShell in Cloud Shell:

From the below information, we can see that Microsoft provides us two cmdlets to work with the cloud shell.

image_12

Get-CloudDrive provides the details of the “Azure File Share” that was created when the cloud shell started. You may continue to use the cloud share. However, if you want a new one, you can dismount and create a new one using the Dismount-CloudDrive cmdlet.

image_11

Note: Once you dismount the Azure file share, your current session will be restarted to set up a new cloud share.

Assumption:

I am assuming that Microsoft is using container service infrastructure to provide a session. You will get the below windows path when you query for the temp drive:

C:\Users\ContainerAdministrator\AppData\Local\Temp

image_11

Note the administrator is a “ContainerAdministrator.” The container here could be a Windows Server or a Windows Container. I am assuming it is a Windows Container since the underlying “image” comes pre-packaged with below tools and a temporary one. A typical use case scenario for Container technology.

image_13

 

If the content is valuable to you, do consider sharing it with your friends and colleagues.

Did I miss out anything? Let me know in the comments section.

 

Download my PowerShell scripts for Free!

 

Azure – First look into “Event Grid Subscriptions”

This blog takes a first look at the “Event Grid” service by Azure. Microsoft announced this service in August 2017. This is still in public preview as of writing this blog.

Azure Event Grid allows you to easily build applications with event-based architectures. You select the Azure resource you would like to subscribe to and give the event handler or WebHook endpoint to send the event to. Event Grid has built-in support for events coming from Azure services, like storage blobs and resource groups. Event Grid also has custom support for application and third-party events, using custom topics and custom webhooks.

You can use filters to route specific events to different endpoints, multicast to multiple endpoints, and make sure your events are reliably delivered. Event Grid also has built in support for custom and third-party events.

event-grid-functional-model

We shall create a simple “Event Grid Subscription” which comes under Event Grid. This Event Subscription will monitor one of Azure Resource Group for events. Trigger an Azure automation runbook, in case if any event occurs on the resource group.

Below is the event types that the event subscription will monitor:

RG_Event_Types.jpg

  1. Search for the “Event Subscription”search_event_subscription
  2. Fill in the required details and click on CREATE. Enter your Azure Automation Runbook’s webhook under the “Subscriber Endpoint”. Thus, your runbook will be triggered when ever any event under the specified runbook occurs.create_event_sub
  3. Once the “Event Subscription” is created. Create a simple resource (like public IP), and we can see that our runbook is triggered. [Open JOBS pane under the runbook to verify that the runbook has run]trigger_runbook

 

Follow this link, if you are looking for Azure APIs to create Event Subscriptions.

Follow this link for more information about Azure Event Grid.

 

Looking for free PowerShell scripts? Check out my Powershell Contributions under Microsoft Technet Script Centre

Azure – How to trigger Azure runbooks with Azure Monitor

Today we shall see how to trigger runbooks using webhooks when an alert is fired from Azure Monitor.

To illustrate this, I have taken a specific example of triggering a runbook, when an Azure virtual machine is de-allocated.

Step 1: Simulate a de-allocation event by Stopping a virtual machine.

Step 2: Check the activity logs for the de-allocation event.

1

 

Step 3: Click on the “+ Add activity log alert”. This will start up a pane to fill out the alert details.

2

 

Step 4: Before filling out the alert details, I encourage you to go ahead and create an empty Runbook. So that you can create the webhook for the Runbook, which is required while filling out the alert details.

3

 

Step 5: Once you click on the “Webhook”, you will get the below pane. Click on “Create new webhook” and fill out the details accordingly. Make sure you copy the URL, this information is not available again once the webhook is created.

4

 

Additionally, you can specify how the webhook is to be run. It will be either “Azure RunAs account” or “Hybrid Worker”

5

 

Step 6: Now go back to the alert, and fill the details:

6.jpg

“Event Initiated By” field can contain blank space to monitor all de-allcoation events. or specify Id upon which an alert is triggered only when that user initiates de-allocation event.

Under Actions, select the “Webhook” action type and specify the webhook URL that we created in the previous step.

This ensures, that when ever a de-allocation event occurs, this alert will be fired. And this alert will in turn trigger the runbook via its webhook.

 

Azure – Configure Storage Spaces for Azure VM for increased disk performance

This blog will walk you through on how to configure Storage Spaces for Azure Virtual Machine (Windows). Finally, we get to see some IOPS benchmarks.

Each data disk (Standard Storage Account) has about 500 IOPS. In this example, we are going to create a Storage Space by attaching 4 data disks to a Standard A2 sized Azure VM. In theory, this should increase the IOPS to 2k. (500 x 4 = 2000)

 

Configuring Storage Spaces for Azure windows VM

Step 1: Attach four data disks to your virtual machine.

From the azure portal, select your virtual machine >> Click on “Disks” >> click on the “+ Add data disk” >> Fill out the details accordingly >> Save the disk.

1

Repeat this process 3 more times and we will have 4 data disks attached to our virtual machine as shown below:

4_disk_attached_azure_portal.PNG

 

Inside the VM, we can see the disks attached:

4_disk_not_initialized

 

 

Step 2: Login to the virtual machine and run the following PowerShell cmdlets. This will configure Storage Space and will create a drive for you.

 

In our example, we will configure one volume. Hence, only one storage pool. If you are implementing SQL Server or any other architecture, you may need more than one storage pool.

Create a new virtual disk using all the space available from the storage pool using a Simple configuration. The interleave is set to 256KB. We are also setting the number of columns to be equal to the number of disks in the pool

Format the disk with NTFS filesystem and a 64KB allocation unit size.

Below is a snippet of the PowerShell console after executing the above cmdlets.

create_storage_space.PNG

Finally, we can see the drive. A drive named “E” will be created with a free space of ~4TB.

e_drive_created.png

 

Benchmark Tests

Obviously, this works. However, I have run IOPS test to have a visual. You may choose any standard benchmark testing tools. To keep it simple, I have used a PowerShell script authored by Mikael Nystrom, Microsoft MVP. This script is a wrapper to the SQLIO.exe. You may download the PowerShell script and SQLIO.exe file, HERE.

 

Download the archive file to your local system and copy it to the server. Extract the contents to any folder.

 

Below is a sample script to estimate IOPS:

.\DiskPerformance.ps1 -TestFileName test.dat –TestFileSizeInGB 1 -TestFilepath F:\temp -TestMode Get-SmallIO -FastMode True -RemoveTestFile True -OutputFormat Out-GridView

Feel free to tweak the parameter values for different results.

Explaination of parameters:

-TestFileName test.dat

The name of the file, it will create the file using FSUTIL, but it checks if it exists and if it does it stops, you can override that with the –RemoveTestFile True

–TestFileSizeInGB 1

Size of the file, it has fixed values, use the TAB key to flip through them

-TestFilepath C:\VMs

The folder can also be an UNC path, it will create the folder so it does not need to exist.

-TestMode Get-SmallIO

There is too test modes Get-LargeIO or Get-SmallIO, you use Get-LargeIO to measure the transfer rate and you use Get-SmallIO to measure IOPS

-FastMode True

Fast mode true runs each test for just 10 seconds, it gives you a hint, if you don’t set it or set it to false it will run for 60 sec (it will take a break for 10 sec between each run)

-RemoveTestFile True

Removes the test file if it exists

-OutputFormat Out-GridView

Choose between Out-Gridview or Format-Table

 

IOPS for C drive on Azure VM [OS Disk]:

C_drive

 

IOPS for D drive on Azure VM [Temporary Disk]:

D_drive

 

IOPS for E drive on Azure VM [Standard data disk]:

E_drive

 

IOPS for F drive on Azure VM [Storage Spaces]:

F_drive

 

We can use this storage strategy when we have a small amount of data but the IOPS requirement is huge.

Example scenario:

You have 500GB of data, and the IOPS for that data exceeds 1K. Storing 500GB of data in one data disk will create IOPS problems since each data disk has a 500 IOPS limit. But, if we combine 4 disks and create a storage space, the IOPS will increase to ~2k [we have to consider latency etc., to have a correct figure]. Since we are using the same Standard A2 virtual machine and Azure charges for the overall data and not per disk, the pricing will be the same.

 

 

AWS – Monitor AWS Windows EC2 instance using Microsoft OMS (Operations Management Suite)

Microsoft is investing a lot of money and effort into OMS (Operations Management Suite). OMS can be used to monitor Windows/Linux machines, not just in Azure, but also in AWS, or any cloud vendor platform for that matter. You can even monitor the servers hosted in your on-premise environment.

Configuring OMS for Azure instance is pretty straightforward. I will walk you through on how to configure OMS on Windows AWS instance.

I already have a OMS workspace (with an Azure subscription)

Step 1: Create and connect to your AWS Windows instance, by following the below link as guidance:

http://docs.aws.amazon.com/codedeploy/latest/userguide/tutorials-windows-launch-instance.html

Step 2: Download the OMS direct agent for Windows Machine.

Option1: If you are using an Azure subscription to manage OMS, then you can find the link to download the direct agent by clicking on:

Selecting your OMS workspace >> Select “Quick Start” >> Select “Computers” >> Select “Download Windows Agent (64 bit)”

2

 

1

 

Option 2: You can download the OMS direct agent, from the OMS portal as well.

Click on the “gear button” (third icon from left) located at the top right-hand corner of the portal >> select “Connected Sources” >> Select “Windows Servers” >> Click on “Download Windows Agent (64 bit)”

3.PNG

Once the “MMASetup-AMD64.exe” file is downloaded to your local desktop. Copy the file to the AWS Windows instance where you are configuring the OMS agent.

Now, double click on the “MMASetup-AMD64.exe” to start installing.

 

4

 

Click “Next”

5

Click on “I Agree” once you have read the legal terms.

6

Select the installation folder, if you are not happy with the default location. Then click “Next”

7

Select the checkbox “Connect the agent to Azure  Log Analytics (OMS)” and then click “Next”.

8

Enter your OMS workspace details. You can find this information from Azure portal or OMS portal. It is the same page from where we download the Direct Agent for Windows.

[Optional] Click on the “Advanced” button, if your server has to go through a proxy server. Make necessary changes and click “Next”. Since I do not use a proxy server to connect to the OMS, I am leaving the fields as blank.

9

Click “Next” on the above page takes you back to the page where you updated the OMS workspace ID and key. Now click “Next” to proceed.

10

Select accordingly, if you need Microsoft updates or not. Then click “Next”.

11

Review your settings/data. Click on “Install”.

12

Now click on “Finish”.

 

Step 3: Verify connectivity to OMS workspace

Open Control Panel >> Select “Microsoft Monitoring Agent”

13

Select “Azure Log Analytics (OMS)” tab.  You can see that your windows agent has successfully connected to the Microsoft Operations Managment Suite service.

14

 

Step 4: Verify log from AWS windows instance to OMS

From the OMS portal, we can see that our AWS windows instance is connected. [WIN-PQ69983CQ24 is my AWS windows instance name]

16

 

A simple Log Search will give us data fetched from the instance.

17

 

 

Azure – Create a windows VM from a generalized image

This blog shows you how to create a windows VM from a  generalized image. This uses un-managed Azure disks.

For this example, I will be using resources deployed on Azure. i.e., generalize an Azure VM, create a image out of it and then create a new Azure VM using the image.

Below are the steps:

  1. Generalize the VM
  2. Capture a VM image from a generalized Azure VM, that we obtain from Step 1
  3. Create a VM from a generalized VHD image in a storage account, that we obtained from step 2

Part 1: Generalize the VM

  1. Remote Desktop to the Azure virtual machine
  2. **Important** Before running the “Sysprep.exe”. Delete the “unattend.xml” file from the “C:\Windows\Panther” folder. If you do not do this, you will encounter “OS Provision time out” exception while creating the VM from this image.

    This is due to the fact that when an image is deployed the unattend.xml file must come from the ISO image that is attached to the Virtual Machine by Windows Azure as part of VM provisioning from an image.

    8.PNG

  3. From the command prompt / powershell, change the directory to: “C:\Windows\System32\Sysprep”
  4. Run “Sysprep.exe”
  5. In the System Preparation Tool, select the option, “Enter System Out-of-Box Experience (OOBE)”. Select the “Generalize” option from the check-box.
  6. Select “Shutdown” from the drop down list, in Shutdown Options.
    1
  7. Once the Sysprep process completes, the VM will shutdown.
  8. From the Azure portal, you can see the status of the VM as “Stopped (Shutdown)”. Use the below powershell cmdlet to fetch the VM status.

    (Get-AzureRmVM -ResourceGroupName manjug_test -Name windowsmachine -Status).Statuses

    2

Part 2: Capture a VM image from a generalized Azure Virtual Machine

  1. De-allocate the VM

    Stop-AzureRmVM -ResourceGroupName manjug_test -Name windowsmachine

    Confirm the status of the VM:

    (Get-AzureRmVM -ResourceGroupName manjug_test -Name windowsmachine -Status).Statuses

    3

  2. Set the status of the VM to “Generalized”

    Set-AzureRmVm -ResourceGroupName manjug_test -Name windowsmachine -Generalized

    4

    Confirm the status of the VM:

    (Get-AzureRmVM -ResourceGroupName manjug_test -Name windowsmachine -Status).Statuses

    5

  3. Create the image by running the below command:

    Save-AzureRmVMImage -ResourceGroupName manjug_test -Name windowsmachine ` -DestinationContainerName images -VHDNamePrefix windowsmachineimage ` -Path C:\Filename.json

    -DestinationContainerName, is the container name where the image will be stored.
    -VHDNamePrefix, is the prefix given to the image.
    -Path, is the path of json file that contains the details of the image that gets created.

    You can get the URL of your image from the JSON file template. Go to the resources > storageProfile > osDisk > image > uri section for the complete path of your image. The URL of the image looks like:

    https://<storageAccountName&gt;.blob.core.windows.net/system/Microsoft.Compute/Images/<imagesContainer>/<templatePrefix-osDisk>.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.vhd

    You can also verify the URI in the portal. The image is copied to a container named system in your storage account.

Part 3: Create a VM from a generalized VHD image in a storage account

  1. Obtain the image uri, from the json file (Part 2, step 3). Or you can fetch this from the Azure portal.

    From the portal:

    9.PNG

    From the JSON file:

    6

  2. Set the VHD uri to a variable. Example:

    $imageURI = “https://manjugtestdisks.blob.core.windows.net/system/Microsoft.Compute/Images/images/windowsmachineimage-osDisk.04a4f0cb-268a-49ea-a0d9-a203c8fa8c51.vhd&#8221;

  3. Create a Virtual Network
    Create the subnet. The following sample creates a subnet named mySubnet in the resource group myResourceGroup with the address prefix of 10.0.0.0/24.

    $rgName = “manjug_test”
    $subnetName = “mySubnet”
    $singleSubnet = New-AzureRmVirtualNetworkSubnetConfig -Name $subnetName -AddressPrefix 10.0.0.0/24

    Create the virtual network. The following sample creates a virtual network named myVnet in the West US location with the address prefix of 10.0.0.0/16.

    $location = “Southeast Asia”
    $vnetName = “myVnet”
    $vnet = New-AzureRmVirtualNetwork -Name $vnetName -ResourceGroupName $rgName -Location $location `
    -AddressPrefix 10.0.0.0/16 -Subnet $singleSubnet

  4. Create a public IP address and network interface

    Create a public IP address. This example creates a public IP address named myPip.

    $ipName = “myPip”
    $pip = New-AzureRmPublicIpAddress -Name $ipName -ResourceGroupName $rgName -Location $location `
    -AllocationMethod Dynamic

    Create the NIC. This example creates a NIC named myNic.

    $nicName = “myNic”
    $nic = New-AzureRmNetworkInterface -Name $nicName -ResourceGroupName $rgName -Location $location `
    -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id

  5. Create the network security group and an RDP rule

    To be able to log in to your VM using RDP, you need to have a security rule that allows RDP access on port 3389

    This example creates an NSG named myNsg that contains a rule called myRdpRule that allows RDP traffic over port 3389.

    $nsgName = “myNsg”

    $rdpRule = New-AzureRmNetworkSecurityRuleConfig -Name myRdpRule -Description “Allow RDP” `
    -Access Allow -Protocol Tcp -Direction Inbound -Priority 110 `
    -SourceAddressPrefix Internet -SourcePortRange * `
    -DestinationAddressPrefix * -DestinationPortRange 3389

    $nsg = New-AzureRmNetworkSecurityGroup -ResourceGroupName $rgName -Location $location `
    -Name $nsgName -SecurityRules $rdpRule

  6. Create a variable for the virtual network

    $vnet = Get-AzureRmVirtualNetwork -ResourceGroupName $rgName -Name $vnetName

  7. Create the Virtual Machine

    The following PowerShell script shows how to set up the virtual machine configurations and use the uploaded VM image as the source for the new installation.

    # Enter a new user name and password to use as the local administrator account
    # for remotely accessing the VM.
    $cred = Get-Credential

    # Name of the storage account where the VHD is located. This example sets the
    # storage account name as “myStorageAccount”
    $storageAccName = “manjugtestdisks”

    # Name of the virtual machine. This example sets the VM name as “myVM”.
    $vmName = “winmachimage”

    # Size of the virtual machine. This example creates “Standard_D2_v2” sized VM.
    # See the VM sizes documentation for more information:
    # https://azure.microsoft.com/documentation/articles/virtual-machines-windows-sizes/
    $vmSize = “Standard_D2_v2”

    # Computer name for the VM. This examples sets the computer name as “myComputer”.
    $computerName = “winmachimage”

    # Name of the disk that holds the OS. This example sets the
    # OS disk name as “myOsDisk”
    $osDiskName = “myOsDisk”

    # Assign a SKU name. This example sets the SKU name as “Standard_LRS”
    # Valid values for -SkuName are: Standard_LRS – locally redundant storage, Standard_ZRS – zone redundant
    # storage, Standard_GRS – geo redundant storage, Standard_RAGRS – read access geo redundant storage,
    # Premium_LRS – premium locally redundant storage.
    $skuName = “Standard_LRS”

    # Get the storage account where the uploaded image is stored
    $storageAcc = Get-AzureRmStorageAccount -ResourceGroupName $rgName -AccountName $storageAccName

    # Set the VM name and size
    $vmConfig = New-AzureRmVMConfig -VMName $vmName -VMSize $vmSize

    #Set the Windows operating system configuration and add the NIC
    $vm = Set-AzureRmVMOperatingSystem -VM $vmConfig -Windows -ComputerName $computerName `
    -Credential $cred -ProvisionVMAgent -EnableAutoUpdate
    $vm = Add-AzureRmVMNetworkInterface -VM $vm -Id $nic.Id

    # Create the OS disk URI
    $osDiskUri = ‘{0}vhds/{1}-{2}.vhd’ `
    -f $storageAcc.PrimaryEndpoints.Blob.ToString(), $vmName.ToLower(), $osDiskName

    # Configure the OS disk to be created from the existing VHD image (-CreateOption fromImage).
    $vm = Set-AzureRmVMOSDisk -VM $vm -Name $osDiskName -VhdUri $osDiskUri `
    -CreateOption fromImage -SourceImageUri $imageURI -Windows

    # Create the new VM
    New-AzureRmVM -ResourceGroupName $rgName -Location $location -VM $vm

  8. Verify that the Virtual Machine was created.

~ If this post helps at-least one person. The purpose is served. ~