Tuesday, March 24, 2015

BIG-IP SCOM Management Pack - Monitoring you want

I’m currently working on F5 BIG-IP SCOM Management Pack. Comtrade, a company I work for, already has a BIG-IQ SCOM Management Pack, which was very well accepted by F5 product management and customers involved in Beta Program. Encouraged by that fact and market demand, we also decided to build a separate BIG-IP SCOM Management Pack. Early Access version will be released in the following weeks.

Let’s start this blog series by describing the main problems I’m tackling.

Purpose of BIG-IP device is to provide optimal access to applications (such as Microsoft Exchange, Lync and SharePoint), both from the user’s point of view and application host’s point of view.  So, we can say that the application is the king. Let’s make sure users can access these applications anytime!

F5 BIG-IP, as a device, can become unavailable, or start behaving unexpectedly. This can be due to a number of hardware related reasons: high CPU usage at some periods, no free storage, expired certificates, expired license and interfaces being down. In these cases, all application services configured on a specific BIG-IP may become unavailable, experience performance issues and have unexpected side-effects.

It is possible to configure BIG-IPs to send email notifications when certain events occur on the BIG-IP system, but it’s by no means trivial, as it is difficult to use this information for reporting and SLA dashboard purposes. It would be better (better as in less work to configure, check health and identify device having a problem and problem’s root cause) if when incident is about to happen (i.e. storage almost full, certificate due to expire, license due to expire, CPU working at high usage level for prolonged times), we can proactively, without specific per-device configuration, be warned about our applications that are in danger. This warning would, of course, come with enough information about the threat and this information would be collected to a single point.

There may be situations when a device becomes unavailable (i.e. stuck during reboot, disconnected from network etc.) and in those cases, in order to optimize your applications availability, it’s important to be aware of the issue, with all the relevant details as soon as possible.

If something might go wrong, we want to know that before it becomes an issue. How this sounds? Do you encounter these problems? What do you think about:
  •  Monitoring BIG-IP device availability?
  •  Making sure applications are working?
  •  Navigating multiple devices in order to understand where application issues are?
Are there any other BIG-IP related problems that make your working day difficult?

To be honest with you, I will not cover all your needs with this first product release, therefore my true intention is to learn from you and help you address issues you are dealing with. So, if you want to discuss  your ideas that can be useful for managing BIG-IP problems or if you are willing to see product in action once it will be ready for a demo, please send me an email at milan.bundalo@comtrade.com or contact me using my Skype ID milan.bundalo.

Wednesday, October 29, 2014

PowerShell - Create CmdLet Hello World

How to create PowerShell CmdLet ?

What is PowerShell CmdLet learn at Microsoft site for CmdLet .

Create class library project.

Reference System.Management.Automation from  PowerShell folder.

Typically  
C:\Program Files (x86)\Reference Assemblies\Microsoft\WindowsPowerShell\3.0\System.Management.Automation.dll on x64 system

CmdLet is class inherited from PSCmdlet class  (System.Management.Automation namespace).

We need some attributes to describe our CmdLet, at least something like:

[Cmdlet(VerbsCommon.Remove, "Controller")]

Decorating class with this attribute will create Cmdlet which can be invoked by typing

Remove-Controller.

PowerShell CmdLet common Verbs and custom Verb

VerbsCommon class from System.Management.Automation contains set of common verbs i.e. Add, Clear,Get,Set and if you want to use verb which is not defined, attribute declaration will look like:

[Cmdlet("Specify", "Controller")]

This will create Cmdlet which can be invoked by typing

Specify-Controller.

CmdLet parameters

If you need parameters in your Cmdlet you will declare them as class properties decorated with Parameter attribute i.e.:

[Parameter(Mandatory = true)]
public int ControllerId {get;set;}

CmdLet ProcessRecord method

In order to perform some work when CmdLet is invoked we need to override method:

protected virtual void ProcessRecord(); 

and we will do it like:

protected override void ProcessRecord()
{
    WriteObject(String.Format("You have entered {0}", ControllerId));
    base.ProcessRecord();
}

Importing PowerShell Module

Start Windows PowerShell and import module by issuing command:

Import-Module pathtodll

If you used non-common verb (like in our example) you will receive warning

WARNING: The names of some imported commands from the module 'PowershellPlayground' include unapproved verbs that might  make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.

Your CmdLet is now available in this specific workspace. If you close Windows PowerShell window and open new, module will not be imported.

Let's run it by typing in PowerShell

Specify-Controller

As we marked ControllerId mandatory PowerShell will prompt:

cmdlet Specify-Controller at command pipeline position 1
Supply values for the following parameters:
ControllerId:

After you enter value you will have your value printed.

Optional CmdLet parameters

Let's change decoration of our parameter:

[Parameter(Mandatory = false)]
public int ControllerId { get; set; }

Rebuild. Import module to workspace and run Specify-Controller.

PowerShell will not prompt to enter parameter value and will print:

You have entered 0

Let's play but more and change parameter decoration once more:

[Parameter(Mandatory = false)]
public int? ControllerId { get; set; }

Rebuild, import module and run

Specify-Controller

Have in mind that rebuild will fail if you have PowerShell workspace with imported dll as PowerShell locks imported file.

PowerShell will print

You have entered

Let's run command with parameter specified:

Specify-Controller -ControllerId 2

Powershell will print:

You have entered 2

Write object from CmdLet

Let's create new class in our project

    public class Response
    {
        public int Number { get; set; }
        public DateTime TimeProcessed { get; set; }
        public string Machine { get; set; }
    }

and change ProcessRecord implementation

protected override void ProcessRecord()
        {
            var response = new Response
            {
                Number = ControllerId,
                TimeProcessed = DateTime.Now,
                Machine = Environment.MachineName
            };
            WriteObject(response);
            base.ProcessRecord();
        }

Rebuild, import and run

Specify-Controller -ControllerId 4

Response will be

Number       TimeProcessed                           Machine
------            -------------                                   -------
     4            29.10.2014 9:39:43                      LIGHTNING

Let's change our Response class

     public class Response
    {
        [DisplayName("Time processed")]
        public DateTime TimeProcessed { get; set; }
        public string Machine { get; set; }
        public int? Number { get; set; }
        public string Property1 { get; set; }
        public string Property2 { get; set; }
        public string Property3 { get; set; }
        public string Property4 { get; set; }
        public string Property5 { get; set; }
        public string Property6 { get; set; }
        public string Property7 { get; set; }
        public string Property8 { get; set; }
        public string Property9 { get; set; }
        public string Property10 { get; set; }
    }

Rebuild, import module and run

Specify-Controller -ControllerId 4

Result is:

TimeProcessed : 29.10.2014 9:45:41
Machine       : LIGHTNING
Number        : 4
Property1     :
Property2     :
Property3     :
Property4     :
Property5     :
Property6     :
Property7     :
Property8     :
Property9     :
Property10    :

We can see that display name is not affecting how PowerShell prints object. We can also see that order of properties will be same as order in which they are specified in class.

CmdLet Parameter from Pipeline

Let's create another Cmdlet

using System.Management.Automation;

namespace PowershellPlayground
{
    [Cmdlet(VerbsCommon.Get,"CalculatedValue")]
    public class ExampleCmdLet2 : PSCmdlet
    {
        [Parameter(Mandatory = true, Position = 1, ValueFromPipeline = true)]
        public Response response { get; set; }

        protected override void ProcessRecord()
        {
            response.Number += 10;
            WriteObject(response);
            base.ProcessRecord();
        }
    }  
}

We have defined one parameter of Response type, mandatory and we have indicated that we will be able to receive this value from pipeline. Rebuild, import module and run:

Specify-Controller -ControllerId 5

Response will be

TimeProcessed : 29.10.2014 9:54:55
Machine       : LIGHTNING
Number        : 5

Now run

Specify-Controller -ControllerId 5 | Get-CalculatedValue

Response will be:

TimeProcessed : 29.10.2014 9:55:11
Machine       : LIGHTNING
Number        : 15

Create PowerShell Snap-in

To create PowerShell Snap-in we need to add extra class to our project. Here's example:

using System.ComponentModel;
using System.Management.Automation;

namespace PowershellPlayground
{
    [RunInstaller(true)]
    public class PowerShellPlayground : PSSnapIn
    {
        public override string Description
        {
            get { return "Playing with PowerShell"; }
        }

        public override string Name
        {
            get { return "PlayingWithPowerShell"; }
        }

        public override string Vendor
        {
            get { return "MCo"; }
        }
    }
}

This class inherits abstract class PSSnapIn and overrides few properties talking about our Snap-in. Class is actual installer so it needs to be marked with RunInstaller(true).

Rebuild solution.

Installing PowerShell Snapin

In order to make PowerShell aware of our Snap-in we need to install it using installutil:

From PowerShell:

Set-Alias installutil C:\windows\Microsoft.NET\Framework64\v4.0.30319\InstallUtil.exe
installutil pathtodll 

Uninstalling PowerShell Snapin

From PowerShell:

Set-Alias installutil C:\windows\Microsoft.NET\Framework64\v4.0.30319\InstallUtil.exe
installutil -u pathtodll 

Check that Snap-In is installed

Execute PowerShell CmdLet

Get-PSSnapin -Registered

You should be able to see following entry

Name        : PlayingWithPowerShell
PSVersion   : 3.0
Description : Playing with PowerShell

Add PSSnapin to workspace

Execute PowerShell CmdLet:

Add-PSSnapin PlayingWithPowerShell

List commands available in CmdLet

Execute PowerShell CmdLet:

Get-Command -PSSnapin PlayingWithPowerShell

Get all command with specific verb

Execute PowerShell CmdLet:
Get-Command -Verb Specify

Read more about Get-Command at Microsoft site.

Get detailed information on CmdLet usage


Execute PowerShell CmdLet:

Get-Help Specify-Controller

Response will be:

NAME
    Specify-Controller

SYNTAX
    Specify-Controller [-ControllerId <int>]  [<CommonParameters>]


ALIASES
    None


REMARKS
    None

Override PowerShell CmdLet

You can override CmdLet by crating one with same name. We will override Get-WSManInstance CmdLet.

Let's create our CmdLet

using System.Management.Automation;

namespace PowershellPlayground
{
    [Cmdlet(VerbsCommon.Get, "WSManInstance")]
    public class GetWsManInstance : PSCmdlet
    {
     
        protected override void ProcessRecord()
        {          
            WriteObject("Hello World!");
            base.ProcessRecord();
        }
    }  
}

Rebuild but do not import.

Start PowerShell and run

Get-WSManInstance

PowerShell will respond with

cmdlet Get-WSManInstance at command pipeline position 1
Supply values for the following parameters:
ResourceURI: 

This is "default" implementation from Microsoft.WSMan.Management module.

Lets import our module:

Import-Module pathtodll

Run


Get-WSManInstance

And PowerShell will respond with

Hello World!

If you have multiple overrides of same CmdLet imported in PowerShell workspace last CmdLet imported will be one that will be executed.

Setup VisualStudio for PowerShell CmdLet development

The questions is: How to setup my Visual Studio in order to debug CmdLet during development.

We will use Visual Studio 2013 for this. All setup is in project properties.

If we are creating PSSnapin we can add following line in Build Events - Post-build event command line

C:\windows\Microsoft.NET\Framework64\v4.0.30319\InstallUtil.exe "$(TargetDir)$(TargetFileName)"

and in Debug - Start Action  - Start external program

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

and in Start Options - Command line arguments

-noexit -command add-pssnapin SnapinName

This setup will automatically install snapin and load it into workspace when we hit F5.

If we are developing module we don't need Post-build event setup and we need to change Start-Options Command line arguments to

 -noexit -command import-module ModulePath

Access Database from PowerShell Module or SnapIn using EF

PowerShell Module or Snapin is typically Class library. This means that we can't use Properties/Configuration file to store Database connection string.

If we are using EF we can use DbContext constructor like i.e.:

public class MyContext : DbContext
    {
        public MyContext ()
            : base(@"Data Source=C:\db\mydb.sdf")
        {
            Configuration.ProxyCreationEnabled = false;
            Configuration.LazyLoadingEnabled = false;          
            Database.SetInitializer(new MigrateDatabaseToLatestVersion              
                                                        <MyContext , MyContextMigrationConfiguration>());
        }
}

Irony grammars

Recently I had task to create domain specific language. Looking for some open source implementation to speed up start I found Irony project .

Here's how I created my grammar.

Creating grammar with Irony


Start by creating class that inherits Irony.Parsing.Grammar.

 public class ATestGrammar : Grammar

You can add some extra information about language which you are creating by Language attribute:

[Language("ATest", "1.0", "Automated tests description")]
public class ATestGrammar : Grammar

We have added name, version and description of our language. This information is displayed in Irony.GrammarExplorer tool included in download.

Next, let's create class constructor where we will describe our grammar

public ATestGrammar() : base(false)
{
}

By calling base(false) we are indicating that our grammar is not case sensitive.

If you want to create extra information for Irony.GrammarExplorer (or your implementation) you can add

GrammarComments = "Some text";

Irony Grammar Comments


Let's define single line and delimited comments :

var SingleLineComment = new CommentTerminal("SingleLineComment", "//", "\r", "\n", "\u2085", "\u2028", "\u2029");
var DelimitedComment = new CommentTerminal("DelimitedComment", "/*", "*/");

NonGrammarTerminals.Add(SingleLineComment);
NonGrammarTerminals.Add(DelimitedComment);

Irony Grammar Elements


Elements are describing grammar structure. In our example we have following elements

var testcase = new NonTerminal("Case");
var id = new NonTerminal("TestCaseId");
var description = new NonTerminal("TestCaseDescription");
var steplist = new NonTerminal("CaseSteps");
var step = new NonTerminal("CaseStep");
var stepIn = new NonTerminal("CaseStepStart");
var call = new NonTerminal("CaseStepCall");
var callArguments = new NonTerminal("CaseStepCallArguments");
var stepDescription = new NonTerminal("CaseStepCallDescription");
var stepReturnValue = new NonTerminal("CaseStepCallReturnValue");
var stepExpectedValue = new NonTerminal("CaseStepCallExcpectedValue");

which are non-terminal (meaning there are children elements).

We must define root non-terminal element of our grammar:

this.Root = testcase;

And we have following terminal elements

var caseId = new StringLiteral("CaseId","\"", StringOptions.None);
var caseDescription = new StringLiteral("CaseDescription", "\"", StringOptions.None);
var callMethod = new StringLiteral("StepCallMethod", "\"", StringOptions.None);
var callMethodArguments = new StringLiteral("StepCallMethodArguments", "'", StringOptions.None);
var callDescription = new StringLiteral("CallDescription", "\"", StringOptions.None);
var callReturnValue = new StringLiteral("CallReturnValue", "\"", StringOptions.None);
var callExpectedValue = new StringLiteral("CallExpectedValue", "\"", StringOptions.None);

Those are elements which do not have children and which will hold values parsed using our grammar.

Irony Grammar Terms


We can treat terms as keywords. Let's define few:

var COLON = ToTerm(":");
var ID = ToTerm("id");
var DESCRIPTION = ToTerm("Description");
var STEP = ToTerm("step");
var CALL = ToTerm("Call");
var CALLARGUMENTS = ToTerm("CallArguments");
var LEFTPARENTIS = ToTerm("(");
var RIGHTPARENTIS = ToTerm(")");
var SAVERETURNVALUEAS = ToTerm("SaveReturnValueAs");
var EXPECTEDVALUE = ToTerm("ExpectedValue");

Irony Grammar Rules


Now we need to define actual grammar rules:

testcase.Rule = id + description + steplist;
id.Rule = ID + COLON + caseId;
description.Rule = DESCRIPTION + COLON + caseDescription;
steplist.Rule = MakePlusRule(steplist, step);
step.Rule = stepIn + call + callArguments + stepDescription + stepReturnValue + stepExpectedValue;
stepIn.Rule = STEP + COLON;
call.Rule = CALL + COLON + callMethod;
callArguments.Rule = Empty | CALLARGUMENTS + COLON + LEFTPARENTIS + callMethodArguments + RIGHTPARENTIS;
stepDescription.Rule = DESCRIPTION + COLON + callDescription;
stepReturnValue.Rule = Empty | SAVERETURNVALUEAS + COLON + callReturnValue;
stepExpectedValue.Rule = Empty | EXPECTEDVALUE + COLON + callExpectedValue;

Ok, so here's how we are reading our rules:

testcase.Rule = id + description + steplist;

Test Case have 3 elements: Id, Description and Step list

id.Rule = ID + COLON + caseId;

Id element have 3 elements: keyword id, keyword : and actual value. Example:

Id: "CheckService"

steplist.Rule = MakePlusRule(steplist, step);

Step list element have multiple Step elements.

callArguments.Rule = Empty | CALLARGUMENTS + COLON + LEFTPARENTIS + callMethodArguments + RIGHTPARENTIS;

Call arguments is either empty or in form

CallArguments: ('"ServiceName", "MachineName" , "Username" , "Password"')

DSL example source file


// Author: Milan Bundalo
// Date created: 10.9.2014
// Test case example
Id: "CheckService"
Description: "Check if service can be restarted."
Step:
Call: "StopService"
CallArguments: ('"Service1", "LIGHTNING", "User1", "Password"')
Description: "Stop Service1 on server Lightning."
                        ExpectedValue: "0"
Step:
Call: "Wait"
CallArguments: ('360000')
Description: "Wait 6*60*1000 milliseconds."
Step:
Call: "StartService"
CallArguments: ('"Service1" , "LIGHTNING", "User1","Password"')
Description: "Start Service1 on server Lightning."
                        ExpectedValue: "0"

Parsing Irony Grammar


Let's create console application to parse DSL example source file. In main method we need to add:

var grammar = new ATestGrammar.ATestGrammar();
var parser = new Parser(grammar);
var specification = System.IO.File.ReadAllText(args[0]);
var result = parser.Parse(specification);
if (result.HasErrors())
    {
        Console.WriteLine(String.Format("{0} at line {1} column {2}", result.ParserMessages[0].Message, result.ParserMessages[0].Location.Line, result.ParserMessages[0].Location.Column));
                throw new ArgumentException("Configuration is not in required format!");
      }

var testcase = result.Root;

Reading parsed Irony Grammar Nodes


Here are few examples  how to position node

var idInfo = testcase.ChildNodes.SingleOrDefault(s => s.Term.Name == "TestCaseId");

var description = testcase.ChildNodes.SingleOrDefault(s => s.Term.Name == "TestCaseDescription");

var caseSteps = testcase.ChildNodes.SingleOrDefault(s => s.Term.Name == "CaseSteps");

and how to read values

var testCaseId = idInfo.ChildNodes.SingleOrDefault(s => s.Term.Name == "CaseId").Token.ValueString;

var testCaseDescription = description.ChildNodes.SingleOrDefault(s => s.Term.Name == "CaseDescription").Token.ValueString;