Tuesday 6 December 2016

Firebug Stopped working!

I rely on Firebug as a Firefox add-in and was seriously dismayed when I installed Firefox 50 to find that Firebug had stopped working.  I believe that this is because from version 50 Firefox is multi-process.

Anyway I found a little work-around that gets Firebug working again.  Install Firequery after you upgrade to Firefox 50 and Firebug will start working again. Simple!

http://firequery.binaryage.com/

Monday 28 November 2016

Chrome Extension for Testing GeoIP

Recently I've been working on a project that used GeoIP.  In particular the site has a location finder map, that uses your location as a starting point.  

Not being able to test different locations was an issue for me.  And then we were asked to develop some personalization that used the State/Region value obtained via GeoIP! 

How do you easily test for different scenarios, and more importantly how would the end user do effective UAT?

Whilst doing some research into the issue I came across the following chrome browser plug-in. It solves the problem by allowing you to override the appropriate browser data that contains the IP address value and trick the GeoIP detection into thinking you are somewhere you are not.  In effect spoofing an IP Address.

Here's a link to the tool:

https://chrome.google.com/webstore/detail/sitecore-analytics-testin/pecalkbdlhhhcoenmcjnmhgnncnkdgak?utm_source=chrome-app-launcher-info-dialog

Once installed you will see a Sitecore logo top-right.  Click on the logo and under GeoIP you will see the name of the selected profile - Default is the one set up for you on install.



Click on drop-down arrow to the right of the Default text, and select Manage Profiles


Here you can then add new profiles.  In the text boxes give it a name and enter the appropriate external IP address and click the Add Profile button:





Now open a new tab and navigate to your Sitecore site, click on the Sitecore logo and select the profile you want to use and then select New Contact. The page will refresh and your IP address will be changed to the selected one.

N.B. One point to note is that in order for this to work, you will need to update the Analytics.ForwardedRequestHttpHeader value in
Sitecore.Analytics.Tracking.config file.  You should use whatever value you have configured in the GeoIP settings in the tool:






Typical choices are X-Forwarded-For or X-Real-IP.

Note that the instructions supplied in the GeoIP settings section are wrong for Sitecore version 8.1 the setting is not in Sitecore.Analytics.config file.




Friday 23 September 2016

Wednesday 11 May 2016

ERROR Error in FileWatcher. Internal buffer overflow

On one installation of Sitecore 8.1 update 2 that I recently did, I noticed lots of errors in the logs like:

5404 15:52:21 ERROR Error in FileWatcher. Internal buffer overflow.
Exception: System.IO.InternalBufferOverflowException
Message: Too many changes at once in directory: D:\inetpub\...\Website\.

As explained here: http://mikael.com/2016/01/sitecore-file-watcher-internal-buffer-overflow-exception/ 

The resolution oddly enough is to move the license file to a subfolder of the data directory.

Email Experience Manager 3.2.1 Rev. 160127 Issue with Sample Package

I recently had to install the Email Experience Manager (ECM) for a client on Sitecore 8.1 update 2, and came across an issue when following the steps to smoke test the installation:

https://doc.sitecore.net/email_experience_manager/configuring_the_delivery_process/performance/testing_exm_performance_in_emulation_mode

The steps although not explicitly saying install the supplied Sample Newsletter package do refer to the same.  So I went ahead and installed the Email Experience Manager Sample Newsletter 3.2.1 rev 160127.zip that is supplied as part of this ECM package.  

N.B. The sample package like the main ECM package is part of a collection of 7 zip files contained in the zip file that you download from  (you'll need a Sitecore Login) the Downloads -> Email Experience Manager Link found on this page: https://dev.sitecore.net/Downloads/Email_Experience_Manager/Email_Experience_Manager_32/Email_Experience_Manager_32_Update1.aspx

When clicking on the Sample Newsletter item I got the following error:

Requested value 'Drafts' was not found. 

The error appears in a popover dialogue that it launched when the item is selected.

I used Resharper's dotPeek and was able to see what was going on:


The error I am seeing is happening in the MessageItemSource class where its calls MessageStateToString - it looks like the sample item is passing in a MessageType of Drafts - which doesn’t exist!

Simple answer is don't install the package until Sitecore fix this issue, but instead go ahead and create your own Sample Newsletter.
 

Tuesday 10 May 2016

Sitecore WFFM Error on 8.1 update 2

On Sitecore 8.1 CD server after using the supplied SwitchMasterToWeb.config and having installed Web Forms For Marketers 8.1 rev 160304 - I get the following error when trying to access a form:

Unable to cast object of type 'System.String' to type 'Sitecore.Analytics.Reporting.ReportDataProvider'

The error occurs because Sitecore is trying to connect to the master Db and is getting a failure because it is not accessible. I suspect that SQL is returning a string error message instead of the expected object.

If you open up the SwitchMasterToWeb.config and comment out the following block it will resolve the issue:

<reporting>
  <dataprovider>
 <datasources>
   <add key="collection">
  <filtersfactory>
    <param desc="definitionDatabaseName" />web
  </filtersfactory>
   </add>
   <add key="reporting">
  <filtersfactory>
    <param desc="definitionDatabaseName" />web
  </filtersfactory>
   </add>
 </datasources>
  </dataprovider>
</reporting>

Wednesday 16 March 2016

Custom Sitecore Fields Part 1 of 2

Custom Fields - Part 1 - A Google Maps Field

Abstract 

One of the best features of Sitecore (in my opinion) is how extensible it is.  Almost every aspect of the CMS can be extended, overwritten or enhanced.  This includes fields.

In this two part  blog I am going to cover some different types of fields you can create and how to put them together.

All of the examples and screenshots will be taken from Sitecore version 8.1, but the fields themselves should work in 7.x as well.

We'll start by looking at how you add a custom field to Sitecore, then move on to two  examples of different types of custom fields and finally show you how implement the fields in Sitecore templates, etc.

This article assumes that you have some basic knowledge of Sitecore development and that you are able to create a new Class Library project in Visual Studio and reference all the the necessary Sitecore dlls (perhaps create a nuGet package with all the Sitecore dlls, so that you can import those straight into your project).

Setting up Sitecore

To add a custom field to Sitecore, you need to navigate to the core database.  

From the dashboard click on the desktop icon:

















Once in the desktop, click on the databases icon bottom right and select core.

Then click on the Sitecore icon bottom left, and from the menu click on Content Editor:

























You should now see a content tree similar to the above.

Navigate to /sitecore/system/Field types.

I recommend creating your own field types sub-folder, and putting your fields in there, that way they will be all grouped together when you come to add them to your templates.

To do this: right-click on the field types node and click on insert and then select folder - give it a meaningful name.

Navigate to your newly created folder and right click on the node, select insert and then select Insert from template.  You need to add an item of the following template type: /sitecore/templates/System/Templates/Template field type.

Once added, the only fields we are interested in are the Assembly and Class fields, these will be filled out once we have created the code for our new field in Visual Studio.

Creating a Google Maps Field

This custom field will enable us to display a Google Maps image on a template and configure it to pick information from an address field on the same item.  This could be used by a content author to verify address information being added to an item.

Here's an example of it being used to show the location of a partner, based on the Partner Address field on the same item:

















To start lets create a new class called GoogleMapField.

It needs to inherit from: Sitecore.Web.UI.HtmlControls.Control, and needs to implement:
Sitecore.Shell.Applications.ContentEditor.IContentField

 
public class GoogleMapField : Control, IContentField
{
}

We need to add some properties which will get populated at run time.


public class GoogleMapField : Control, IContentField
{
     //Constants
     private const int MAP_HEIGHT = 300;
     private const int MAP_WIDTH = 600;
     private const string CONTENT_EDITOR_SAVE = "contenteditor:save";
     private const int DEFAULT_ZOOM = 14;
     private const string OVERRIDEZOOM = "OverrideZoom";
     private const string ISDEBUG = "Debug";
     private const string ADDRESSFIELDNAME = "AddressField";

     private Image _mapImageCtrl;

     private int _mapZoomFactor = DEFAULT_ZOOM;
     private bool Debug { get; set; }

     //Needed for a custom field 
     public string Source { get; set; }
     public string ItemID { get; set; }

     //Parameter properties
     protected int OverrideZoom { get; private set; }
     protected string AddressFieldName { get; private set; }

}

Next we'll create a method to parse the parameters  that will be passed into it from the source field in the template where it is used:



In this example the source field parameters are: &Debug=false&AddressField=Partner Address.
  • Debug [Optional] - turns on off display of the of the Google URL above the map.
  • AddressField [Required] - Name of the field containing the address information
  • OverrideZoom [Optional] - An integer value used to override the default zoom level of 14 

private void ParseParameters(string source)
{
     if (string.IsNullOrEmpty(source))
     {
         return;
     }

     var parameters = new UrlString(source);
     if (!string.IsNullOrEmpty(parameters.Parameters[OVERRIDEZOOM]))
     {
         int zoomValue;
         int.TryParse(parameters.Parameters[OVERRIDEZOOM], out zoomValue);

         if (zoomValue < 0 || zoomValue > 20) zoomValue = DEFAULT_ZOOM;

         OverrideZoom = zoomValue;
      }

      if (!string.IsNullOrEmpty(parameters.Parameters[ADDRESSFIELDNAME]))
      {
          AddressFieldName = parameters.Parameters[ADDRESSFIELDNAME];
      }

      if (!string.IsNullOrEmpty(parameters.Parameters[ISDEBUG]))
      {
           Debug = MainUtil.GetBool(parameters.Parameters[ISDEBUG], false);
      }
}
 
Next we need to add a method that extracts the address information and parses it into a format that we can push to Google: 

private string GetGeoData(Item item, ref string centre)
{
       var location = "";

       location = item[AddressFieldName].Replace("
", ", ");

       location = HttpUtility.UrlEncode(location);

       if (string.IsNullOrEmpty(location))
       {
           centre = location;
       }
       return location;
}
Next we need to add a method that will generate the URL:

private string GetGoogleMapImageUrl(string location, string centre)
{
    return
    string.Format(
          !string.IsNullOrEmpty(location)
               ? "http://maps.googleapis.com/maps/api/staticmap?&q={0}&zoom={1}&size={2}x{3}&sensor=false&maptype=roadmap&markers=color:blue%7Clabel:1%7C{0}"
               : "http://maps.googleapis.com/maps/api/staticmap?center={4}&zoom={1}&size={2}x{3}&sensor=false&maptype=roadmap&markers=color:blue%7Clabel:1%7C{0}",
               location, _mapZoomFactor, MAP_WIDTH, MAP_HEIGHT, centre);
}

Finally we override the Render method to put it all together:

protected override void Render(HtmlTextWriter output)
 {
     if (!string.IsNullOrEmpty(Value))
     {
          int.TryParse(Value, out _mapZoomFactor);
     }

     ParseParameters(Source);
     if (!string.IsNullOrEmpty(AddressFieldName))
     {
          base.Render(output);

          var db = Factory.GetDatabase("master") ?? Factory.GetDatabase("web");

          var id = !string.IsNullOrEmpty(ItemID)
               ? ItemID
               : ControlAttributes.Substring(ControlAttributes.IndexOf("//master/", StringComparison.Ordinal) + 9,
                 38);
          var item = new ID(id).ToSitecoreItem(db);

          //render other control

           var centre = "";
     
           var location = GetGeoData(item, ref centre);


           _mapImageCtrl = new Image
           {
               ID = ID + "_Img_MapView",
               CssClass = "imageMapView",
               Width = MAP_WIDTH,
               Height = MAP_HEIGHT,
               ImageUrl = GetGoogleMapImageUrl(location, centre)
            };
            _mapImageCtrl.Style.Add("padding-top", "5px");

            if (Debug)
            {
                output.Write(
                    "<div style="background-color: silver; color: white;">{0}</div>",_mapImageCtrl.ImageUrl);
            }

           _mapImageCtrl.RenderControl(output);
      }
}

What about the buttons? - For those we need to override the HandleMessage method:

public override void HandleMessage(Message message)
{
    if (!string.IsNullOrEmpty(Value))
    {
        int.TryParse(Value, out _mapZoomFactor);
    }

    string messageText;
    if ((messageText = message.Name) == null)
    {
        return;
    }

    if (messageText.Trim() == "contentfile:zoom")
    {
        _mapZoomFactor = _mapZoomFactor + 1 > 20 ? _mapZoomFactor : _mapZoomFactor + 1;
        Value = _mapZoomFactor.ToString();
        Sitecore.Context.ClientPage.Dispatch(CONTENT_EDITOR_SAVE);
        return;
    }

    if (messageText == "contentfile:zoom-")
    {
        _mapZoomFactor = _mapZoomFactor - 1 < 0 ? _mapZoomFactor : _mapZoomFactor - 1;
        Value = _mapZoomFactor.ToString();
        Sitecore.Context.ClientPage.Dispatch(CONTENT_EDITOR_SAVE);
        return;
    }

    if (messageText == "contentfile:reset")
    {
        _mapZoomFactor = _mapZoomFactor - 1 < 0 ? _mapZoomFactor : _mapZoomFactor - 1;
        Value = DEFAULT_ZOOM.ToString();
        Sitecore.Context.ClientPage.Dispatch(CONTENT_EDITOR_SAVE);
        return;
    }
    base.HandleMessage(message);
}

Finishing off Sitecore Field Setup

Now that we have created our Google Maps field, lets go back to Sitecore and finish setting up the field.
















As  mentioned before fill out the Assembly and Class fields with the appropriate information taken from your solution.

Next we need to add the three buttons:



















You need to fill out the following for each of the buttons:
  • The Display Name - Text you want to appear in Sitecore
  • Message  - This should match the Message Text value used in the HandleMessage method above.  You should also add the (id=$Target) to tell Sitecore to target the current item. 
  • Show in Field Editor (not shown) - This should be checked.

 

How could we enhance this?

For one project I worked on, we imported a geo database for North America into Sitecore.  We added the Google Maps Field to the template used to store the geo data items.  I then added some enhancements to enable the Google Maps Field to try and get the information from different fields on the item:
  • Lat & Long
  • Post Code
  • Full Address
private static string GetGeoData(BaseItem item,  ref string centre)
{
    double lat;
    double lng;
    double.TryParse(item["latitude"], out lat);
    double.TryParse(item["longitude"], out lng);
    var location = string.Format("{0},{1}", lat, lng);
            
    if (lat == 0.00 && lng == 0.00)
    {
         location = item["postcode"];
    }
    if (string.IsNullOrEmpty(location))
    {
         centre = string.Format("{0}{1}{2}{3}{4}{5}",
             !string.IsNullOrEmpty(item[LOCALITY]) ? item[LOCALITY] + "," : ""
             , !string.IsNullOrEmpty(item[REGION4]) ? item[REGION4] + "," : ""
             , !string.IsNullOrEmpty(item[REGION3]) ? item[REGION3] + "," : ""
             , !string.IsNullOrEmpty(item[REGION2]) ? item[REGION2] + "," : ""
             , !string.IsNullOrEmpty(item[REGION1]) ? item[REGION1] + "," : ""
             , !string.IsNullOrEmpty(item[COUNTRY]) ? item[COUNTRY] : "");
    }
    return location;
 }

Hopefully this will give you some ideas on how you too, could use this field!

I have the full source (including the Sitecore Package) here if you would like to download it: Sitecore Fields

In the next article we will look at a more complex field and also how to use these fields in Sitecore.

Thursday 28 January 2016

Sitecore FakeDB and real world testing

One of the biggest issues in my opinion with Sitecore at present, is that it is relatively hard to unit test anything you build for it.  This is because none of the integral pieces (Item, Database, Security, etc.) have interfaces, are often sealed classes or have no parameter-less constructors, making it extremely hard to mock.  There are ways around this, such as stand-alone mocks Microsoft Fakes, but I have never found this particularly easy to implement and they don't work well ReSharper's test runners.

Obviously components that don't integrate with Sitecore directly (controllers for example) or where perhaps you can use Glass Mapper can be mocked out, to some extent.

I recently came across Sitecore FakeDB (https://marketplace.sitecore.net/Modules/Sitecore_FakeDb.aspx) and wondered if this might be the answer to my problems.

I wondered if we could test something substantial like a pipeline processor?

The code examples below assume that:
  • You are familiar with how to set up the configuration for Sitecore FakeDB (see the documentation tab on the above Sitecore Marketplace link).
  • You are familiar with xUnit (https://xunit.codeplex.com/)
  • You are familiar with mocking  and/or duck casting technologies.
  • You have a basic understanding of Sitecore.
I also used a combination of NSubstitute (http://nsubstitute.github.io/) and Moq (https://github.com/Moq/moq4), as Moq had issues setting up the GetItem<T> method found on Glass Mapper's SitecoreService class.

N.B. Moq kept reporting no such extension method - I don't believe that GetItem<T> is an extension method - but I needed to get going so I used NSubstitute instead.

Brief:

I want to write a test for a pipeline processor, without having to make too many changes to the existing code.

I have an item resolver; an HttpRequestProcessor that looks at the current URL passed via arguments, and makes a decision to redirect to another or not.

All HttpRequestProcessors are passed HttpRequestArgs in a Process method which contains contextual information such as the current URL, etc.

The code:

Here is my process method (the starting point for my pipeline processor):

public override void Process(HttpRequestArgs args)
{

   Assert.ArgumentNotNull(args, "args");
   if (Context.Item != null || Context.Database == null || Context.Database.Name.ToLowerInvariant() == "core" || args.Url.ItemPath.Length == 0)
    {
       return;
    }

   if (args.Url.FilePath.Contains("/sku/"))
    {
        GetSkuItem(args);
    }

   if (!args.Url.FilePath.Contains("/brands/")) return;
   Context.Item = GetBucketItem(args);
}

So the first issue I hit was how to construct the HttpRequestArgs, which is in itself is a sealed class (so cannot be mocked) and is constructed using a combination of both HttpRequest (not base) and HttpResponse (not base) both of which are hard to mock out.

This is what I came up with:

private HttpRequestArgs CreateHttpRequestArgs(string url)
{
  _redirect = new StringBuilder();

  var response = new HttpResponse(new StringWriter(_redirect));
  var request =
      new HttpRequestArgs(
          new System.Web.HttpContext(new HttpRequest("", url, ""),
          response), HttpRequestType.End);

  var dynMethod = request.GetType().GetMethod("Initialize", BindingFlags.NonPublic | BindingFlags.Instance);
      dynMethod.Invoke(request, null);
  return request;
}

The important part here is the small piece of reflection at the end, that calls the private Initialize method on the HttpRequestArgs object, which amongst other things populates the URL on the object itself.

The StringWriter above is passed a StringBuilder, which we will use later. 

Now my test methods can use this to construct the HttpRequestArgs and send it through to the processor.

So, lets start with something really simple, no database (anything else will require us to start using Sitecore FakeDB).

[Fact]
public void Null_Database_Item()
{
  var request = CreateHttpRequestArgs("http://myhost/");

  _bucketItemResolver.Process(request);

  Sitecore.Context.Item.Should().BeNull();
}

N.B. The _bucketItemResolver class is initialized in the constructor for this class

Now lets pass in context item, which should also fail at the first if in the above Process method where it checks for a null context item.

[Fact]
public void A_Context_Item()
{
   using (var db = new Db {new DbItem("Home") {{"Title", "Welcome!"}}})
   {

     var request = CreateHttpRequestArgs("http://myhost/");

     var homeItem = db.GetItem("/sitecore/content/home");
     Assert.Equal("Welcome!", homeItem["Title"]);

     Sitecore.Context.Item = homeItem;

     _bucketItemResolver.Process(request);

     Sitecore.Context.Item.Should().Be(homeItem);
    }
}

In the above code, we construct the Fake Db with a single item.  We set the context to be that item, and when the test is run the Process method checks for a null context and halts any further processing immediately because that condition fails.

A more complex test:

One of the URL checks I have, is one that looks for the following pattern: '/sku/'. 

The processor does the following if it finds a URL pattern match in the GetSkuItem method:
  1. Splits up the file-path to determine if there is a SKU number after the pattern.
  2. Does a product search with SKU against a custom index to find the product.
  3. Uses a link provider to build the URL to that product
  4. Redirects to the new URL
private void GetSkuItem(HttpRequestArgs args)
{
   try
   {
      var filepath = args.Url.FilePath;

      var splitPath = filepath.ToLowerInvariant().Split('/').RemoveWhere(p => p.Equals(string.Empty));
      var pathPaths = splitPath as string[] ?? splitPath.ToArray();
      if (!pathPaths.Any() || pathPaths.Count() <= 1) return;
      var productSearchString = pathPaths[1];
      string redirectUrl;
      var originalUrl = args.Url.FilePath;
      lock (RedirectionCache)
      {
          if (!RedirectionCache.TryGetValue(originalUrl, out redirectUrl))
          {
              var product = ProductsSearchIndex.GetProductBySku(productSearchString);
              if (product != null)
              {
                   redirectUrl = new ProductLinkProvider().GetItemUrl(product.GetItem(),
                     (UrlOptions) UrlOptions.DefaultOptions.Clone());

                   if (!string.IsNullOrWhiteSpace(redirectUrl))
                   //Add new url to the cache.
                   if (!RedirectionCache.ContainsKey(originalUrl))
                      RedirectionCache.Add(originalUrl, redirectUrl);
               }
           }
        }

        if (!string.IsNullOrWhiteSpace(redirectUrl)) // There is a redirection match
        {
            try
            {
                args.Context.Response.RedirectPermanent(redirectUrl, true);
            }
            catch (ThreadAbortException)
            {
               Log.Info(string.Format("Permanent Redirection being applied for {0} to {1}", originalUrl, redirectUrl),
                     this);
            }
            catch (NullReferenceException)
            {
               //Used for testing
               args.Context.Response.Write(redirectUrl);
               throw;
            }
         }
      }
      catch (ThreadAbortException)
      {

      }
      catch (Exception ex)
      {
         Log.Error(ex.Message, ex, this);
      }
} 

Here is the test that I wrote:

[Fact]
public void Its_A_Sku()
{
   var index = Substitute.For<Sitecore.ContentSearch.ISearchIndex>();
   Sitecore.ContentSearch.ContentSearchManager.SearchConfiguration.Indexes.Add("product_master_search_index", index);

   using (var db = new Db
           {
              new DbItem("Home") {{"Title", "Welcome!"}},
              new DbItem("Globals/Import/products/10074-test-toy")
                {
                  {"Name", "10074-test-toy"},
                  {"SKU", "10074"},
                  {"Product Name", "10074 Test Toy"},
                  {"Brand", "{989FA9CC-0522-4C6F-AECD-B11F6AC84CFB}"}
                }
            })
    {
        var homeItem = db.GetItem("/sitecore/content/home");
        Assert.Equal("Welcome!", homeItem["Title"]);

        var request = CreateHttpRequestArgs("http://myhost/sku/10074");

        var searchResultItem = Substitute.For<ProductSearchResultItem>();

        searchResultItem.GetItem()
            .Returns(db.GetItem("/sitecore/content/Globals/Import/products/10074-test-toy"));
        searchResultItem.SKU = "10074";
        searchResultItem.Product_Name = "10074 Test Toy";
        searchResultItem.Name = "10074-test-toy";

        var mockglassProduct = new Moq.Mock<GlassProduct>();

        mockglassProduct.Setup(p => p.TemplateId).Returns(IImported_ProductConstants.TemplateId.Guid);
        mockglassProduct.Setup(p => p.Product_Name).Returns(searchResultItem.Product_Name);
        mockglassProduct.Setup(p => p.Name).Returns(searchResultItem.Name);
        mockglassProduct.Setup(p => p.Brand).Returns(new Imported_Brand{Name = "Test Brand"});

        var mockSitecoreService = Substitute.For<ISitecoreService>();

        mockSitecoreService.GetItem<GlassProduct>(Arg.Any<Guid>())
                           .Returns(mockglassProduct.Object);


        ProductLinkProvider.SitecoreService = mockSitecoreService;

        index.CreateSearchContext()
            .GetQueryable<ProductSearchResultItem>()
            .Returns((new[] {searchResultItem}).AsQueryable());

        _bucketItemResolver.Process(request);

        Sitecore.Context.Item.Should().BeNull();

        _redirect.ToString().Should().Be(string.Format("/brands/{0}/{1}", mockglassProduct.Object.Brand.Name, mockglassProduct.Object.Name));
     }
}

  • I recreate the search index using Sitecore FakeDB.
  • Set up the in-memory database with two items.
  • Mock up the Glass Mapper SitecoreService to return the expected search result item.
  • Set the index to return the correct item from the in-memory DB (make sure the values are correct as linQ to Sitecore will be performed against the item).
  • Execute the processor.
  • And finally check the StringBuilder to see if it has the redirected URL in it.

 I had to make a couple of changes to the existing code to get this test to work. 

  1. I had to add the NullReferenceException catch as there was no way to get the HttpResponse which is used to do the redirect to delegate and do something else instead.  If the processor arguments used HttpResponseBase I would have been able to mock out a delegate method to call. 

    Its possible that the redirect might be able to be done with some other Sitecore Utility methods, but as these would undoubtedly be static, again it would be hard to mock them out.

    I took the opportunity here to populate the StringBuilder with a value - not the neatest but at least I can test a result.

  2. In the ProductLinkProvider class we use the SitecoreService.  Unfortunately this is resolved from DI using Kernel.Resolve (found on a static helper class), so I had to add a property to the class, which I could use in testing, to set up my mocked version of the service and have the code check for a valid service in that property. 

    N.B. If I had had time, I may have been able to rewrite the provider so that the SitecoreService was injected in properly.
Conclusions

All in all I am very happy with Sitecore FakeDb and will be using it to write more comprehensive tests in the future.  I give it two thumbs up!

If in the future Sitecore can make some minor changes with testing in mind, then that would be the icing on the cake.

Wednesday 20 January 2016

Building Lucene Search Index - Order of templates does matter!

I've just spent a frustrating 30 minutes trying to figure out why a search of mine wasn't returning the results I expected.

Here's the json that gets passed to my API that is used to search against the search index:

{"pages":
    ["c159f9f7-77b6-42c2-9b35-50279001987e",
     "8c24e1a1-986e-4c01-9c2f-447b89805fd7",
     "b52b3044-f7b0-406d-b3ac-6be5b8148a7f",
     "f2726c5f-3fbc-4b25-8d53-693c0eb50b93",
     "2319ea68-74c6-4ba3-af7b-02a6b766f311",
     "a70eb547-bab6-44c0-97ca-b749ba333561",
     "2685a1f2-7aac-4e99-ba9d-5b05667140a7",
     "6385cf29-2133-4c70-a247-ec0be744db64",
     "37816c10-fb96-45b6-b123-cb815926df97"
    ],"language":"en","PageSize":10}


In my search I pass in a number of GUIDs (pages) which are matched against Ids in the index.  Sounds simple enough!  I had four templates added to my index configuration, three of which where pages and one (the last in the list) was inherited by all the pages:
       
<include hint="list:IncludeTemplate">
  <!--Product Overview Page--> 
  <templateId>{1A1706B9-95AF-408D-ACF6-A158315333F4}</templateId>
  <!--Product Module Detail Page--> 
  <templateId>{1263F902-B309-47B1-BCE5-25305167E15C}</templateId>
  <!--Product Module Landing Page--> 
  <templateId>{ECA7823D-4BA9-4995-836B-85FA90F00481}</templateId>
  <!--Related Product Details--> 
  <templateId>{57F36C1B-9292-4386-96DB-8B87A2FBBE28}</templateId>
</include>


All worked fine until I added another page template id to the list (which also inherited the same template as the other pages).  I added it as the last item in the list:

         
<include hint="list:IncludeTemplate">
  <!--Product Overview Page--> 
  <templateId>{1A1706B9-95AF-408D-ACF6-A158315333F4}</templateId>
  <!--Product Module Detail Page--> 
  <templateId>{1263F902-B309-47B1-BCE5-25305167E15C}</templateId>
  <!--Product Module Landing Page--> 
  <templateId>{ECA7823D-4BA9-4995-836B-85FA90F00481}</templateId>
  <!--Related Product Details--> 
  <templateId>{57F36C1B-9292-4386-96DB-8B87A2FBBE28}</templateId>
  <!--Alternative Text--> 
  <templateId>{82B30165-9698-4E79-8445-9F2BEF315382}</templateId>
</include>


Now if my search json just included page ids for pages that were of the new page template type then my query would return what I expected.  Even if I added some additional page ids but the unique count per template was less than the count for my new one, it still worked as expected.

But as soon as I sent in a request where there were a larger number of one of the original template types, then the result excluded my new template type.  It didn't matter where in the list I put the Ids for new template type they always got excluded. - Weird or what!

As soon as I changed the order of my templates in the index configuration it all worked as expected:

         
<include hint="list:IncludeTemplate">
  <!--Alternative Text--> 
  <templateId>{82B30165-9698-4E79-8445-9F2BEF315382}</templateId>
  <!--Product Overview Page--> 
  <templateId>{1A1706B9-95AF-408D-ACF6-A158315333F4}</templateId>
  <!--Product Module Detail Page--> 
  <templateId>{1263F902-B309-47B1-BCE5-25305167E15C}</templateId>
  <!--Product Module Landing Page--> 
  <templateId>{ECA7823D-4BA9-4995-836B-85FA90F00481}</templateId>
  <!--Related Product Details--> 
  <templateId>{57F36C1B-9292-4386-96DB-8B87A2FBBE28}</templateId>
</include>


For brevity here is the query I used to return the search results:

  using (var searchContext =
                    ContentSearchManager.GetIndex(RelatedProductsSearchIndexName).CreateSearchContext())
            {
                var query = GetBaseQueryForOtherType<RelatedProductSearchResultsItem>(searchContext, language);

                var predicate = PredicateBuilder.False<RelatedProductSearchResultsItem>();

                var predicateCategory = PredicateBuilder.False<RelatedProductSearchResultsItem>();

                if (relatedProductsRequest.Pages.Any())
                {
                    predicateCategory =
                        relatedProductsRequest.Pages.Select(item => new ID(item))
                            .Aggregate(predicateCategory, (current, id) => current.Or(p => p.ItemId == id));

                    predicate = predicate.Or(predicateCategory);
                }

                query = query.Where(predicate);

                if (relatedProductsRequest.ProductId != Guid.Empty)
                {
                    query = query.Where(p => p.ProductPageID == relatedProductsRequest.ProductId);
                }

                count = query.Count();

                return query.Page(relatedProductsRequest.PageNumber, relatedProductsRequest.PageSize)
                    .GetResults()
                    .Hits.OrderByDescending(h => h.Score)
                    .Select(h => h.Document)
                    .ToList();
            }

Friday 8 January 2016

Using Sitecore to manage external images

We have just gone live with a new site where a large number of the product images are hosted by a third party.  In fact the whole product data set is imported and contains image items that just have a URL to the images themselves.

The issue with these images was that they are massive in size and cannot be loaded into the web site without some sort of re-sizing and compression.  Unfortunately the image hosting site, does not have any functionality that would allow us to resize the image at source - i.e. it is not a DAM.

We wanted to leverage Sitecore to do this for two reasons:
  1. Sitecore has a very robust re-sizing mechanism that is easy to invoke and is familiar to all Sitecore developers.

  2. We wanted the images to be cached on the server - which Sitecore does for all media items by default.
To get Sitecore to process the image, we need to leverage the existing media library functionality.  This is done by using a media library item as the hook or kicking off point for the processor.  We then use that media item's URL where we need to display an external image, and we add some additional parameters to the URL that can then be picked up by a custom image processor.  

In our case we have an image in the media library called CustomContent.  

We have also designed our processor so that if there is no external image, the CustomContent image will be shown instead.



Just to give you an idea of what we are talking about here is an image URL:
/~/media/images/CustomContent.ashx?externalImage=https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZ9hdJMNqLxS9LO-wqL5S2la1_D9naH77vul12s10qI8PTzCb10F0qypgWMTyU_xGXI_3jV31vziVwLPGcKmb5vajxeWXhXVmfmqpxIalraqVnpilsJHp4_V-DOzAoNVhXz6xSLUo7VuF1/,27da3a5fe74dc9dc/IMG_0030.JPG&h=180&w=180&id=3f36195e-8dbe-45c5-8ee0-18c2ab130eef&hash=7FC9C5E5044978E99335A9BF4FD3E8B56237E17B

Important points to note:
  1. Extra Parameters including externalImage and id

  2. The image URL can point to any external image
The hash in the URL is added by Sitecore for media request protection, this is a new feature in Sitecore 7.5 and above.  If enabled you will need to register your parameters in the Sitecore config file: Sitecore.Media.RequestProtection.config, and generate the hash when building your image URL.  This means that the URL will then become tamper proof.

In your URL helper / resolver do something like:

HashingUtils.ProtectAssetUrl(
  string.Format("/~/media/images/CustomContent.ashx?externalImage={0}&h={1}&w={2}&id={3}", url,
  source.Height, source.Width, source.Item.ID.ToGuid().ToString("D")));

The HashingUtils class can be found by using Sitecore.Resources.Media.

You will need to add your parameters to the protectedMediaQueryParameters section of the Sitecore.Media.RequestProtection.config file, as follows:


 
So now for the code:

First off the processor has an entry method called Process, in here we accept a single argument parameter of GetMediaStreamPipelineArgs, which we can then examine for our custom parameters:

public void Process(GetMediaStreamPipelineArgs args)
{
    if (args.Options.CustomOptions["externalImage"] != null || 
        args.Options.CustomOptions["id"] != null)
    {
             GetExternalImage(args);
    }
} 

If we find the custom parameters, we can call the main piece of functionality which will get the image re-size and/or compress it and render it through the media request pipeline.

private void GetExternalImage(GetMediaStreamPipelineArgs args)
{
    try
    {
         var client = new WebClient();
         var result = client.OpenRead(new 
            Uri(args.Options.CustomOptions["externalImage"]));

         if (result == null)
         {
                args.AbortPipeline();
                return;
         }
         var bm = CreateBitMapFromStream(result);
         var quantizer = new WuQuantizer();
        if (args.Options.Width != 0 && args.Options.Height != 0)
         {
                 using (var newImage = ScaleImage(bm, args.Options.Width, 
                           args.Options.Height))
                 {
                     using (var quantized = quantizer.QuantizeImage(new Bitmap(newImage)))
                     {
                         var stream = new MemoryStream();
                         quantized.Save(stream, ImageFormat.Png);
                         args.OutputStream = new MediaStream(stream, "png", 
                              args.MediaData.MediaItem);
                     }
                 }
          }
          else
          {
                  using (var quantized = quantizer.QuantizeImage(bm))
                  {
                      var stream = new MemoryStream();
                      quantized.Save(stream, ImageFormat.Png);
                      args.OutputStream = new MediaStream(stream, "png", 
                          args.MediaData.MediaItem);
                  }
          }
       }
       catch (Exception ex)
       {
           Log.Error(ex.Message, ex, this);
           args.AbortPipeline();
       }
}

We use WuQuantizer to do the png compression which does a better job than .net's native compression.

https://www.nuget.org/packages/nQuant/ 

Just for completeness here are the two private methods referenced in the above:

private static Bitmap CreateBitMapFromStream(Stream result)
{
    byte[] imageBytes;
    using (var ms = new MemoryStream())
    {
        var count = 0;
        do
        {
            var buf = new byte[1024];
            count = result.Read(buf, 0, 1024);
            ms.Write(buf, 0, count);
        } while (result.CanRead && count > 0);
        imageBytes = ms.ToArray();
    }

    // Now you can use the returned stream to set the image source too
    var image = new MemoryStream(imageBytes);
    var bm = (Bitmap) Image.FromStream(image);
    return bm;
}

public static Image ScaleImage(Image image, int maxWidth, int maxHeight)
{
    var ratioX = (double) maxWidth/image.Width;
    var ratioY = (double) maxHeight/image.Height;
    var ratio = Math.Min(ratioX, ratioY);

    var newWidth = (int) (image.Width*ratio);
    var newHeight = (int) (image.Height*ratio);

    var newImage = new Bitmap(newWidth, newHeight);

    using (var graphics = Graphics.FromImage(newImage))
        graphics.DrawImage(image, 0, 0, newWidth, newHeight);

    return newImage;
}

Finally implementing the processor, we need to add our processor to the getMediaStream pipeline, and we do that we creating the following config file:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <getMediaStream>
        <processor patch:after="processor[@type='Sitecore.Resources.Media.GrayscaleProcessor, Sitecore.Kernel']"
            type="MyProject.Library.Processors.CustomImageProcessor, MyProject.Library" />
      </getMediaStream>
    </pipelines>
  </sitecore>
</configuration>

Saturday 2 January 2016

Testing in software development and parallels with the real world

I have often thought that we can see many parallels to software development in other professions and walks of life.

The other day we had a plumber come out to our house to fit a water purification system (in the crawl).  At some point during the install he mentioned that he would need to fit a pressure reducing valve as the water pressure in the house was too high for his equipment - for the sake expediency I agreed.  At the time I was surprised that after completing the install he never once set foot in the house.  He handed over his invoice and went on his way, the only thing he said was that if there were any issues please call the office.

A little while later I went to run a faucet (tap) in the house and was surprised (but with hindsight perhaps not really!) that the water came out with a dribble.  I tried the showers and again the pressure was so low that there was no way you could wash under flow of water.  I called their office, and the because the plumber was on another job, the owner had to come (1 hour round trip) spend 15 minutes adjusting (and testing!) to finally get it right.  

What struck me here, was that this was not the first time that this pattern of events had happened to me when calling out a trade to the house.  I am also sure after speaking to friends that this has happened frequently for others as well.  I later found out there are regulations that lay out the kind of tests a plumber should be performing - most importantly to ensure that someone doesn't get scolded!  So why were these not followed?  Do they need to be scripted out?

I suspect that the plumber had done this so many times, that he felt that his assumptions were as good as doing the tests.  Sound Familiar? - But as we know assumptions are the mothers of all mistakes!*

But I know from experience that if I have a set tests laid out before me that require a single click to run, I will use them, and more often than not I will add to and update them as and when necessary.  I also know that developers shy away from writing tests, often siting that it takes too much time.  But is that really their call? Feels to me that we need a culture shift...

The implications for my plumber, also apply to developers:
  1. Customer/end-user gets frustrated/annoyed and may not use them again
  2. Business has additional expense to fix the issue
  3. Business owner/key staff are taken away from running his business
  4. Legal implications if injured, and so on.
I have come across many businesses where the software development teams behave just like my plumber.  They churn out code with little or no testing with a subconscious expectation that QA or (more expensively) the business will test their code.   They make assumptions about how things should operate and rarely have any concrete evidence (unit tests) that their code does in fact work!  

Any business that builds software and relies solely on QA/UAT as their only line of defense will end up with a high turn over of senior staff, software that doesn't perform/doesn't work period/fails frequently, and lazy programmers.

If you get:
  • Your software developers to spend the extra time writing unit tests.
  • Implement a CI platform
  • Have a UAT environment that as closely as possible matches production 
  • Have developers/deployment teams prepared to smoke test deploys based on a set of scripts.
  • Have scripts prepared for UAT
Then:
  • Issues will be caught earlier in the process
  • UAT becomes an easy step-by-step process - a sign off.
  • You reduce the cost to the business 
  • You reduce the strain on the most important people in your business.
  • And more importantly less failures once the code is in production
*Paraphrasing the Marcus Penn (Everett McGill) from the movie Under Siege 2