From feeling to insight

A while ago I blogged about starting a new crawl with PowerShell in this post, but today I got a bit further, each rollout I do on my dev machine a whole new site collection gets created using PowerShell, after that there is some test content provisioned (also using PowerShell), and finally the search settings are added. It kinda feels like everything is done through PowerShell. The point however is that I not only want to start a new crawl, but make sure that ‘old’ content is deleted. Step by step; I create some Crawled properties, create some Managed Metadata properties, create some scopes and finally delete the old content and crawl the new one.

The first part creating crawled properties and metadata properties i used this blog post. Getting me something like the following, that checks if the crawled property exists and if so gets it, otherwise creates it, then setting it into a metadata property that can be used. Nice to know is that the –propset property even though the documentation says its optional is in fact required!

  1: # Get or create SPEnterpriseSearchMetadataCrawledProperty 
  2: if (Get-SPEnterpriseSearchMetadataCrawledProperty 
  3:         -SearchApplication $searchapp -Name "proptocreate" 
  4:         -ea "silentlycontinue") {
  5:   $crawlprop = Get-SPEnterpriseSearchMetadataCrawledProperty 
  6:           -SearchApplication $searchapp -Name "proptocreate" 
  7: } else {
  8:   $crawlprop = New-SPEnterpriseSearchMetadataCrawledProperty 
  9:           -SearchApplication $searchapp -VariantType 31 
 10:           -Name proptocreate -IsNameEnum $false -PropSet "00130329-0000-0130-c000-000000131346"
 11: }
 12: 
 13: if (Get-SPEnterpriseSearchMetadataManagedProperty 
 14:         -SearchApplication $searchapp -Identity "MetadataPorpertyExample" 
 15:         -ea "silentlycontinue") {
 16:    write-host -f Green "MetaProperty already exists, 
 17:           we delete it so it can be reacreated "
 18:    $prop = Get-SPEnterpriseSearchMetadataManagedProperty 
 19:           -SearchApplication $searchapp -Identity "PublishingPageContent"
 20:    $prop.DeleteAllMappings()
 21:    $prop.Delete()
 22:    $searchapp.Update()
 23: } else {}
 24: 
 25: write-host -f Green "Try to create MetaProperty"
 26: $prop = New-SPEnterpriseSearchMetadataManagedProperty
 27:    -SearchApplication $searchapp -Name "PublishingPageContent" -Type 1
 28: $prop.EnabledForScoping = $true
 29: $prop.Update()
 30: 
 31: New-SPEnterpriseSearchMetadataMapping -SearchApplication 
 32: $searchapp -ManagedProperty $prop -CrawledProperty $crawlprop

After that i can create scopes (my colleague Peter found it out), where I check if a scope already exists and if so delete it so it can be recreated again:

  1: if (Get-SPEnterpriseSearchQueryScope -SearchApplication $searchapp 
  2:         -Identity "ScopeExample" -ea "silentlycontinue") {
  3:   write-host -f Green "Scope already exists, 
  4:            we delete it so it can be reacreated "
  5:   $scope= Get-SPEnterpriseSearchQueryScope -SearchApplication 
  6:             $searchapp -Identity "ScopeExample" 
  7:   $scope.Delete();
  8:   $searchapp.Update();
  9: } else {}
 10: 
 11: # Create "ScopeExample" scope
 12: $scope = New-SPEnterpriseSearchQueryScope -Name "ScopeExample" 
 13:    -Description "Doorzoek alle informatie" -SearchApplication $searchapp 
 14:    -DisplayInAdminUI $true
 15: 
 16: New-SPEnterpriseSearchQueryScopeRule -RuleType AllContent 
 17:   -Url $url -scope $scope

And after that in finally can reset my index, and make sure everything is found:

  1: Write-Host -f Green "Delete current index ...";
  2: $searchapp.Reset($true, $true)
  3: 
  4: Write-Host -f Green "Start indexing and scopecompilation ";
  5: $searchapp.StartScopesCompilation()
  6: 
  7: $CrawlContents = Get-SPEnterpriseSearchServiceApplication 
  8:     | Get-SPEnterpriseSearchCrawlContentSource
  9: 
 10: foreach ($crawlcontent in $CrawlContents) 
 11:     { $crawlcontent.StartFullCrawl(); }

I just ran into a nice SharePoint problem that is being described and resolved over here. Apparently there is a job trying to access your HOSTS file, in order to add some lines for the Search, and if it doesn’t have sufficient rights it complains about it. Whenever you decide to add users and give them ‘fulltrust’ it just deletes the host file..

So whenever you see your log filling up with:

Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance

Reason: Access to the path ‘C:\WINDOWS\system32\drivers\etc\HOSTS’ is denied.

Or:

Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance

Reason: Access to the path ‘C:\WINDOWS\system32\drivers\etc\HOSTS’ is denied.

Make sure that COMPUTERNAME\WSS_ADMIN_WPG has modify permissions on your HOSTS file. Whenever the HOSTS file is already gone, just create a new empty file with notepad and save it as HOSTS without an extension. (Make sure to add the COMPUTERNAMWSS_ADMIN_WPG with modify permissions after that again).

Today I found myself stuck with a server having the FacetedSearch v 3.0 installed on it, along with the v 2.5, and as expected it has some conflicts, and after an hour trying to fix it, I decided to reinstall everything.

A few minutes later I had a clean install of v 2.5, and it seems to works fine, except for the SearchFacets WebPart kept giving me an object reference not set to an instance of an object. After beating myself for another 30 minutes I realized that the WebPart properties where all blank. So after changing them (important one is the one about the scope, it has to be valid), it all worked fine.

So after some testing, it seems that whenever you have the .webpart files already in your library they got replaced, but without any default settings in it. And that's something that is nice to know, and can save you quite some time.

So another minor post about search. Last post I explained the use of metadata property mappings, and their strength allowing you to show more information without a lot of changes. However I forgot to mention the fact that u can actually show them pretty easy in your existing search results.

Assuming you created your Metadata properties you can go to your search page, the XSLT of your resultswebpart to:

<?xml version="1.0" encoding="UTF-8"?>

      <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

            <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>

                  <xsl:template match="/">

                        <xmp><xsl:copy-of select="*"/></xmp>

                  </xsl:template>

</xsl:stylesheet>

(More info on that on a previous post on this blog) .

Next step will be a minor change in Results Query Options, here you will have a property for Selected Columns, editing it will show you the normal columns used in your search results before the end </Columns> you can easily add a new <Column Name=”YourMetaDataProperty” />. By using the above XLST you can save the page search, and check real-time whether your column exists and contains information.

You might have to play with capitalization a bit, ‘cause sometimes the column name differs from the metadata property.

As mentioned in the last post about Search, in the following posts there will be some more advanced features. In this post there will be some details about the your search properties (and using them in  your URL). While in the next post there will be some info about the Search Result Hidden Object, that allows you to sort your complete result set (and not only by the default relevance option).

Depending on how you use your SharePoint environments you can either choose for standard search WebParts (giving you only the options described in the previous post) or create your own webparts (described in the next post). Whatever option you may choose you will have to deal with showing information in you result set. Something that is easily obtained through the use of Metadata property mappings. This is a nice feature hidden in your SSP under search settings.  It contains a list of properties on your environment,  and the possibility to make a new one (making new ones allows you to map them to the BDC or other values existing on your environments).

Each item has a nice checkbox “Use in scopes” checking it will not only allow you to use this property to exclude/include items based on the property value but also allows it to be used in your URL.

Small example:

Most of you knows that passing the ?k=Test parameter to your search page will get a result set containing the keyword test. Passing the next url to your search page will not only get items containing the keyword test, but also match the Author : ?k=test+Author:”SystemAccount”

Each managed metadata property you create can be used within your URL to filter down search results (actually it’s how Faceted Search works).

Knowing that unleashes a lot of potential to your search solutions.

A small real live example:

You have a WebPart that gets items and folders. By searching in that WebPart you get redirected too your search page with a custom scope and you have the search results you want (easy doable with a k=<textbox value>&s=<set property>.

Now we would like to search not only the scope (that contains all items), but also we would like to search within a folder , the trick here is finding the metadata property containing the url (or folder in our case). Besides the fact that it took ages to find out what property it was, it’s pretty simple. The property seems to be Basic:9, so we made a new Metadata property and mapped it to Basic:9, that leaves us with a property containing the URL and all we had to do was to expand our k=<textbox value> with a +MetadataProppertyContainingBasic:9:”<the current folder>”. That actually returns only items of the folder where in.