Sharepoint Server 2010 Search Architecture

Sharepoint Server 2010 Search Architecture

Overview

Search in Microsoft® SharePoint® Server 2010 is re-architected with new components to create greater redundancy within a single farm and to allow scalability in multiple directions. Each of the components that make up the query architecture and the crawling architecture can be scaled out separately based on the needs of an organization. More information:

Query architecture:

The query architecture includes query components, index partitions, and property databases.

About index partitions:

-         An index partition is a logical portion of the entire index. The index is the aggregation of all index partitions.

-         Index partitions are associated with query components. You deploy a query component that is associated with a particular index partition to a specific server. In this way, index partitions are spread across query servers. For example, in a farm with three index partitions and one query component per partition, each query component contains one-third of the total index.

-         Deploying query components that are associated with index partitions across different servers creates faster query architecture because the processing power of multiple query servers is used to respond to queries.

-         Index partitions can be associated with one or more query components. Multiple query components (mirrors) for a given index partition can be deployed across query servers to achieve redundancy. Typically, two query components are configured for each index partition, and these query components reside on different query servers to achieve redundancy of the index partition.

Crawl architecture

The crawl architecture includes several components that can be scaled out based on crawl volume and performance requirements:

-         Crawl component — multiple crawl components can be deployed to crawl simultaneously. Each crawl component is associated with a crawl database. Crawl components reside on application servers. Crawl components produce portions of the index (per index partition) and propagate them to the servers that are running the query components associated with the given index partition.

-         Crawl database — Manages crawl operations and stores crawl history. You can assign multiple crawl components to each crawl database for redundancy. In this case, each crawl component will crawl different content during a crawl.

-         Property database — Also considered part of the query architecture; stores properties for crawled data. The number of required property databases depends on the volume of content that is crawled and the amount of metadata that is associated with the content.

Search Flow

The Step Search:

1.         Upload document or create new item

2.         The crawl process works: When those Full crawl process starts, The Start address of the search source moved to queue. iFilter opens files and Content index created on crawl server. Then the Index moved in batches to query server and the relevant Data written to crawl.

The crawler uses protocol handlers and iFilters as follows:

a.     The crawler retrieves the start addresses of content sources and calls the protocol handler based on the URL’s prefix.

b.    The protocol handler connects to the content source and extracts system-level metadata and access control list information.

c.     The protocol handler identifies the file type of each content item based on the file name extension and calls the appropriate iFilter associated with that file type.

d.    The iFilter extracts content, removing any embedded formatting, and then retrieves content item metadata.

e.     Content is parsed by one or more language-appropriate word breakers and is added to the content index, also called the full-text index. Metadata and access control lists are added to the Search database.

Additional Reading:

-                 Manage Crawl Rules:  (http://go.microsoft.com/fwlink/?LinkID=197051&clcid=0x409)

-                 Best Practices for using crawls: (http://go.microsoft.com/fwlink/?LinkID=197052&clcid=0x409)

3.         Written to Property databases

4.         User input keyword

5.         The query flow: the WFE serving the call uses the associated search service application proxy to connect to a server running the Query and Site Settings Service also known as the Query Processor.  It uses WCF for this communication. The Query Processor will connect to the following components to gather results merges\security trims and return results back to WFE: Query Component (holds entire index or partition of an index) Property Store DB (holds metadata\properties of indexed content) Search Admin DB (holds Security Descriptors\Configuration data). Then WFE displays search results to the user.

How to Develop a Custom Timer Job in SharePoint 2010 using Visual Studio 2010

1. Open Visual Studio 2010 | New Project | Empty SharePoint Project | its name TimerJob2010

2. Deploy as Farm Solution

3. Add new Class, its name TimerJobClass.cs

4. Inherit class from Microsoft.SharePoint.Administration.SPJobDefinition class. And code as

// -----------------------------------------------------------------------

// <copyright file="TimerJobClass.cs" company="">

// TODO: Update copyright text.

// </copyright>

// -----------------------------------------------------------------------

namespace TimerJob2010

{

    using System;

    using System.Collections.Generic;

    using System.Linq;

    using System.Text;

    using Microsoft.SharePoint.Administration;

    using Microsoft.SharePoint;

    /// <summary>

    /// TODO: Update summary.

    /// </summary>

    public class TimerJobClass:SPJobDefinition

    {

        public TimerJobClass():base()

        {

        }

        public TimerJobClass(string jobName, SPService service, SPServer server, SPJobLockType targetType)

            : base(jobName, service, server, targetType)

        {

            this.Title = "List Timer Job 2010";

        }

        public TimerJobClass(string jobName, SPWebApplication webapp)

            : base(jobName, webapp, null, SPJobLockType.ContentDatabase)

        {

            this.Title = "List Timer Job 2010";

        }

        public override void Execute(Guid contentDBId)

        {

            //get a reference to the current site collection's content database

            SPWebApplication webApplication = this.Parent as SPWebApplication;

            SPContentDatabase contentDB = webApplication.ContentDatabases[contentDBId];

            //Geta reference to the "Student" list in the rootweb of the first site collection in the Content DB

            SPList listJob = contentDB.Sites[0].RootWeb.Lists["Student"];

            //Create a new list item and update the item

            SPListItem newList = listJob.Items.Add();

            newList["Title"] = "32806";

            newList["Firstname"] = "Siddartha";

            newList["Lastname"] = "Reddivari";

            newList["Gender"] = "Male";

            newList["Address"] = "Bangalore";

            newList.Update();

        }

    }

}

5: Right click on Feartures | Add Feature

Step 6: You can rename Feature as

7: Right click on Feartures | Add Event Receiver

8. Code as

using System;

using System.Runtime.InteropServices;

using System.Security.Permissions;

using Microsoft.SharePoint;

using Microsoft.SharePoint.Security;

using Microsoft.SharePoint.Administration;

namespace TimerJob2010.Features.TimerJobFeature

{

    /// <summary>

    /// This class handles events raised during feature activation, deactivation, installation, uninstallation, and upgrade.

    /// </summary>

    /// <remarks>

    /// The GUID attached to this class may be used during packaging and should not be modified.

    /// </remarks>

    [Guid("43bb3273-ef57-4bc3-bff1-52d8679f74c8")]

    public class TimerJobFeatureEventReceiver : SPFeatureReceiver

    {

        const string ListTimerJob = "ListTimerJob";

        // Uncomment the method below to handle the event raised after a feature has been activated.

        public override void FeatureActivated(SPFeatureReceiverProperties properties)

        {

            SPSite site = properties.Feature.Parent as SPSite;

            //make sure that the job is not already registered

            foreach (SPJobDefinition job in site.WebApplication.JobDefinitions)

            {

                if (job.Name == ListTimerJob)

                    job.Delete();

            }

            //Install the Job

            TimerJobClass listLoggerJob = new TimerJobClass(ListTimerJob, site.WebApplication);

            SPMinuteSchedule schedule = new SPMinuteSchedule();

            schedule.BeginSecond = 0;

            schedule.EndSecond = 59;

            schedule.Interval = 5;

            listLoggerJob.Schedule = schedule;

            listLoggerJob.Update();

        }

        // Uncomment the method below to handle the event raised before a feature is deactivated.

        public override void FeatureDeactivating(SPFeatureReceiverProperties properties)

        {

            SPSite site = properties.Feature.Parent as SPSite;

            //Delete the job

            foreach (SPJobDefinition job in site.WebApplication.JobDefinitions)

            {

               if( job.Title==ListTimerJob)

                job.Delete();

            }

        }

    }

}

Step 9: Right click Feature | View Designer and change the scope to "Site"

Step 10: Build project and Deploy it.

Step 11: Go to Central Admin | Check job status.custom timer job created, click to timer job and configure

Open your web application | Open your list and see item was created

SharePoint Interview Questions and Answers: IIS, WP, AppPool

What is IIS?

IIS (Internet Information Server) is one of the most powerful web servers from Microsoft Corporation that is used to host the Asp.Net web application. IIS has its own ASP.NET process engine to handle the ASP.NET request. So, when request comes from client to server, IIS takes the request and process it and send response back to the clients.

IIS means when request comes from client to the server a lot of operation is performed before sending response to the client. This is about how IIS process the request.

What is Worker Process?

Worker process (w3wp.exe) runs the ASP.NET application in IIS. This process is responsible to manage all the request and response that are coming from the client system. All the ASP.NET functionality runs under the scope of worker process. When request comes to the server from a client worker process is responsible to generate the request and response.

In single word, we can say worker process is the heart of ASP.NET web application runs on IIS.

What is Application Pool?

Simply to say about what is application pool is: A group of one or more URLs are served by a particular worker process or set of worker processes.

Application pool is the container of worker process. Application pools are used to separate sets of worker processes that share same configuration. Application pools enable a better security, reliability, and availability for any web application.

The worker process servers as the process boundary that separates each application pool so that when one worker process or application is having an issue or recycles, other applications or worker processes are not affected. This makes sure that a particular web application doesn’t impact other web application as they are configured into different application pool.

Application pool with multiple worker process is called “Web Garden”.

http://msdn.microsoft.com/en-us/library/gg552610.aspx

 

Why are application pools important?

They provide a way for multiple sites to run on the same server but still have their own worker processes and identity.

Main Point to Remember:

•             Isolation of Different Web Application.
•             Individual worker process for different web application.
•             More reliably web application.
•             Better Performance.

How to find the application pool account for a web application?

Go to IIS settings (type inetmgr in command mode) and select the web application on left pane and click on the “Basic Settings” on the right pane.

List of fields in a list SharePoint programmatically

Below post will help you to get the list of columns that is displayed when you view the items in the list programtically:

var fields = from SPField f in list.Fields                     where !f.Hidden && f.AuthoringInfo == "" &&                                   f.Type! = SPFieldType.Attachments &&                                   f.Title! = "SharingID" &&                                   f.Type! = SPFieldType.WorkflowStatus                     orderby f.Title                     select new { ListItem = new ListItem(f.Title, f.Id.ToString()),                                 Readonly = f.ReadOnlyField, FieldType = f.Type };

 Note: we will get the null result to the “f.AuthoringInfo” in SharePoint 2007 but we can get string empty result to the “f.AuthoringInfo” in SharePoint 2010.

 So, use: f.AuthoringInfo == null [In SP2007] & f.AuthoringInfo == ""[In SP2010]

Output: We can get the list of viewable lists by default once create a sample list in SharePoint.

SPField’s to the dropdown List programmatically

By default, we can get the number of built-in fields or internal fields if you try to retrieve the fields in a list. My below post will help to add only the custom fields to the dropdown list by avoiding all the internal, built-in and hidden fields.

using (SPSite site = new SPSite(SPContext.Current.Site.ID)) {     SPWeb web = site.OpenWeb(SPContext.Current.Web.ID);     web.AllowUnsafeUpdates = true;     SPList list = web.Lists[new Guid(listId)];     var fields = from SPField f in list.Fields                         where !f.Hidden && f.AuthoringInfo == "" &&                                       f.Type != SPFieldType.Attachments &&                                       f.Type != SPFieldType.WorkflowStatus                         orderby f.Title                         select new { ListItem = new ListItem(f.Title, f.Id.ToString()),                                                Readonly = f.ReadOnlyField, FieldType = f.Type }; ddlDestfield.DataSource = from g in fields where !g.Readonly && (g.FieldType == SPFieldType.Note || g.FieldType == SPFieldType.Text)                                               select g.ListItem; ddlDestfield.DataBind(); ddlDestfield.Items.Insert(0, new ListItem("[Select]", "none")); }

Note: Set f.AuthoringInfo == null [In SP2007] and Set f.AuthoringInfo == "" [In SP2010]

Output: