Archive for the ‘IT Pro’ Category

Extracting Zip Files using PowerShell

Tuesday, July 17th, 2012

To help troubleshoot some SharePoint issues, I had a need to analyze some log files that were contained in multiple zip files; one log file per zip file. Since there were several hundred zip files to extract, I figured PowerShell could help! There are several posts I found with example scripts for how to perform this operation. I took pieces from many posts, added some COM object clean-up code, and wrapped this in a function. I hope it helps you in your scripting activities!

The code below is the function I’m using to extract a single zip file. It leverages the Windows Shell COM object (i.e. Windows Explorer) to extract the files by exploiting the fact that Windows Explorer treats zip files as folders. Therefore, the operation can leverage existing file copy methods between two folders.

Latest Code Not Working Quite Right?

Tuesday, July 3rd, 2012

I’m going to start this post with an example. See if you have ever found yourself in this situation when working on developing updates for previously deployed code; most often in a test and/or development environment:

1. I start out with an existing feature receiver that uses a web-scoped feature. The feature is already deployed to my development environment. To make this example very simple to illustrate the issue – this feature receiver will set the web’s title to an arbitrary value. The title will be reset back to its original state when the feature is deactivated.

2. Activating the feature on my test site reveals the following site title change.

3. Now I’m going to illustrate a change to my previously deployed code. Let’s say for example that I wanted the title to be changed to “*** NEW WEB TITLE ***” instead of “EXISTING WEB TITLE”. I’ll make the change on Line 16 of the code above and create a new WSP for deployment.

4. I then open a new PowerShell window and execute the following six PowerShell commands using the same window. These commands will deploy the latest code and reactivate the feature on my test site so I can see the latest change.

5. I will then visit the site to see the change…

What happened? The title didn’t change?! What is wrong?

The problem occurred during Step 4. When the first command deactivated the feature, that cmdlet called the SharePoint DLLs which in turn called the SMayes.SharePoint assembly. This loaded the assembly in the PowerShell.exe process. Once the assembly is loaded, it will not be unloaded.

That’s the issue here: Once a DLL is loaded in a .NET process, it will not and cannot be unloaded!

So what? This means that when I executed the Enable-SPFeature cmdlet, PowerShell was still referring to my old code that was loaded earlier and therefore was not using my updated code! Opening a new PowerShell instance and executing the activation in that instance will cause the latest code to be used since the new process will be getting the updated DLL from the GAC.

In summary: Whenever you are deploying code updates, the proper way to execute the redeployment would include closing the current PowerShell process and opening a new PowerShell process as soon as new assemblies are deployed. This will ensure that further commands, such as feature activations, will not use outdated code.

Note that while the issue I described above pertains specifically to a feature receiver, it would also equally pertain to an event receiver, timer job, or other piece of code within a deployed assembly called by the SharePoint API. This issue is also a general issue across the board whenever a process (PowerShell, W3WP, OWSTimer, etc.) is using a custom-built assembly. This is why, when deploying new assemblies, W3WP is typically recycled. Also ensure that OWSTimer is recycled if there are any timer jobs being deployed.

Happy coding!

Adding a PowerShell Snap-in

Tuesday, May 29th, 2012

A typical scenario when developing PowerShell scripts for SharePoint is to need the use of the SharePoint PowerShell snap-in. This snap-in provides all of the great SharePoint cmdlets that have quickly become critical to any SharePoint developer or administrator’s toolkit.

Some scripts may assume that the script will be executed within the SharePoint 2010 Management Shell – which is simply a standard PowerShell window with some extras: including the SharePoint PowerShell snap-in. However, what if an unsuspecting or unexperienced administrator executes the SharePoint script in a standard PowerShell instance?

To cover this circumstance, most script developers will choose to add the snap-in at the beginning of the script using the following code.

While this seems sufficient, there is an issue if the snap-in has already been added. This would easily occur if the script was run within the SharePoint 2010 Management Shell. PowerShell reports the following error pertaining to the fact that the snap-in has already been added:

Add-PSSnapin : Cannot add Windows PowerShell snap-in Microsoft.SharePoint.PowerShell because it is already added. Verify the name of the snap-in and try again.
At line:1 char:13
+ Add-PSSnapin < <<< Microsoft.SharePoint.PowerShell -PassThru + CategoryInfo : InvalidArgument: (Microsoft.SharePoint.PowerShell:String) [Add-PSSnapin], PSArgumentException + FullyQualifiedErrorId : AddPSSnapInRead,Microsoft.PowerShell.Commands.AddPSSnapinCommand

A typical solution to this issue I’ve seen is to instruct PowerShell to ignore the error using the following code.

That bothers me. I should be looking to suppress only the error where the snap-in is already added. What if I get another unrelated error? Therefore, instead of instructing PowerShell to silently continue, I would prefer the following code below. This code first checks for the snap-in before adding it and also allows for unrelated errors to still be reported.

It should be noted that this code could be used with any snap-in; not just SharePoint’s PowerShell snap-in. Here is a function one could use for any snap-in.

SharePoint Diagnostic Studio Logon Failure

Monday, February 6th, 2012

I was introduced to the SharePoint Diagnostic Studio 2010 at the Microsoft SharePoint Conference 2011 in Anaheim, CA. I was very excited about its capabilities and have just gotten around to testing it in the last couple of days.

After configuring PowerShell remoting between my laptop and my development server, I set out to create a new project in the SharePoint Diagnostic Studio and see what it could do. Upon creating a new project, I entered my credentials. After a few moments, I received the following message.

It appears the new project installation was not successful. Click OK to see the error message

After clicking OK, the error log is opened. If it does not open for you, my log was at %HOMEPATH%\Documents\SharePoint Diagnostic Studio\Server Extensions\Error.log In the error log, I saw the following message.

Logon failure: unknown user name or bad password.

After getting quite paranoid about being up early in the morning with my two-month old and perhaps not remembering any of my passwords, I started to investigate things on the server to see if I could figure out what was going on. What I found has to do with my configuration of CredSSP authentication.

With CredSSP authentication, there are two preferred/secure ways of submitting credentials: Kerberos authentication and Certificates. These required extra configuration of PowerShell remoting that I haven’t quite gotten to just yet. Therefore, my remoting capability was relying on the third and less secure way of submitting credentials: NTLM.

It turns out, after investigating my server’s security log, the SharePoint Diagnostic Studio (or perhaps the underlying PowerShell commands) are attempting to first authenticate using Kerberos. That fails and I would expect that since I didn’t do all of the necessary configuration for Kerberos to work. However, it then falls back to NTLM. To my surprise it does not use the credentials I had specified. Rather, it was using my local laptop credentials!

To trick any application in to sending other credentials when it wants to send the credentials of the currently logged in user, you can use the Runas command which allows one to execute any program under another user. For this instance, SharePoint Diagnostic Studio shouldn’t run under the remote account. It should run under the local account, but use the remote account for any remote access. Thankfully, Runas has just such a provision – its /netonly switch! The following command will open SharePoint Diagnostic Studio under the local account while using the remote account to access the remote server.

runas /netonly /user:DOMAIN\USERNAME "C:\Program Files\Microsoft\SharePoint 2010 Administration Toolkit\SPDIAG.EXE"
(The Runas command will ask for a password before launching SharePoint Diagnostic Studio.)

Have fun!

Access Denied when Enabling PowerShell Remoting

Thursday, February 2nd, 2012

Quick tip: Are you receiving Access Denied messages when you are attempting to run the Enable-PSRemoting cmdlet? I’ve received them on machines connected to a domain, even when I’m running in an elevated PowerShell window. An easy trick is to open a PowerShell window as the built-in administration account on the machine. Not sure why, but for whatever reason, it seems to work!

VirtualBox Unidentified Network

Thursday, February 2nd, 2012

Thanks to Oisin Grehan and his Nivot Ink blog for providing the foundation of this post!
VMWare VMNET Adapters Triggering Public Profile for Windows Firewall

I use Oracle’s VirtualBox to run x64 SharePoint virtual machines from my laptop. I’ve also noticed an Unidentified Network in my Windows 7 list of networks. That is caused by VirtualBox’s Host-Only Network Adapter. It wasn’t harming anything at the time so I left it alone.

However, I later attempt to enable PowerShell remoting on my host laptop for work with SharePoint scripting. Upon doing so, I was greeted with the following error message while attempting the Enable-PSRemoting cmdlet:

Set-WSManQuickConfig : WinRM firewall exception will not work since one of the network connection types on this machine is set to Public. Change the network connection type to either Domain or Private and try again.

Another helpful error message! That seems easy enough; Windows makes it very easy to modify the settings for each individual network adapter to Private, Work, or Public depending on your personal preference. However, this is not the case with an unidentified network. With an unidentified network, Windows sticks to its Public settings and will not change it.

So can I now not enable PowerShell remoting since I can’t remove the Public designation of VirtualBox’s unidentified network? No! VirtualBox’s Host-Only network isn’t really a true network connection at all. It is an endpoint adapter. Kudos to Oisin Grehan for developing a nice PowerShell script that will solve the issue by telling Windows, via the registry, that the network adapter is an endpoint device and not a true external network connection. This will cause Windows to stop treating the VirtualBox Host-Only adapter as a network and thus remove the unidentified network (and its public designation) from my list of networks. Problem solved! I’ve modified Oisin’s script to account for VirtualBox’s Host-Only instead of VMware adapters.

Note: This script will need to be executed every time VirtualBox is updated because the update will replace the existing adapter and cause the settings in the registry to be lost.

Managing Customizations to ASP.NET & SharePoint Browser Definitions

Wednesday, November 2nd, 2011

This article’s purpose is to discuss the best practices around managing customizations to ASP.NET’s Browser Definitions. For more details around ASP.NET’s browser definition platform, please see the Browser Definition File Schema (browsers element) article on MSDN. A quick overview of ASP.NET Browser Definitions can be found in the side panel.

ASP.NET Browser Definition Overivew
In any IIS site, including SharePoint sites, Microsoft provides a highly configurable platform for defining the various capabilities and mobile adapters of a browser. To facilitate this, Microsoft uses what is called a Browser Definition File which is an XML-based file that defines browsers, what makes that browsers different from the rest (the Identification), and browser properties – capabilities and mobile adapters. Microsoft provides several definition files out-of-the-box with ASP.NET that define various browsers. Then, individual web applications can provide their own Browser Definition Files that will supplement or override the out-of-the-box files. ASP.NET uses this information to tailor page rendering based on the browser making the page request.

ASP.NET allows for multiple files that all have the same schema. There are two sets of files that ASP.NET parses:

  1. Predefined Browser Definition Files – This set of files is specific to the version of the .NET Framework being used in the web application and contains the out-of-the-box ASP.NET browser definition files.
  2. Application-Level Browser Definition Files – This set of files is specific to each web application and contains the web application specific browser definition files.
    App_Browsers directory within the web application’s web root folder (e.g. inetpub or inetpub\wss\VirtualDirectories\80)

Each individual browser is defined using a <browser> tag. Within this tag, there are several other allowed tags that define how a browser is uniquely identified and what unique attributes should be defined for that browser. Browser definitions follow an inheritance hierarchy. New definitions must define a parent definition or existing definitions can be modified using the definition’s ID.

ASP.NET loads each browser definition file by concatenating them – each browser file contains a series of browser tags. The predefined files are processed first based on the ASP.NET version being used followed by the application-level files. Within each file location, ASP.NET seemingly processes each file in alphabetical order by file name. This means that if there are multiple definitions for the same browser, it seems that the definition in the last file will take precedence.

SharePoint chose to implement two browser definition files that are deployed locally to each SharePoint web application’s App_Browsers directory. This is important as SharePoint deploys its custom browser definitions in the same way that other applications using SharePoint or ASP.NET should deploy them – as separate .browser files deployed to the application-level App_Browsers directory. These two files, which may look familiar, are:

  • compat.browser, and
  • compat.moss.browser (If using SharePoint Server editions)

As the Browser Definition File Schema article in MSDN stresses, the predefined files that ship with the .NET Framework should never be modified – ever! This is because they could be overwritten by updates, patches, or service packs; causing any customizations to be lost.

As an extension to this rule, never modify the application-level browser definition files that ship with SharePoint. Why? The same reason as with the predefined files. Future SharePoint updates, patches, or service packs could overwrite these files; causing any customizations to be lost.

So what is left if customizations are required? Thanks to the architecture of these browser definition files, Microsoft allows an application to define its own browser definition files. This is relatively straightforward, but it does require knowledge of the browser definition files you will be overriding. I will cover three scenarios:

  1. Adding an entirely new browser definition
  2. Appending information to an existing browser definition
  3. Modifying an existing browser definition

As far as I know, there is no way to remove a browser definition. However, I’m not sure there would be a need or desire to do so.

Adding an entirely new browser definition
To add a new browser definition, add the definition’s browser element to a new browser definition file. Deploy this file locally into each web application’s App_Browsers folder. This new file’s definition(s) will be combined with the other predefined and application-level definitions.

Appending information to an existing browser definition
To append capabilities, mobile adapters, or other information to an existing browser definition, add a new browser tag to a new browser definition file that uses refID to reference the existing browser’s ID. Add the new information within this browser tag. This definition will be combined with the existing browser’s definition, which could be in another file, and append the new information.

Modifying an existing browser definition
To modify a capability, mobile adapter, or other information already defined as part of an existing browser definition, there are a few options. The first would be to use a similar process to appending information. It is possible to define browser definitions in a new file, using refID to reference the existing browser’s ID, and define the same information with different values. The second would be to implement a new browser, with no additional identification, with the parent browser set to the browser that should be modified.

For example, if the IE browser should be modified to be treated as a mobile device, then the following browser definitions could be used:

  • Option 1:
    <browser refID="ie">
            <capability name="isMobileDevice" value="true" />
  • Option 2:
    <browser id="ieMobile" parentID="ie">
            <capability name="isMobileDevice" value="true" />

However, note that if multiple browser definitions exist that reference the same browser ID and each has a different definition of the same information, then the last definition in the last file alphabetically in the application-level will be used.

For example, assume Option 1 was implemented in both MyApp1.browser & MyApp2.browser and that each of those browser files was deployed to the application-level App_Browsers folder. MyApp1 defines IsMobileDevice=True and MyApp2 defines IsMobileDevice=False. Since MyApp2 comes last alphabetically, it will take precedence and IsMobileDevice will be False. This underscores the need for applications to consider the environment’s existing customizations when developing custom browser definitions. In this example, the MyApp2 team, presumably deploying after MyApp1, should have seen MyApp1’s existing customization and implemented a new child browser (Option 2) to avoid a conflict or change MyApp1’s customization. The MyApp2 team would need to be aware that either way that is chosen for implementation will affect the entire web application, so they should consult the MyApp1 team!

To properly manage customizations to the browser definitions that ship with ASP.NET and SharePoint, applications being deployed to a SharePoint environment should make use of their own browser definition files that are deployed to the application-level App_Browsers folder. However, if the environment is planning on host multiple sets of customizations or applications that will require changes to the browser definitions, I would encourage a single file for each web application to ensure that applications changes are developed properly and do not conflict with each other.

Never modify any browser definition file not directly managed by the application being deployed. To implement the changes to existing browsers within the application browser definition files, it is best to use the refID attribute of the browser tag.

When deploying, changes to the App_Browsers folder should automatically be picked up by IIS and not require any sort of application pool recycle or IISReset.

Expanding Warm Up Script Triggers

Wednesday, September 7th, 2011

This post is a follow up to a post I wrote a few months ago around triggering a warm up script for SharePoint to execute only when a specific application pool is recycled. You may want to read that post before this one: “Application Pool”-Specific Warm Up Scripts

After using what I recommended in the previous post, I realized there were still issues with this approach. To step back and explain the larger picture, I had a scenario where I needed to execute a warm up script whenever the application pool was freshly started. The script is responsible for filling application-level cache. Therefore, I want to execute the script any time the application pool will be freshly started. This means not just when it is recycled, but also in instances like server reboots. I have previously described how to do this using the Windows Task Scheduler to trigger the script to execute based on certain events being logged.

Side Note: I did brainstorm ideas to trigger execution when the application pool started, instead of triggering when the application pool recycled, etc. Regrettably, I was unable to find any Windows events that are logged when the application pool starts. Therefore, that paradigm could not be leveraged. However, one could feasibly create a custom extension to the OOTB SharePoint HttpApplication class (SPHttpApplication) and add logic to one of the class’ methods or events to trigger the warm up script.

This method would be a bit expensive in terms of the need for extra custom development, impacts to the entire web application due to the need to customize the IIS web application’s HttpApplication class, extra deployment steps, interoperability issues with other global.asax customizations, etc. Evaluate this carefully before proceeding with the suggestion above.

As it would be too difficult to trigger when the application pool is started (see the Side Note), I needed to continue triggering the event based on what would occur immediately prior to the application pool being started. I found three things that need to be used:

  1. Server Start/Reboot
  2. IIS Reset
  3. Application Pool Recycle

Server Start/Reboot
Whenever the server is started, one may wish to trigger the warm up script. If the server will take requests or the script will perform a function desired soon after the server is started, then this trigger can be used. Fortunately, the Windows Task Scheduler allows one to add a trigger that will begin the task At startup. Therefore, one can easily add this trigger!

IIS Reset
Whenever IIS is reset, then all of the application pools will be shut down. If requests will come shortly after resetting IIS, it would make sense to use this as a trigger for the warm up script. After analyzing all of the events that the IISreset utility could log, the only one that applies to the situation of needing to track the application pool’s pending start would be when IIS is freshly started: ID 3201. All of the other events described above pertain to states where the warm up script would not need to be executed.

To add event 3201 as a trigger to the task scheduler, you can use the following settings:

  • Begin the task: On an event
  • Log: System
  • Source: IIS-IISReset
  • Event ID: 3201

Note that, per the TechNet links below, Microsoft no longer supports using the IISreset utility in IIS 7.0 & 7.5.

IISreset Event Reference for Windows Server 2008 / IIS 7.0

IISreset Event Reference for Windows Server 2008 R2 / IIS 7.5

Application Pool Recycle
This is the key event that need to be tracked. The application pool’s recycle settings in IIS are quite extensive! Each reason for recycling produces a different event. This is the major area I realized that I was lacking from the previous post.

IIS logs an event when the application pool recycles

  • (5074) automatically on a regular time interval,
  • (5075) automatically when a defined number of requests have been fulfilled,
  • (5076) automatically at specific times,
  • (5077) automatically when a defined amount of virtual memory is used or exceeded by the worker process,
  • (5078) automatically when an ISAPI extension reports an unhealthy condition,
  • (5079) manually by an administrator,
  • (5080) automatically upon making configuration changes,
  • (5081) automatically due to detected problems with the IIS configuration store,
  • (5117) automatically when a defined amount of private memory is used or exceeded by the worker process, or,
  • (5186) automatically when inactive.

Parameters for automatic recycles can be configured in the IIS Manager. After considering each event, if the warm up script needs to perform any function that should always be in place while the application pool is running, then each of these events should have a trigger set against it.

The good news is that we can use the notion from the previous post to configure the trigger. (“Application Pool”-Specific Warm Up Scripts) One simply needs to include all of the extra IDs in the event filter.

Example Event Filter for an example application pool SharePoint – 32767
Don’t forget to replace SharePoint – 32767 with your application pool’s name!
  <Query Id="0" Path="System">
    <Select Path="System">
      and (EventID=5074 or EventID=5075 or EventID=5076 or EventID=5077 or EventID=5078
           or EventID=5079 or EventID=5080 or EventID=5081 or EventID=5117 or EventID=5186)]
      and EventData[Data[@Name="AppPoolID"]="SharePoint - 32767"]]

Important Note: There are also settings in IIS to control if IIS generates recycle event log entries for each of the events listed above! For any warm up script triggers to function properly, ensure that all of the scnearios are set to generate events!

Application Pool Recycling Event Reference for Windows Server 2008 / IIS 7.0

Application Pool Recycling Event Reference for Windows Server 2008 R2 / IIS 7.5

In closing, don’t forget to include any necessary triggers to your own particular warm up script’s scheduled task in Windows so that it executes whenever desired! My suggestions are to include the server start/reboot, the IIS Reset event (3201), and the application pool recycle events (5074-5081, 5117, & 5186).

Thanks to Greg Rosati for providing the links to the event information on TechNet that led to the creation of this follow up post!

Discard Check Outs by the System Account

Wednesday, August 3rd, 2011

SharePoint has a wonderful check-in/check-out system for any library with minor versioning enabled. Users with the Override Check Out permission on a particular library have the powerful ability to discard or check in a document, page, etc. that is checked out to another user. This can typically be easily achieved via any of the library’s views using the out-of-the-box user interface.

Any user with the proper permissions can check in or discard the check out using the item's edit menu

If the System Account is the account that has the item checked out, users are no longer available to check in the item or discard the check out from the library’s views, even with the Override Check Out permission!

The options to check in or discard the check out are unavailable in the item's edit menu

However, there is an area where this can be done: Site Content and Structure. One can navigate there by selecting Manage Content and Structure under the Site Actions menu. If you find the same item in that area, you will be able to check in the item or discard the check out.

The missing options are available in the Site Content and Structure views

Moving an Event Source to a Different Windows Event Log

Wednesday, August 3rd, 2011

It is typically best practice when developing .NET applications, including SharePoint customizations, to create an event source for Windows Event Logging while installing the application. Each event source on a Windows computer is tied to a specific log upon registration. I recently provided guidance on how to move an event source to use its own brand new event log. The following lines of PowerShell can do this quickly. Unless your .NET application has the event log hardcoded into itself, which it shouldn’t because the event source should be registered to a log during installation, then the move shouldn’t require any code changes.

I found that I had to reboot the machine after executing the above lines of PowerShell for this change to fully take effect.

Update – I also have had the need to update the event log properties. To do so, use the Limit-EventLog cmdlet. The following code limits the MyNewOrExistingWindowsEventLog event log to a size of 20 MB (20*1024*1024 or 20,971,520 bytes) and tells Windows to overwrite old entries with new entries as needed.