Blog2mind

Permissions, permissions, permissions

peter@bring2mind.net — Mon, 13 Feb 2012 22:37:33 GMT

This blogpost is about disk permissions and asp.net applications like DotNetNuke. Although there are probably many posts like this I write this because permissions, or more precisely the lack of them, are the root cause of many support requests. And a little knowledge is all that would have been needed to avoid the situation.

Background: the worker process and its app pool

An asp.net application is just that: an application. It is a program running on your server calculating what HTML to spew out to the requesting browser. This is quite different from serving html in the classic way. There the server just streams files from your server’s hard disk over the wire. No processing as such. Just “serving”. But asp.net applications (and their predecessor ASP and alternatives such as PHP) require the server to process instructions of a program that will tell it what to send to the client. This has given us the data driven web site and the whole Web 2.0 revolution.

To do this processing the server starts up the so-called worker process. You can actually see this process in action if you’re on the server which is serving asp.net pages and bring up the Task Manager and flip to the “Processes” tab:

They’re called “w3wp.exe” (for WWW Worker Process) and in this screenshot you can see two of them. By the way: this is a really useful place to check the health of your site. It is probably always my first stop. You can quickly see if the worker process is clipping the CPU (i.e. it’s at 99% CPU) meaning it is consuming all processing power of your server, or if it is consuming a lot of memory (usually not a reason to panic).

The worker process encapsulates the code that is running for asp.net. Because the server is starting this up autonomously based on http requests, this process is boxed into an App Pool. The app pools are kept in a list in your IIS manager:

The app pools govern how much the worker process can consume from the server like in terms of memory for instance (lest they complete consume the server and bring it down). The app pool, therefore, protects the server from potentially hazardous code in the asp.net application. The app pool also gives the worker process its identity. It is the latter we are concerned with here but we’ll come back to this in a bit. What you need to know at this point is the existence of the worker process and the app pool, the relationship between the two and where you can find them.

Windows disk permissions

This should not be a big mystery. All items on your server’s hard disk (folders, files) have a set of permissions that commonly inherit down the folder/file tree. Right click on any folder on your server and select properties and you get a popup of that folder’s properties with a “Security” tab. Go to this tab and you’ll see something like this:

The concept is simple here. You can specify users and roles with the level of access to this resource. So giving someone Read but not Write access will allow them to see and download/open documents, but not change those documents or upload new ones. This paradigm is pervasive in computing. What you may not realize is that when the worker process (i.e. the asp.net code) accesses the hard disk it needs to have a “face” for the system to authenticate. It’s “identity” needs to have the right access level to what it wants to do on disk. The way that is implemented is through the identity of the aforementioned App Pool that is responsible for taming the worker process.

App Pool Identity

The app pool a worker process belongs to specifies which identity it will assume when it goes to Windows. Check out the advanced settings in IIS Manager for an App Pool:

So this app pool runs as “NetworkService” when accessing the disk. Note that the default for this depends on the IIS flavour. Commonly you’ll see “NETWORK SERVICE”. What is important to keep in mind though is that these default accounts are local to your server. I.e. NETWORK SERVICE on server SERV-A in your network is a different account than NETWORK SERVICE on SERV-B on your network. Even though they have exactly the same name. Well, actually their full names would be “SERV-A\NETWORK SERVICE” and “SERV-B\NETWORK SERVICE” but we don’t always see it like that in the UI. Keep this in mind!

The bottom line is that your app pool should have access to the files of your ASP.NET application. And write access as well in the case of an app like DotNetNuke as it is able to install extensions (modules, skins etc) which means writing files to the application’s file system. Let’s examine that in more detail.

DotNetNuke and its Installer

DotNetNuke’s most powerful feature is its ability to install and upgrade extensions which are uploaded as zip files. DNN’s installer unzips those and, based on the information presented in an XML manifest, it begins to write the contents of that zip file to the server’s hard disk. It will write files only inside it’s own application’s directory. Never beyond that. And that’s good. You want the server to encapsulate the application and not have the risk of some rogue module writing executable code to your Windows files. The installer typically writes ascx files to a folder it will create under DesktopModules. These files are template files that tell the server how to flow the HTML. Next to these ascx files it will also typically write one or more dll files to the bin folder. This folder is the most important folder of your asp.net’s application. It is where the compiled code sits. Waiting to be invoked by the worker process as the ascxs tell it to go to them and do some fancy processing.

The installer performs another very important function which will have some bearing on the problems we will describe below. That is the running of SQL scripts. The module programmer packages the zip file with a bunch of scripts that prepare the database so that the application may do its work. These scripts are incremental. That is: they are run in order and during an upgrade DNN’s installer works out which scripts it needs to run to upgrade the module. Keep this nugget of info in mind as we examine the common failure scenarios.

What you need to know by now is that the installer writes files to disk and very importantly to the bin folder of the DotNetNuke application.

Common Failure Scenarios

The focus of this post is on errors in disk permissions. These permissions can lead to serious malfunctions in your application.

After Server Migration

Servers die. Or you need a bigger/better etc server. There comes a moment in the life of your website that you’ll need to move it. So you (x)copy over the files of the asp.net app, register the app in IIS and away you go. But this is where the problem begins. The files with FULL permissions for “SERV-A\NETWORK SERVICE” are now running on SERV-B. And the asp.net app is now running as “SERV-B\NETWORK SERVICE” although all you saw was “NETWORK SERVICE”. If you’re lucky (or not depending on how you look at it) your site will still have read access for the new identity and the site will fire up correctly. But what will now happen when we try to install, or worse upgrade, a module? Check below.

After Restored Backup

Murphy’s law has it that a site will only die when we haven’t got a backup. But just in case you do have one please read this. Your backup might not have the full permission set of the files stored with it. It all depends a bit on your backup solution. What is also common is a combination of this and the above. I.e. a server has died and we restore to a new server from our last backup (pfew). Again what you need to keep in mind is that the permission set may not be right for the worker process’ identity.

Common Failure Effects, Symptoms and Solutions

These are common results from mishaps after the above.

Module Install Failure

Keeping in mind the application is unable to write to one or more places of the application’s file system it is easy to see that the installer would run scripts but not be able to write files. Although the installer is pretty smart this is still something that can happen and I’ve had customers report in with installation problems resulting from this. The biggest problem here is that DNN’s installer will have already run the SQL scripts to install the module but the actual installation failed. Now, when the user tries to install again the installer will re-run the scripts leading to all kinds of spectacular SQL failures as the SQL objects are already there that the installer should be creating. This is messy but ultimately easily solvable. Manually rip out all objects related to the module from SQL and correct the disk permissions then install again.

The main reason this scenario is not that dangerous is that (1) modules are usually (if well programmed) encapsulated so they will not damage other bits of the system and (2) there is no user data yet as we are just installing. So you can afford to horse around with the bits that came with the module provided you don’t touch the rest of the application.

Module Upgrade Failure

This is the real nightmare scenario. And it is not uncommon. This is also the main reason that I advocate making a backup of your installation before you upgrade a module. Why is this scenario so bad? Let me explain. The installer will begin to upgrade the database objects of the module first. At this time the data may well be transformed (columns being deleted, added and data being moved to other tables for instance). In short this is typically irreversible. DNN has no concept of uninstall largely due to the complexity of this bit. The scripts run first and then the file copying begins. So now the server has to begin adding files, overwriting files and possibly deleting files. Presuming this has become impossible due to a permissions issue, we are now left with module code (in the bin folder for instance) that thinks it is version A and the database tables thinking we are now version B. The installer will bomb out of the install process with “cannot overwrite file XYZ” or “cannot access ABC” and it won’t register in DNN that the module updated. Even though the scripts have run just fine.

The effects are generally that the module will fail spectacularly with messages of the kind “Column XYZ could not be found in table ABC” etc. The error is easy to spot but harder to solve. In fact you are going to have to do one of two things. You can mimic the DNN installer and continue where it left off. But for that you need a lot of DNN knowledge. In short you’ll need to (1) copy the files from the zip to their correct destination, (2) update the versions in Packages and DesktopModules tables and (3) update the module definitions if there are any changes. The latter can be quite elaborate work. And then we don’t know if upgrade code should be run from within the module. If it does then … well, we’re stuck. But this is relatively rare, as are changes to the module definitions, so just doing 1 and 2 above might do the trick.

But really the best way out of this is to roll back to the last backup. As you followed my instructions you have a backup ready to be restored. And then you just set the permissions correctly and try the upgrade again.

Internal Module Failures

Many modules use the file system. Certainly my Document Exchange does so. So having permissions on disk can be crucial to the operation of these modules. Keep this in mind. Typically you’ll see error messages starting with “Could not find a part of the path” or “Could not access”.

Setting Disk Permissions

So how to set these permissions correctly?

Look up the identity of the worker process in the app pool settings
Open up the folder of your DNN installation and select properties and go to the “Security” tab
Click “Edit …”
Click “Add …”
Type in the name you looked up under 1 (you may need to Google on alternatives at this point if you are still not having luck)
Make sure to transfer these new permissions to all children
Verify a couple of levels down on a random file. Are permissions as they should?

Final Note

First and foremost I am a programmer and not a Windows administrator. What is in this post is the result of me having to solve installation issues over the years. I will not claim this is the definitive guide to asp.net permissions debugging. But it is a good overview of the causes of a number of issues I’ve dealt with in DNN/DMX support. And it is not uncommon unfortunately. Though the above may at times be considered common knowledge, I know it is not. And I also know it is hard to explain in a paragraph or two. Hence this post so I can point you here.

More ...

Document Exchange 6 is here

peter@bring2mind.net — Tue, 29 Nov 2011 18:36:19 GMT

Today, Document Exchange 6 (DMX6 in short) has been uploaded to this site and Snowcovered. After Months of programming, debugging, tweaking, and … a conference (DNN World in Orlando) I am very proud to announce that DMX 6 is final. Although this release is somewhat behind the internal schedule, it was not opportune to release earlier. Next to some changes in the data model and codebase there are numerous visible enhancements. Allow me to guide you round this new version.

First you’ll notice the familiar 3 panel Ajax UI has been replaced with a 2 panel version (the details panel was sacrificed). You can still get the 3 panel UI, though, by selecting it in the module’s settings. You’ll also notice the menu’s have been slightly rearranged and the Admin and Help menus have been stuck to the right. Both these changes were made among other to speed up the interface. The right-hand menus don’t need to be refreshed so they stay static. The left-hand menu is context-aware and you’ll see it refresh.

Below I highlight some of the bigger visible changes to this version of DMX.

Thumbnails

Files can now have thumbnails. If the file is an image type the thumbnail is auto-generated. This mechanism can be extended so in the future we could add auto thumbnail generation for other file types. The thumbnails by default will be stored as jpgs (with a hashed name) in a directory in the portal directory. This is a (tiny) security hole as someone who happens to know the hashed name could access this thumbnail. So we provide as option the ability to have thumbnails stored in DMX itself, so secure. But this creates an overhead, so our default (and recommended) setting is regular storage. Read more in the help file on thumbnails.

Document Administrators

Up until now portal administrators had specific rights in DMX like assigning approval permissions, hard deleting, etc. This can now be delegated to any number of roles who become “document managers”. Users in the selected roles can do most of what the portal admins could before. Find this on the main options screen under application permissions.

Subscription management

Document Administrators can manage who is subscribed to an item through a subscription management screen. This also allows them to add users to items as subscribers.

Silverlight upload

A new upload provider has been added based on Silverlight. This control allows the user to select multiple files with his/her own browser and this control takes care of the rest.

Mail documents to others

Up until now DMX included a function whereby users could email themselves a document (called MailMe). A completely new mailing engine has now been integrated into DMX allowing users to send documents to others. Because spam is such a menace to our society and we wish to avoid adding vulnerabilities through which users may spam, we have added a bunch of parameters to restrict what users can do. First the ability to mail is only available if the portal administrator allows it. Then the administrator specifies who is allowed to use the mailing engine and to whom they may email. We feel enough guarantees are in place to avoid spam but to allow you to distribute DMX content to your users through this powerful mechanism. We hope you like it.

Documentation

All DMX documentation has been revised for this release. Note most help files are found under the help menu of the module.

Other enhancements

Several other enhancements have been implemented in this release. The full list is as follows:

New Feature: Files can have thumbnails
New Feature: Ability to specify people with Document Admin rights which gives them almost all admin capabilities
New Feature: Subscription manager allows admin to examine and control subscriptions to items and hence notifications
New Feature: Language files can now be downloaded directly from Bring2mind without the need to go to the site.
New Feature: New upload control using Silverlight allows multiple file selection
New Feature: Integrated mail application allows users to mail documents to others with many options
New Feature: New default view UI now only shows 2 panels. The old 3 panel UI is optional.
New Feature: The details of a file can be seen by selecting "properties" from the context menu
New Feature: Doc Admins can recover hard deleted files from the "Graveyard"
Enhancement: Revised view template management allows template sharing across the portal
Enhancement: Revised view template syntax to allow for more powerful and complex templates (incl. Javascript)
Enhancement: Search engine revision should make it less prone to errors
Enhancement: Revision of the way content localization is implemented in the database
Enhancement: Improvements to IPortable should make it less prone to errors when used in portal templates
Enhancement: Lucene search engine update to 2.9. Supports highlighting as well.
Change: Code namespace reorganization
Change: Dependency to DNN 5.2.3

Install

Installing is simple and like any other DotNetNuke extension. Please download the revised installation manual for details.

Upgrading

If you’re running a previous version of DMX and you have a DMX license with a service period (check the license page of the module on the module’s menu) that extends until 29 November 2011 (or later) you can install this version free of charge over your old version. Please read the following carefully to make sure you don’t get stuck with a broken site.

Major releases have a potential major impact on a site, so one takes more precautions than otherwise. In this iteration the codebase has not been overhauled as much as with DMX 4 and 5, but it has been significantly reshuffled to make it more manageable toward the future. Also, the multilingual implementation has been altered which impacts most aspects of the module. This can only be done during a major upgrade as most users will be aware of the significance and take necessary precautions. These precautions are no different than any release, but this time we really emphasize you:

Make an inventory of any DMX Extensions (e.g. Shortlist) or any code that you have that relies on DMX in some way (e.g. the DMX browser for the RadEditor). If you have any you must make sure you have DMX 6 compatible versions for these extensions before you continue.
Pick a quiet moment (evening/weekend)
Backup your installation (i.e. files and database)
Install the module (and any extensions you need after that)
Refresh your browser to refresh css and js files from the site
Run a couple of sanity checks (i.e. Does it come up? Can you add and edit documents? etc.)
If you find anything odd: describe in detail anything you have seen (screenshots etc), let us know and roll back

We have run numerous upgrade tests and are confident you will not run into any issues. But as I’ve illustrated in this blogpost, there is no substitute for a live site. So please take these precautions.

Where to get it

You can download the module directly from this site here: http://www.bring2mind.nethttp://www.bring2mind.nethttp://www.bring2mind.net/Downloads.aspx?EntryId=1589. You can also buy licenses through the Bring2mind site or renew your license. Note there is a 10% discount here on subsequent licenses. Alternatively you can go to Snowcovered to buy DMX or use the new integrated extension gallery in DNN 6.

The module needs to be activated and can be activated in trial mode which lasts for 30 days. Please read the instructions in the installation manual.

Finally

We hope you like this new release. We thank our customers who have been with us for all these years and we think this version really does improve the experience of the module. Let us know what you think and feel free to like our product on Facebook or on Snowcovered. Personally I am proud to be part of the larger DotNetNuke project and where it has gotten us so far. At the conference I was able to meet with some of you and as always those are the highlights of my year. I look forward to 2012!

More ...

DotNetNuke World and DMX 6 Release Candidate

peter@bring2mind.net — Mon, 31 Oct 2011 14:22:04 GMT

Just a few days ago the snowplow poles have gone into the ground along our roads.This means two things: 1. the snow is coming, and 2. it’s conference season. Every year around this time of the year there is the main US DotNetNuke conference. This year it’s called DotNetNuke World. It’s in Orlando on November 9-11 and I will be attending. And just like 2007 and 2009 I will be presenting as well.

DMX 6 goes beta

peter@bring2mind.net — Thu, 29 Sep 2011 13:10:33 GMT

I’ve kept rather quiet lately as I’ve been working hard on this new version of Document Exchange. But just now I feel this is now at a stage where it can begin being released in the wild. DMX 6 builds on the strong features of DMX 5 and adds a bunch more. While at the same time effort has been devoted to try and slim down where possible.

(Http) Modules

peter@bring2mind.net — Fri, 12 Aug 2011 13:13:32 GMT

To our second installment on this two part series on handlers and modules in ASP.NET web applications, their significance and how to begin debugging situations where things have gone haywire. Like I mentioned in the previous post on handlers: modules and handlers go together like peas and carrots. The main difference is the following: whereas a handler runs only when ASP.NET has decided it should handle the incoming request and ASP.NET stops after it has found the handler, the module always runs on every request and ASP.NET continues. So the way that looks is that upon a web request ASP.NET first passes the request to each and every module in the list of modules and then tries to find a handler for it and passes the request on. Most crucially a module can alter a request as it passes through it or set the stage for other components before they run. For the remainder of this post I assume you will have seen/read through the previous post and you are familiar with Fiddler, web requests and responses and IIS 6 and 7 differences in the web.config.

DotNetNuke modules

Below is the modules section (IIS 7) of a default DNN site.

If you’re interested look up one of these in the source code and see what it does. Did you know that:

DNN’s Url rewrite module also initializes PortalSettings
DNN’s membership module takes care of user login, roles, authentication cookies, and user language

Basics of an http module

A module can flag to ASP.NET to be kicked into action at the beginning and/or the end of a request. ASP.NET hands control over to the module and it can look at the request, and change it (at begin request) or the response (typically at end request). A common example of the former is url rewriting. A Url rewiter changes the url at the beginning of the request. Url rewriters are used to make web urls more human friendly and get better results in search engines. Lets take the example of a request that asks for resource “/Blog/ThisWonderfullBlogpost.aspx”. By analyzing the resource string and looking things up in a database the url rewriter is now able to deduce that this is a request for page id 3 and blog post id 18. It then substitutes the found values and the request continues its journey in ASP.NET as “/Default.aspx?tabid=3&itemid=18” for instance. Another common example of logic in a module that fires at the beginning of a request is to add variables and their values to the application property bag. These can later be used by the application. An example of this is the variable PortalSettings which holds all the current DotNetNuke’s portal settings in an object. This way subsequent processing doesn’t need to recreate those all the time. Instead they are present throughout the request to both the DNN main application as well as the extensions.

An example of a module that latches on to the end of processing is a compressor. A compressor changes the response (output) of the server by running it through for instance a GZip compression component.

If you want to find out more about http modules check out the various web resources about these and/or examine the DotNetNuke source code for some great examples.

Debugging modules

Errors in modules lead to different symptoms than errors in handlers. A routing issue with handlers usually breaks stuff immediately. A module failure can hide itself better. But the approach in debugging is still similar to what was mentioned in the handlers post. We start by examining the web traffic for hidden error messages and routing issues. We also should not forget to look at the DNN event log to see if we spot exceptions that have come out of a module. It helps at this point to have a rough idea of what each module does.

Upload components

A special mention here about upload components. There are a number of upload components (Telerik, ComponentArt, NeatUpload, etc) that use Ajax callbacks to stream an upload to the server. What the component does is chunk the file that is being uploaded into small bits and upload one by one using Ajax and using the server’s responses to drive a progress bar/indicator. This has several advantages, notably user feedback and better memory management on the server. Typically these upload components use a module to get the stream of uploaded bytes. So when debugging upload issues, check out the modules in the web.config and make sure the necessary one is there and running (failed request tracing as described in the previous post can help you see running modules as well). Try removing non-essential ones to see if this helps. I have witnessed one upload component “eat” the bytes destined for the right upload component before it got to it which led to failed uploads as the upload component that had consumed the upload didn’t know what to do with it and the expecting upload component was left empty handed.

Finally

As mentioned before you need to use your knowledge of IIS (in these two posts), the application and Google to work your way through any issue with handlers and modules. It is a skill which you’ll develop and you’ll come to appreciate. Due to the nature of DotNetNuke extension programming, a developer cannot guarantee that all goes well. The configuration is not under our control and it is impossible to foresee what happens in the future. These two posts hope to help you cover some ground in this respect.

More ...

Handlers

peter@bring2mind.net — Fri, 12 Aug 2011 12:20:40 GMT

This post is the first of a two part series about handlers and modules and is an attempt to offload what I know about handlers to you. Knowing what handlers are and how they work is what separates the savvy from the not-so-savvy when it comes to IIS administration. Although very few modules in the DotNetNuke ecosystem use/rely on handlers, those that do experience a steady stream of support calls along the lines of “hey, XYZ doesn’t work” and it is down to a handler issue. Examples of modules that use handlers are Ultra Video Gallery and Document Exchange. DotNetNuke uses handlers itself as well. So what is a handler? In a nutshell a handler is piece of code that handles a web request. To understand why it’s good to know more about this you need to look closer at http traffic (i.e. the bits and bytes that go back and forth between your browser and the server).

Document Exchange 05.03.08 Released

peter@bring2mind.net — Mon, 20 Jun 2011 12:12:48 GMT

While work on DMX 6 is making good progress here's a bugfix update of DMX 5.3.

Document Exchange 05.03.07 Released

peter@bring2mind.net — Thu, 03 Mar 2011 13:58:13 GMT

This release fixes an important notifications issue in 5.3.6 that cropped up, plus a couple of minor things.

Document Exchange 05.03.06 Released

peter@bring2mind.net — Tue, 15 Feb 2011 17:09:50 GMT

It has been a while since the last release. With the release of DNN 05.06.01 a minor issue cropped up and we've harvested a couple of other fixes in the meantime.

Document Exchange 05.03.05 Released

peter@bring2mind.net — Fri, 17 Dec 2010 12:43:56 GMT

A critical bug crept into the DMX 5.3.4 codebase. This version repairs that.