Posts Tagged iis

Roundup of ISAPI rewrite filters for IIS

Posted by J.D. in Ask J.D., tech tips on January 22nd, 2010

Here is a roundup of ISAPI rewrite filters for IIS, they let the webserver take a human friendly request like http://www.jdhodges.com/photos/1002 and rewrite it to a url that a dynamic scripting engine (such as ASP) can understands like http://www.jdhodges.com/photos.asp?photoID=1002 This benefits users who now have a legible URL and especially search engines like Google which are much more likely to index a page which appears to be static than a dynamic looking URL string:

Reasons your site may not be included: Your pages are dynamically generated. We are able to index dynamically generated pages. However, because our web crawler can easily overwhelm and crash sites serving dynamic content, we limit the amount of dynamic pages we index.
Google information for webmaster [google.com]

Another use for URL rewriting is to deal with “hotlinks”. Bob pointed me to a site nice site discussing hotlinks which are bascially links from other websites directly to your content, used for display on their page. I’ve had people do this in forums where they like a photo of mine and instead of linking to the particular page it’s hosted on and sending folks my way, they link directly to the image and use my bandwidth for their gain :-( Here’s an example of someone doing that to me. It’s not a huge deal and thankfully I put the jdhodges.com URL on all of my images… Following the table is a new program called coldlink that is specifically targeted at preventing damage from hotlinks. First, here is another description of hotlinks:

…the practice of building web pages that contain unauthorized content links, known as a hotlink, direct link, or remote link to files hosted by another site. Notice that we said content links and not navigation links that lead to another site. Content links are file references that the browser fetches to draw the page such as images, style sheets, scripts or even complete web pages that are rendered within a frame. In other words, these are embedded content or embedded objects within an html page .

The result of hotlinking is that the offending site is able to present it’s pages without paying for the bandwidth needed to serve up the stolen content. The victim site ends up paying the bandwidth expense for serving up the files without gaining any page views.
Source: wordworx.com

IIS Rewrite is the granddady of rewrite filters and it is the first thorough implemenation that I came across. Now ISAPI_Rewrite appears to be taking the lead but they are still very comparable products. I would like to mention that I have corresponded personally with IIS Rewrite’s creator in the past and he was very helpful and responsive. I don’t have personal experience with ISAPI_Rewrite but many other users are atarted to praise the features it brings to the table.

Below is a table that I compiled attempting to compare the available products, I tried not to come to any incorrect conclusions but for some of the less well know rewriters out there you will want to do some more of your own research. If you don run across and errors or you have a new rewriter to add to the list please email me at jd_hodges_ii AT hotmail.com Finally, if you find this information useful and end up purchasing a filter, please let them know that you found out about them here. Thanks for reading!

Single license Price Demo available? Support Virtual Servers? mod_rewrite script compatibility? regular expressions? Requires reboot for changes to take effect? Automated install? Support Forums? Thorough documentation?
IIS Rewrite $99* 200 manipulations, then must restart web server Yes Yes/Yes Yes No No Yes
ISAPI_Rewrite $69 30 day* * Yes Yes/Yes No Yes Yes Yes
OpUrl $45 14 day No? No?/Yes? Yes No Yes No
URL Replacer Free N/A No? No?Yes? Yes No No No
URL Rewrite $22**** No. No? No/No Yes? No No No
*limited time, $199 regular price.
** “Lite” version available as well for free.
***Only the “Lite” version supports automated install.
**** 22 Euros
? means I’m not sure

ColdLink

I wasn’t aware of an IIS solution to the hotlinking problem, but Bob knew of one called ColdLink which claims to “puts hotlinks on ice” hehe, now that’s a nice slogan!

ColdLink Bandwidth Protection Software gives you instant relief when hotlinks from bandwidth bandits get you hot under the collar.

The ColdLink iis isapi filter or apache module transparently maps and rewrites protected urls dynamically in realtime within every page served from your site. Your site continues to work normally, even when the pages are linked from search engines or other sites. Users can even bookmark your pages for later return because the page will contain new valid links.
Source: coldlink.com

Update: For those wanting to check if their current anti-hotlinking measures are working, here is an online anti-hotlinking testing tool. Just enter the URL of a protected image, on their easy to use web form and hit “check now” Finally, here is an example of a hotlink protected image now go to the original source page of that image to view it in its original context(you may have to refresh your browser.) Pretty nifty huh?! I’m not sure what software they’re using but it works well :-)

IIS Rewrite

What I’ve been using on my site for the past year, it has easily supported over a million rewrites without giving me an problems.

IISRewrite is a rule-based rewriting engine that allows a webmaster to manipulate URLs on the fly in IIS.

URLs are rewritten before IIS has handed over the request to be processed, so requests for HTML files, graphics, program files, and even entire directory structures can be rewritten before they are passed to ASP scripts for processing.

[. . . ]

IISRewrite is a stripped down implementation of Apache’s mod_rewrite modules for IIS. Webmasters who have used Apache’s mod_rewrite in the past will find that much of the configuration and functionality is the same.

OpUrl

ISAPI URL Rewrite

An ISAPI filter that provides powerful, integrated URL re-writing for IIS. OpUrl has many benefits, including helping to ensure search engines crawl even the dynamic parts of your site. Most search engine crawlers don’t index dynamic pages, e.g. page.asp?item=1 so OpUrl allows you to use static URLs instead. The functionality is very similar to Apache’s mod_rewrite

ISAPI Rewrite

ISAPI_Rewrite is a powerful URL manipulation engine based on regular expressions. It acts mostly like Apache’s mod_Rewrite, but is designed specifically for Microsoft’s Internet Information Server (IIS). ISAPI_Rewrite is an ISAPI filter written in pure C/C++ so it is extremely fast. ISAPI_Rewrite gives you the freedom to go beyond the standard URL schemes and develop your own scheme.

Here is what Joshua thought about ISAPI_Rewrite:

I’ve tried ISAPI_ReWrite (www.isapirewrite.com) and I like it. It seems comparable to IISReWrite, but here are some features I really like:
1. It’s free in it’s most basic form (one set of rules for every domain on the box). You have to buy it to set up rules for virtual domains, but it’s cheap.
2. The help documentation is really complete in regards to explaining the RegEx rules. I actually use the documentation as my daily RegEx reference.
3. Their support forums are decent. I got a couple answers within 24 hours.
4. * When you edit the config file, the changes are immediate * That means no having to restart anything when a rule changes. I know that IIS ReWrite requires a reboot of IIS every time you make a rule set change. The reason I like not having to restart anything is because I plan on making an online application to create rules on the fly.

Over on Web Master World forums another user also had good things to say about ISAPI_Rewrite.

ISAPI_Rewrite would be my suggestion. We’ve tested both IIS Rewrite and ISAPI. The ISAPI_Rewrite product outperforms the others due to its ease of configuration and a bunch of other little extras that come in handy. One of the biggest benefits we’ve found with ISAPI_Rewrite is that you don’t have to restart IIS each time you make a change to the .ini file

The ISAPI_Rewrite featureset sounds complete, and the documentation is thorough but not moreso than IIS_Rewrite.

URL Replacer

A new freeware rewriter that I just found out about, I don’t have any experience with it but if you’re looking for something to try on the cheap it might be your ticket.

The ISAPI filter replaces defined parts of URL from browser. It enables url to scripts (.asp, .cgi, .idc) with parameters look like static html pages or specify exact download filename generated by script.

Update 2006.March.28

LinkFreeze Description from the Helicon Tech website:

LinkFreeze is a fast and easy solution to optimize dynamic or database-driven website for search engines. It optimizes all dynamic links on your site removing all unwanted characters and making links look like static files. […] the main benefit of LinkFreeze is that you don’t need to change anything in your application to make it search engines friendly. LinkFreeze will modify all links directly before the server will send response to the client. So everybody – users, search engine crawlers or other robots will see the same nice statically looking links. And when somebody will navigate to this link LinkFreeze will intercept this request on the server and rewrite URL back to the dynamic form, so your applications will still thinking it is working with old dynamic links.

HotLink Blocker Description from the Helicon Tech website:

Hotlink Blocker is an IIS plug-in that will protect your web site from bad people stealing your traffic by directly linking to the content on your server. If you have an image collection, video or document archives, Flash games or any other downloadable content HotlinkBlocker is a right product for you. After installing HotlinkBlocker your site will continue working as before, all search engines and backward links will still be valid, all user’s favorites will remain working. But if someone tries to embed your stuff directly on their site, they will only see an error message or your site’s logo. You can save tons of traffic and thus the money. You can even turn these hotlinks into real visitors by redirecting hotlink request to your site’s homepage.

Updated 2003.Dec.05 with anti-hotlinking testing tool

, , , , ,

1 Comment

301 Redirects on IIS A Tale of Two URLs

Posted by J.D. in Ask J.D., tech tips on November 22nd, 2003

301 Redirects on IIS A Tale of Two URLs

Most websites, including mine, have the same content for their domainname as well as for www+their domain name. For instance at my site in the past if you went to jdhodges.com you’d get the same thing as www.jdhodges.com. The problem with this is that Google and other search entries treat www.yourdomain.com and yourdomain.com as two completely different sites, this can result in your pagerank being split between multiple sites and duplicate content showing up on the search engines.

Here is how to make sure that www.yourdomain.com is the only address people and search engines will ever get even if they try to access yourdomain.com without the www.

Do a 301 redirect via the IIS management console and “A redirection to a URL”

For this exampledomain to perform the redirect we do the following

  1. Open Internet Services Manager and create a web site for exampledomain.com
    1. Make the content directory for the site exampledomain.com the same directory, i.e. c:\inetpub\wwwroot, that you use for your www domain i.e. www.exampledomain.com
  2. Test loading a page from http://exampledomain.com this should work because we haven’t setup the redirect yet.
  3. Open exampledomain.com site in Internet Services manager.
  4. Go to the properties of the exampledomain.com site
  5. Go to the Home Directory tab:
  7. Change the option button “When connecting to this resource the content should come from” to "A redirection to a URL"
  8. Enter your full URL http://www.exampledomain.com
  9. Check the checkbox that says “A permanent redirection for this resource.”
  10. Your screen should look something like this:

301 Redirect via ASP code

If you don’t have access to the IIS management console or you want to do it on a specific ASP page, here is the ASP code to insert:

<%
Response.Status = "301 Moved Permanently"
Response.addheader "Location", "http://www.yourdomain.com/"
Response.End
%>

You would put that code in the default.asp document (or whatever your asp index file is) that is in the root directory of yourdomain.com server, when people type that domain in their browser it will load your default.asp file and run the redirect code which will take them to www.yourdomain.com

Comments:

 

1. Stewart commented on 11/23/2003 2:18:25 PM:

This is interesting. I thought Google and the others would treat the URLs the same (assuming you did an A record for the short, and a CNAME pointed at A for “www.”). I looked around and can’t find any info on this — do you have one?

I tried “link:vardaman.org” and “link:www.vardaman.org” at Google and got identical results. But it appears that evey site linking to me uses “www.”

I’ve noticed a lot of SE’s are really sloppy on 301 redirs. I moved from .ASP to .PHP well over a year ago, and used mod_rewrite to 301 them, but some like AllTheWeb still show .ASP.

Stewart Vardaman

2. J.D. commented on 11/23/2003 7:20:08 PM:

With totally different domains pointing to the duplicate content, Google will penalize after a point. So you are right Stewart that Google must treat specially domains with www and non-www pointing to the same thing, otherwise half off the sites out there would be getting penalized for duplicate content.
If you look at the two URLs of a page like my friend Ronnie at www.ronniewilliams.net and ronniewilliams.net then the PageRanks (viewed via the Google toolbar) will be different. Another example, before my 301 redirect, jdhodges.com used to have a PR1 and www.jdhodges.com PR of 5.
You came up with a great solution Stewart, in using CNAMEs but no one really knows what google does with the issue, and I don’t think they do CNAMEs because otherwise I would expect the same PR from both www and non-www.

3. Stewart commented on 11/24/2003 1:11:29 AM:

Thanks for the info! I personally haven’t looked at my PR (never installed the Google Toolbar), but it’s something I should be paying attention to.

I noticed this a bit later: I happened to recall that many links I see to Reason magazine have WWW left off. So I tried “link:” queries at Google:

link:reason.com = 3,260 results
link:www.reason.com = 4,450 results

There’s some duplication between the two, but WOW.

Your 301 redirect solution is a good practice is well worth adopting.

Also on this subject, NPR has the stupidest method:

http://npr.org/

They use Javascript to redirect to WWW. Some 14% of internet users have JS disabled:

http://www.thecounter.com/stats/2003/May/javas.php

And robots like googlebot obviously are going to choke on JS.

Thanks again for the idea!

4. Ronnie commented on 11/24/2003 3:27:26 PM:

I never noticed I had 2 page ranks.. is there a solution for those who do not host their website on servers they control to fix this?

5. J.D. commented on 11/24/2003 3:31:02 PM:

You could actually point your DNS cname records for ronniewilliams.net to someone else’s server, and let them host the 301 redirect back to your www.ronniewilliams.net page. Let me know if you want me to do that Ronnie.

6. Stewart commented on 11/27/2003 2:48:50 PM:

FYI if you’re running Apache you can do this with mod_rewrite (in the main config if you control the server, or .htaccess if you have shared hosting). Mine looks like this:

RewriteEngine On
RewriteCond %{HTTP_HOST} ^vardaman.org$
RewriteRule ^(.+) http://www.vardaman.org/$1 [L,R=301]

This takes care of subdirs too (http://site.com/stuff/here/there.html)

7. mike commented on 4/10/2004 11:31:06 PM:

I understand different URLs with the same domain can have different pageranks, because they are allowed to be different sites. Sometimes by coincidence ranks are the same: compare www.house.info and plain
house.info ). I have some questions I hope you can answer. Is it necessary to have a
unique IP number dedicatedd to your domain and site in order to have good pagerank?
How can high PR domains can be used to increase rank of another site, without developing a site? Redirection doesn’t help. Does CName help? Last question: If
CName doesn’t resolve to intended domain and instead resolves to a different domain hosted on the same server, what can one conculde in terms of PR?

8. mike commented on 4/10/2004 11:31:07 PM:

Above I meant compare http://www.house.info and http:// house.info, sorry for confusion.

9. MonIca KEndal commented on 4/10/2004 11:31:08 PM:

I meant h.i.

10. J.D. commented on 4/13/2004 11:31:06 PM:

Hi Mike, good questions and observations Smiley

You’re right about pages (URLs) having the same PageRank seemingly “by coincidence” but you should be able to determine fairly precisely WHY each URL has a certain PR.

The most useful way I know to analyze why a URL has a high or low PR is to look at the backlinks of the URL (backlinks are the pages that point “back” to the URL you’re viewing.)

Different search engines will have different sets of backlinks, and the methods for determing baclinks varies. I wrote the Googlizer to check backlinks or search engine saturation (saturation indicates how many pages from a particular domain are indexed in a search engine, i.e. how “saturated” the search engine is with that domain.)

If two URLs have the same PR, then each URL should theoretically have a similar number or quality of inbound links (backlinks)

An example of this, say widget1.com has TEN PR5 backlinks and www.widget1.com has ONE PR6 backlink. Both widget1.com and www.widget1.com may both end up having a PR4 or PR5 (the point is that inbound PR quality as well as quantity plays a role in determining PR.) If this wasn’t clear, just post and we’ll try to do some better examples Smiley

I don’t think CNAME will help. The easiest way to transfer PR is via a text link. The link should be FROM the high PR page and TO the low PR page.

Can you give me a specific example on the last question you had (about cname not resolving…)

11. mike commented on 4/14/2004 11:31:06 PM:

Thanks for answers. Maybe I should ask last one in a different way. Two domains and their sites are hosted on one server:
site A, and site B. Site B is mine. Webhoster told me that site B doesn’t have unique IP resolving to site B. If I Cname another domain, say C, to B; it is redirected to A. I am wondering if I (site B) am losing any rank because of this situation.

, , ,

No Comments