UrlScan on cloud

1-click AWS Deployment 1-click Azure Deployment

Overview

UrlScan 3.1 is a security tool that restricts the types of HTTP requests that IIS will process. By blocking specific HTTP requests, the UrlScan 3.1 security tool helps to prevent potentially harmful requests from reaching applications on the server. UrlScan 3.1 is an update to UrlScan 2.5 supports IIS 5.1, IIS 6.0 and IIS 7.0 on Windows Vista and Windows Server 2008.

One of the best ways to keep potentially malicious Internet traffic from attacking your Internet Information Services (IIS) Web server is to keep it from getting to the Web server service. To help protect users from malicious webpages, Microsoft and other browser vendors have developed filters that keep track of sites that host malware and phishing attacks and display prominent warnings when users try to navigate to them. One tool Microsoft created a few years back to help protect users from malicious webpages

URLScan is a security tool that restricts the types of HTTP requests that IIS will process. URLScan scans incoming URL requests and associated data. It uses a series of rules to determine whether the information in each request is potentially dangerous, or contains information not normally expected. To help you diagnose any potential problems and any attempts to upset your server, URLScan can also log requests—including the offending request data. By blocking specific HTTP requests, the URLScan security tool helps to prevent potentially harmful requests from reaching applications on the server.

Using this tool allows much greater control over what requests an IIS Web server responds to and helps reduce the systems susceptibility to certain types of known attacks and methods used by viruses, worms, and hackers. While URLScan technologies (such as built in Request Filtering Module) are built in to IIS 7 or newer versions of IIS, it is still a valuable tool for systems that are running IIS 6.0 and below. For reference, below is a list of the operating systems and their default IIS version:

Operating System	Internet Information Server (IIS) Version
Windows XP	IIS 5.1
Windows Server 2003	IIS 6.0
Windows Vista	IIS 7.0
Windows Server 2008	IIS 7.0
Windows Server 2008 R2	IIS 7.5
Windows 8	IIS 8.0
Windows Server 2012	IIS 8.0

The filters in URLScan are based upon rules that the administrator configures. Administrators may configure URLScan to reject HTTP requests based on the following criteria:

The HTTP request method or verb
The file name extension of the requested resource
Suspicious URL encoding
Presence of non-ASCII characters in the URL

Presence of specified character sequences in the URL
Presence of specified headers in the request

Because URLScan works as a filter before the information is passed on to the script or application that handles the request, it can act as a buffer, so you don’t have to modify your existing code. Therefore, if a request is identified as being a potential risk, the script can immediately return an HTTP 404 message to the client, without the information ever reaching the script. This help to protect the script, your Web site and your server.

If you are using older Microsoft technologies such as IIS 6.0 on Windows XP or Windows Server 2003 then I encourage you to run URLScan to help protect against attackers trying to compromise your web server. Please note that that Migrate to Windows 7 or Windows 8 ASAP., For more information on URLScan, please check out these helpful resources:

REJECTED-BY-URLSCAN

URLScan v3.1 is a security tool that restricts the types of HTTP requests that Internet Information Services (IIS) will process. By blocking specific HTTP requests, URLScan helps prevent potentially harmful requests from being processed by web applications on the server.

If you receive a HTTP Error 404.0 – Not Found with a physical path containing Rejected-By-UrlScan you will need to request from us to disable this filter for your domain

URLScan.io: the best way to scan any website

Intel gathering and data reconnaissance are one of the first steps in any security investigation. There are a lot to use in your daily infosec tasks — and we will keep sharing our best tools with you.

, Focusing on analyzing all possible details about any established HTTP connection, site content, relations with other sites and much more.

Once you input your URL and hit the “Scan” button (public and privates scans are free!) it will launch a lot of automated tests against all the elements, services and connections during the page load.

Analyzed elements include programming technologies like HTML, CSS, Javascript or images. It also records cookies, DOM content, server IPs, linking domains, among many other interesting details.

It also includes other interesting security-oriented features like detecting malicious content phishing, as well as hacked pages.

All the details you need :

URLScan.io does all the scanning and analyzing in the background using Google Chrome in a perfect headless environment.

All the details you can imagine are recorded by the tool, and once finished you can start reviewing the information displayed on the results page.

Stay in the loop with the best infosec news, tips and tools

Features you will love

Let’s see what else URLScan.io has to offer:

HTTP Request data analysis: allowing you to discover how many requests were made, and displaying how many requests were made using the secure HTTPS protocol vs the traditional HTTP.

Inside the HTTP tab you will be able to discover total number of HTTP transactions, and the details for each one of them ordered by Method Protocol, Server Response Status, Resource Path, Size, Time to complete, Mime-Type and IP address and location.

Domain interaction: it also shows how many domains the website is interacting with, including social networks and third-party javascript/images/CSS resources.

Website technologies: detect which website technologies are used by the website, perfect to detect CMS like WordPress, Joomla, Drupal, as well as for popular programming frameworks and CDN networks.

Subdomain detection: detect how many subdomains are being used to serve the information for different sources.

SSL Certificate detection: nowadays having an SSL certificate is very important, and this function will help you to find which SSL certificates are being used, as well as SSL provider, among other details.

IP address detection: is also a useful detail that will help to know where the site is hosted (exact geographic location where the information was served), and what is the ISP behind the IP address.

Link structure: where do this site link to? Find the most important internal and external links for the analyzed page.

Malicious page detection: while it is not an antivirus/antimalware tool, it does integrate pretty well with Google Safe Browsing to alert users from possible malware, cryptojacking, and phishing attacks.

Related Scans: this is a very interesting feature, that allows you to find similar scans by various attributes. For example, for alexa.com, it was able to find out 91 hits for the same domain name, and 119285 hits for the same ASN, as you see below:

IP information: urlscan.io has also the ability to scan all the IP addresses involved in the actual HTTP transaction, showing you the information easily. In our test against alexa.com revealed 17 different IPs in 2 countries, showing up ASN assigned number and PTR records as well:

Screenshot of actual page: get the latest screenshot from the page at the time of the analysis, see how it looks like right now.

Further lookup: Directly jump to other useful resources that make your life easier:

SecurityTrails (Domains & IPs)
VirusTotal
AbuseIPDB

Cymon

The SecurityTrails DNS, Domain and will allow you to unveil IP address information — find related domains in no matter of time:

This will help you to reveal even more information about certain domains, detecting popular subdomains, neighbours domain names and IP addresses used by the main website.

urlscan.io has become a great tool for security professionals and beginners who are looking for effective ways to investigate website details, as well as to inspect for possible malicious content.

Now it’s your turn, in the same way as URLScan.io integrated their security platform with our powerful intelligent toolkit, now you can do the same with your apps… Sign up for today, or explore any website using our manual scans available at our web page.

UrlScan 3.1 is a security tool that restricts the types of HTTP requests that IIS will process. By blocking specific HTTP requests, the UrlScan 3.1 security tool helps to prevent potentially harmful requests from reaching applications on the server. UrlScan screens all incoming requests to the server by filtering the requests based on rules that are set by the administrator. Filtering requests helps secure the server by ensuring that only valid requests are process

Installation

Run UrlScan v3. 1 MSI installer for either x86 or x64 version depending on your platform. On successful installation you should have a “UrlScan” folder with UrlScan. dll under %windir%\system32\inetsrv and additionally under %windir%\syswow64\inetsrv as well for x64 installations

Run UrlScan v3.1 MSI installer for either x86 or x64 version depending on your platform.
On successful installation you should have a “UrlScan” folder with UrlScan.dll under %windir%\system32\inetsrv and additionally under %windir%\syswow64\inetsrv as well for x64 installations.

Folder above will also contain the configuration file, UrlScan.ini. In the x64 case, only the system32 directory will have the UrlScan.ini file since the filter will turn off redirection when attempting to access UrlScan.ini, so both versions of the filter will access the same configuration under system32 directory
Upgrading from UrlScan v2.5 or UrlScan v3.0 will leave your old UrlScan.ini file intact.
The default installation will install UrlScan as a global filter only. Please refer to the Setting Up UrlScan v3.1 section for details.
UrlScan is required to be the highest priority filter for it to function properly. The MSI installer will do so for the global filter case, but if you are installing UrlScan as a site filter you will need to set UrlScan as the highest order filter.

Setting up UrlScan v3.1

UrlScan v3.1 can be set up as either a global filter or site level filter. A global filter is invoked for every HTTP request to the IIS server on which it is set up. A site level filter is invoked only for requests to a particular site on the IIS server. UrlScan v3.1 supports using the filter as both a global and site filter in conjunction, with the goal of having global rules in the global filter and application specific rules in the site filter.

In every case, the filter will read the UrlScan.ini configuration file from the same location that it loads UrlScan.dll. If you install UrlScan.dll filter from c:\foo and your IIS worker process loads the filter from this location, the configuration that will be applied to that instance of UrlScan is loaded from c:\foo\UrlScan.ini.

Global Filter

The default installation of UrlScan v3.1 installs the filter as a global filter. See the Site Filter section on how to setup your filter as a site filter

If you are upgrading from UrlScan v2.5 or from UrlScan v3.0 your old UrlScan.ini file will be persisted and all your old settings will apply. If you want to check out the new defaults for the UrlScan.ini file, download the new UrlScan.ini file

The default logging directory is the “logs” directory underneath the “UrlScan” directory where your global filter is installed. Change this to point to wherever you want your log files to be located. Make sure that IIS worker processesn for some common configurations.

Site Filter :

Upon installation of UrlScan v3.1 as a site filter requests for that particular site will run through both the site and the global filter if the global filter is not un-mapped. There would be two instances of the filter running in this case.

If you have a site called “Site1” setup on the server under c:\site1 folder and you want to use UrlScan v3.1 as a site filter for this site, here is how you would set it up.

Copy UrlScan.dll and UrlScan.ini from the default installation location to c:\site
Modify c:\site1\UrlScan.ini with all the options that you want for your site filter. The default UrlScan.ini file that you can has the options listed with comments on what they do.
Register c:\site1\UrlScan.dll as a site filter. Run “inetmgr” and drill down to “Site1” in your left hand pane. For IIS 5.1 and 6.0, right-click on “Site1” and select “Properties” then go to the “ISAPI Filters” tab and add a new filter and point the executable to c:\site1\UrlScan.dll. For IIS 7.0 and above, you should see an ISAPI Filters icon under the IIS category and double-clicking this will bring up an “Add…” action which you can then point the executable to c:\site1\UrlScan.dll
Using the up arrow key, move the UrlScan filter to the top to make it the highest priority filter. This step is essential for the filter to function correctly.

UrlScan 1.0 was the first version of UrlScan that Microsoft released as an ISAPI filter sample that helped reduce the attack surface for IIS versions 4.0 and 5.0. UrlScan 1.0 allows Web server administrations to define the list of HTTP verbs, headers, file name extensions, and character sequences that are allowed or denied on their servers. UrlScan 1.0 consists of the UrlScan ISAPI filter named UrlScan.dll and a configuration file named UrlScan.ini.

Installing UrlScan 1.0

UrlScan 1.0 required manual installation for IIS. The following steps will lead you through the steps to install UrlScan as a global ISAPI filter.

Copy UrlScan.dll and UrlScan.ini into a local directory on the server. While it is not required, the suggested directory is:
%SystemRoot%\System32\Inetsrv\UrlScan

Open the Internet Information Services (IIS)

Open the Windows Control Panel.
Double-click Administrative Tools.

Double-click Internet Information Services.

Open the global-level properties for your version of IIS:

For IIS 4.0 or IIS 5.0:

Right-click the server name and select Properties.
Ensure that WWW Service is displayed in the Master Properties drop down list, and click Edit.

For IIS 5.1 or IIS 6.0:

Right-click the Web Sites node and select Properties.

Choose the ISAPI Filters tab.

Click Add.

When the ISAPI filter properties dialog is displayed, enter the following information:

Enter “UrlScan” in the Filter Name box.
Enter the full path to the UrlScan.dll file in the Executable box.

Click OK to close each dialog that is open.

Restart the web service.

Using UrlScan 1.0

You configure UrlScan’s operation by setting options in the UrlScan.ini file. This file should reside in the same directory as UrlScan.dll, and it contains the sections and options that are listed below.

Note

For performance reasons, UrlScan only reads the UrlScan.ini file when IIS first loads the ISAPI filter. If you make changes to the UrlScan.ini file, you will need to stop and start the World-Wide-Web Publishing Service before your changes will be effective.

Warning: The default options built into UrlScan.dll will result in a configuration that will reject all requests to the server, therefore it is necessary to provide a UrlScan.ini file for IIS to serve HTTP requests when you are using UrlScan. A sample UrlScan.ini file is provided that contains several recommended settings to defend against known attacks on IIS servers.

UrlScan.ini Sections

[Options] Section

The [Options] section of a UrlScan.ini file contains a list of name/value pairs that define the general behavior for UrlScan. A few of the settings enable or disable other sections in the UrlScan.ini file.

Enabling or Disabling other UrlScan.ini Sections

TABLE 1

UseAllowVerbs

Allowed values are 0 or 1. The default value for UseAllowVerbs is 1. If set to 1, UrlScan will read the [AllowVerbs] section of the UrlScan.ini file and reject any request containing an HTTP verb that is not explicitly listed. If set to 0, UrlScan will read the [DenyVerbs] section of the UrlScan.ini file and reject any request containing an HTTP verb listed. Note: The [AllowVerbs] section is case sensitive, but the [DenyVerbs] section is not case sensitive.

UseAllowExtensions

Allowed values are 0 or 1. The default value for UseAllowExtensions is 1. If set to 1, UrlScan will read the [AllowExtensions] section of the UrlScan.ini file and reject any request where the file name extension associated with the URL is not explicitly listed. If set to 0, UrlScan will read the [DenyExtensions] section of the UrlScan.ini file and reject any request where the file name extension associated with the request is listed. Note: The [AllowExtensions] and [DenyExtensions] sections are both case insensitive.

URL Scanning Options

TABLE 2

NormalizeUrlBeforeScan	Allowed values are 0 or 1. The default value for NormalizeUrlBeforeScan is 1. If set to 1, UrlScan will do all of its analysis on the request URLs after IIS decodes and normalizes them. If set to 0, UrlScan will do all of its analysis on the raw URLs as sent by the client. Note: Only advanced administrators who are very knowledgeable about URL parsing should set this option to 0, as doing so will likely expose the IIS server to canonicalization attacks that bypass proper analysis of the URL extensions.
VerifyNormalization	Allowed values are 0 or 1. The Default value for VerifyNormalization is 1. If set to 1, UrlScan verifies normalization of the URL. This action will defend against canonicalization attacks, where a URL contains a double encoded string in the URL. For example, the string “%252e” is a double encoded dot ‘.’ character because “%25” decodes to a ‘%’ character, the first pass decoding of “%252e” results in “%2e”, which can be decoded a second time into a single dot ‘.’ string. If set to 0, UrlScan will not verify normalization of the URL. Note: This option is dependent on the NormalizeUrlBeforeScan option.
AllowHighBitCharacters	Allowed values are 0 or 1. The default value for AllowHighBitCharacters is 1. If set to 1, then UrlScan will allow any byte to exist in the URL. If set to 0, then UrlScan will reject any request where the URL contains a character outside of the ASCII character set. Note: This feature can defend against UNICODE or UTF-8 based attacks, but will also reject legitimate requests on IIS servers that use a non-ASCII code page.
AllowDotInPath	Allowed values are 0 or 1. The default value for AllowDotInPath is 1. If set to 1, UrlScan will allow requests that contain multiple instances of the dot (.) character in the URL. If set to 0, UrlScan will reject requests that contain multiple instances of the dot (.) character in the URL. Note: Because UrlScan operates at a level where IIS has not yet parsed the URL, it is not possible to determine in all cases whether a dot character denotes the extension or whether it is a part of the directory path or filename of the URL. For the purposes of extension analysis, UrlScan will always assume that an extension is the part of the URL beginning after the last dot in the string and ending at the first question mark or slash character after the dot or the end of the string. Setting AllowDotInPath to 0 defends against the case where an attacker uses path info to hide the true extension of the request; for example, “/path/TruePart.asp/FalsePart.htm”. Setting AllowDotInPath to 0 also causes UrlScan to deny any request that contains a dot in a directory or file name.

Logging Options

TABLE 3

EnableLogging

Allowed values are 0 or 1. The default value for EnableLogging is 1. If set to 1, UrlScan will log its actions in a file called UrlScan.log that will be created in the same directory that contains UrlScan.dll. If set to 0, UrlScan will not log any activity.

PerProcessLogging

Allowed values are 0 or 1. The default value of PerProcessLogging is 0. If set to 1, UrlScan will append the process ID of the IIS process that is hosting UrlScan.dll to the log file name; for example, UrlScan.1234.log. This feature is helpful for IIS versions that can host filters in more than 1 process concurrently, such as IIS 6.0 and later. If set to 0, UrlScan will log all activity in UrlScan.log.

HTTP Server Header Manipulation

TABLE 4

RemoveServerHeader

Allowed values are 0 or 1. The default value for RemoveServerHeader is 1. If set to 1, UrlScan will remove the server header on all responses, and the value of AlternateServerName will be ignored. If set to 0, IIS will return the default server header on all responses. Note: This feature is only available if UrlScan is installed on IIS 4.0 or later.

AlternateServerName

Allowed value is a string. The default value for AlternateServerName is an empty string. If you specify a value for AlternateServerName and RemoveServerHeader is set to 0, then IIS will replace its default “Server:” header in all responses with the AlternateServerName value. If RemoveServerHeader is set to 1, the value of AlternateServerName will be ignored. Note: This feature is only available if UrlScan is installed on IIS 4.0 or later.

Example [Options] Section

The following example [Options] section configures several recommended settings for UrlScan:

ConsoleCopy

[Options]UseAllowVerbs=1 ; Use the [AllowVerbs] section.UseAllowExtensions=0 ; Use the [DenyExtensions] section.NormalizeUrlBeforeScan=1 ; Normalize a URL before processing.VerifyNormalization=1 ; Normalize a URL twice and reject request if a change occurs.AllowHighBitCharacters=0 ; Deny high bit characters in URL.AllowDotInPath=0 ; Deny dots that are not file name extensions.RemoveServerHeader=0 ; Do not remove the “Server” header from response.EnableLogging=1 ; Log UrlScan activity.PerProcessLogging=0 ; Do not create log files for each process.

[AllowVerbs] Section

The [AllowVerbs] section contains a list of HTTP verbs or methods. If UseAllowVerbs is set to 1 in the [Options] section, UrlScan will reject any request that contains an HTTP verb that is not explicitly listed in this section. The entries in this section are case sensitive.

Example [AllowVerbs] Section

The following example [AllowVerbs] section configures UrlScan to allow basic HTTP functionality:

ConsoleCopy

[AllowVerbs]HEAD ; Allow HTTP feature discovery.GET ; Allow most HTTP requests.POST ; Allow posting data to applications.OPTIONS ; Allow HTTP feature discovery.

To use this example, you would need to set UseAllowVerbs to 1 in the [Options] section.

[DenyVerbs] Section

The [DenyVerbs] section contains a list of HTTP verbs or methods. If UseAllowVerbs is set to 0 in the [Options] section, UrlScan will reject any request that contains a verb that is listed in this section. The entries in this section are not case sensitive.

Example [DenyVerbs] Section

The following example [DenyVerbs] section configures UrlScan to deny several of the HTTP methods that are not required for basic HTTP functionality, such as WebDAV methods:

ConsoleCopy

[DenyVerbs]TRACE ; Deny HTTP tracing.PUT ; Deny uploading files.DELETE ; Deny deleting files.MKCOL ; Deny creating folders/collections.COPY ; Deny copying files.MOVE ; Deny moving files.LOCK ; Deny locking resources.UNLOCK ; Deny unlocking resources.PROPFIND ; Deny property queries.PROPPATCH ; Deny setting properties.SEARCH ; Deny protocol-based searches.

To use this example, you would need to set UseAllowVerbs to 0 in the [Options] section.

[DenyHeaders] Section

The [DenyHeaders] section contains a list of request headers in the form “header-name:”. Any request containing a request header listed in this section will be rejected. The entries in this section are not case sensitive.

Example [DenyHeaders] Section

The following example [DenyHeaders] section configures UrlScan to deny several HTTP headers that are used with WebDAV requests:

ConsoleCopy

[DenyHeaders]Translate: ; Allow HTTP feature discovery.If: ; Allow most HTTP requests.Lock-Token: ; Allow posting data to applications.

[AllowExtensions] Section

The [AllowExtensions] section contains a list of file name extensions in the form of “.ext“. If UseAllowExtensions=1 is set in the [Options] section, then any request containing a URL with an extension not explicitly listed here is rejected. The entries in this section are not case sensitive.

Example [AllowExtensions] Section

The following example [AllowExtensions] section configures UrlScan to allow several static content types:

ConsoleCopy

[AllowExtensions].htm ; Allow HTML files..html ; Allow HTML files..txt ; Allow text files..jpg ; Allow JPEG graphics..jpeg ; Allow JPEG graphics..gif ; Allow GIF graphics.

To use this example, you would need to set UseAllowExtensions to 1 in the [Options] section.

[DenyExtensions] Section

The [DenyExtensions] section contains a list of file name extensions in the form of “.ext“. If UseAllowExtensions=0 is set in the [Options] section, then any request containing a URL with an extension listed here is rejected. The entries in this section are not case sensitive.

Example [DenyExtensions] Section

The following example [DenyExtensions] section configures UrlScan to allow several static content types:

ConsoleCopy

[DenyExtensions].asp ; Deny ASP requests..asa ; Deny ASA requests..inc ; Deny include files..cdx ; Deny certificate requests..cer ; Deny certificate requests..config ; Deny configuration files..exe ; Deny executable files..bat ; Deny batch files..cmd ; Deny batch files..com ; Deny executable files..htw ; Deny Index Server hit highlighting..ida ; Deny HTTP-based Index Server administration..idq ; Deny Index Server queries..htr ; Deny legacy IIS password changing requests..idc ; Deny legacy database access requests..shtm ; Deny Server Side Includes..shtml ; Deny Server Side Includes..stm ; Deny Server Side Includes..printer ; Deny Internet-based printing..ini ; Deny configuration files..log ; Deny log files..pol ; Deny policy files..dat ; Deny configuration files..mdb ; Deny Microsoft Access databases..ldb ; Deny Microsoft Access lock files..mdf ; Deny Microsoft SQL databases..ldf ; Deny Microsoft SQL log files.

To use this example, you would need to set UseAllowExtensions to 0 in the [Options] section.

[DenyUrlSequences] Section

The [DenyUrlSequences] section contains a list of character sequences that UrlScan will deny if they appear in a URL. For example, two dots “..” indicate a parent path, which a hacker might try to use to gain access to files outside of a Web site’s content area.

Example [DenyUrlSequences] Section

The following example [DenyUrlSequences] section configures UrlScan to deny several URL sequences that could be used to attack your server:

ConsoleCopy

[DenyUrlSequences].. ; Deny directory traversals../ ; Deny trailing dot on a directory name.\ ; Deny backslashes in URL.: ; Deny access to alternate streams.% ; Deny escaping after normalization.& ; Deny running multiple CGI processes on a single request.

Features

Major Features of UrlScan:

New installer allows UrlScan 3.1 to be installed on IIS 5.1, IIS 6.0, and IIS 7.0
Create “deny” rules independently to the query string, all headers, or a particular header.
A global DenyQueryString section in configuration lets you add deny rules for query strings with the option of checking the un-escaped version of the query string.
A global AlwaysAllowedUrls section in configuration lets you specify safe URLs that will bypassall URL based checks.
A global AlwaysAllowedQueryStrings section in configuration lets you specify safe query strings that will bypass all query string checks.
Escape sequences (e.g., %0A%0D) can be used in deny rules so it is possible to deny CRLF and other sequences involving non-printable characters.
Multiple UrlScan instances can be installed as site filters, each with its own configuration and rules (UrlScan.ini).
Configuration (UrlScan.ini) change notifications are propagated to IIS worker processes.
Enhanced W3C formatted logging gives descriptive configuration errors in the Remarks header.

Installation Instructions For Windows

A) Click the Windows “Start” button and select “All Programs” and then point to UrlScan

B) RDP Connection: To connect to the operating system,

1) Connect to virtual machine using following RDP credentials :

Hostname: PublicDNS / IP of machine
Port : 3389

Username: To connect to the operating system, use RDP and the username is Administrator.
Password : Please Click here to know how to get password .

C) Other Information:

1.Default installation path: will be on your root folder “C:\Windows\system32\inetsrv\urlscan”
2.Default ports: