Mapping Web Sites with Omniscope

Enter a web URL and Omniscope will crawl and map it, importing keywords and content

New in 2.5... this page in progress

In Omniscope 2.5 and beyond you have the ability to generate a website map and view the site and associated data on content and structure in Omniscope.

To generate a website map you need to provide Omniscope with one or more 'seed' URL's. Omniscope will 'crawl' and then download the contents of each of these URLs and extract all the links. The content of each of the linked pages is then downloaded and the content extraction process is repeated. The resulting data set contains the details of all of the pages downloaded and the connections that exist between these pages. This can be visualised in Omniscope using the Network View to show a 'map' of all the pages, with each node in the map representing a page and each line representing a link between from one page to another.

A website map can prove useful in many different scenarios:

• You can use the website map functionality to get a quick understanding of your site structure. For example, you can quickly determine which pages haven't been modified in the last few months or determine the number of 'clicks' a user has to use to reach a particular page.
• You can download and save a website map and use it as an 'Offline browser'for a particular website; allowing you to view the content for eachpage even when you're not connected to the internet.
• By selecting the 'Include Google Analytics data' option you can import Google Analytics information for each page inside the website map. This can be used to provide additional data such as the number of clicks on each page and the bounce rate.
• Omniscope automatically extracts the top keywords in each page. You can use this to get an understanding of the themes in a particular page without having to read the content or you can filter the data by a particular set of keywords, allowing you to retrieve pages of interest.

To generate a web site map select File > Open online source > Website map. 

URL(s) - Enter one or more URL's to use as a starting point for the website map. Link depth (same host) - determines the maximum depth ofthe website map, i.e. the maximum number of pages out from a startingURL. Generating a website map for a very large site can take a longtime. You can use this setting in combination with the 'Max pages'setting to reduce the number of pages processed and speed up theoverall map generation time. Include external pages - most web sites include links to pages on other sites.  When ticked, this option includes pages from other sites referenced by links. Enforce tree structure -  when checked, Omniscope willensure the resultant map reflects a proper tree structure. If selected, and during the map generation process Omniscopeencounters a page that has already been processed, it is not added to thelist of links. The diagram below shows the difference between anenforced and a non-enforced website map.

Import Google Analytics data - when checkedOmniscope will attempt to retrieve Google Analytics information for anyof the pages encountered that belong to one of the sites associatedwith a Google Analytics account to which you have access. If you select this option then youwill have to enter a Google Analytics username and password. Moreinformation on the Google Analytics functionality within Omniscope canbe found here.
Start date - only available if the 'Import Google Analytics data' option is checked. Select the start date for the Google Analytics data.
End date - only available if the 'Import Google Analytics data' option is checked. Select the end date for the Google Analytics data.

Enforce tree structure selected		Enforce tree structure unselected