Tagged with geo-spatial - Visokio Forums

GIS: Importing from Esri ArcGIS Online

richard — Mon, 07 Jul 2014 11:11:18 -0400

Omniscope's DataManager supports importing data layers from Esri ArcGIS Online. These layers can then be used in all Omniscope views or added to Map Views as overlays.

The following example assumes that you already have an ArcGIS Online account with at least one feature layer configured. You can sign up for an account by clicking on "Esri account help" in the DataManager block noted below and following the link.

INSTRUCTIONS

Create a new file in Omniscope and go to DataManager.
Under "Feeds", drag on a new 'ArcGIS feature layers' block.
Enter your ArcGIS Online username and password then click connect
Configure the search criteria then click 'Search' to find relevant Feature Services.
Choose one or more Feature Services then select the layers you would like to import into Omniscope.
Load the data into Omniscope then drag a Map View into Omniscope to see the postcode areas.

Mapping: GIS Resources for Omniscope Map View - importing shapefiles/overlays

daniel — Fri, 12 Feb 2016 10:30:49 -0500

Posting for reference and also to look at perhaps getting Omniscope to be more efficient and able to handle the sources directly. Currently most of these resources take a bit of time to load up in the Omniscope map view due to their size and granularity.

UK specific
- Ordnance Survey Open Data BoundaryLine administrative low level (high quality): https://www.ordnancesurvey.co.uk/business-and-government/products/boundary-line.html
- Edina for admin/political/nhs boundaries: https://census.edina.ac.uk/bds.html

Worldwide
- Openstreetmap: http://wiki.openstreetmap.org/wiki/Shapefiles#Download_shapefiles

ArcGIS:
http://forums.visokio.com/discussion/2637
The ArcGIS API will have some of those boundaries that can be accessed with a free public account you just have to sign up for a trial to get some login information for the software. Also every year ArcGIS provides a couple of online MOOCs where you can get some extra help regarding their products.

To get these into Omniscope you need to
1) Convert OSGB shapefiles into WGS version shapefile/kml - import/load up your shapefile into your GIS package of choice and set the current projection to OSGB, then go into the save as/export options and change the resulting output into WGS projection.
I would suggest you save as .kmls as they appear to be the leanest option.

2) Generalise/clip the shapefiles to reduce the load time (also reduces accuracy), load up the shapefile in you GIS package, find the generalisation options either inbuilt in some packages i.e. model builder with spatial analyst for ArcGIS or install plugins for GrassGIS/QGIS like generalizer/simplify. Just google how to use each package for tutorials/videos on GIS users who use them.

https://plugins.qgis.org/plugins/generalizer/
https://plugins.qgis.org/plugins/simplipy/

Other options would be to use programming/scripting like in R or Python to do the same generalisation with their own specific libraries but I find the GIS packages will be far quicker to process them.

Unfortunately there is only so much you can generalise/simplify, every iteration of this will erode the quality and accuracy of your boundaries so you have to make the call on how much you want it to simplify to and just do it by trial and error. Including a shapefile in Omniscope significantly bumps up the size of the total Omniscope iok file and affects the loading time and the amount of RAM your machine will also determine what loads.

Idea: Mapping: German postcode zones (1-dig, 2-dig) available?

dvo_D — Thu, 07 Oct 2010 14:01:41 -0400

Hi all - on the map it would be great, if the postcode zones were visible. As such a layer currently seems not be available, I'd like to add a pic in the background, showing the current map semi-transparent, jpg + map grouped for zoom.
-> any idea?
Caution: attached postzone map might not be free, it's googled...

GIS: Geometric Buffer Filtering - Selected Data on Map + related views

dpawley — Thu, 31 Jul 2014 05:37:09 -0400

Hi

Was having a play with Geometric Buffers today and these are great! The only issue is that what I really want to see is a Map Visual but also the map summarised in a table.

I have a buffer layer of 260 points across the UK with a 10km radius. This is overlayed on to another data file of 400,000 individual address with different classifications, i.e. Pubs, Shops etc. You can construct a pivot table along the Map that when you select an area the pivot table is changed to that area. However, what I want is a pivot table that is all the areas listed and what split of Pubs to shops are in their areas.

Does this make sense? I think you need to have the Buffer file as part of the dataset but when you have overlapping areas this proves difficult.

Any ideas or tips would be great.

Danny

GIS: Map View - Overlays (e.g. boundaries)

richard — Wed, 09 Jul 2014 08:33:42 -0400

Omniscope's Map View supports adding multiple data layers, known as overlays.

In this example we are going to analyse data relating to cities in the USA and add an overlay of USA state areas to improve the visualisation.

ATTACHMENTS

USA cities.zip - Contains an Esri Shapefile of USA city data, including latitude/longitude locations.
USA states.zip - Contains an Esri Shapefile of USA state areas.

INSTRUCTIONS

Download and unzip "USA cities.zip" and "USA states.zip".
Create a new file in Omniscope and go to DataManager.
First we need to load the USA cities data into Omniscope. Drag "USA_cities.shp" on to the DataManager workspace.
Expand the block and next to "Overlay identifier", select "CITY". This means that Omniscope will use the city name to uniquely identify every record.
Next to "Metadata fields included", select all fields. This will load all meta-data from the Esri Shapefile.
Execute the block and load it into Omniscope.
Now that the city data is loaded we can begin exploring it. Go to the 'DataExplorer' tab.
Change the Bar/line View to a Map View, which will display the location of each city.
The cities are now visible on the map but we can add an overlay of the USA state areas to summarise the data. Drag the "USA_states.shp" onto the Map View and select 'Open as a map overlay'.
The next dialog allows you to customise how the data will be imported. This contains the same options you saw in the DataManager block earlier. Next to "Overlay identifier", select "STATENAME" to use USA state names as unique identifiers for each area in the file. Next to "Metadata fields included", select all fields. This will load all meta-data from the Esri Shapefile. Click "OK".
Now we need to change the colour of the states. In the Map View toolbar, go to the "Overlay" menu and select the downward pointing arrow next to "USA_state.shp". This will display a panel containing advanced options for your overlay layer.
Under "Outline", change "Colour mode" to "Fixed" and then choose a new colour from the new drop-down. This will update the outline colours of all USA state areas. More information on colour modes is available at the end of these instructions.
Under "Fill", change "Colour by (overlay dataset)" to "Subregion". This uses the "Subregion" field in your overlay dataset (USA_state.shp) to generate category colours for each area.
You can also use data from your currently loaded dataset (USA_cities.shp) to colour the USA state areas. To do this use the same overlay configuration menu and under "Fill", change "Colour mode" to "By current dataset".
You'll notice there is a message 'No "Linked data" field selected for the current dataset.'. For Omniscope to link these two datasets together we need a common field with values found in both datasets. Our sample SHP files both contain state names with common values so select "State" next to the "Join on (current dataset)" control to configure the link. We now have a basic relationship configured between the USA_cities dataset and the USA_state dataset.
You might have noticed that some states, e.g. Wyoming, have disappeared from the map. By default after you have created this link between 2 datasets, Omniscope will not display any overlay areas which do not have related data in your currently loaded dataset. To disable this behaviour uncheck "Filter overlay data on join".
The overlay is now configured to display state areas coloured by the number of linked records in the cities dataset. You can colour by a field in the current dataset by selecting from the drop-down next to "Colour by (current dataset)". Select "Pop2007" to use the population statistics for 2007. You can choose a function to summarise all the population statistics by selecting from the "Function" drop-down but we will leave this as "Sum".
Your map is now displaying each USA state coloured by the total population across all cities within its area. Notice how California, Texas and New York all have same colour yet they have total populations of ~17.2m, ~10.5m and ~9.1m respectively. This is because we are using the original colour range from the "Pop2007" field which ranged from ~100k to ~8.3m. Any values above or below these limits will generate the same colour, which is misleading for our example. We can dynamically generate new colour limits by going back to the overlay "Fill" options and enabling '"Fit to data" colour range'.

ADVANCED
In the Map View overlay functionality you will come across terms like "Overlay ID", "Overlay identifier", "Data type". Below are explanations for each of these terms and how they are used by Omnsicope. If you are using KML or Shapefile sources then most of these fields will be configured automatically.

Data types

Point - A dataset containing pairs of latitude/longitude coordinates, e.g. a shop location.
Line - A dataset containing connected, and unfilled/unclosed, chains of points, e.g. a hurricane path or road.
Area - A dataset containing connected, and filled/closed, chains of points, e.g. a post code area or country border.

Colour modes

Fixed - Choose a single colour which will be applied to all data.
By current dataset - Generate colours using a field from the currently loaded Omniscope dataset.
By overlay dataset - Generate colours using a field from the embedded overlay dataset.

Fields

Overlay identifier - Field containing unique identifiers for each point, line or area in the input dataset, e.g. a field containing country names or postal area codes.
Overlay sub-path identifier - This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.
Longitude - Field containing WGS84 longitude values.
Latitude - Field containing WGS84 latitude values.

GIS: Map View - Spider plots

richard — Tue, 08 Jul 2014 10:11:34 -0400

Omniscope's Map View supports creating spider plots between points in your currently loaded dataset (AKA "current dataset") and points in an overlay dataset.

In this example we are going to create a spider plot linking the company's distribution centres to each of the customers awaiting a delivery.

ATTACHMENTS

Customer deliveries.csv - A CSV file containing the location's of customers and the name of the distribution centre where the delivery will originate.
Distribution centres.csv - A CSV file containing the location's of a company's distribution centre..

INSTRUCTIONS

Open "Customer deliveries.csv" in Omniscope.
Change the Bar/line View to a Map View, which will display the location of each customer.
Now we need to add the second CSV file as an additional layer in the Map View. Drag "Distribution centres.csv" onto the Map View and select 'Open as a map overlay'.
The next dialog allows you to customise how the data will be imported. This contains the same options you would see in a DataManager block for the same file type. Click "OK".
Now we can configure how the Map View should interpret the distribution centre data. We need to tell Omniscope what type of data the file contains, e.g. point, line or area, and also which fields contain latitude/longitude values etc. See the ADVANCED section at the end of this post for more information. For now just configure the settings to match the screenshot shown below and click "OK".
The distribution centres are now displayed alongside the curent dataset's customer locations but they all look the same, so we need to customise the view. In the Map View toolbar, go to the "Label" menu and select "Customer" to add some labels.
Now we need to change the colour of the distribution centres. In the Map View toolbar, go to the "Overlay" menu and select the downward pointing arrow next to "Distribution centres.csv". This will display a panel containing advanced options for your overlay layer.
Click on "Fixed colour" and choose another colour to update the distribution centres.
Click on "Spider plot" to begin enabling spider plots. You'll notice there is a message 'No "Linked data" field selected for the current dataset.'. In order for Omniscope to link these two datasets together we need a common field with values found in both datasets. Our sample CSV files both contain a "Distribution Centre" field with common values so select that next to the "Join on (current dataset)" control. We now have a basic spider plot connecting each distribution centre to it's respective customers.
You might have noticed that some distribution centres have disappeared from the map. By default after you have created this link between 2 datasets, Omniscope will not display any overlay points which do not have related data in your currently loaded dataset. To disable this behaviour uncheck "Filter overlay data on join".

Point - A dataset containing pairs of latitude/longitude coordinates, e.g. a shop location.
Line - A dataset containing connected, and unfilled/unclosed, chains of points, e.g. a hurricane path or road.
Area - A dataset containing connected, and filled/closed, chains of points, e.g. a post code area or country border.

Fields

Overlay identifier - Field containing unique identifiers for each point, line or area in the input dataset, e.g. a field containing country names or postal area codes.
Overlay sub-path identifier - This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.
Longitude - Field containing WGS84 longitude values.
Latitude - Field containing WGS84 latitude values.

GIS: iGeolise routing operation

richard — Tue, 08 Jul 2014 08:50:53 -0400

Omniscope's DataManager supports generating routes between pairs of points using a service provided by iGeolise. For example, this could be used to generate customer delivery routes from a distribution centre.

The following example assumes that you already have an iGeolise Travel Time account. You can sign up for an account by clicking on "iGeolise account help" in the DataManager block noted below and following the link.

ATTACHMENTS

Customer deliveries.csv - A CSV file containing the deliveries to be made from a company's distribution centre to various customers across London.

INSTRUCTIONS

Create a new file in Omniscope and go to DataManager.
Drag "Customer deliveries.csv" onto the DataManager workspace.
Create a new "iGeolise Travel Time" operation and connect it to your workflow. This block will use the point data from the CSV and generate routes from the distribution centre to each customer location.
Enter your iGeolise Travel Time application ID and key.
Choose the "Routing" service.
Change the "Transport type" to "Driving" so the service will calculate results based routes which the company's delivery vans can travel on.
Enter a maximum journey duration, e.g. 60 minutes. Any destinations which cannot be reached in that time will be filtered out.
Go to the 'Fields' tab to configure. We need to configure which fields this block should use to identify the origin and destination of each route:
- Point identifier - Field containing unique identifiers for each point in the input dataset. In our example this would be the names of the distribution centres or customers.
- Longitude - Field containing WGS84 longitude values.
- Latitude - Field containing WGS84 latitude values.
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the routes.

GIS: Geometric filter operation

richard — Mon, 07 Jul 2014 13:04:14 -0400

Omniscope's DataManager supports filtering of latitude/longitude points against geographic areas. For example, this could be used to filter all customers outside a sales catchment area.

ATTACHMENTS

"UK major cities.csv" - A CSV file containing the locations of major cities in the United Kingdom.
"English postal areas.kml" - A KML file containing the boundaries of all postal areas in England.

INSTRUCTIONS

Create a new file in Omniscope and go to DataManager.
Drag "UK major cities.csv" and "English postal areas.kml" onto the DataManager workspace.
In the "English postal areas.kml" block, next to "Overlay identifier", select "Name". This means that Omniscope will use the postal area names to uniquely identify postal area shapes in the data.
Create a new "Geometric filter" operation. This block requires first a point dataset and second an area dataset to function. So connect the "UK major cities.csv", then the "English postal areas.kml" (please note that the point data must be connected first).
Select a Filter Type:
- All points (don't filter) - All points will be returned from the block. Use this in conjunction with 'Add "Areas" field' (see next step).
- Points inside any area - Keep only the points which fall inside at least one area shape.
- Points outside all areas - Keep only the points which do not fall inside any area shape.
Check 'Add "Areas" field' to add a new field and merge the "Overlay identifier" from the Area dataset into your new Point dataset. In our example this will add the postal area codes from the "English postal areas.kml" into the new cities dataset.
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the filtered point data.

ADVANCED
The "Geometric filter" block also contains additional controls for configuring which fields contain the latitude/longitude values and the fields for identifying individual features (shapes) in the input dataset. If you are using KML or Shapefile sources then these fields will be configured automatically. Below are explanations for each option:

Overlay identifier - Field containing unique identifiers for each point or area in the input dataset, e.g. a field containing country names or postal area codes.
Overlay sub-path indentifier - This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.
Longitude - Field containing WGS84 longitude values.
Latitude - Field containing WGS84 latitude values.

GIS: Geometric buffer operation

richard — Mon, 07 Jul 2014 12:28:04 -0400

Omniscope's DataManager supports buffering of geographic shapes by applying a fixed distance to extend/contract the original geometry. For example, this could be used to generate a sales area around a shop using a Euclidean (straight line) distance in all directions.

Geometric buffers can be applied to 3 types of shape data in Omniscope;

Point - A dataset containing pairs of latitude/longitude coordinates, e.g. a shop location.
Line - A dataset containing connected, and unfilled/unclosed, chains of points, e.g. a hurricane path or road.
Area - A dataset containing connected, and filled/closed, chains of points, e.g. a post code area or country border.

Below are instructions for buffering these 3 types of shape data. Please download and extract the contents of the ".zip" file:

POINT

Create a new file in Omniscope and go to DataManager.
Drag "United Kingdom cities.csv" onto the DataManager workspace. This contains 5 major cities in the United Kingdom with latitude/longitude coordinates.
Connect a "Geometric buffer" operation to the workflow.
Choose "Point" as the "Input data type". (For KML and Shapefile sources, this will be configured for you.)
Enter a distance (km) to buffer around each input point. By reducing the "Quality" you will increase the speed of this block but the output dataset will contain less records and the buffered outline will not be as smooth.
Check 'Include metadata' if you would like the output dataset to contain all other fields found in the input, i.e. "Country name".
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the buffered points.

LINE

Create a new file in Omniscope and go to DataManager.
Drag "Hurricane Katrina.csv" onto the DataManager workspace. This contains coordinates for the path of Hurricane Katrina in 2005.
Connect a "Geometric buffer" operation to the workflow.
Choose "Line" as the "Input data type". (For KML and Shapefile sources, this will be configured for you.)
Enter a distance (km) to buffer around the line (hurricane path). By reducing the "Quality" you will increase the speed of this block but the output dataset will contain less records and the buffered outline will not be as smooth.
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the buffered line.

AREA

Create a new file in Omniscope and go to DataManager.
Drag "United Kingdom.kml" onto the DataManager workspace. This contains coordinates defining a simplified boundary of the United Kingdom..
Connect a "Geometric buffer" operation to the workflow.
Ensure "Area" is selected as the "Input data type". (For KML and Shapefile sources, this will be configured for you.)
Choose your method of buffering:
- Expand area - The original area plus an outside perimeter.
- Contract area - The original area less an inside perimeter.
- Outside perimeter - The area within a certain distance outside of the perimeter.
- Inside perimeter - The area within a certain distance inside of the perimeter.
- Outside + inside perimeter - The area within a certain distance either side of the perimeter.
Enter a distance (km) to buffer around the United Kingdom. By reducing the "Quality" you will increase the speed of this block but the output dataset will contain less records and the buffered outline will not be as smooth.
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the buffered line.

ADVANCED
The "Geometric buffer" block also contains additional controls for configuring which fields contain the latitude/longitude values and the fields for identifying individual features (shapes) in the input dataset. If you are using KML or Shapefile sources then these fields will be configured automatically. Below are explanations for each option:

Overlay identifier - Field containing unique identifiers for each point, line or area in the input dataset, e.g. a field containing country names or postal area codes.
Overlay sub-path identifier - This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.
Longitude - Field containing WGS84 longitude values.
Latitude - Field containing WGS84 latitude values.

GIS: Importing an ESRI Shapefile

richard — Mon, 07 Jul 2014 08:00:00 -0400

Omniscope's DataManager supports importing geographical data from ESRI Shapefiles. Use the attached files and following steps for a basic example;

ATTACHED FILES

"postcode areas.shp" (inside ZIP) Contains geometries for each feature (shape) in the Esri Shapefile
"postcode areas.dbf" (inside ZIP) Contains meta-data for each feature (shape) in the Esri Shapefile
"postcode areas.prj" (inside ZIP) Contains the coordinates' projection, e.g. UTM, WGS84.

INSTRUCTIONS

Download the attached ".zip" file and extract the ".shp", ".dbf" and ".prj" files into the same folder.
Create a new file in Omniscope and go to DataManager.
Drag the "postcode areas.shp" file onto the DataManager workspace (or drag a "Data file" block onto the workspace and browse to the "postcode areas.shp" file). The ".dbf" and ".prj" files will be automatically detected.
The Esri Shapefile DataManager block contains 2 configurable options
- "Overlay identifier" - The field which contains unique identifiers for each feature (shape) in the file. In our Shapefile this would be the names of the postal areas, e.g. AB for Aberdeen, SW6 for South West London. If your data does not include such a field then selecting "Create new field" will allow Omniscope to generate internal unique identifiers using a field named "Overlay ID".
- "Meta data fields included" - The names of all fields which will be loaded from the Esri Shapefile into the block.
Load the data into Omniscope then drag a Map View into Omniscope to see the postcode areas.

ADVANCED
You will notice a field has also been created named "Overlay sub-path ID". This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.

GIS: Importing a KML (Keyhole Markup Language) file

richard — Mon, 07 Jul 2014 07:49:27 -0400

Omniscope's DataManager supports importing geographical data from KML (Keyhole Markup Language) files. Use the attached file and following steps for a basic example;

ATTACHED FILE

"postcode areas.kml" (inside ZIP) Contains geometries and meta-data for each feature (shape)

INSTRUCTIONS

Download the attached ".zip" file and extract the ".kml" file.
Create a new file in Omniscope and go to DataManager.
Drag the "postcode areas.kml" file onto the DataManager workspace (or drag a "Data file" block onto the workspace and browse to the "postcode areas.kml" file).
The KML DataManager block contains 3 configurable options:
- "Overlay identifier" - The field which contains unique identifiers for each feature (shape) in the file. In our KML file this would be the names of the postal areas, e.g. AB for Aberdeen, SW6 for South West London. If your data does not include such a field then selecting "Create new field" will allow Omniscope to generate internal unique identifiers using a field named "Overlay ID".
- "Meta data fields included" - The names of all fields which will be loaded from the KML file into the block.
- "Shape types included" - KML files can contain a combination of points, lines and areas. Use this option to filter which shape types you want to load.
Load the data into Omniscope then drag a Map View into Omniscope to see the postcode areas.

Idea: Custom map markers-image sets, colours and shapes

peter_balfe — Thu, 27 Mar 2014 13:31:16 -0400

Hi,

I have been asked to produce a Map view report by our marketing team. Basically we wish to show on the map all our locations. We will also show on the same map the locations of a partner of ours. I was able to get this to show by the usual methods. But what the marketing team would like is to show a small graphic instead of the default marker. So for all our locations we show a small graphic and for all the partner we show their graphic (small logos) This was we can show the complementary locations of both our sites.

Can you let me know how I can do this?

Mapping: Importing Geospacial polygon shapes?

davedunckley — Tue, 21 Jan 2014 08:56:09 -0500

Hi,

Is this supported?

Dave

Demo: Some geospatial demo files to share

daniel — Tue, 16 Jul 2013 07:59:43 -0400

Just a few demos I've created and finally gotten around to organising and putting them up on github, they're all to do with spatial data.

You can download them from github at:
https://github.com/ucfnlam/OmniscopeDemos?source=c

Or from my blog which has webstart links when you click on the icons:
http://spatiametrics.wordpress.com/omniscope-data-visualisations/

Map View: Geometric buffer operation (2.8+)

steve — Fri, 10 May 2013 12:23:26 -0400

Omniscope 2.8 Beta introduces the Geometric Buffer operation in DataManager.

Use this to transform geographic data imported typically from KML files or ESRI Shapefiles.

It includes the ability to convert point data such as building locations into circular areas representing a specified distance from that location (stored as a series of point records with a shared ID). You might use this to find the areas representing 5km from tube/subway stations, for example.

It also allows you to convert area (polygon) or line (polyline) data such as coastlines or zip code boundaries by enlarging outwards, subtracting inwards, or creating a "channel" either/both sides of the outline, according to a specified distance. You might use this to find the area representing 20 miles from the coast, for example.

The resulting geographic "areas" can be used as an overlay in the Map view.

Please be aware that buffering can multiply the volume of data by several 100 times, since each point can introduce many points to describe the area around the point/line. Also note that while point buffering is true to the Earth's shape, line/area buffering is done in Euclidean space, so is only accurate at the local level and becomes completely inaccurate when you have areas/lines covering a large proportion of the world. Only supports WGS84 coordinates.