Tagged with shapefiles - Visokio Forums

Mapping: GIS Resources for Omniscope Map View - importing shapefiles/overlays

daniel — Fri, 12 Feb 2016 10:30:49 -0500

Posting for reference and also to look at perhaps getting Omniscope to be more efficient and able to handle the sources directly. Currently most of these resources take a bit of time to load up in the Omniscope map view due to their size and granularity.

UK specific
- Ordnance Survey Open Data BoundaryLine administrative low level (high quality): https://www.ordnancesurvey.co.uk/business-and-government/products/boundary-line.html
- Edina for admin/political/nhs boundaries: https://census.edina.ac.uk/bds.html

Worldwide
- Openstreetmap: http://wiki.openstreetmap.org/wiki/Shapefiles#Download_shapefiles

ArcGIS:
http://forums.visokio.com/discussion/2637
The ArcGIS API will have some of those boundaries that can be accessed with a free public account you just have to sign up for a trial to get some login information for the software. Also every year ArcGIS provides a couple of online MOOCs where you can get some extra help regarding their products.

To get these into Omniscope you need to
1) Convert OSGB shapefiles into WGS version shapefile/kml - import/load up your shapefile into your GIS package of choice and set the current projection to OSGB, then go into the save as/export options and change the resulting output into WGS projection.
I would suggest you save as .kmls as they appear to be the leanest option.

2) Generalise/clip the shapefiles to reduce the load time (also reduces accuracy), load up the shapefile in you GIS package, find the generalisation options either inbuilt in some packages i.e. model builder with spatial analyst for ArcGIS or install plugins for GrassGIS/QGIS like generalizer/simplify. Just google how to use each package for tutorials/videos on GIS users who use them.

https://plugins.qgis.org/plugins/generalizer/
https://plugins.qgis.org/plugins/simplipy/

Other options would be to use programming/scripting like in R or Python to do the same generalisation with their own specific libraries but I find the GIS packages will be far quicker to process them.

Unfortunately there is only so much you can generalise/simplify, every iteration of this will erode the quality and accuracy of your boundaries so you have to make the call on how much you want it to simplify to and just do it by trial and error. Including a shapefile in Omniscope significantly bumps up the size of the total Omniscope iok file and affects the loading time and the amount of RAM your machine will also determine what loads.

GIS: Geometric buffer operation

richard — Mon, 07 Jul 2014 12:28:04 -0400

Omniscope's DataManager supports buffering of geographic shapes by applying a fixed distance to extend/contract the original geometry. For example, this could be used to generate a sales area around a shop using a Euclidean (straight line) distance in all directions.

Geometric buffers can be applied to 3 types of shape data in Omniscope;

Point - A dataset containing pairs of latitude/longitude coordinates, e.g. a shop location.
Line - A dataset containing connected, and unfilled/unclosed, chains of points, e.g. a hurricane path or road.
Area - A dataset containing connected, and filled/closed, chains of points, e.g. a post code area or country border.

Below are instructions for buffering these 3 types of shape data. Please download and extract the contents of the ".zip" file:

POINT

Create a new file in Omniscope and go to DataManager.
Drag "United Kingdom cities.csv" onto the DataManager workspace. This contains 5 major cities in the United Kingdom with latitude/longitude coordinates.
Connect a "Geometric buffer" operation to the workflow.
Choose "Point" as the "Input data type". (For KML and Shapefile sources, this will be configured for you.)
Enter a distance (km) to buffer around each input point. By reducing the "Quality" you will increase the speed of this block but the output dataset will contain less records and the buffered outline will not be as smooth.
Check 'Include metadata' if you would like the output dataset to contain all other fields found in the input, i.e. "Country name".
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the buffered points.

LINE

Create a new file in Omniscope and go to DataManager.
Drag "Hurricane Katrina.csv" onto the DataManager workspace. This contains coordinates for the path of Hurricane Katrina in 2005.
Connect a "Geometric buffer" operation to the workflow.
Choose "Line" as the "Input data type". (For KML and Shapefile sources, this will be configured for you.)
Enter a distance (km) to buffer around the line (hurricane path). By reducing the "Quality" you will increase the speed of this block but the output dataset will contain less records and the buffered outline will not be as smooth.
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the buffered line.

AREA

Create a new file in Omniscope and go to DataManager.
Drag "United Kingdom.kml" onto the DataManager workspace. This contains coordinates defining a simplified boundary of the United Kingdom..
Connect a "Geometric buffer" operation to the workflow.
Ensure "Area" is selected as the "Input data type". (For KML and Shapefile sources, this will be configured for you.)
Choose your method of buffering:
- Expand area - The original area plus an outside perimeter.
- Contract area - The original area less an inside perimeter.
- Outside perimeter - The area within a certain distance outside of the perimeter.
- Inside perimeter - The area within a certain distance inside of the perimeter.
- Outside + inside perimeter - The area within a certain distance either side of the perimeter.
Enter a distance (km) to buffer around the United Kingdom. By reducing the "Quality" you will increase the speed of this block but the output dataset will contain less records and the buffered outline will not be as smooth.
Execute the block, load the data into Omniscope and then drag a Map View into Omniscope to see the buffered line.

ADVANCED
The "Geometric buffer" block also contains additional controls for configuring which fields contain the latitude/longitude values and the fields for identifying individual features (shapes) in the input dataset. If you are using KML or Shapefile sources then these fields will be configured automatically. Below are explanations for each option:

Overlay identifier - Field containing unique identifiers for each point, line or area in the input dataset, e.g. a field containing country names or postal area codes.
Overlay sub-path identifier - This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.
Longitude - Field containing WGS84 longitude values.
Latitude - Field containing WGS84 latitude values.

GIS: Importing an ESRI Shapefile

richard — Mon, 07 Jul 2014 08:00:00 -0400

Omniscope's DataManager supports importing geographical data from ESRI Shapefiles. Use the attached files and following steps for a basic example;

ATTACHED FILES

"postcode areas.shp" (inside ZIP) Contains geometries for each feature (shape) in the Esri Shapefile
"postcode areas.dbf" (inside ZIP) Contains meta-data for each feature (shape) in the Esri Shapefile
"postcode areas.prj" (inside ZIP) Contains the coordinates' projection, e.g. UTM, WGS84.

INSTRUCTIONS

Download the attached ".zip" file and extract the ".shp", ".dbf" and ".prj" files into the same folder.
Create a new file in Omniscope and go to DataManager.
Drag the "postcode areas.shp" file onto the DataManager workspace (or drag a "Data file" block onto the workspace and browse to the "postcode areas.shp" file). The ".dbf" and ".prj" files will be automatically detected.
The Esri Shapefile DataManager block contains 2 configurable options
- "Overlay identifier" - The field which contains unique identifiers for each feature (shape) in the file. In our Shapefile this would be the names of the postal areas, e.g. AB for Aberdeen, SW6 for South West London. If your data does not include such a field then selecting "Create new field" will allow Omniscope to generate internal unique identifiers using a field named "Overlay ID".
- "Meta data fields included" - The names of all fields which will be loaded from the Esri Shapefile into the block.
Load the data into Omniscope then drag a Map View into Omniscope to see the postcode areas.

ADVANCED
You will notice a field has also been created named "Overlay sub-path ID". This is used internally within Omniscope to identify nested geometries within each shape. For example a shape for the United Kingdom could not be drawn with one continuous line. We use this field to breakdown the records into numerous islands which make up the country.

Mapping: Importing Geospacial polygon shapes?

davedunckley — Tue, 21 Jan 2014 08:56:09 -0500

Hi,

Is this supported?

Dave

Map View: New GIS data files - Shapefiles, KML (2.8+)

steve — Fri, 10 May 2013 12:13:13 -0400

From Omniscope 2.8 Beta, you can now import data from ESRI Shapefiles (.shp) and Google Earth (.kml) files.

These files typically contain geographic data, such as points, polygons and polylines, with additional metadata.

Omniscope allows you to import this data as a flat table of fields and records, take advantage of the powerful ETL DataManager merging and transforming operations, and/or open in Omniscope for exploring and analysing in the Map and other views.

When imported into Omniscope, the data is treated no differently to non-geo data. You can filter and aggregate, for example. Typically two fields will contain the lat/lon coordinates of each point or vertex, and other fields might exist for metadata or path ID purposes.