Tagged with version_2.8 - Visokio Forums

Connector: Google BigQuery data source (2.8+)

chris — Wed, 10 Oct 2012 06:42:32 -0400

Hi -In the latest version of Omniscope 2.8 you will see that we have added a "Google BigQuery" feed.

Google BigQuery is a cloud database that allows you to analyse very large data-sets (apparently billions of rows). The Omniscope feed allows you to download and query data in a similar way to any other database.

At the moment we have only added a BigQuery feed, although in the future we plan to integrate BigQuery into the DataManager bookmarks and also add a BigQuery data publisher.

Please let us know if you have any feedback/ideas/problems using this functionality.

Formulae: "Product" aggregation/measure function (2.8+)

steve — Tue, 09 Oct 2012 05:02:44 -0400

From tonight's build of 2.8 (2.8 b337+) you will be able to choose "Product" alongside "Sum", "Mean", "Range", etc. to apply to numeric fields when aggregating and as measures. You will need to click "Common Functions" in the function menu to reveal it. You will also be able to use SUBSET_PRODUCT in your formulae.

This is a somewhat specialist-use function and can result in very high, unusuable values. One typical use is investment analysis, where periodic returns (like 1.10 for 10% periodic return) need to be multiplied many times to obtain the return over much longer periods defined as subsets of rows using the SUBSET(X) clause of the the SUBSET_PRODUCT function.

Scheduler: Troubleshooting running as a Windows Service (2.8+)

mustafa — Mon, 13 Aug 2012 07:52:42 -0400

This blog explains how to diagnose what to do if you are having problems trying to run Scheduler as a Windows service.

Make sure you have followed the steps here.
Check the troubleshooting page.
Search the forums for similar issues.
Follow the workflow in the attached PDF to diagnose the problem.

Note: If you are trying to diagnose then the above still applies, however, for point 1 refer to here. Also, when going through the attached PDF workflow you do not need to follow the "Visokio Windows Service" section.

Formulae: Exporting formulae for documentation/audit (2.8+)

davedunckley — Fri, 10 Aug 2012 05:02:22 -0400

Is there a way of writing out all the formula in a file, for documention?

Content View: Native Browser JavaScript API (2.8+)

steve — Fri, 24 Aug 2012 13:04:55 -0400

We've been busy working on building a Native Browser API for the Content View in 2.8.

This allows advanced users with JavaScript development skills to effectively build a custom view in Javascript which runs in your system browser, embedded inside the Omniscope Content View, with the page source editable in the Content View as usual.

This is very much work in progress, and subject to change following feedback before we release 2.8 in public beta. Please experiment and post below, but don't build any live reports that depend upon the new behaviour.

But can't I already use Javascript in the Content View?

The Content View already supports Javascript pre-processing. In your page source, you have special <# #> tags indicating the start and end of script fragments. Omniscope then, behind the scenes, executes the Javascript in a browser-less environment to produce the final page text.

This is directly equivalent to the way a PHP/JSP/ASP/etc. web server might "execute" a server-side page to produce the HTML document that is sent to the client. We chose to use Javascript on the "server" side, but note that there is no HTML DOM or browser Window object, for example; this is pure "headless" Javascript.

In this case, the "server" is the Omniscope Content View back-end, in-memory on your desktop, and the "client" is the page renderer, either Java within Omniscope, or a separate native browser process embedded in the Omniscope window.

New source editing enhancements

When editing in "source mode" in the Content View, we now have full syntax highlighting of your page source. This makes scripting and HTML coding much, much easier.

Note that if you're using script pre-processing, advanced formulae, or the native browser, you will only be allowed to use source mode, rather than "design mode" for editing your page.

New menu items in the view's Tools menu:

"Use native browser"

This affects how the HTML page content is rendered, after it has been evaluated (e.g. applying any formula insertion or script pre-processing).

This is deselected by default, matching 2.7 behaviour, meaning the page is displayed using Omniscope's built-in rich text display (via Java, based on HTML 3.2, without any support for in-page Javascript).

When selected, the page is displayed by embedding the system web browser (typically IE on Windows), allowing "client-side" scripts to be executed in the browser's Javascript engine, with full access to the HTML DOM, etc. This allows you to build Javascript interactivity and do other native browser behaviours such as embed media.
"View in external browser"

Click this to open the same page in an external browser. This allows you to debug your page in a full separate browser; we recommend Google Chrome for its developer tools out-of-the-box. The embedded browser, if IE, has very limited support for debugging.

Note that Javascript native dialogs (alert/confirm/prompt) are disabled in the embedded browser, because it can often lead to a complete freeze of the application. You should use alternative Javascript frameworks instead.
"Refresh on change"

By default, the page content is rebuild and the browser refreshed when cell edits or filtering occur. Untick this option to disable. You will need to code in a different means of triggering updates to any data you may have loaded via API / AJAX calls.

Even if ticked, viewing in an external browser will never refresh automatically on change; you will need to manually reload the external browser.

AJAX requests

You can choose to use this native browser option to add some animations or general pizzazz to your Content Views, or more extensively by using the API to request and process data.

We currently provide two APIs. At a low level, an HTTP service is provided should you wish to roll your own AJAX calls and perhaps use different or incompatible JS libraries to those used by the high-level API.

The high-level API is enabled by inserting a pre-processing snippet into your page source in the HTML HEAD tag, and provides Javascript objects and methods to 'call' the enclosing Omniscope Content View.

Low-level API

The page itself is served up by "page.html" within an auto-generated unique URL, which changes each time the view is opened. Use "View in external browser" to see the URL. But you should not need to know this, since the other HTTP services are available in a relative URL to this page. For example, from within "page.html", you can use AJAX and hit the "meta.json" URL (without any further URL qualifiers).

All requests return HTTP status code 200 if successful.

page.html

Returns the page content as text/html after applying pre-processing and evaluating formulas.

No request parameters.
eval.json

Evaluates a JS expression on the back-end, with access to formula functions.

Accepts a POST request with text/plain upload body (NOT regular key/value form data), containing a Javascript expression to evaluate against the Content View's dataset as per "script pre-processing". This can be used to query arbitrary formula functions such as subset_max("fieldname"), for example.

Returns JSON { "result": value } where value is the result of the expression; can be a complex object or array.
meta.json

Retrieves metadata about the view's data.

No request parameters.

Returns JSON { "fieldCount" : 13, "recordCount": 1052, "fieldNames": ["field1","field2",...], "fieldTypes": ["text","text","date","integer","decimal",...] }
cells.json

Retrieves one or more cell values.

Accepts a POST request with the following form key/value pairs:
"fields": a JSON array of field names; if omitted, all fields are retrieved
"firstRow": The first row to retrieve, 0-indexed, must be in-bounds; if omitted, zero is used
"numberOfRows": The number of rows to retrieve; if omitted, all remaining rows are retrieved; automatically capped by the last row in the dataset

Returns JSON { "result": [ [cell00, cell10, cell20], [cell01, cell11, cell21], [cell20, cell21, cell22] ] }
(where the 2-dimensional array's dimensions reflect the filtered data results, doesn't contain a row for field names, and is indexed [column][row]).
edit.json

Writes back one or more cell edits to the data table.

Accepts a POST request with the same key-value pairs as "cells.json", plus:
"data": a JSON two-dimensional array, in the same format as returned by "cells.json" under the "result" property.

Returns the empty JSON {} to indicate success.

This will only work in licensed editions where editing cell data is permissioned - i.e. not in the free Viewer. It will propagate a change to all views, and you will immediately see the result. You will need to untick "Refresh on change" if you want to avoid the same Content View (which originated the call) from reloading your page.

Distribution: Custom Jar now has web browser support (for Web View and Content View) (2.8+)

antonio — Mon, 03 Sep 2012 07:21:50 -0400

You can now embed web browser support and have a fully working Web and Content view in Custom JAR using from the download page (through clicking "Customise" link under the Download button).

Note the following:

Functionality is only available from 2.8 b276.
The total size of the JAR will increase due to the weight of the libraries used to provide the native browser integration.
Custom JAR you create will only be available on our server for the current week. It is recommended that you download this JAR and store it locally.
To embed your own IOK and Branding you must use Omniscope Custom JAR functionality, for more information see here. You must use the new Custom JAR as the source JAR file.

Output: Database Tables - Date/Time values (2.8+)

sunicha_sherriff — Thu, 16 Aug 2012 07:16:03 -0400

Hi - At the first,the content is as in picture "Original content" and then I would like to update it to my database using operation "Database Output" and select Publish method: Update data.

But the result is Omniscope change my content that you can see in picture "Result Content" and see at column Platts Created. It changed from "Oct 1, 2010 5:42:26 PM" to "Oct 1, 2010 12.00.00 AM". Omniscope 2.8 build 258(Alpha) *64

Best Regards - Palm

Menus: Simpler "Advanced settings" (2.8+)

steve — Wed, 15 Aug 2012 12:58:47 -0400

From tonight, the "Settings > Advanced" menu has been restructured in 2.8. Power users who access settings in these menus should find them much easier to navigate.

Note that the "Performance" section (containing very rarely needed advanced settings) is only shown if "Settings > Advanced > Feature availability > Debugging, diagnostics and performance" is enabled.

Scheduler: Import/export/copy/paste actions, tasks & recurrences (2.8+)

steve — Tue, 31 Jul 2012 10:45:19 -0400

Tonight's build of 2.8 brings some simple time-saving improvements to the Scheduler UI.

You can now import/export/copy/paste any actions, tasks and recurrences.

From 2.8 b226 onwards.

Formulae: new TRENDVALUE function (2.8+)

steve — Thu, 02 Aug 2012 09:57:46 -0400

In tonight's build of 2.8, the new TRENDVALUE function has been added.

This allows you to work out the Y value for a line of best fit (linear trend) given any two numeric/date fields. This is equivalent to what you see in the Graph view when you show a line of best fit.

For an example, see here:
http://forums.visokio.com/discussion/comment/6116/#Comment_6116

Selection: Filters brushing (2.8+)

steve — Wed, 01 Aug 2012 13:29:08 -0400

For a long time now Omniscope has supported "Selection brushing" (previously called just "Brushing").

With selection brushing, when you select data in one view, other views update to show the same data, with a shadow of the original data shown behind.

Omniscope 2.8 now supports "Filter brushing", available via opt-in to any views configured to show "All data". With this, as you filter, the view updates to show the filtered data, with a shadow of the original data. See example below.

Selection brushing is enabled by default. Filter brushing is not.

To explore filter brushing in a given view:

Choose View toolbar > Subset > All data
Choose View toolbar > Tools > Brushing > Filters brushing

Note that selection and filters brushing are now configurable at a per-view level (as above) as well as a global level (Main toolbar > Settings > Brushing)

Please note that this feature is still in development and may evolve further in the coming days.

Example:

Scripting: Using arrays of data (2.8+)

steve — Mon, 16 Jul 2012 11:23:32 -0400

Version 2.8, available shortly to alpha partners, introduces a new function "dataArray", for use in Javascript. It retrieves a two-dimensional array of cell data for bulk processing and transformation.

You can use Javascript in Omniscope in the SCRIPT function in formulae, and to do HTML pre-processing in the Content view.

Syntax:

dataArray(fields, rowSubset, includeFieldNames)

Argument "fields": An optional array of field names. If omitted, all fields are retrieved.
Argument "rowSubset": An optional data subset, as returned by the subset function.
Argument "includeFieldNames": if true, the first 'row' in the returned 2d array 'table' will be the field names.

Returns: A two-dimensional array of cell data, indexed as [column][row], with indexes numbered from 0 to n-1.

See attached example, using dataArray inside a Content View.

Formulae: Field/Cell direct access functions CELL, etc. (2.8+)

steve — Tue, 17 Jul 2012 09:56:49 -0400

These 4 new formula functions allow you to access arbitrary cells in the data. Also available in scripting.

CELL
FIELDCOUNT
FIELDNAME
FIELDNUMBER

Formula examples to obtain cell values

CELL([Price], 53)

CELL(FIELDNAME(3), CURRENTROW()-1)

Javascript preprocessing example to list all field values


for (n = 1; n<=fieldCount(); n++) {
  out.println(fieldName(n)+": "+cell(fieldName(n), 1)+"
");
}

DataManager: "Custom Script" Operation (2.8+)

steve — Tue, 17 Jul 2012 08:36:04 -0400

Omniscope 2.8 introduces a new operation called "Custom Script".

The "Custom Script" operation allows you to execute arbitrary Javascript to transform one or more inputs into a new output.

The result of the Javascript must be a two-dimensional array, indexed as [column][row] (i.e. [field][record]). You can optionally choose to provide the field name as the first row in this 'table'.

The inputs are accessible via "input1", "input2", etc. and also via "inputs[0]", "inputs[1]". These objects provide access to all regular Omniscope Javascript functions, such as:


input1.recordCount()
input[0].subset_sum("Price")

Here is an example script which simply ignores inputs, and produces static data:


[
  ["1, 1", "1, 2", "1, 3"],
  ["2, 1", "2, 2", "2, 3"],
  ["3, 1", "3, 2", "3, 3"]
]

Here is another which returns a 2x1 table of metadata about two inputs:


[
[
input1.recordCount()
],
[
input2.recordCount()
]
]

Here is another which does a sideways concatenation of multiple inputs:


working = new Array();
for (i in inputs) {
  var input = inputs[i].dataArray(null, null, true);
  for (n in input) working.push(input[n]);
}
working;

See attached example. Requires 2.8 or later.

Date/Time: Cross-time zone publishing fix (2.8+)

steve — Thu, 21 Jun 2012 11:53:40 -0400

Omniscope 2.8, available shortly, fixes a known issue with publishing IOK files with date fields to recipients in different regions.

2.7 and earlier

Dates in Omniscope are modelled as absolute points in time (known as "unix" or "epoch" time) and are displayed in the local time zone in whatever format is configured for the field - e.g. "month-year", stored as the instant at the beginning of the month in question.

Additionally, different regions have different "first day of week" settings, leading to different week number calculations.

Typically this can lead to dates appearing offset by a day, or week histograms starting on the wrong day of the week (Sunday vs. Monday), when an IOK file from 2.7 is opened on a PC in a different region.

Until your publishing and end-user installations have been updated to Omniscope 2.8, you will need to either author the IOK file in the recipient's time zone, or will need to store dates as hidden text fields with a TEXTTODATE formula field configured to recalculate on open.

2.8 and later

As with 2.7, dates continue to be modelled as absolute points in time. But now, IOK files carry the originator's timezone and calendar week settings with them.

Time zone handling

Omniscope 2.8 now captures a fixed time zone when you create a file, which is the system time zone at that point. It also captures the current time zone when you first save a legacy file (from 2.7 or earlier) using 2.8+, although that setting will be lost if you subsequently save in 2.7.

When a time zone-aware file (saved in 2.8+) is subsequently opened, it continues to work in the same originating time zone. It doesn't matter whether you view or edit it on another PC the other side of the world, the time zone is still fixed to the originating time zone.

All date fields are recorded internally, manipulated, and displayed, in that time zone. This means that you'll always see the same date calculations regardless of local system time zone.

Unfortunately this will have no effect for clients still using 2.7 or earlier until they upgrade to 2.8. See workarounds above, meanwhile.

Regional settings dialog

You can use Settings > Advanced file settings > Regional settings to:

Review all regional settings and related.
Change the calendar week settings according to the target audience, if needed.
Change the setting and convert any date fields. Typically you wouldn't normally need to do this; the conversion isn't exhaustive. However, if your file has the wrong setting and has "hidden date offsets" due to being created in a "pre-June-2.8" version of Omniscope, you can correct it.
Diagnose "hidden offsets" (symptoms of a legacy file with the wrong time zone setting).

Calendar week settings

In the same dialog, Omniscope also captures some calendar week settings. This is for a similar reason to time zone. Calendar week settings vary by locale (Saturday/Sunday/Monday as first-day-of-week), yet your formulas must be able to perform week-of-year settings without the local PC settings affecting the result.

As per time zone, these settings are captured and frozen when you create a file, according to the typical regional settings for your PC's locale. The dialog also explains what are normal settings for different locales.

Language and data locale

The pre-existing settings "Language" and "Data locale" still work as before. They are installation settings, which aren't saved as properties of a file. But they no longer have any impact on most date calculations / manipulations / storage. Previously they affected calculations in rare situations.

"Language" means the interface language, and has no effect on data manipulation or display.

"Data locale" means that (by default) if you're in France, you'll see comma for decimal point and French textual month/day names.

Please be aware that if you are using formulae to process numbers or dates inside text values, you should provide an explicit locale ID in the formula function arguments. Without this, textual month/day names, and decimal point / thousand separator characters, will follow the rules of the local installation's "data locale" setting.

Integration: ILF link files with parameter overrides (2.8+)

steve — Mon, 19 Mar 2012 12:51:04 -0400

A previous blog described how a 3rd party application can launch Omniscope to open a file with customised DataManager parameters:

http://forums.visokio.com/discussion/1376/new-command-line-arguments-for-datamanager-parameters/p1

We have now extended this concept to ILF link files.

(An ILF file is like a shortcut, which doesn't contain any real data. Instead it instructs Omniscope to open a file from a URL written inside the ILF file. You would use ILF files typically if you want to force Omniscope to open an IOK file directly from the IOK file's URL, rather than allow a web browser upon click to download the IOK file to a temporary folder first. This would allow you to use domain locking, for example.)

Use the new Server Edition XML action "Create Link File" to create ILF files which customise DataManager parameters. As with command-line parameters from the above linked post, you can specify the name of DataManager parameters and the value to use, which overrides the current value in the file. Using this method, you can override an unlimited number of parameters.

For example, the attached two ILF files instruct Omniscope to open the IOK file attachment from the above linked post.
- The first overrides the parameters to show the first 5 "abbey" tickers.
- The second overrides the parameters to show the first 3 "hbos" tickers.

Idea: Table View-Formatting; Alignments in cells/headers?

hgross — Mon, 13 Feb 2012 06:52:51 -0500

It would be great if table cell content and headers could be centered or aligned right or left (maybe through right clicking on the table header row).

Thanks!

Idea: Dynamic view titles?

edtclarke — Mon, 03 Oct 2011 06:04:39 -0400

Does anyone know if it is possible to add dynamic text to a view title? We use this a lot in Content views, for example display the contents of a variable, but it would be really helpful if you could do this in the title of a view -Thanks

New: Native support for SSH tunnels and SFTP/SCP file transfers

antonio — Thu, 03 May 2012 12:52:38 -0400

Allow Omniscope to establish SSH tunnels for secure connections to remote services such as intranet databases HTTP servers and data files.

Using port forwarding to allow existing Omniscope connectors to connect to remote services via an Omniscope-established SSH tunnel, including FTP over SSH.

Using SCP (Secure Copy over SSH) and SFTP (SSH File Transfer Protocol) to allow Omniscope to upload or download data from remote files.

Connectors: Salesforce Test/Sandbox instead of Production org?

cidarm — Mon, 07 May 2012 12:56:28 -0400

Hello,

I am trying to publish (output) to the Salesforce Sandbox/QC environment. Users cannot specify the test.salesforce.com link when logging into the non-production environment. Thoughts? Any suggestions or comments you have would be much appreciated.

Thanks, Cid

DataManager: "Field metadata" operation (2.8+)

steve — Mon, 02 Apr 2012 06:28:49 -0400

New in 2.8: the "Field metadata" operation.

This is a very simple operation currently without any configuration options. For every input field, it creates an output record describing that field, with attributes such as Name, Type, Min, Max, etc.

DataManager: More attributes in IOK "File metadata" block (2.8+)

steve — Wed, 11 Apr 2012 15:17:58 -0400

New in 2.8, the "File metadata" data source (available in DataManager and from "File > Open folder of files/media") now includes basic metadata from within IOK files:

- Record count
- Field count
- Field names, comma-separated
- Source file/URL/summaries, comma-separated
- Tab names, comma-separated (includes hidden tabs).

Layout: Formulas in page/view headers (2.8+)

steve — Wed, 18 Apr 2012 06:11:06 -0400

2.8, available shortly to alpha partners, now supports formulas in page and view headers. So, if you are using view headers (via view "Tools > Toolbars and styles > Header"), you can now insert formulas, which behave just as with formulas in page/view footers and in the Content view.

Formulas in headers currently don't have a wizard and have to be entered directly using {{double curly brackets}} when editing the header as follows:
This view has {{ RECORDCOUNT() }} records showing
or
Data for category "{{ SUBSET_SINGLETON([My Category Field]) }}"

Formulae: Fuzzy text matching-Soundex & Levenshtein (2.8+)

chris — Thu, 12 Apr 2012 11:52:43 -0400

Hi,

In Omniscope 2.8 we have added the ability to perform fuzzy matching in the Record/Field filter operations and in the Merge/Join block.

In both cases you can select either the Soundex or Levenshtein matching routines. Please let us know if there are any other routines you would like us to include.

In the Record/Field filter blocks you can perform fuzzy matching by selecting a fuzzy match routine from the operation drop-down. So whereas before you might have chosen "Filter records where Employee name = 'Angelina Joline'", you can now create the rule "Filter records where Employee name fuzzy matches 'Angelina Joline". Depending on the matching routine selected this may match records "Angelina Joyline", "Angelina "Joleen" and other similar names.

In the "Merge/Join" block you can specify the fuzzy match routine in the operation drop-down for each join criterion specified.

Please let us know if you have any questions.

DataManager: Database source field picker (2.8+)

chris — Wed, 11 Apr 2012 13:06:29 -0400

In Omniscope 2.8 we have now added a field picker to the Database configuration options. This allows you to select which fields to import without having to customise the SQL statement.

DataManager: "Batch append links" merging feed (2.8+)

chris — Tue, 03 Apr 2012 06:58:45 -0400

In Omniscope 2.8 we have added a new merging feed, "Batch append links". This feed requires one or more URL's as an input. It will then append the data obtained from each of these URL's into a single Omniscope tabular data set. It is very similar to the Batch Append operation, the difference being that in "Batch Append" the list of URL's are generated dynamically based on a selected directory.

Formulae: ROUND function (2.8+)

antonio — Mon, 02 Apr 2012 11:59:52 -0400

Available from 2.8, the ROUND function is equivalent to that in Excel, i.e. rounds a number to a specified number of digits.

Syntax:
ROUND(number, num_digits)
where:
- 'number' is the number that you want to round.
- 'num_digits' is the number of digits to which you want to round the number argument. Default 0 if not specified.

Samples:
ROUND(123.7825, 2) = 123.78
ROUND(123.456, 1) = 123.5
ROUND(123.456, -2) = 100)

Idea: Pivot View 2- formatting options?

schergr — Fri, 16 Mar 2012 16:17:54 -0400

How is the Pivot 2 view coming? The capability is great, just wondering if you might be adding formatting features for font sizes, etc... as it current is kinda ugly ;)

I know..l know, it's still experimental, but you can't expect to keep us away from great stuff like this ;)

Pivot View: Improvements to Pivot 2 View?

indranildatta — Thu, 26 Jan 2012 07:21:27 -0500

I've been recently getting a few questions from our prospects about multi-column pivots, so I revisited the Pivot 2 View and was pleasantly surprised to see the development in this view.

I'll take the opportunity to enlist a few enhancement requests that I have on Pivot 2:

1. An option to turn on/off the font sizing based on measure value.
2. An option to turn on/off grids inside the pivot table.
3. Options to place the vertical pane labels on top/bottom, and similarly a left/right option for the horizontal pane label..
4. Option to set "No background color". Currently, even if I set the bg colour to white, it is set as grey.
5. Seperate options to set the horizontal/vertical axis pane label orientations.

I think once we have these options, this view can completely replace the Excel pivot table option, so anything you can do to prioritise is highly appreciated.

Cheers!
Indranil

Scripting with JavaScript

steve — Tue, 26 Jul 2011 04:00:08 -0400

Omniscope 2.6 has experimental support for JavaScript in formulas. Further ideas:

- from JavaScript in a formula, the ability to access arbitrary cells and metadata in the data, and the ability to execute nested formulas

- in DataManager, a JavaScript operation block, which provides access to the input datasets and their cells & metadata, and create an arbitrary output dataset.