Visokio website     Downloads     Video tutorials     KnowledgeBase  
'Pause' button for DataManager - Visokio Forums
'Pause' button for DataManager
  • Guy_Cuthbert        Guy_Cuthbert August 20, 2010 2:59PM
    When dealing with large datasets in DataManager (e.g. 1m+ rows) many modifications to the 'data map' can perform slowly due to repeated attempts to implement changes as they occur, for example: adding new formulae in a Filed Manager block - each edit causes an "Executing..." process. With small data sets this is not a major issue (although can be time-consuming if complex SUBSET-based formulae are added), but on large datasets it can be 5-10 minutes between adding formula whilst the whole formula set is recalculated - the workaround is to add lots of separate blocks, but this becomes hard to manage. Note: this issue isn't restricted to Field Manager blocks... that's just the challenge I'm wrestling with at present!

    Could we have a "pause" button in the DataManager view (or across Omniscope if possible, although I know I have floated that idea before!) - with 'pause' depressed the calculation engine would sleep, allowing a number of changes to be made, before re-engaging normal Omniscope behaviour.

    On large/complex models this would save many hours of processor time and twiddling of thumbs / making coffees :)

    Thanks

    Guy
    Atheon Analytics Ltd
    w: www.atheonanalytics.com
    e: guy.cuthbert@atheon.co.uk
    t: +44 8444 145501
    m: +44 7973 550528
    s: guycuthbert
  • 3 Comments
  •     steve August 20, 2010 3:09PM
    Guy, we're aware of this issue, but haven't yet found the best solution.

    This might help, though...

    We've recently added a "Random sample" operation which allows you to work with 1000 (configurable) random records. We've also added an "Input switch" operation. Both these allow you to prototype using a random sample of data, then flick a switch at the end.

    You can also flick the switch automatically using the scheduler and a DataManager parameter (tonight's build required), allowing you to prototype permanently with a subset, and have the server automatically work with the full dataset.
  • Guy_Cuthbert        Guy_Cuthbert August 20, 2010 3:12PM
    Steve - those sound like welcome additions... I will be sure to download the new version on Sunday :)
    Atheon Analytics Ltd
    w: www.atheonanalytics.com
    e: guy.cuthbert@atheon.co.uk
    t: +44 8444 145501
    m: +44 7973 550528
    s: guycuthbert
  •     steve August 25, 2010 4:38PM
    Guy, try tonight's build of 2.6. It has the option to turn off "Propagate block changes downstream" in the view tools menu for DataManager. I think this is precisely what you want.

Welcome!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Apply for Membership