Visokio website     Downloads     Video tutorials     KnowledgeBase  
Performance: DataManager - Long multi-block flows slow? - Visokio Forums
Performance: DataManager - Long multi-block flows slow?
  •     Mees December 9, 2012 3:45AM
    Hi there - We have an IOK file that is only used as DataManager file. The number of DM blocks is large (around 50) but the number of records in Omniscope (the blue block) is only 50k and 7 fields. It takes a very long time for the file to open and when it is open it reacts very slow.

    It uses up to 10GB RAM to open on a 20GB RAM 64bit server?

    Question: does the number of DM blocks affect the iok files performance? Even when there is hardly any data in?

    Thanks, A
  • 2 Comments
  •     tjbate December 9, 2012 10:02AM
    Arjan - When you say that the file is a transformational DataManager-only file, that implies that your Output blocks are not primarily the blue, in-memory portable IOK Omniscope visual discovery file, but primarily other file types, such as relational database tables, spreadsheets etc.

    If you do not plan to make any use of this transformed data set as an Omniscope DataExplorer IOK Report file, or as an input Source IOK file in other downstream file chains, then do NOT connect the blue Omniscope IOK output box into the chain, and even delete the blue box from the data flow in the workspace. This will greatly reduce the maximum RAM footprint of transformational (ETL-only) files.
  •     Mees December 9, 2012 4:04PM
    thanks Tom, will do.

    Have also split the iok files and appears to improve performance as well, just for info...

Welcome!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Apply for Membership