Visokio website     Downloads     Video tutorials     KnowledgeBase  
Batch Processing - Visokio Forums
Batch Processing
  •     mburgess November 30, 2010 10:22AM
    Hi

    i just had a question before spending time building a report

    we have a data source we are getting daily in xls format - approx 7,000 rows x 5 fields

    we are looking to use batch processing to create on table for a years (i.e. 365 files) data initially (and then continue to update going forward with future files)

    would there be any issues using batch processing with this large amount of source files? is there a better way other than batch processing for Visokio to cope with this many data source files

    thanks
    Matt
  • 1 Comment
  •     steve November 30, 2010 1:19PM
    There may be an issue with the total volume of data (~2 million records per year), depending on your PC configuration. You may need 8gb of memory or so.

    You should use DataManager to batch append the files. This should take no time to set up and experiment with. If you find there are performance issues with this, let us know; you may be able to work around them by, for example, batch appending a month at a time, and saving the results as an IOK file, then batch appending the IOK files.

Welcome!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Apply for Membership