Automation: File processing via Batch input/output?

cmollo · May 6, 2013 4:01PM

Is there a way to apply the same operations to a set of files?

For example, I have 10 files in a directory: file01.csv, file02.csv, ... file10.csv. These files all have the same columns (fields): field1, field2, field3. For each file I want to create a fourth field, field4, that is the sum of field2 and field3. This will create 10 new files, file01_new.csv, file02_new,csv, ... file10_new.csv in either the same directory or some other directory (doesn't matter). How do I do this?

Is there a way to run Ominscope in "batch mode," i.e. from a command line?

I looked at the Batch Append Files block... it is kind of what I am looking (in that it inputs a bunch of files from a directory) but I do not want to append the files together.

I also looked at the Batch Output block, but that did not look like what I need.

Thanks!
Chris

tjbate · May 7, 2013 5:15AM

Chris - You can use Batch Append....a Source field will be added on import noting which original .csv file each row came from....once appended you can use Field Organiser to add new transformational formulae fields to the combined row set...then, using Batch Output, you can specify the 10 output files to be delivered/saved by using the Batch Control IOK file filter setting to include in each new .csv file created/exported only the rows in the original files using the Source field added on import.

cmollo · May 7, 2013 9:19AM

Thanks! I got your suggestion to work.

My example, however, did not capture the full situation. In the "real world" there could be hundreds of files. Creating a Batch Output configuration file with hundreds of rows in it might be a lot of work. Is there any other approach? Please let me know. Thanks!

Welcome!

Categories

Ideas Parade

Tagged