Martyn - Are you using a Batch Append Connector pointing at a single folder containing all 53 (and growing) files?

Hi Tom- yes I am.

Is this time-sliced data, in other words...historic (weekly?) files that never change, and only the latest week's new 'timeslice' added to the folder?

It's once a year data for 53 different countries (4000 rows and 200 columns each). But I use Visokio file to identify data errors in a country file and then go and corrent it. It may only be a single country error, but when I refresh it reads them all, even though only one has changed.

Given that any one of these concurrent country files could be corrected at any time, one thing you might do is create a 'pyramid' of Batch Appends, grouping the Country files by Region. This means that instead of one global Batch Append folder, you would have one Batch Append folder by Region, i.e. Europe folder and another folder for North America...then a final Append block that appends all the regional data sets.

If you do this, editing the data in only the UK country file should trigger a reload/re-caching of only of the files in the Europe regional file...but not in all the other regions.

Thanks Tom, might even go to country level with the folders. Will try it.