Ben - Yes, if you are working on configuring a file for the first time, and the source data files are not changing, then you should turn off refresh of the Source blocks.

You can set different refresh behaviour for each of the Source blocks using the blue spanner/wrench block options drop-down on the right side of the Source blocks. From the drop-down, choose Update options) and set it to "Never", while you're working on the workflow (other alternatives are Auto-detect and Scheduled periods). After completion/testing switch the Update options settings to something appropriate.

Hi Paola

I have tried
A.Unselecting background refresh in the file menue
B.Selecting refresh to "Never" on the SQL block itself

However in both cases when i open the file and add in another field in the right datamanager stream, when this stream then joins onto the left datamanager stream it updates the whole left hand side stream

Would you be able to post or email us directly a demo file?
thanks

If you open the file and then click "Open with stale data"
Now in Field organiser 4 if you add or delete a column
You will then notice that the whole upstream block on the LHS updates

It would be better example if you connected it to a database on the LHS that takes a long/slow time to query

I have one question related to this subject.

I have a source1.iok file that is the only input of a second source2.iok file.

I save the source2.iok file with the above indicated 'never' option activated on the source1.iok input block and I prepare a scheduled process in which I refresh and save the source1.iok file and, later, I refresh and save the source2.iok file.

Which is the behaviour of the scheduled process? Will the source2.iok file be correctly updated at the end of the scheduled process also with the 'never' option activated?

Thanks

Enrico - Good question...

When you have an IOK file (1) as the source of another IOK file (2), refreshing file (2) will always retrieve data from what is contained within file (1). The refresh option 'never' applies to whether you at the same time refresh data in (1) from its own source, which might typically be a CSV or Excel file.

So typically you might have a data flow chain:
CSV --> IOK1 --> IOK2

If as stated, IOK2 has refresh option "Never" configured in its DataManager source block referencing the 'upstream' IOK1, and you schedule a "file action" upon IOK2 that refreshes from source and saves, it will read latest data from IOK1 and flow it through the IOK2 DM workflow. In this case, even if the original source CSV file has newer data, these changes will be ignored; the data set 'cached' in IOK1 will be used instead.

If instead IOK2 has refresh option set to "Always", the same scheduled "file action" task will go all the way upstream and first read latest data from CSV file, then flow it through any DM sequence modelled in IOK1, before using the fully updated data in the IOK2 workflow. Hence the CSV latest data will be reflected correctly, and any stale 'cached' data in IOK1 will not be used.

Hi, I'm having the same problem when trying to work on an Omniscope file with Upstream operations. This file has queries that take a very long time to run and I am trying to debug something downstream however whenever I try to look at what is happening downstream it triggers a full upstream refresh, paralyzing the work I am doing.

In previous versions of Omniscope I would take a snapshot of a block and then feed the downstream area I am working on off of the snapshot. However when running 2.9 plus b1765, even snapshotting causes a full upstream refresh.

Hi, we could not reproduce this behaviour, but would suggest that you take a snapshot of the first block below (Append or Merge ) and then set its' refresh behaviour to "Never". This should definitely cut off the connection with the database, while preserving the data.