When dealing with large datasets in DataManager (e.g. 1m+ rows) many modifications to the 'data map' can perform slowly due to repeated attempts to implement changes as they occur, for example: adding new formulae in a Filed Manager block - each edit causes an "Executing..." process. With small data sets this is not a major issue (although can be time-consuming if complex SUBSET-based formulae are added), but on large datasets it can be 5-10 minutes between adding formula whilst the whole formula set is recalculated - the workaround is to add lots of separate blocks, but this becomes hard to manage. Note: this issue isn't restricted to Field Manager blocks... that's just the challenge I'm wrestling with at present!
Could we have a "pause" button in the DataManager view (or across Omniscope if possible, although I know I have floated that idea before!) - with 'pause' depressed the calculation engine would sleep, allowing a number of changes to be made, before re-engaging normal Omniscope behaviour.
On large/complex models this would save many hours of processor time and twiddling of thumbs / making coffees :)
Guy, we're aware of this issue, but haven't yet found the best solution.
This might help, though...
We've recently added a "Random sample" operation which allows you to work with 1000 (configurable) random records. We've also added an "Input switch" operation. Both these allow you to prototype using a random sample of data, then flick a switch at the end.
You can also flick the switch automatically using the scheduler and a DataManager parameter (tonight's build required), allowing you to prototype permanently with a subset, and have the server automatically work with the full dataset.
Guy, try tonight's build of 2.6. It has the option to turn off "Propagate block changes downstream" in the view tools menu for DataManager. I think this is precisely what you want.