DataManager: Validate Data block-curious behaviour?

cmollo · February 8, 2013 9:09AM

Hi. I am evaluating Omniscope Desktop 2.7 build 459 (x64) on a Windows 7 computer. I was experimenting with the Validate Data block. A have a File Source block that feeds in to the Validate Data block. The File Source block is connected to a CSV file. I created several rules in the Validate Data block (in the following screenshot only 4 of the 5 rules I created are visible). At first all the records pass validation. Then I changed the CSV so one record failed one of the rules: the Co2_improvement field of record #6 is greater than 1. The validation then fails, as expected. See the following screenshot (or attached "ss3" file).

Questions:
1. I click the Update button on the Validate Data block, but the update does not seem to be successful... the Update button remains red and the Co2_improvement field of record #6 shows the old value. Why is this?
2. Is there a way to determine what rule(s) is failing and on what record(s)?
3. Why does the preview show "unavailable" for all records after record #10 (see following screenshot)? It would make sense if the preview showed "unavailable" for all records after record #5 (since record #6 is the record with the invalid data).

Thanks!
Chris

tjbate · February 8, 2013 11:29AM

Chris - Some partial answers to some of these questions:

You have set the validation rules to accept/pass records only if ALL the rules are met, you should try changing this to other options. Currently Omniscope caches for preview display purposes only the first 10 records. When you changed one of your values to fail one rule, while your Accept rule was still set to ALL, the update is not succeeding for some reason so the preview visualisation cache still shows what was previously in the incoming data set for information.

cmollo · February 8, 2013 1:38PM

Thanks tjbate for the quick reply! Just now I tried the other "accept" options... they do not give me what I want... I want to only accept records that match all the rules. It would be nice if there was a "Accept records that do NOT match any of these rules" option... that way I could see which records are failing (which would partially answer my second question).

Chris

tjbate · February 15, 2013 7:13AM

Chris - Remember that you can connect any number of Validate Data Blocks to look at the data flow, and each creates its own branched flow, so that Accepts can go one way, and Rejects can go another way, and flow out to export files for correction/writeback loops, alerts etc.

cmollo · March 4, 2013 8:43AM

Sorry for the delay in replying. I saw your comment when you posted it... I just forgot to reply at that time. Anyhow, thanks for the additional input... I think it will be useful.

	omniscope_ss1.jpg	102K
	omniscope_ss2.jpg	28K
	omniscope_ss3.jpg	98K
	omniscope_ss4.jpg	98K

Welcome!

Categories

Ideas Parade

Tagged