Deduplicate, but select "latest" records based on date field

schergr · February 22, 2012 11:31AM

Is it possible to deduplicate data, but ensure that the record included in the result set is the "latest" single record as defined by a specific date field. In effect, comparing each of the duplicates and saying, show me the "latest" one only?

chris · February 22, 2012 12:37PM

Hi,

You can do this fairly easily in DataManager.

Connect your data to a "Sort" block and select the date field. This will ensure the most recent data is at the top.

Now connect a "De-duplicate" block and select the field(s) that you want to de-duplicate on. The first unique record will be selected for each field combination, and because you have ordered the data by date the first record will always be the latest.

I've attached a screenshot of an example workflow.

steve · February 22, 2012 2:18PM

Alternatively, choose "Output: All records" and tick "Create instances field". You can then filter by the "Instances" field. But Chris' suggestion is probably easier.

Welcome!

Categories

Ideas Parade