Connectors: DoubleClick/DFA long download times?

Alk · March 16, 2011 10:14AM

Hi all, I've got a problem with a DoubleClick feed taking literally hours to download. Is there any good solution or work-around anyone has in mind with regards to a method to download the data incrementally - without changing the custom date range (and appending) or having to update it every # days exactly?

I thought of having one DoubleClick block with all the data downloaded once and an incremental block which downloads the month's data, appends the blocks and removes duplicates, but this can cause incorrect data to show so is not a solution.

I need to download the update every 3-10 days and the wait for it to download all the data is painful.
Download speed directly from DART goes at around 80kb/s, so this is likely to be why it takes so long, but I am looking for a solution which will make the download size less by limiting the number of dates I need to download and also retaining the old information in Omniscope.

chris · March 16, 2011 10:38AM

Hi,

Thanks for the question.

What is the reason for the DART download speed being so low? Is it a problem with the DART server or your network connection?

If there's nothing you can do about the download speed the best solution would be schedule the data download during the night. You can use the Visokio Enterprise Scheduler application to refresh the DoubleClick data at a specific time. This would typically be the best way to handle any workflow sequence that takes a long time to execute and that needs executing on a regular basis.

Chris

Alk · March 16, 2011 11:55AM

Hi Chris,

Thanks for the prompt reply.

I am not sure if it is that speed for everyone or only on my network, but when I download from DoubleClick's website, it runs at around that speed. I presume Omniscope is getting a similar download speed.
A ping to dfaapi.doubleclick.net and reportcentral.doubleclick.net is averaging at 135ms. Web-browsing speed is and downloading from other websites is fine, so I do not think it is a major network issue. Downloading a month's worth of data takes around 30mins, so was hoping I can get away with that.

I am using the Professional Desktop version of Omniscope rather than Enterprise Server, so will not be able to download the data over night unless I leave my computer on or begin the download in the morning and hope it completes by the time I leave office.

Alk · March 17, 2011 7:22AM

Strange, Omniscope downloaded all the content today in 30mins. Whereas yesterday and a few other attempts before that it didn't complete or took many hours. May have hung or dropped packets.

Downloading from http://reportcentral.doubleclick.com/ is still taking its time at 60-100kB/s and the ping time from my computer is similar to a few other networks. Must be the upload speed which is slow.

Welcome!

Categories

Ideas Parade

Tagged