DataManager: Automatically import/export database tables?

October 9, 2012 6:39AM

I'm trying to use Omniscope DataManager ETL workspace data flows to create a tool for importing several tables into a Database Structure as an automated process. It does not seem possible to manage this import and output sequence automatically?

I defined a data flow like this: Open database -> organize fields -> validate data -> publish data using a database table output only. My data flow does not include a load into the blue ouput block representing the in-memory DataExplorer output for visualisation/presentation/reporting. Could we have the ability to launch this type of ETL data flow in batch mode (silent or no prompts doesn't allow the publish process run automatically)? Please if you have any ideas, let me know. - Alfonso

tjbate · October 9, 2012 9:20AM

Alfonso - Have you included the Scheduler task 'Publish DataManager block' in the Scheduler Task List? You do not have to load the data set into the Omniscope in-memory visualisation block in order to publish a database table using this Scheduler command.

OmarKhan · October 12, 2012 5:26AM

Hi Thomas,

This works good for creating a table on my database, but is very slow and I find creating the table directly was faster.

tjbate · October 12, 2012 6:24AM

Omar - There are on-going performance improvements in priority areas reported by clients. For example, we have converted all the database inserts to create/update a table from 1 record to 100 records at a time, for those databases that support this.

We regularly receive unresponsive process reports and use these to prioritise the de-bottlenecking of real-world data flows. Your unresponsive application timing settings can be used to pinpoint and report slow processes.

chris · October 12, 2012 6:26AM

Omar,

Would it be possible to demonsrate your database performance problem to us via a screen-sharing session?

Please contact us to arrange.

October 14, 2012 5:02AM

Thanks for comments,
I haven't Server License for this project however, I cannot use the Scheduler Task List. But ok, It's the way..

tjbate · October 14, 2012 7:15AM

Alfonso - Your organisation already has a Server Edition license but has not yet exercised the right acquire one spare Server key for the price of an internal use Deskop. Spare Server keys are used for failover/back-up, development/test environments and peak load time-constrained publishing. You should be developing/testing all your public-facing publishing solutions with a spare Server key.

October 14, 2012 9:57AM

Thank you, I'll ask for it.

Welcome!

Categories

Ideas Parade

Tagged