Scaling: Text character/string length limit per cell?

mattyh · June 5, 2014 5:23AM

I'm planning to import a fair amount of data from an API (currently only testing with small amounts so far) using the download link text feed. I was wondering if anyone knows if there is a character limit for a single cell in Omniscope (as all data will initially be in one cell)?

Thank you, Matt

paola · June 5, 2014 11:11AM

This should be possible, but it's worth testing. You should be able to import large amount of data into single cell via an API, in the DataManager space. Once imported, in the same space you should use text mining and other operations to trim/filter and re-distribute the cell content before it is moved further.

mattyh · June 6, 2014 4:36AM

Thanks Paola,

I've used de-tokenise which is working well so far, is that ok or will that struggle when there's around 300k records to go through?

paola · June 6, 2014 7:12AM

Please see the demo that loads RSS feed from the Reuters news into Omniscope. (on open click refresh data option to see the latest news stories). On the DataManager tab you can see the news stories loaded and displayed inside the table.
http://forums.visokio.com/discussion/2478

What would you like to do once the long text is inside the cell?
I attach here list of Omniscope text mining functions that might come useful.

mattyh · June 6, 2014 7:19AM

Thanks Paola,

That RSS iok looks fantastic.

De-tokenise does the job for me and I managed to find a page with 5 milllion plus characters on which Omniscope downloaded with no problem at all, so it looks like it will be perfectly possible.

Welcome!

Categories

Ideas Parade

Tagged