Importing Data using R

Cheyam · November 7, 2016 12:29PM

Hello,

I am trying to use R to import data. This is done via a COM library. The R code works fine in both R-Studio and OmniScope. The issue, however, is that cells with commas in them are getting split into separate columns. Testing the number of dimensions in my final dataframe "output_data" we have correct numbers however when the data is displayed in "preview" mode in the R input block we have an extra 2 columns. Any cells which have text containing commas in them are being split into new fields and the data which should have been in those fields is getting pushed to the right.

I believe the issue is with the R package which Omniscope uses to interact with R. That in converting a dataframe data type in R into a table which Omniscope understands, that it assumes the data is in comma separated format when it really shouldn't do this (its already been put into a data structure being a dataframe). It takes commas in text and splits them into new columns. This is an undesired result.

Does anyone have some advice on how this can be addressed.

Thank you,
Chey

daniel · November 8, 2016 9:23AM

You may need to check if it runs in standard R like that as well, you may have some setting in R studio correcting it. In which case you may need to enable that same setting in your R script or add lines into the script like search/replace to clean out commas first.

Another way is to define the character separation to tab or pipe if possible.

nils · November 8, 2016 10:51AM

Hi Cheyam,

Thanks for letting us know. I'm looking into it and will report back once I know what is going on.

Best,
-Nils

Cheyam · November 8, 2016 11:23AM

Hi Daniel,

Just ran and checked the dimensions in normal R and it seems to run fine (i.e. no values getting pushed into additional columns). I'm guessing maybe during the conversion between a dataframe in R and Omniscope, the package used by Omniscope flattens the data into a comma separated piece of text and then reads it in as a csv? I have made a workaround by enclosing all cell values in double quotes and then removing these with the Omni "Search/Replace" block after the data is retrieved.

Thanks for looking into this Nils. Let me know if you find out anything.

Thank you,
Chey

carlosmartinmari · December 22, 2016 11:31AM

You could use a workaround in the meantime... replacing the commas by some combination of characters, which I guess the importing treats as a csv file and that's where the problem comes from, and later replace it again in visokio with a search/replace block.

	Final line of R code and dimensions of dataframe Output_Data.PNG	7K
	Data being split up into additional (now 24 cols vs 22) columns somewhere between dataframe and Omni table.PNG	26K

Welcome!

Categories

Ideas Parade

Tagged