Phyton Using numpy.genfromtxt to read a csv file with strings containing commas python using numpy,python use numpy array,python use numpy

You can use pandas (the becoming default library for working with dataframes (heterogeneous data) in scientific python) for this. It's read_csv can handle this. From the docs:

quotechar : string

The character to used to denote the start and end of a quoted item. Quoted items 
can include the delimiter and it will be ignored.

The default value is ". An example:

In [1]: import pandas as pd

In [2]: from StringIO import StringIO

In [3]: s="""year, city, value
   ...: 2012, "Louisville KY", 3.5
   ...: 2011, "Lexington, KY", 4.0"""

In [4]: pd.read_csv(StringIO(s), quotechar='"', skipinitialspace=True)
Out[4]:
   year           city  value
0  2012  Louisville KY    3.5
1  2011  Lexington, KY    4.0

The trick here is that you also have to use skipinitialspace=True to deal with the spaces after the comma-delimiter.

Apart from a powerful csv reader, I can also strongly advice to use pandas with the heterogeneous data you have (the example output in numpy you give are all strings, although you could use structured arrays).

Answer:1

The problem with the additional comma, np.genfromtxt does not deal with that.

One simple solution is to read the file with csv.reader() from python's csv module into a list and then dump it into a numpy array if you like.

If you really want to use np.genfromtxt, note that it can take iterators instead of files, e.g. np.genfromtxt(my_iterator, ...). So, you can wrap a csv.reader in an iterator and give it to np.genfromtxt.

That would go something like this:

import csv
import numpy as np

np.genfromtxt(("\t".join(i) for i in csv.reader(open('myfile.csv'))), delimiter="\t")

This essentially replaces on-the-fly only the appropriate commas with tabs.

Answer:2

If you are using a numpy you probably want to work with numpy.ndarray. This will give you a numpy.ndarray:

import pandas
data = pandas.read_csv('file.csv').as_matrix()

Pandas will handle the "Lexington, KY" case correctly

Answer:3



  1. python using numpy
  2. python use numpy array
  3. python use numpy library
  4. python with numpy
  5. python with numpy online
  6. python with numpy download
  7. python with numpy and pandas
  8. python with numpy and scipy

NLTK version 3.4.5. Python 3.7.4. OSX version 10.14.5. Upgrading the codebase from 2.7, started running into this issue just now. I've done a fresh no-cache reinstall of all packages and extensions, ...

NLTK version 3.4.5. Python 3.7.4. OSX version 10.14.5. Upgrading the codebase from 2.7, started running into this issue just now. I've done a fresh no-cache reinstall of all packages and extensions, ...

I'm trying to pickle an object of a (new-style) class I defined. But I'm getting the following error: >>> with open('temp/connection.pickle','w') as f: ... pickle.dump(c,f) ... Traceback ...

I'm trying to pickle an object of a (new-style) class I defined. But I'm getting the following error: >>> with open('temp/connection.pickle','w') as f: ... pickle.dump(c,f) ... Traceback ...