-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[io] csv parser cannot parse arraymancer csv files #530
Comments
Vindaar
added a commit
to Vindaar/Arraymancer
that referenced
this issue
Oct 26, 2021
Merged
Vindaar
added a commit
that referenced
this issue
Oct 27, 2021
* change CSV parser to directly parse into a Tensor Otherwise we would have to copy the `seq` we parse into for mem copyable types after parsing. * [io] replace CSVParser based line counter by memfiles counter * [io] add simple readCsv test & add note to docstring about #530 * [io] extend tests with semicolon example, fixup empty line test * [io] remove TODO note * fix line counting for quoted fields in CSV files * [tests] add test case for quoted field in CSV file
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Related to #162 but deserves to be its own issue.
Given the existence of
readCsv
andtoCsv
it's not unexpected to think that arraymancer could parse its own csv files. Unfortunately, this is not possible as the csv files generated by arraymancer contain the dimension information for each index as two columns. Thus, parsing an arraymancer csv file leads to a(N+1)xM
tensor, whereN
is the number of dimensions of the original tensor and M the size of the tensor.We should have an option in
readCsv
isTensor
or something like this that knows about the arraymancer layout and parses it back to the original layout.The text was updated successfully, but these errors were encountered: