Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support nested schemas #45

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Support nested schemas #45

wants to merge 2 commits into from

Conversation

jcrobak
Copy link
Owner

@jcrobak jcrobak commented Oct 1, 2016

Rather than flattening schemas and adding a '.' between paths
in the schema (e.g. foo.bar), support the schema path as a
first-class object (for schema operations, at least). This
is an experimental implementation and likely has bugs. But it
supports some simple cases.

This implementation changes behavior. Specifically:

  • DictReader() now has a flatten argument that defaults to
    False. If Flatten is false, DictReader will read nested data
    as {'foo': {'bar': 1}} instead of as {'foo.bar': 1}.
  • Likewise, this is the new default behavior for the command-line
    tool with --format json. This can be changed with --flatten.

Known issues:

  • Repetition-levels still aren't supported. A file with arrays
    will break.
  • nulls aren't interpretted at the level. (e.g.: {"foo": null}
    will be interpetted as {"foo": {"bar": null}} if foo has
    a child of bar.

Refs: #32

Since we didn't support repetition levels previously, this hadn't
been tested. But we were reading the levels in the wrong order.
Rather than flattening schemas and adding a '.' between paths
in the schema (e.g. `foo.bar`), support the schema path as a
first-class object (for schema operations, at least). This
is an experimental implementation and likely has bugs. But it
supports some simple cases.

This implementation changes behavior. Specifically:
 * `DictReader()` now has a `flatten` argument that defaults to
   `False`. If Flatten is false, DictReader will read nested data
   as `{'foo': {'bar': 1}}` instead of as `{'foo.bar': 1}`.
 * Likewise, this is the new default behavior for the command-line
   tool with `--format json`. This can be changed with `--flatten`.

Known issues:
 * Repetition-levels still aren't supported. A file with arrays
   will break.
 * nulls aren't interpretted at the level. (e.g.: `{"foo": null}`
   will be interpetted as `{"foo": {"bar": null}}` if `foo` has
   a child of `bar`.

Refs: #32
@jcrobak jcrobak mentioned this pull request Oct 1, 2016
@martindurant
Copy link

"Read repetition levels before definition levels." - thank you for pointing this out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants