Best way to iterate through model's fields to retrieve type?

[GregDungca]

From here #2322, @PrettyWood suggested that I use __fields__. I notice that there's a type_ and outer_type_ attribute on the ModelField instance that I can use. Ideally I'd simply be able to retrieve the type defined in the model definition.

I have some follow-up questions:

1. Is outer_type_ always guaranteed to be the type defined for the field?
2. Is type_ always the "inner type"? ex. str in List[str]?
3. How does the value for type in the __repr__() representation of ModelField get populated?
4. An alternate approach is using get_type_hints() from the typing module. I like the format that it outputs (single type for object for the field), and I don't have to access private attributes. I was worried though that it might output some fields internal to Pydantic or something unexpected.

Some code for reference:

In [79]: class ExampleModel(BaseModel):
    ...:     some_list_of_ints_field: List[int]
    ...:     some_list_of_str_field: List[str]
    ...:     some_optional_field: Optional[str]
    ...:     some_dict_field: Dict[str, str]
    ...:

In [80]: from typing import get_type_hints

In [81]: get_type_hints(ExampleModel)
Out[81]:
{'some_list_of_ints_field': typing.List[int],
 'some_list_of_str_field': typing.List[str],
 'some_optional_field': typing.Union[str, NoneType],
 'some_dict_field': typing.Dict[str, str]}

In [82]: ExampleModel.__fields__
Out[82]:
{'some_list_of_ints_field': ModelField(name='some_list_of_ints_field', type=List[int], required=True),
 'some_list_of_str_field': ModelField(name='some_list_of_str_field', type=List[str], required=True),
 'some_optional_field': ModelField(name='some_optional_field', type=Optional[str], required=False, default=None),
 'some_dict_field': ModelField(name='some_dict_field', type=Mapping[str, str], required=True)}

In [86]: [(f.type_, f.outer_type_) for _, f in ExampleModel.__fields__.items()]
Out[86]:
[(int, typing.List[int]),
 (str, typing.List[str]),
 (str, str),
 (str, typing.Dict[str, str])]

[samuelcolvin]

Best solution at this level is to read the code, it's just python and it's always the true reference. If you still have questions, feel free to ask.

[gdungca-fn]

It looks like outer_type_ will be the original type except for the Optional case. https://github.com/samuelcolvin/pydantic/blame/master/pydantic/fields.py#L519 This will work for me.

What does the trailing underscore convention in outer_type_ and type_ signify? Internal variable? I feel a bit uneasy about relying on these and __fields__ in my implementation. That's why I was considering using typing.get_type_hints() instead.

[PrettyWood]

Using get_type_hints should work if you just want the type but you'll get extra annotations __slots__, private attributes, ...
Using outer_type_ is problematic for Optional and could be solved by using allow_none but it feels hacky.
Personally I would simply use a mix of both like this

from typing import get_type_hints, Any, Dict, Optional, Type

from pydantic import BaseModel


class A(BaseModel):
    x: int

class B(A):
    y: int

class M(BaseModel):
    z: int

class N(B, M):
    u: Optional[str] = 'qwe'

    _v: int

    class Config:
        underscore_attrs_are_private = True


print(get_type_hints(A))
print(get_type_hints(B))
print(get_type_hints(M))
print(get_type_hints(N))
# {'__slots__': typing.Tuple[str, ...], 'x': <class 'int'>}
# {'__slots__': typing.Tuple[str, ...], 'x': <class 'int'>, 'y': <class 'int'>}
# {'__slots__': typing.Tuple[str, ...], 'z': <class 'int'>}
# {'__slots__': typing.Tuple[str, ...], 'z': <class 'int'>, 'x': <class 'int'>, 'y': <class 'int'>, 'u': typing.Union[str, NoneType], '_v': <class 'int'>}


def get_fields_annotations(m: Type[BaseModel]) -> Dict[str, Any]:
    default_annotations = get_type_hints(m)
    return {field_name: default_annotations[field_name] for field_name in m.__fields__}

print(get_fields_annotations(A))
print(get_fields_annotations(B))
print(get_fields_annotations(M))
print(get_fields_annotations(N))
# {'x': <class 'int'>}
# {'x': <class 'int'>, 'y': <class 'int'>}
# {'z': <class 'int'>}
# {'z': <class 'int'>, 'x': <class 'int'>, 'y': <class 'int'>, 'u': typing.Union[str, NoneType]}

Hope it helps!

[gdungca-fn]

@PrettyWood thanks! Very nice and creative workaround. I like that it overcomes the shortcomings of both approaches.

[samuelcolvin]

@gdungca-fn can you mark this as answered.