Optional[Type] with default value None is not nullable

[jjshinobi]

Hi,

According to https://docs.pydantic.dev/latest/migration/#required-optional-and-nullable-fields
we should be able to specify the class:

class MyModel(SparkBase):

    someProperty: Optional[str] = None
...

But, if we do so, the property is not nullable because it has the default value (

pydantic-spark/src/pydantic_spark/base.py

Line 143 in d61c660

"nullable": "default" not in metadata and value.get("anyOf") is not None,

).
Note from the pydantic docs:

A field annotated as typing.Optional[T] will be required, allowing for a value of None. 
It does not mean that the field has a default value of None. (This is a breaking change from V1.)

Currently, we can only use someProperty: Optional[str] and we have to explicitly set None to the property in the object that is being validated.

[arcaputo3]

As a workaround we use someProperty: Optional[str] = Field(default_factory=lambda: None). This repo isn't being maintained as well as, for example, https://github.com/mitchelllisle/sparkdantic, so I'd take a look there as well.