Fix pydantic v2 pydantic_model_creator nullable field not optional

[waketzheng]

Fix pydantic v2 pydantic_model_creator nullable field not optional(#1454)

How Has This Been Tested?

1. Tests for examples/fastapi passed
2. Tests for examples/blacksheep not work, because it does not support pydantic V2

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added the changelog accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[coveralls]

Pull Request Test Coverage Report for Build 6497193654

• 4 of 5 (80.0%) changed or added relevant lines in 1 file are covered.
• No unchanged relevant lines lost coverage.
• Overall first build on fix-pydantic-model-creator-error at 88.67%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
tortoise/contrib/pydantic/creator.py	4	5	80.0%

Totals
Change from base Build 6479987928:	88.7%
Covered Lines:	5756
Relevant Lines:	6408

---

💛 - Coveralls

[plusiv]

This PR solve a lot of things... I think it should be taken as hotfix @long2ice .

[long2ice]

Thanks! Since 0.20.0 was released, so changelog should be in new section 0.20.1

[plusiv]

I was reviewing @waketzheng 's code. And, I humbly submit that in order to set as nullable ForeignKeyFieldInstance and OneToOneFieldInstance, the following portion of the code:

tortoise-orm/tortoise/contrib/pydantic/creator.py

Lines 352 to 365 in ba593c3

	if (
	field_type is relational.ForeignKeyFieldInstance
	or field_type is relational.OneToOneFieldInstance
	or field_type is relational.BackwardOneToOneRelation
	):
	is_to_one_relation = True
	model = get_submodel(fdesc["python_type"])
	if model:
	if fdesc.get("nullable"):
	json_schema_extra["nullable"] = True
	if fdesc.get("nullable") or field_default is not None:
	model = Optional[model] # type: ignore
	properties[fname] = model

Should be set to:

        if (
            field_type is relational.ForeignKeyFieldInstance
            or field_type is relational.OneToOneFieldInstance
            or field_type is relational.BackwardOneToOneRelation
        ):
            is_to_one_relation = True
            model = get_submodel(fdesc["python_type"])
            if model:
                if fdesc.get("nullable"):
                    json_schema_extra["nullable"] = True
                    # set default_factory to None in the field config
                    fconfig["default_factory"] = lambda: None
                if fdesc.get("nullable") or field_default is not None:
                    model = Optional[model]  # type: ignore

                # create property with that config
                properties[fname] = (model, Field(**fconfig))

[waketzheng]

@plusiv After set default_factory to None in the field config, unittest failed at tests/contrib/test_pydantic.py

[plusiv]

properties[fname] = (model, Field(**fconfig))

Hi again @waketzheng !

I've cloned your changes and tested with the changes that I suggested before, and figure out that the reason is because the hardcoded schema used in the test assertion has so many errors since the relational fields are not set in the proper way to make nullable relationships fields/objects in the schema.

To explain my point, let's take a look to the models Event and Reporter

tortoise-orm/tests/testmodels.py

Lines 74 to 113 in c030c9c

	class Reporter(Model):
	"""Whom is assigned as the reporter"""
	id = fields.IntField(pk=True)
	name = fields.TextField()
	events: fields.ReverseRelation["Event"]
	class Meta:
	table = "re_port_er"
	def __str__(self):
	return self.name
	class Event(Model):
	"""Events on the calendar"""
	event_id = fields.BigIntField(pk=True)
	#: The name
	name = fields.TextField()
	#: What tournaments is a happenin'
	tournament: fields.ForeignKeyRelation["Tournament"] = fields.ForeignKeyField(
	"models.Tournament", related_name="events"
	)
	reporter: fields.ForeignKeyNullableRelation[Reporter] = fields.ForeignKeyField(
	"models.Reporter", null=True
	)
	participants: fields.ManyToManyRelation["Team"] = fields.ManyToManyField(
	"models.Team", related_name="events", through="event_team", backward_key="idEvent"
	)
	modified = fields.DatetimeField(auto_now=True)
	token = fields.TextField(default=generate_token)
	alias = fields.IntField(null=True)
	class Meta:
	ordering = ["name"]
	def __str__(self):
	return self.name

Since the field alias of the model Event is set to be null, the schema looks like:

tortoise-orm/tests/contrib/test_pydantic.py

Lines 191 to 198 in c030c9c

	"alias": {
	"anyOf": [
	{"maximum": 2147483647, "minimum": -2147483648, "type": "integer"},
	{"type": "null"},
	],
	"nullable": True,
	"title": "Alias",
	},

But, with the nullable reporter field of the same Event model doesn't happen the same in its schema:

tortoise-orm/tests/contrib/test_pydantic.py

Lines 84 to 99 in c030c9c

	"pydantic__main__tests__testmodels__Reporter__leaf": {
	"additionalProperties": False,
	"description": "Whom is assigned as the reporter",
	"properties": {
	"id": {
	"maximum": 2147483647,
	"minimum": 1,
	"title": "Id",
	"type": "integer",
	},
	"name": {"title": "Name", "type": "string"},
	},
	"required": ["id", "name"],
	"title": "Reporter",
	"type": "object",
	},

Notice that the element {"type": "null"}, in the anyOf property list and the property "nullable": True, are missing in the schema definition.

I think that re-generating the hardcoded schema with the changes that I suggested before and replacing the old ones with the new ones, should make the test run without problems. I could help with this if you like.

[plusiv]

Hello again @waketzheng !

I was reviewing the function pydantic_model_creator, and I'm wondering why did you just added IntField and TextField in:

https://github.com/waketzheng/tortoise-orm/blob/ba593c3888741122b832e19a45d4504a951fa7ae/tortoise/contrib/pydantic/creator.py#L415-L424

This is causing setting as required IntFields and TextFields, even if marked as nullable.

Note: I'm just trying to understand the project logic in order to collaborate in a future.

[waketzheng]

properties[fname] = (model, Field(**fconfig))

Hi again @waketzheng !

I've cloned your changes and tested with the changes that I suggested before, and figure out that the reason is because the hardcoded schema used in the test assertion has so many errors since the relational fields are not set in the proper way to make nullable relationships fields/objects in the schema.

To explain my point, let's take a look to the models Event and Reporter

tortoise-orm/tests/testmodels.py

Lines 74 to 113 in c030c9c

	class Reporter(Model):
	"""Whom is assigned as the reporter"""
	id = fields.IntField(pk=True)
	name = fields.TextField()
	events: fields.ReverseRelation["Event"]
	class Meta:
	table = "re_port_er"
	def __str__(self):
	return self.name
	class Event(Model):
	"""Events on the calendar"""
	event_id = fields.BigIntField(pk=True)
	#: The name
	name = fields.TextField()
	#: What tournaments is a happenin'
	tournament: fields.ForeignKeyRelation["Tournament"] = fields.ForeignKeyField(
	"models.Tournament", related_name="events"
	)
	reporter: fields.ForeignKeyNullableRelation[Reporter] = fields.ForeignKeyField(
	"models.Reporter", null=True
	)
	participants: fields.ManyToManyRelation["Team"] = fields.ManyToManyField(
	"models.Team", related_name="events", through="event_team", backward_key="idEvent"
	)
	modified = fields.DatetimeField(auto_now=True)
	token = fields.TextField(default=generate_token)
	alias = fields.IntField(null=True)
	class Meta:
	ordering = ["name"]
	def __str__(self):
	return self.name

Since the field alias of the model Event is set to be null, the schema looks like:

tortoise-orm/tests/contrib/test_pydantic.py

Lines 191 to 198 in c030c9c

	"alias": {
	"anyOf": [
	{"maximum": 2147483647, "minimum": -2147483648, "type": "integer"},
	{"type": "null"},
	],
	"nullable": True,
	"title": "Alias",
	},

But, with the nullable reporter field of the same Event model doesn't happen the same in its schema:

tortoise-orm/tests/contrib/test_pydantic.py

Lines 84 to 99 in c030c9c

	"pydantic__main__tests__testmodels__Reporter__leaf": {
	"additionalProperties": False,
	"description": "Whom is assigned as the reporter",
	"properties": {
	"id": {
	"maximum": 2147483647,
	"minimum": 1,
	"title": "Id",
	"type": "integer",
	},
	"name": {"title": "Name", "type": "string"},
	},
	"required": ["id", "name"],
	"title": "Reporter",
	"type": "object",
	},

Notice that the element {"type": "null"}, in the anyOf property list and the property "nullable": True, are missing in the schema definition.

I think that re-generating the hardcoded schema with the changes that I suggested before and replacing the old ones with the new ones, should make the test run without problems. I could help with this if you like.

@plusiv You are right! Unittest should be updated.

[waketzheng]

Hello again @waketzheng !

I was reviewing the function pydantic_model_creator, and I'm wondering why did you just added IntField and TextField in:

https://github.com/waketzheng/tortoise-orm/blob/ba593c3888741122b832e19a45d4504a951fa7ae/tortoise/contrib/pydantic/creator.py#L415-L424

This is causing setting as required IntFields and TextFields, even if marked as nullable.

Note: I'm just trying to understand the project logic in order to collaborate in a future.

This line is to respect to unittest and compare with old version logic, after tests code updated, if can be removed.

[plusiv]

This line is to respect to unittest and compare with old version logic, after tests code updated, if can be removed.

Good! Seems razonable now!

@plusiv You are right! Unittest should be updated.

Got it, I think it can be updated on this PR, or you think a new one is necessary @long2ice ??

I am willing to assist if you agree.

[plusiv]

Hey everyone, Did this PR got stuck? I think those are very importan changes.

[long2ice]

Thanks! Just need resolve conflicts.

[waketzheng]

Thanks! Just need resolve conflicts.

conflicts fixed, but need you to rerun ci action manually.

[long2ice]

Thanks!