Output of serialization and PK extraction does not match Primary type for composite PK

[RDSM-CM]

Describe the bug
I'm not sure if this is a bug or if I'm not understanding these types and methods correctly.

The actual output of serialization and primary key extraction does not match the Primary type for a composite primary key.

Utils.extractPK() should return Primary but for a composite primary key, it returns an object that contains the primary key components.

WrappedEntity.toJSON()/toPOJO() should return EntityDTO with Refs transformed into Primarys, but the output does not match the expected EntityDTOs. I've tried it with and without populating relations.

To Reproduce

1. Define entities:

@Entity()
export class Category {
  @PrimaryKey()
  public id: string

  @OneToMany(() => Article, (attr) => attr.category, {
    cascade: [Cascade.ALL],
  })
  public articles = new Collection<Article>(this)
}

@Entity()
export class Article {
  @PrimaryKey()
  public id: string

  @ManyToOne(() => Category, {
    primary: true,
    ref: true,
  })
  public category: Ref<Category>

  public [PrimaryKeyType]?: [string, string]

  @OneToMany(() => ArticleAttribute, (attr) => attr.article, {
    cascade: [Cascade.ALL],
  })
  public attributes = new Collection<ArticleAttribute>(this)
}

@Entity()
export class ArticleAttribute {
  @PrimaryKey()
  public id: string

  @ManyToOne(() => Article, {
    primary: true,
    ref: true,
  })
  public article: Ref<Article, "id" | "category">

  public [PrimaryKeyType]?: [string, [string, string]]
}

2. PK and plain entity types for reference:

// type CategoryPK = string
export type CategoryPK = Primary<Category>

// type ArticlePK = readonly [string, string]
export type ArticlePK = Primary<Article>

// type ArticleAttributePK = readonly [string, [string, string]]
export type ArticleAttributePK = Primary<ArticleAttribute>

/**
 type PlainCategory = {
    id: string;
    articles: EntityDTO<Article>[];
 }
 */
export type PlainCategory = EntityDTO<Category>

/**
 type PlainArticle = {
    id: string;
    category: string;
    attributes: EntityDTO<ArticleAttribute>[];
 }
 */
export type PlainArticle = EntityDTO<Article>

/**
 type PlainArticleAttribute = {
    id: string;
    article: readonly [string, string];
 }
 */
export type PlainArticleAttribute = EntityDTO<ArticleAttribute>

3. Create entities:

    const category = new Category()
    category.id = "category_1"

    const article = new Article()
    article.id = "article_1"

    const articleAttribute = new ArticleAttribute()
    articleAttribute.id = "articleAttribute_1"

    category.articles.add(article)
    article.attributes.add(articleAttribute)

Expected vs actual behavior

1. Utils.extractPK()

    const categoryPK = Utils.extractPK<Category>(category)

    // 'category_1' as expected
    console.dir(categoryPK)

    // true as expected
    console.dir(Utils.isPrimaryKey<Category>(categoryPK, false))

    const articlePK = Utils.extractPK<Article>(article)

    // { id: 'article_1', category: 'category_1' } does not match Primary<Article>
    console.dir(articlePK)

    // false despite being the result of extractPK
    console.dir(Utils.isPrimaryKey<Article>(articlePK, true))

    // true since it matches Primary<Article>
    console.dir(Utils.isPrimaryKey<Article>(["article_2", "category_2"], true))

    const articleAttributePK =
      Utils.extractPK<ArticleAttribute>(articleAttribute)

    /**
     {
      id: 'articleAttribute_1',
      article: { id: 'article_1', category: 'category_1' }
     }
     * does not match Primary<ArticleAttribute>
     */
    console.dir(articleAttributePK)

    // false despite being the result of extractPK
    console.dir(Utils.isPrimaryKey<ArticleAttribute>(articleAttributePK, true))

    // true since it matches Primary<ArticleAttribute>
    console.dir(
      Utils.isPrimaryKey<ArticleAttribute>(
        ["articleAttribute_2", ["article_2", "category_2"]],
        true,
      ),
    )

2. WrappedEntity.toJSON() / toPOJO()

Unpopulated:

    const wrappedCategory = wrap(category)

    // { id: 'category_1', articles: [ 'article_1' ] }
    console.dir(wrappedCategory.toJSON(), { depth: 6 })

    /**
     {
      id: 'category_1',
      articles: [
        {
          id: 'article_1',
          category: { id: 'category_1' },
          attributes: [
            {
              id: 'articleAttribute_1',
              article: { id: 'article_1', category: { id: 'category_1' } }
            }
          ]
        }
      ]
     }
     */
    console.dir(wrappedCategory.toPOJO(), { depth: 6 })

    const wrappedArticle = wrap(article)

    /**
     {
      id: 'article_1',
      category: { id: 'category_1', articles: [ 'article_1' ] },
      attributes: [ 'articleAttribute_1' ]
     }
     */
    console.dir(wrappedArticle.toJSON(), { depth: 6 })

    /**
     {
      id: 'article_1',
      category: {
        id: 'category_1',
        articles: [
          {
            id: 'article_1',
            attributes: [ { id: 'articleAttribute_1', article: { id: 'article_1' } } ]
          }
        ]
      },
      attributes: [
        {
          id: 'articleAttribute_1',
          article: {
            id: 'article_1',
            category: { id: 'category_1', articles: [ { id: 'article_1' } ] }
          }
        }
      ]
     }
     */
    console.dir(wrappedArticle.toPOJO(), { depth: 6 })

    const wrappedArticleAttribute = wrap(articleAttribute)

    /**
     {
      id: 'articleAttribute_1',
      article: {
        id: 'article_1',
        category: { id: 'category_1', articles: [ 'article_1' ] },
        attributes: [ 'articleAttribute_1' ]
      }
     }
     */
    console.dir(wrappedArticleAttribute.toJSON(), { depth: 6 })

    /**
     {
      id: 'articleAttribute_1',
      article: {
        id: 'article_1',
        category: {
          id: 'category_1',
          articles: [
            {
              id: 'article_1',
              attributes: [ { id: 'articleAttribute_1' } ]
            }
          ]
        },
        attributes: [ { id: 'articleAttribute_1' } ]
      }
    }
     */
    console.dir(wrappedArticleAttribute.toPOJO(), { depth: 6 })

Populated:

    category.articles.populated(true)
    article.category.populated(true)
    article.attributes.populated(true)
    articleAttribute.article.populated(true)

    /**
     {
      id: 'category_1',
      articles: [
        {
          id: 'article_1',
          category: { id: 'category_1' },
          attributes: [
            {
              id: 'articleAttribute_1',
              article: { id: 'article_1', category: { id: 'category_1' } }
            }
          ]
        }
      ]
     }
     */
    console.dir(wrappedCategory.toJSON(), { depth: 6 })

    /**
     {
      id: 'category_1',
      articles: [
        {
          id: 'article_1',
          category: { id: 'category_1' },
          attributes: [
            {
              id: 'articleAttribute_1',
              article: { id: 'article_1', category: { id: 'category_1' } }
            }
          ]
        }
      ]
     }
     */
    console.dir(wrappedCategory.toPOJO(), { depth: 6 })

    /**
     {
      id: 'article_1',
      category: {
        id: 'category_1',
        articles: [
          {
            id: 'article_1',
            attributes: [ { id: 'articleAttribute_1', article: { id: 'article_1' } } ]
          }
        ]
      },
      attributes: [
        {
          id: 'articleAttribute_1',
          article: {
            id: 'article_1',
            category: { id: 'category_1', articles: [ { id: 'article_1' } ] }
          }
        }
      ]
     }
     */
    console.dir(wrappedArticle.toJSON(), { depth: 6 })

    /**
     {
      id: 'article_1',
      category: {
        id: 'category_1',
        articles: [
          {
            id: 'article_1',
            attributes: [ { id: 'articleAttribute_1', article: { id: 'article_1' } } ]
          }
        ]
      },
      attributes: [
        {
          id: 'articleAttribute_1',
          article: {
            id: 'article_1',
            category: { id: 'category_1', articles: [ { id: 'article_1' } ] }
          }
        }
      ]
     }
     */
    console.dir(wrappedArticle.toPOJO(), { depth: 6 })

    /**
     {
      id: 'articleAttribute_1',
      article: {
        id: 'article_1',
        category: {
          id: 'category_1',
          articles: [
            {
              id: 'article_1',
              attributes: [ { id: 'articleAttribute_1' } ]
            }
          ]
        },
        attributes: [ { id: 'articleAttribute_1' } ]
      }
     }
     */
    console.dir(wrappedArticleAttribute.toJSON(), { depth: 6 })

    /**
     {
      id: 'articleAttribute_1',
      article: {
        id: 'article_1',
        category: {
          id: 'category_1',
          articles: [
            {
              id: 'article_1',
              attributes: [ { id: 'articleAttribute_1' } ]
            }
          ]
        },
        attributes: [ { id: 'articleAttribute_1' } ]
      }
     }
     */
    console.dir(wrappedArticleAttribute.toPOJO(), { depth: 6 })

Versions

Dependency	Version
node	14.17.3
typescript	4.9.4
@mikro-orm/core	5.6.8
@mikro-orm/mysql	5.6.8

[B4nan]

Both Utils.extractPK and Utils.isPrimaryKey are private/internal APIs, you shouldn't be using them at all - will mark them as internal so it's clear. And nope, they don't play together, they are not supposed to. Both are rather legacy methods I would very much like to get rid of, as they don't work with entity metadata.

If you want to get a PK out of an entity instance, do wrap(entity).getPrimaryKey() instead.

Regarding the Primary type, would be good if you can try v6, I made a lot of changes there connected to PK inference. But it won't be always correct, the type is meant to be used for the FilterQuery mainly, which supports various ways on its own. Primary type for composite key is represented as a tuple, not an object, but FilterQuery supports both.

I don't really understand your "Expected vs actual behavior" for the serialization part, the surrounding code (that is missing in there) is very important for serialization - it depends on how you load the entity, not just on the category.articles.populated(true) calls, these are inferred based on the populate option when calling em.find.

WrappedEntity.toJSON()/toPOJO() should return EntityDTO with Refs transformed into Primarys, but the output does not match the expected EntityDTOs. I've tried it with and without populating relations.

I think we do this on purpose, the composite FK is always represented as object in DTO, as the tuple is only internal ORM thing (so we can distinguish a FK from an entity object), and DTO is escaping these ORM-specific boundaries. I guess this part should be fixable, will see what I can do in v6.

[RDSM-CM]

Both Utils.extractPK and Utils.isPrimaryKey are private/internal APIs, you shouldn't be using them at all - will mark them as internal so it's clear. And nope, they don't play together, they are not supposed to. Both are rather legacy methods I would very much like to get rid of, as they don't work with entity metadata.

If you want to get a PK out of an entity instance, do wrap(entity).getPrimaryKey() instead.

Thank you for the clarification.

Regarding the Primary type, would be good if you can try v6, I made a lot of changes there connected to PK inference.

I can't upgrade to Node 16 at the moment, so I can't try v6 at this time.

One of my issues is that Primary does not represent the actual output of PK extraction and entity serialization. Whether the primary key is a tuple or an object is not that important to me, but I would like to have a type that actually represents it.
With the current (5.6.8) version, wrappedArticleAttribute.getPrimaryKey() returns the primary key as an object, but the return type of the method claims it's a tuple.
Reference.createFromPK(ArticleAttribute, pk) seems to accept both object and tuple, but if I want to manually build a Primary to pass to it, it has to be a tuple.

I don't really understand your "Expected vs actual behavior" for the serialization part, the surrounding code (that is missing in there) is very important for serialization - it depends on how you load the entity, not just on the category.articles.populated(true) calls, these are inferred based on the populate option when calling em.find.

In my example I'm serializing the entities right after creating them. I tried the manual populate() mainly to see if that would affect what I was trying to do, see below:

WrappedEntity.toJSON()/toPOJO() should return EntityDTO with Refs transformed into Primarys, but the output does not match the expected EntityDTOs. I've tried it with and without populating relations.

I think we do this on purpose, the composite FK is always represented as object in DTO, as the tuple is only internal ORM thing (so we can distinguish a FK from an entity object), and DTO is escaping these ORM-specific boundaries. I guess this part should be fixable, will see what I can do in v6.

I want to serialize Refs as their primary key, regardless of whether they're populated. If EntityDTO in v6 accurately reflects that those primary keys will be objects if they are composite primary keys, that's fine.

If v6 makes the Primary and EntityDTO types more accurate to what the methods actually return, that would be great.

[B4nan]

The Primary type will stay as is, it should be a tuple for composite keys, but it shouldn't be used in EntityDTO (and maybe other places) probably - for composite FKs it should be rather replaced with EntityDTO<Child> limited to the PKs only.