-
-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Supporte Metadata like description #373
Comments
I have investigated this use case. As far as I understand my code, the current implementation of Valibot does not limit this functionality. This feature can already be added by third party libraries by providing pipeline actions like import * as v from 'valibot';
/**
* Index metadata type.
*/
type IndexMetadata<TInput> = {
async: false;
_parse(input: TInput): v.PipeActionResult<TInput>;
type: 'index';
};
/**
* Creates a pipeline metadata action to indicate that the field should be indexed.
*
* @returns A metadata action.
*/
function index<TInput>(): IndexMetadata<TInput> {
return {
type: 'index',
async: false,
_parse(input) {
return v.actionOutput(input);
},
};
}
/**
* Description metadata type.
*/
type DescriptionMetadata<TInput> = {
async: false;
_parse(input: TInput): v.PipeActionResult<TInput>;
type: 'description';
description: string;
};
/**
* Creates a pipeline metadata action that adds a description the field.
*
* @param description The description to be added.
*
* @returns A metadata action.
*/
function description<TInput>(description: string): DescriptionMetadata<TInput> {
return {
type: 'description',
description,
async: false,
_parse(input) {
return v.actionOutput(input);
},
};
}
// Create user entity schema
const UserEntitySchema = v.object({
name: v.string([
index(),
description('The name of the user.'),
v.maxLength(30),
]),
// ...
});
// Read index metadata
const shouldBeIndexed = !!UserEntitySchema.entries.name.pipe?.find(
(action) => 'type' in action && action.type === 'index'
); The more important thing for us as a community is to decide if this functionality belongs in the scope of Valibot, and if the library should get a first-class metadata API. If we decide to do so, we should discuss if we want to add this to our current pipeline feature as shown above, if we want to rename and reimplement the pipeline feature so that it is not necessary to include I look forward to hearing what you think. |
@fabian-hiller You're absolutely right. Over the past few days, I've built the valibot-mikro library for building mikro entities with the help of valibot. /**
* Mikro property meta.
*/
export type PropertyMeta<Entity = any, TInput = any> = BaseValidation<TInput> & {
/**
* The validation type.
*/
type: "mikro_property"
/**
* The property meta.
*/
meta?: Partial<EntitySchemaProperty<TInput, Entity>>
} This works fine, but it feels a little twisted. I strongly agree with rename and reimplement the pipeline feature to support pure metadata. In my practice, I realized that there should be no need for an abstract const User = object({
name: number([property(), index(), columnName('a_strange_column_name')])
}) But by using metadata as a wrapper, the code starts to get confused, const User = object({
name: metadata(number(), [property(), index(), columnName('a_strange_column_name')])
}) Such an approach may could be even more messy: const User = object({
name: columnName(property(index(number())), 'a_strange_column_name')
}) As a community, could we also consider building in some generic metadata like |
I also noticed that not every schema can accept pipe arguments, such as Do you use a plan to complement the pipe parameter for all schemas? /**
* Base schema type.
*/
export type BaseSchema<TInput = any, TOutput = TInput> = {
/**
* store the metadata.
*/
meta?: BaseMetadata[];
/**
* Whether it's async.
*/
async: false;
/**
* Parses unknown input based on its schema.
*
* @param input The input to be parsed.
* @param info The parse info.
*
* @returns The parse result.
*
* @internal
*/
_parse(input: unknown, info?: ParseInfo): SchemaResult<TOutput>;
/**
* Input and output type.
*
* @internal
*/
_types?: { input: TInput; output: TOutput };
}; The advantage of this is that the metadate can be completely ignored when running validation for better performance. |
I like the idea of putting the metadata under its own key. However, this would require that we put the metadata in a different array to separate from the pipe, or we would have to filter and process the pipeline when creating the schema to extract the metadata. Also, I think it would be nice to access the metadata directly via const User = object({
// Note: This would require us to always specify the pipeline, even if it is empty
name: string([], [index(), description('Lorem ipsum')]),
age: number([minValue(10)], [index(), description('Lorem ipsum')]),
}) Another idea is to provide the metadata directly as an object instead of an array of functions. I suspect this will also result in the smallest bundle size. const User = object({
name: string([], { index: true, description: 'Lorem ipsum' }),
age: number([minValue(10)], { index: true, description: 'Lorem ipsum' }),
}) |
I totally agree with the use of key-value pairs to store metadata. const User = object({
name: string([index(), description('Lorem ipsum')]),
age: number([minValue(10), index(), description('Lorem ipsum')]),
}) To do this we need to do some extra work when creating the schema, which fortunately is fairly simple. Also, I noticed that const User = object({
name: string(message("The name is illegal"), index(), description('Lorem ipsum')),
age: number(minValue(10), index(), description('Lorem ipsum')),
}) This will result in a huge break change. const User = object({
name: string({ description:"Lorem ipsum", index: true }),
age: number({ message:"too young", description:"Lorem ipsum", index: true },[minValue(10)]),
}) Taking into account compatibility, development experience and packaging size, I think this is a very good solution. We also need to expose the // valibot
export interface Metadata {
description?: string
message?: string
} Users need to add additional global type declarations when installing third-party libraries: // env.d.ts
import { Metadata } from 'valibot'
import { MikroMetadata } from 'valibot-mikro'
declare module 'valibot' {
export interface Metadata extends MikroMetadata {}
} |
Thanks again for the details! I will look into this and get back to you with the results. |
I agree that the DX is very nice this way. The downside for me is the internal implementation because it requires us to filter and process the pipeline on schema creation. This would increase the bundle size and slow down the startup performance. Since I expect that the metadata feature will only be used by a small fraction of users, I am not sure if I want to go this way.
I see your point. On the other hand,
When I first implemented Valibot in July 2023, I considered this approach. In the end I decided against it because it makes it much harder to distinguish the pipe arguments and limits the API for complex schema functions like
This might work, and I will consider it. It might make it harder to handle and process custom error messages, which is an important feature of the library. Also, I am not sure if
Great idea! I agree! Although I have argued against your suggestions, this comment is not a final decision, and I welcome your feedback. In the end, my goal is to work with the community to create a great schema library. If we decide to add the
const User = object({
name: string({ index: true, description: 'Lorem ipsum' }),
age: number([minValue(10)], { index: true, description: 'Lorem ipsum' }),
}) |
I strongly agree with adding the metadata object as the last optional argument of each schema function. Click to show code/**
* Creates an object schema.
*
* @param entries The object entries.
* @param metadata The schema metadata.
*
* @returns An object schema.
*/
export function object<TEntries extends ObjectEntries>(
entries: TEntries,
metadata?: Metadata
): ObjectSchema<TEntries>;
/**
* Creates an object schema.
*
* @param entries The object entries.
* @param pipe A validation and transformation pipe.
* @param metadata The schema metadata.
*
* @returns An object schema.
*/
export function object<TEntries extends ObjectEntries>(
entries: TEntries,
pipe?: Pipe<ObjectOutput<TEntries, undefined>>,
metadata?: Metadata
): ObjectSchema<TEntries>;
/**
* Creates an object schema.
*
* @param entries The object entries.
* @param message The error message.
* @param pipe A validation and transformation pipe.
* @param metadata The schema metadata.
*
* @returns An object schema.
*/
export function object<TEntries extends ObjectEntries>(
entries: TEntries,
message?: ErrorMessage,
pipe?: Pipe<ObjectOutput<TEntries, undefined>>,
metadata?: Metadata
): ObjectSchema<TEntries>;
/**
* Creates an object schema.
*
* @param entries The object entries.
* @param rest The object rest.
* @param pipe A validation and transformation pipe.
* @param metadata The schema metadata.
*
* @returns An object schema.
*/
export function object<
TEntries extends ObjectEntries,
TRest extends BaseSchema | undefined
>(
entries: TEntries,
rest: TRest,
pipe?: Pipe<ObjectOutput<TEntries, TRest>>,
metadata?: Metadata
): ObjectSchema<TEntries, TRest>;
/**
* Creates an object schema.
*
* @param entries The object entries.
* @param rest The object rest.
* @param message The error message.
* @param pipe A validation and transformation pipe.
* @param metadata The schema metadata.
*
* @returns An object schema.
*/
export function object<
TEntries extends ObjectEntries,
TRest extends BaseSchema | undefined
>(
entries: TEntries,
rest: TRest,
message?: ErrorMessage,
pipe?: Pipe<ObjectOutput<TEntries, TRest>>,
metadata?: Metadata
): ObjectSchema<TEntries, TRest>;
export function object<
TEntries extends ObjectEntries,
TRest extends BaseSchema | undefined = undefined
>(
entries: TEntries,
arg2?: Metadata | Pipe<ObjectOutput<TEntries, TRest>> | ErrorMessage | TRest,
arg3?: Metadata | Pipe<ObjectOutput<TEntries, TRest>> | ErrorMessage,
arg4?: Metadata | Pipe<ObjectOutput<TEntries, TRest>>,
arg5?: Metadata
): ObjectSchema<TEntries, TRest> This looks like it will come with more maintenance costs, other than that the solution is perfect! |
Which metadata properties should Valibot ship by default? Can you create a list for me? |
We should refer to existing popular standards such as annotations for /**
* Schema metadata type.
*/
export interface SchemaMetadata<T = any> {
/**
* The name of the schema.
*/
name?: string;
/**
* A brief description of the schema.
*/
description?: string;
/**
* The instance value of the schema should not be used and the schema may be removed in the future.
*/
deprecated?: boolean;
/**
* The `examples` is a place to provide an array of examples that validate against the schema.
*/
examples?: T[];
} Of course, we can also strictly follow the json-schema specification. Then we will have the following build-in metadata fields: /**
* Schema metadata type.
*/
export interface SchemaMetadata<T = any> {
/**
* The title of the schema.
*/
title?: string;
/**
* A brief description of the schema.
*/
description?: string;
/**
* The `examples` is a place to provide an array of examples that validate against the schema.
*/
examples?: T[];
/**
* The `readOnly` keyword indicates that the value of the instance is managed exclusively by the owning authority, and attempts by an application to modify the value of this property are expected to be ignored or rejected by that owning authority.
*/
readOnly?: boolean;
/**
* The `writeOnly` keyword indicates that the value is never present when the instance is retrieved from the owning authority.
*/
writeOnly?: boolean;
/**
* The instance value of the schema should not be used and the schema may be removed in the future.
*/
deprecated?: boolean;
} |
What about SQL properties like index, unique and primary key? |
I don't think SQL properties should be included in the valibot package itself. This is because when dealing with a specific business, the situation is complex and varied. https://sequelize.readthedocs.io/ It is quite difficult to adapt In order for valibot to be a generalized schema builder, I think it needs to be used to create various community packages such as // env.d.ts
import { SchemaMetadata } from 'valibot'
import { MikroMetadata } from 'valibot-mikro'
declare module 'valibot' {
export interface SchemaMetadata extends MikroMetadata {}
} |
I'm trying to implement this feature, and the current solution is to use metadata as the last parameter of the schema function. Here is the array() declaration/**
* Creates a array schema.
*
* @param item The item schema.
* @param pipe A validation and transformation pipe.
* @param metadata The schema metadata.
*
* @returns A array schema.
*/
export function array<TItem extends BaseSchema>(
item: TItem,
pipe?: Pipe<Output<TItem>[]>,
metadata?: SchemaMetadata<Input<TItem>>
): ArraySchema<TItem>;
/**
* Creates a array schema.
*
* @param item The item schema.
* @param message The error message.
* @param pipe A validation and transformation pipe.
* @param metadata The schema metadata.
*
* @returns A array schema.
*/
export function array<TItem extends BaseSchema>(
item: TItem,
message?: ErrorMessage,
pipe?: Pipe<Output<TItem>[]>,
metadata?: SchemaMetadata<Input<TItem>>
): ArraySchema<TItem>;
/**
* Creates a array schema.
*
* @param item The item schema.
* @param metadata The schema metadata.
*
* @returns A array schema.
*/
export function array<TItem extends BaseSchema>(
item: TItem,
metadata?: SchemaMetadata<Input<TItem>>
): ArraySchema<TItem>;
export function array<TItem extends BaseSchema>(
item: TItem,
arg2?: SchemaMetadata<Input<TItem>> | Pipe<Output<TItem>[]> | ErrorMessage,
arg3?: SchemaMetadata<Input<TItem>> | Pipe<Output<TItem>[]>,
arg4?: SchemaMetadata<Input<TItem>>
): ArraySchema<TItem> const schema2 = array(number(), 'Error', [length(1), includes(123)]); For the above code, TypeScript, gives the following error message:
But the following code works fine: const schema2 = array(number(), 'Error', [length(1), includes(123)], {});
const schema3 = array(number(), 'Error', [length(1), includes(123)], undefined); If you have a spare moment, check out my code implementation here To avoid the above, I think we should use the current This approach also makes our schema function much simpler: Here we only need two overloads/**
* Creates a array schema.
*
* @param item The item schema.
* @param pipe A validation and transformation pipe.
*
* @returns A array schema.
*/
export function array<TItem extends BaseSchema>(
item: TItem,
pipe?: Pipe<Output<TItem>[]>
): ArraySchema<TItem>;
/**
* Creates a array schema.
*
* @param item The item schema.
* @param metadata The schema metadata or error message.
* @param pipe A validation and transformation pipe.
*
* @returns A array schema.
*/
export function array<TItem extends BaseSchema>(
item: TItem,
metadata?: ErrorMessage | SchemaMetadata<Input<TItem>>,
pipe?: Pipe<Output<TItem>[]>
): ArraySchema<TItem>;
export function array<TItem extends BaseSchema>(
item: TItem,
arg2?: SchemaMetadata<Input<TItem>> | ErrorMessage | Pipe<Output<TItem>[]>,
arg3?: Pipe<Output<TItem>[]>
): ArraySchema<TItem> const schema2 = array(number(), 'Error', [length(1), includes(123)]);
const schema3 = array(number(), { message: 'Error', description: "Lorem ipsum" }, [length(1), includes(123)]); Let's compare the scenarios in detail: that is, placing
|
I would not add properties like
Weren't the SQL properties the main reason for this feature? Are the properties not standardized? I thought that a Valibot schema could then be sufficient to generate SQL commands. In general: Where and how do you plan to use this metadata feature?
How would you implement |
Yes, in fact, there is no standard.
I want to use it as the alternative of reflect-metadata There are lots of problems when declaring schema in TypeScript's
Specifically, I have two use cases:
The whole community has been trying to move away from
This is especially true in the world of GraphQL: pothos, nexus, gqtx, grats. As you can see, there are so many schema builders in the community. The problem is that the current schema builder ecosystem is separate and not as universal as I hope that |
/**
* Returns message and pipe from dynamic arguments.
*
* @param arg1 First argument.
* @param arg2 Second argument.
*
* @returns The default arguments.
*/
export function defaultArgs<TPipe extends Pipe<any> | PipeAsync<any>>(
arg1: ErrorMessage | SchemaMetadata | TPipe | undefined,
arg2: TPipe | undefined
): [ErrorMessage | undefined, TPipe | undefined, SchemaMetadata | undefined] {
if (Array.isArray(arg1)) return [undefined, arg1, undefined];
if (typeof arg1 === 'string' || typeof arg1 === 'function')
return [arg1, arg2, undefined];
return [arg1?.message, arg2, arg1];
} Compare this to the implementation when I use metadata as the last parameter: /**
* Returns message and pipe from dynamic arguments.
*
* @param args The arguments.
*
* @returns The default arguments.
*/
export function defaultArgs<TPipe extends Pipe<any> | PipeAsync<any>>(
...args: (SchemaMetadata | TPipe | ErrorMessage | undefined)[]
): [ErrorMessage | undefined, TPipe | undefined, SchemaMetadata | undefined] {
let message: ErrorMessage | undefined;
let pipe: TPipe | undefined;
let metadata: SchemaMetadata | undefined;
for (const arg of args) {
if (typeof arg === 'string' || typeof arg === 'function') {
message = arg;
} else if (Array.isArray(arg)) {
pipe = arg;
} else if (typeof arg === 'object') {
metadata = arg;
}
}
return [message, pipe, metadata];
} |
Thank you very much! I see a problem. The |
Hi, I may change the implementation of If we also remove the I would therefore prefer option 1. Nevertheless, we should also take a look at the DX. |
I could imagine that the first optional argument is always the // With message and metadata
const Schema = pipe(
string('Text ...', { description: 'Text ...' }),
minLength(1)
);
// With just metadata
const Schema = pipe(
string({ description: 'Text ...' }),
minLength(1)
); |
I prefer to use // With the new `pipe` function
const UserSchema = pipe(
object({
id: pipe(string(), primaryKey(), columnName(user_id)),
name: pipe(string(), unique()),
bio: pipe(string(), description('Text ...')),
}),
table('users')
)
// With metadata argument
const UserSchema = object(
{
id: string({ primaryKey: true, columnName: 'user_id' }),
name: string({ unique: true }),
bio: string({ description: 'Text ...' }),
},
{ tableName: 'users' }
) Another important point is that when using valibot as a pure schema builder, the ORM itself contains solid validation, and it is not very useful to validate data on top of the ORM. The only extra work for the developer is to declare the extra metadata fields while installing // env.d.ts
import { MikroSchemaMetadata } from 'valibot-mikro'
import 'valibot'
declare module 'valibot' {
export interface SchemaMetadata extends MikroSchemaMetadata {}
} |
Thank you very much for your feedback! I will get back to you as soon as I have made the proposed changes. Then we can discuss the details of the metadata feature implementation together. |
I've found that having TypeScript accurately infer the metadata type is very helpful in checking the correctness of the program. const Cat = pipe(
object({
id: pipe(string(), primaryKey(), columnName('user_id')),
name: pipe(string(), unique()),
loveFish: pipe(boolean(), description('Does the cat love fish?')),
}),
name('Cat')
);
expectTypeOf<typeof Cat['name']>().toEqualTypeOf<string>()
const CatEntity = toMikroEntity(Cat); // It should pass
const Dog = object({
id: string(),
name: string(),
loveFish: boolean(),
})
const DogEntity = toMikroEntity(Dog); // It should fail with a type error because Dog does not have a name This does not seem to be achievable using metadata arguments. So now I think pipe is a better design. Another idea is to add the export interface BaseSchema {
// ...
with<T>(modifier: (schema: this) => T): T;
// ...
}
function _with<T>(this: BaseSchema, modifier: (schema: this) => T): T {
return modifier(this);
} const Cat = object({
id: string().with(primaryKey()).with(columnName('user_id')),
name: string().with(unique()),
loveFish: boolean().with(description('Does the cat love fish?')),
}).with(name('Cat')); This is more readable than pipe, but adds a little bit of bundle size. |
I don't understand what
Can you explain this in more detail? |
I have a detailed implementation of When In order for TypeScript to do this hint, we need to carry more detailed types on the valibot schema, For example, carrying a function name(
value: string
): <T extends object>(x: T) => T & { name: string } {
return (x) => {
x.name = value
return x
}
}
// with pipe
const Cat = pipe(
object({
id: pipe(string(), primaryKey(), columnName('user_id')),
name: pipe(string(), unique()),
loveFish: pipe(boolean(), description('Does the cat love fish?')),
}),
name('Cat')
);
expectTypeOf<typeof Cat['name']>().toEqualTypeOf<string>() Using pipe, we can modify the type of schema, in this example, we've added the extra // with metadata argument
const Cat = object(
{
id: string(),
name: string(),
loveFish: boolean({ description: 'Does the cat love fish?' }),
},
{ name: Cat }
)
// Is this possible?
expectTypeOf<(typeof Cat)['name']>().toEqualTypeOf<string>()
expectTypeOf<
(typeof Cat)['entries']['loveFish']['description']
>().toEqualTypeOf<string>() |
Thank you for the details! I am quite busy with my studies at the moment. I will probably get back to you next week. |
This is still on my list. I will get back to you as soon as I have the time. |
Discussed in #368
Originally posted by xcfox January 14, 2024
Valibot is a super cool Schema Builder! I am attracted by Valibot's modular design.
I plan to use Valibot as an Entity Schema Builder for MikroORM.
So far, I am using
TypeScript
'sclass
andDecorators
to define MikroORM's Entity like this:Because TypeScript's
class
doesn't support multiple inheritance, I have to define thename
field repeatedly.Maybe I can use Valibot and custom pipelines to solve this problem:
Here I use many custom pipelines:
primaryKey
,property
,index
,unique
,description
. However, the current version of Valibot's pipeline only supportsValidation
andTransformation
. I don't think these custom pipes belong to eitherValidation
orTransformation
.Once
Metadata
is supported, Valibot has the ability to shine in all scenarios where a Schema needs to be defined, such as graphql, typegoose, typeorm and not just for data validation likezod
. Valibot's modularity, combined withMetadata
, makes for a very powerful Schema Builder! All projects using TypeScript can use Valibot to define Schemas, and the development experience will be far superior to that ofclass
andDecorators
.The text was updated successfully, but these errors were encountered: