Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typescript Types / Zod types Error Using Firecrawl #992

Open
HYAZEUS opened this issue Dec 18, 2024 · 1 comment
Open

Typescript Types / Zod types Error Using Firecrawl #992

HYAZEUS opened this issue Dec 18, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@HYAZEUS
Copy link

HYAZEUS commented Dec 18, 2024

Describe the Bug
There appears to be a mismatch between Firecrawl's TypeScript type definitions and its API expectations for schema validation. The TypeScript types expect a Zod schema (ZodType<any, ZodTypeDef, any>), but the API expects a JSON Schema format with specific requirements for the required array property.

To Reproduce
Steps to reproduce the issue:

  1. Create a schema for Firecrawl's extract option
  2. Try to satisfy both TypeScript types and API requirements:
    const schema = {
      type: 'object',
      properties: {
        identifier: { type: 'string' }
      },
      required: ['identifier']
    };
    
    await firecrawl.scrapeUrl(url, {
      extract: {
        schema: schema // TypeScript error here
      }
    });
  3. Observe TypeScript errors about missing Zod properties
  4. Try using Zod schema instead, observe API runtime errors about missing required array

Expected Behavior
The TypeScript types should align with the API's actual requirements. Either:

  1. The API should accept Zod schemas as indicated by the TypeScript types, OR
  2. The TypeScript types should reflect that the API expects JSON Schema format

Environment

  • OS: macOS 24.1.0
  • Firecrawl Version: Latest from @mendable/firecrawl-js
  • Node.js Version: Latest LTS

Logs
TypeScript Error:

Type '{ type: string; properties: { identifier: { type: string; }; ... } }' is missing the following properties from type 'ZodType<any, ZodTypeDef, any>': _type, _output, _input, _def, and 34 more.

Runtime Error:

Failed to scrape URL. Status code: 500. Error: (Internal server error) - 400 Invalid schema for response_format 'websiteContent': In context=(), 'required' is required to be supplied and to be an array including every key in properties. Missing 'identifier'.

Additional Context

  • The issue appears in the schema validation for the extract option of the scrape endpoint
  • We've tried both Zod schemas and JSON Schema formats
  • Using type assertions works around the TypeScript error but doesn't resolve the underlying type definition issue
  • This affects any application trying to use strongly-typed schemas with Firecrawl
@HYAZEUS HYAZEUS added the bug Something isn't working label Dec 18, 2024
@HYAZEUS
Copy link
Author

HYAZEUS commented Dec 18, 2024

Sorry team. I'm self hosting, set this up rather quick, I assume thats the problem. Any confirmation would be great, excited to use. Leaving this issue here so others that might run into it, know what the problem is

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant