Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] headers don't pass to the api in crawl url #909

Open
TChengZ opened this issue Nov 18, 2024 · 2 comments
Open

[Bug] headers don't pass to the api in crawl url #909

TChengZ opened this issue Nov 18, 2024 · 2 comments
Labels
bug Something isn't working question Further information is requested

Comments

@TChengZ
Copy link

TChengZ commented Nov 18, 2024

Describe the Bug
Provide a clear and concise description of what the bug is.
i crawl the url 'https://www.xchuxing.com/car?bid=203', in this page, it will send a request like
image
to get some information to show on the web,but the crawl result don't have the above message,
i figure out there must have some cookie to simulate the log in state,and i use postman to check that once i add the cookie below for the request(/all-brand?bid=203), it can return the information
image

and i past the cookie data to the pageOptions,but still can't crawl the full message from the website
To Reproduce
Steps to reproduce the issue:

  1. Configure the environment or settings with '...'
  2. Run the command '...'
  3. Observe the error or unexpected output at '...'
  4. Log output/error message

Expected Behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots or copies of the command line output to help explain the issue.

Environment (please complete the following information):

  • OS: [e.g. macOS, Linux, Windows]
  • Firecrawl Version: [e.g. 1.2.3]
  • Node.js Version: [e.g. 14.x]

Logs
If applicable, include detailed logs to help understand the problem.

Additional Context
Add any other context about the problem here, such as configuration specifics, network conditions, data volumes, etc.

@TChengZ TChengZ added the bug Something isn't working label Nov 18, 2024
@nickscamara
Copy link
Member

Are you self hosting it? Will take a look into it.

@nickscamara nickscamara added the question Further information is requested label Dec 6, 2024
@TChengZ
Copy link
Author

TChengZ commented Dec 6, 2024

Are you self hosting it? Will take a look into it.

yes, self hosting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants