Post to add with Python, I am missing a component?

[gerroon]

Hi

I was able to put a basic Python script with the points and helps from @pirate however this script fails to actually submit the url. I do not see Archivebox grabbing the contents of the url, but I see a "200" response.

I think either the headers are missing something or the data is wrong. I am wondering if I could get some tips on it. I reverse engineered the browser extension payload, but I am not experienced in this stuff.

I originally tried the Flask server route Pirate recommended but I was not able to figure out how to do post scheduling since I want to actually pass a file with urls in it, and posting multiple urls was overwhelming the Flask server I think.

archivebox_1  | "POST /add HTTP/1.1" 301 0
archivebox_1  | "GET /add/ HTTP/1.1" 200 7084

url = 'http://ARCHIVE-IP/add'

headers = {
'Accept': '*/*',
'Accept-Language': 'en-US,en;q=0.9',
'Cache-Control': 'no-cache', 
'Connection': 'keep-alive',
'Content-Type': 'multipart/form-data; boundary=----WebKitFormBoundarynGFBZipixJtBmPUi',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
}

data = {
'url': 'https://www.digitalocean.com/community/tutorials/how-to-install-run-connect-to-jupyter-notebook-on-remote-server',
'tag': 'saved',
'depth': '0',
# 'parser': 'url_list',
"extractors": "single-file,screenshot,dom,readability,media",
}

r = requests.post(url, headers=headers, data=data)

[mAAdhaTTah]

Add a trailing slash to the url. The server is redirecting and 301 redirects change the method from POST -> GET.

[gerroon]

Thanks, that did it.