HTTP Tracker client: Print unrecognized responses in JSON

[josecelano]

Parent issue: #669
Depends on: #673

Not all tracker responses are always the same for the same request. Some trackers can include only mandatory fields. The example below is a wrong response because the scrape request should include the downloaded field. However, there could be other cases when the tracker response is valid but is not exactly the same response as in the Torrust Tracker. If we want to support other trackers we should be flexible with valid responses that don't match our resposnses.

Announce Request

When you run the HTTP tracker client with a Torrust Tracker:

cargo run --bin http_tracker_client announce http://127.0.0.1:7070 9c38422213e30bff212b30c360d26f9a02136422

you receive a response like this:

{
  "complete": 1,
  "incomplete": 0,
  "interval": 1800,
  "min interval": 900,
  "peers": [
    {
      "ip": "111.222.111.222",
      "peer id": [
        45,
        113,
        66,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        48,
        49
      ],
      "port": 17548
    }
  ]
}

Scrape Request

When you run the HTTP tracker client with a Torrust Tracker:

cargo run --bin http_tracker_client scrape http://127.0.0.1:7070 9c38422213e30bff212b30c360d26f9a02136422

you receive a response like this:

{
  "9c38422213e30bff212b30c360d26f9a02136422": {
    "complete": 1,
    "downloaded": 4,
    "incomplete": 0
  }
}

Compatibility Problem

$ cargo run --bin http_tracker_client scrape http://open.acgnxtracker.com:80 9c3842
2213e30bff212b30c360d26f9a02136422 | jq
    Finished dev [optimized + debuginfo] target(s) in 0.08s
     Running `target/debug/http_tracker_client scrape 'http://open.acgnxtracker.com:80' 9c38422213e30bff212b30c360d26f9a02136422`
thread 'main' panicked at src/shared/bit_torrent/tracker/http/client/responses/scrape.rs:143:60:
called `Result::unwrap()` on an `Err` value: MissingFileField { field_name: "downloaded" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Alternative Tracker request:

http://open.acgnxtracker.com/scrape?info_hash=%44%3C%76%02%B4%FD%E8%3D%11%54%D6%D9%DA%48%80%84%18%B1%81%B6

Alternative Tracker response:

scrape.zip

{
   "files": {
      "<hex>44 3C 76 02 B4 FD E8 3D 11 54 D6 D9 DA 48 80 84 18 B1 81 B6</hex>": {
         "incomplete": 0,
         "complete": 32
      }
   }
}

Torrust Tracker request:

https://tracker.torrust-demo.com/scrape?info_hash=%44%3C%76%02%B4%FD%E8%3D%11%54%D6%D9%DA%48%80%84%18%B1%81%B6

Torrust Tracker response:

scrape-torrust.zip

{
   "files": {
      "<hex>44 3C 76 02 B4 FD E8 3D 11 54 D6 D9 DA 48 80 84 18 B1 81 B6</hex>": {
         "complete": 1,
         "downloaded": 0,
         "incomplete": 0
      }
   }
}

NOTICE: the alternative tracker does not include the downloaded attribute.

In these cases, the client should print the JSON response anyway. But we need a generic serialisation from Becnode to JSON.

Implementation

The implementation requires changing these functions:

async fn announce_command(tracker_url: String, info_hash: String) -> anyhow::Result<()> {
    let base_url = Url::parse(&tracker_url).context("failed to parse HTTP tracker base URL")?;
    let info_hash =
        InfoHash::from_str(&info_hash).expect("Invalid infohash. Example infohash: `9c38422213e30bff212b30c360d26f9a02136422`");

    let response = Client::new(base_url)
        .announce(&QueryBuilder::with_default_values().with_info_hash(&info_hash).query())
        .await;

    let body = response.bytes().await.unwrap();

    let announce_response: Announce = serde_bencode::from_bytes(&body)
        .unwrap_or_else(|_| panic!("response body should be a valid announce response, got: \"{:#?}\"", &body));

    let json = serde_json::to_string(&announce_response).context("failed to serialize scrape response into JSON")?;

    println!("{json}");

    Ok(())
}

async fn scrape_command(tracker_url: &str, info_hashes: &[String]) -> anyhow::Result<()> {
    let base_url = Url::parse(tracker_url).context("failed to parse HTTP tracker base URL")?;

    let query = requests::scrape::Query::try_from(info_hashes).context("failed to parse infohashes")?;

    let response = Client::new(base_url).scrape(&query).await;

    let body = response.bytes().await.unwrap();

    let scrape_response = scrape::Response::try_from_bencoded(&body)
        .unwrap_or_else(|_| panic!("response body should be a valid scrape response, got: \"{:#?}\"", &body));

    let json = serde_json::to_string(&scrape_response).context("failed to serialize scrape response into JSON")?;

    println!("{json}");

    Ok(())
}

We first try to deserialize the encoded response into our data structures:

    let announce_response: Announce = serde_bencode::from_bytes(&body)
        .unwrap_or_else(|_| panic!("response body should be a valid announce response, got: \"{:#?}\"", &body));

If that operation fails, then we try to serialize to a generic json.

You can get a list of HTTP trackers from https://newtrackon.com/. The client should print all the responses from all those trackers.