saldl.1.txt

SALDL(1)
=========
:doctype: manpage

NAME
----

saldl - A lightweight well-featured CLI downloader optimized for speed and
early preview.


SYNOPSIS
--------

*{manname}* <<OPTIONS,[OPTIONS]>> 'URL'


DESCRIPTION
-----------

*{manname}* is a CLI downloader based on libcurl. +

By default, it splits a download into  fixed-sized chunks and download
them in-order with multiple concurrent connections.

Many <<OPTIONS,OPTIONS>> are available to change the default behavior.


Files and Dirs
---------------

'<filename>.part.sal'::
  The incomplete/partial file that will eventually be renamed to '<filename>'.

'<filename>.ctrl.sal'::
  This file contains the minimal information required about the download
  progress to allow resuming.

'<filename>.tmp.sal/'::
  By default, *{manname}* uses temporary files to store chunks before they are
  merged. Those files are created in this dir.


Chunk Progress States
----------------------

*4*::
  Chunk already merged in '<filename>.part.sal'

*3*::
  Chunk downloaded, but still not merged. +
  +
  This state should be transient. If it's not, your system probably has
  load problems (CPU/IO/memory).

*2*::
  Chunk download is in progress.

*1*::
  Chunk is queued for download. +
  +
  This is a transient state you shouldn't see.
  Unless you have serious load/congestion problems in your system.

*0*::
  Chunk still not queued for download.

[NOTE]
=============
When resuming from a previous session, an incomplete chunk would
initially have state '0'. That doesn't mean it will be emptied and
downloaded from scratch. Sub-chunk resume will work as expected.
=============


[[OPTIONS]]
OPTIONS
---------

[[gen-opts]]
General
~~~~~~~~

*-d, --dry-run*::
  Do not download, only display information.

*--get-info*::
  Do not download, only print specific informational to stdout. +
  The option can be passed multiple times with different arguments.

  ::
  (Valid arguments are "file-name", "file-size", and "effective-url".)

*--force-get-info*::
  Get and print information, even if filename already exists.

*--mirror-url*::
  If the mirror is valid, Assign half the connections to it.


  A mirror URL is considered valid when compared to URL if: :::

   1. Both do not have the same effective URL.
   2. Both have the same content size.
   3. Both support ranges.
   4. Both are not FTP.
   5. Both are not using content encoding. Or both do, but we will not
      decode/decompress.

*--fatal-if-invalid-mirror*::
  Fatally fail if the mirror is invalid. Only a warning is displayed by default.

*--stdout*::
  Write/Pipe output to *stdout*.

  Everything else will behave in the same way except: :::

   1. The pseudo output filename "STDOUT" will be used.
   2. Chunks will only be merged (written to stdout) in order.
   3. Resume is disabled.
   4. No state is saved in a '<filename>.ctrl.sal' file.
   5. If single mode is used, timeouts and other non-fatal response
      errors will become fatal.

  ::
  If *-m/--memory-buffers* are used, saldl will not create or write to
  any files directly.

*-m, --memory-buffers*::
  Use memory buffers instead of temp files for downloading chunks. +
  This mode increases memory overhead. And we lose sub-chunk resumability.
  But it might help performance in some situations, especially when IO is
  the bottleneck.

*--read-only*::
  Don't create files or write anything to disk.
  This should be only used to test network performance.

*--no-mmap* (default mode only)::
  Read chunk files into buffers instead of using mmap(). +
  This option hurts performance and should only be used for
  debugging.

[[ui-opts]]
UI Options
~~~~~~~~~~~

*-i 'interval', --status-refresh-interval='interval'*::
  Minimum refresh interval between status updates in seconds.
  Increasing the default should help if terminal emulation is slow.
  (*default*: '0.3')

*-C, --no-color*::
  Disable colors in output. +
  +
  Pass it twice (*-CC*) to disable all other formattings and escape
  codes (auto-set if not tty).

*-V , --verbosity*::
  Increase verbosity level. +
  (See <<v-levels,*Verbosity Levels*>> for details.)

*--verbose-libcurl*::
   Enable libcurl's verbose output. Regardless of verbosity level.

*--no-status*::
   Disable progress status output. (For debugging purposes only)

Resume Options
~~~~~~~~~~~~~~~

*-r, --resume*::
  resume download. +
  (requires a '<filename>.ctrl.sal' to exist with a matching filesize)

*-f, --force*::
  If not resuming, and '<filename>.part.sal' exists, truncate the file
  and start over.

[[ch-sz-conn]]
Chunk sizes and connections
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*-S, --single*::
  a single chunk single connection mode (wget-like).

*-s 'size', --chunk-size='size'*::
  Set size for chunks in bytes. <<unit-suf,*A unit suffix*>> can
  be used.
  (*default*: '1m')

*-c 'num', --connections='num'*::
  number of concurrent connections.
  (*default*: '6')

*-l 'num', --last-chunks-first='num'*::
  the number of last chunks that should be downloaded first.
  (*default*: '0')

*-L 'size', --last-size-first='size'*::
  The minimum size that should be downloaded first at the end of
  the file. <<unit-suf,*A unit suffix*>> can be used. +
  This should be useful when chunk-size is not known beforehand.
  e.g. when *-a/--auto-size* is used. +
  This option overrides *-l/--last-chunks-first*.
  (*default*: '0')

*-F, --allow-ftp-segments*::
  Due to a certain limitation in the information provided by libcurl, single
  mode is force-enabled by default with FTP links. This option overrides that
  behavior.

*--random-order*::
  Download chunks in random order.

*--merge-in-order*::
  Only merge chunks in-order. +
  +
  This works as if we were piping to the output file, which could be useful
  with some storage devices.

[WARNING]
================
1. It does not make sense to use *--merge-in-order* with *--last-chunks-first*
   or *--last-size-first*. As merging those chunks will be delayed. Even if
   they were downloaded early.
2. If memory buffers *--merge-in-order* are both used, completely downloaded
   chunks could be lost due to delayed merges if the download was interrupted.
================

*-a 'num', --auto-size='num'*::
  increase chunk size so that chunk progress can fit in 'num' lines.

*-w, --whole-file*::
  work like typical download accelerators
  (no. of chunks = no. of connections).


Filename Options
~~~~~~~~~~~~~~~~

*-o 'filename', --output-filename='filename'*::
  Use this filename instead of the detected name.

*-n, --no-path*::
  assume path is relative, replace '/' and ':' with '_'.

*-D 'dirname', --root-dir='dirname'*::
  Prepend filename with this path.

*-g, --filename-from-redirect*::
  Get filename from redirected URL instead of the original URL.

*-G, --keep-GET-attrs*::
  keep GET attributes at the end of a filename.
  (This option has no effect if *-o*/*--output-filename* is used)

*-A, --no-attachment-detection*::
  Do not use Content-Disposition attachment filename if present.

*-t, --auto-trunc*::
  truncate filename if name or path is too long.

*-T, --smart-trunc*::
  same as *-t/--auto-trunc* but tries to keep the file extension.
  (overrides *-t*/*--auto-trunc*)


Network Options
~~~~~~~~~~~~~~~~

*-6, --resolve-ipv6*::
  Only resolve to IPv6 addresses.

*-4, --resolve-ipv4*::
  Only resolve to IPv4 addresses.

[NOTE]
================
If both *-6* and *-4* are passed, what's passed last will take precedence.
================

*-R 'bandwidth', --connection-max-rate='bandwidth'*::
  maximum rate per connection in bytes/s. <<unit-suf,*A unit suffix*>>
  can be used.
  (*default*: '0' [unlimited])

*-O, --no-timeouts*::
  disable all timeouts.

*--timeout-connection-period*::
  period (in seconds) before a connection timeout is triggered.
  (*default*: '10')

*--timeout-low-speed-period*::
  period (in seconds) before a low-speed timeout is triggered.
  (*default*: '10')

*--timeout-low-speed*::
  minimum rate per connection in bytes/s.
  <<unit-suf,*A unit suffix*>> can be used.
  (*default*: '512B/s')

*-x 'proxy', --proxy='proxy'*::
  set proxy.

*-X 'proxy', --tunnel-proxy='proxy'*::
  similar to --proxy but tunneling all traffic through it.

*-N, --no-proxy*::
  disable all proxies even if set in the environment.

*--skip-TLS-verification*::
  Skip TLS/SSL verification. Use at your own risk.


Custom Request Options
~~~~~~~~~~~~~~~~~~~~~~~~

*-E, --auto-referer*::
  auto set referer in case of a redirect.

*-e 'referer', --referer='referer'*::
  set referer.

*-M 'local-file', --since-file-mtime='local-file'*::
  Sets the 'If-Modified-Since' header with the last modification time
  of the passed local file.
  +
  This option has no effect if *-I*/*--no-remote-info* was passed.

*-Y 'date-expression', --date-cond='date-expression'*::
  Sets 'If-Modified-Since' header according to the date expression
  passed. Consult *curl_getdate(3)* for details.
  +
  If the expression starts with '+++'-'+++', then 'If-Unmodified-Since' is
  set.
  +
  This option overrides *-M*/*--since-file-mtime*.
  +
  This option has no effect if *-I*/*--no-remote-info* was passed.


*-z, --compress*::
  Request a compressed response. Single mode would be forced if the server
  supports compression. Unless *-Z*/*--no-decompress* was passed.

*-Z, --no-decompress*::
  Disable decompression, whether compression was requested or forced by
  the server.

*-K 'cookie-file', --cookie-file='cookie-file'*::
  File to read cookies from .

*-k 'cookies', --inline-cookies='cookies'*::
  Set cookies.

*-p 'post-data', --post='post-data'*::
  Send 'post-data' in a simple POST request (no multipart).

*-P 'raw-post-data', --raw-post='raw-post-data'*::
  Send 'raw-post-data' as-is including headers (supports multipart).

[WARNING]
================
*--post* and *--raw-post* will double-post by default. Unless
*-I*/*--no-remote-info* is also passed.
================

*-U, --no-user-agent*::
  Don't set user agent (disables default agent).

*-u 'agent', --user-agent='agent'*::
  set user agent.

The following settings should not be generally used, but might help in rare
cases:

*-H, --custom-headers='header(s)'*::
  Add one or more custom headers to the request. Or suppress an auto one.
  Those headers would be sent to the remote server. If you need to send them
  to the proxy, use *--proxy-custom-headers*. +
  +
  This option can be passed multiple times.
  +
  If only one header is passed. It must not end with '\r\n'. libcurl will
  add this for us. Multiple headers must be separated by '\r\n'. And the last
  on must also not end with an '\r\n'. +

  *Three Header formats are acceptable:* :::
   1- Header with info ::::
    'Name: info'
   2- Header without info ::::
    'Name;'
   3- Suppress default header ::::
    'Name:'

*--proxy-custom-headers='header(s)'*::
  Works like *--custom-headers*. But the headers are passed to the proxy
  instead of the remote server.

*--use-HEAD*::
  Use 'HEAD' instead of 'GET' to retrieve download information from
  servers.

*-I, --no-remote-info*::
  Just 'GET'. Don't do anything else.
  (*-S*/*--signle* is forced, *-r*/*--resume* is disabled)

*--no-http2*::
  Don't try HTTP/2 requests.

*--http2-upgrade*::
  Try HTTP/2 requests with plain HTTP.

*--no-tcp-keep-alive*::
  Don't send TCP keep-alive probes.

*--assume-range-support*::
  Don't use this. The wrong data will probably be downloaded.

ENVIRONMENT
------------

In addition to all environment variables affecting *libcurl(3)*.
The following variable(s) can affect *{manname}*:

*SALDL_EXTRA_ARGS*::
  Append these arguments to argv.


[[unit-suf]]
Unit Suffixes
-------------

<<OPTIONS,OPTIONS>> that take an argument representing a byte value
can be suffixed with one of the following letters:

 ;;
  *K* or *k*::
    multiply size by 1024 (KiB).
  *M* or *m*::
    multiply size by 1024*1024 (MiB).
  *G* or *g*::
    multiply size by 1024*1024*1024 (GiB).

[[v-levels]]
Verbosity Levels
------------------

*{manname}* only shows essential information with the default verbosity
level. You can pass <<ui-opts,*-V*/*--verbosity*>> multiple times to
increase verbosity.

[NOTE]
===============
Verbosity levels are incremental. Each level includes all messages from
lower levels.
===============

*-V*::
  Show non-fatal errors.

*-VV*::
  Show warnings.

*-VVV*::
  Show informational messages.

[WARNING]
==============
The following levels are noisy. They should only be used for
debugging/tracing purposes.
==============

*-VVVV*::
  Show debug messages.

*-VVVVV*::
  Enable verbose output in libcurl.

[NOTE]
==============
This can be enabled independently with
the <<ui-opts,*--verbose-libcurl*>> option.
==============

*-VVVVVV*::
  Show debug messages from event loops.


EXIT STATUS
-----------
*0*::
    Success.

*1*::
   A 'fatal' error happened. A helpful message should explain the error.

WWW
----

https://saldl.github.io


BUGS
-----
https://github.com/saldl/saldl/issues

AUTHOR
------
Mohammad AlSaleh (MoSal@Github).

COPYING
-------
Copyright \(C) 2014-2015 Mohammad AlSaleh. Free use of this software is
granted under the terms of the GNU Affero General Public License (AGPL).