SYNOPSIS
saldl [OPTIONS] URL
DESCRIPTION
saldl is a CLI downloader based on libcurl.
By default, it splits a download into fixed-sized chunks and download them in-order with multiple concurrent connections.
Many OPTIONS are available to change the default behavior.
Files and Dirs
- <filename>.part.sal
-
The incomplete/partial file that will eventually be renamed to <filename>.
- <filename>.ctrl.sal
-
This file contains the minimal information required about the download progress to allow resuming.
- <filename>.tmp.sal/
-
By default, saldl uses temporary files to store chunks before they are merged. Those files are created in this dir.
Chunk Progress States
- 4
-
Chunk already merged in <filename>.part.sal
- 3
-
Chunk downloaded, but still not merged.
This state should be transient. If it’s not, your system probably has load problems (CPU/IO/memory). - 2
-
Chunk download is in progress.
- 1
-
Chunk is queued for download.
This is a transient state you shouldn’t see. Unless you have serious load/congestion problems in your system. - 0
-
Chunk still not queued for download.
Note
|
When resuming from a previous session, an incomplete chunk would initially have state 0. That doesn’t mean it will be emptied and downloaded from scratch. Sub-chunk resume will work as expected. |
OPTIONS
General
- -d, --dry-run
-
Do not download, only display information.
- --get-info
-
Do not download, only print specific informational to stdout.
The option can be passed multiple times with different arguments. -
(Valid arguments are "file-name", "file-size", and "effective-url".)
- --mirror-url
-
If the mirror is valid, Assign half the connections to it.
- A mirror URL is considered valid when compared to URL if:
-
-
Both do not have the same effective URL.
-
Both have the same content size.
-
Both support ranges.
-
Both are not FTP.
-
Both are not using content encoding. Or both do, but we will not decode/decompress.
-
- --fatal-if-invalid-mirror
-
Fatally fail if the mirror is invalid. Only a warning is displayed by default.
- --stdout
-
Write/Pipe output to stdout.
- Everything else will behave in the same way except:
-
-
The pseudo output filename "STDOUT" will be used.
-
Chunks will only be merged (written to stdout) in order.
-
Resume is disabled.
-
No state is saved in a <filename>.ctrl.sal file.
-
If single mode is used, timeouts and other non-fatal response errors will become fatal.
-
-
If -m/--memory-buffers are used, saldl will not create or write to any files directly.
- -m, --memory-buffers
-
Use memory buffers instead of temp files for downloading chunks.
This mode increases memory overhead. And we lose sub-chunk resumability. But it might help performance in some situations, especially when IO is the bottleneck. - --read-only
-
Don’t create files or write anything to disk. This should be only used to test network performance.
- --no-mmap (default mode only)
-
Read chunk files into buffers instead of using mmap().
This option hurts performance and should only be used for debugging.
UI Options
- -i interval, --status-refresh-interval=interval
-
Minimum refresh interval between status updates in seconds. Increasing the default should help if terminal emulation is slow. (default: 0.3)
- -C, --no-color
-
Disable colors in output.
Pass it twice (-CC) to disable all other formattings and escape codes (auto-set if not tty). - -V , --verbosity
-
Increase verbosity level.
(See Verbosity Levels for details.) - --verbose-libcurl
-
Enable libcurl’s verbose output. Regardless of verbosity level.
- --no-status
-
Disable progress status output. (For debugging purposes only)
Resume Options
- -r, --resume
-
resume download.
(requires a <filename>.ctrl.sal to exist with a matching filesize) - -f, --force
-
If not resuming, and <filename>.part.sal exists, truncate the file and start over.
Chunk sizes and connections
- -S, --single
-
a single chunk single connection mode (wget-like).
- -s size, --chunk-size=size
-
Set size for chunks in bytes. A unit suffix can be used. (default: 1m)
- -c num, --connections=num
-
number of concurrent connections. (default: 6)
- -l num, --last-chunks-first=num
-
the number of last chunks that should be downloaded first. (default: 0)
- -L size, --last-size-first=size
-
The minimum size that should be downloaded first at the end of the file. A unit suffix can be used.
This should be useful when chunk-size is not known beforehand. e.g. when -a/--auto-size is used.
This option overrides -l/--last-chunks-first. (default: 0) - -F, --allow-ftp-segments
-
Due to a certain limitation in the information provided by libcurl, single mode is force-enabled by default with FTP links. This option overrides that behavior.
- --random-order
-
Download chunks in random order.
- --merge-in-order
-
Only merge chunks in-order.
This works as if we were piping to the output file, which could be useful with some storage devices.
Warning
|
|
- -a num, --auto-size=num
-
increase chunk size so that chunk progress can fit in num lines.
- -w, --whole-file
-
work like typical download accelerators (no. of chunks = no. of connections).
Filename Options
- -o filename, --output-filename=filename
-
Use this filename instead of the detected name.
- -n, --no-path
-
assume path is relative, replace / and : with _.
- -D dirname, --root-dir=dirname
-
Prepend filename with this path.
- -g, --filename-from-redirect
-
Get filename from redirected URL instead of the original URL.
- -G, --keep-GET-attrs
-
keep GET attributes at the end of a filename. (This option has no effect if -o/--output-filename is used)
- -A, --no-attachment-detection
-
Do not use Content-Disposition attachment filename if present.
- -t, --auto-trunc
-
truncate filename if name or path is too long.
- -T, --smart-trunc
-
same as -t/--auto-trunc but tries to keep the file extension. (overrides -t/--auto-trunc)
Network Options
- -6, --resolve-ipv6
-
Only resolve to IPv6 addresses.
- -4, --resolve-ipv4
-
Only resolve to IPv4 addresses.
Note
|
If both -6 and -4 are passed, what’s passed last will take precedence. |
- -R bandwidth, --connection-max-rate=bandwidth
-
maximum rate per connection in bytes/s. A unit suffix can be used. (default: 0 [unlimited])
- -O, --no-timeouts
-
disable all timeouts.
- -x proxy, --proxy=proxy
-
set proxy.
- -X proxy, --tunnel-proxy=proxy
-
similar to --proxy but tunneling all traffic through it.
- -N, --no-proxy
-
disable all proxies even if set in the environment.
- --skip-TLS-verification
-
Skip TLS/SSL verification. Use at your own risk.
Custom Request Options
- -E, --auto-referer
-
auto set referer in case of a redirect.
- -e referer, --referer=referer
-
set referer.
- -M local-file, --since-file-mtime=local-file
-
Sets the If-Modified-Since header with the last modification time of the passed local file.
This option has no effect if -I/--no-remote-info was passed. - -Y date-expression, --date-cond=date-expression
-
Sets If-Modified-Since header according to the date expression passed. Consult curl_getdate(3) for details.
If the expression starts with '-', then If-Unmodified-Since is set.
This option overrides -M/--since-file-mtime.
This option has no effect if -I/--no-remote-info was passed. - -z, --compress
-
Request a compressed response. Single mode would be forced if the server supports compression. Unless -Z/--no-decompress was passed.
- -Z, --no-decompress
-
Disable decompression, whether compression was requested or forced by the server.
- -K cookie-file, --cookie-file=cookie-file
-
File to read cookies from .
- -k cookies, --inline-cookies=cookies
-
Set cookies.
- -p post-data, --post=post-data
-
Send post-data in a simple POST request (no multipart).
- -P raw-post-data, --raw-post=raw-post-data
-
Send raw-post-data as-is including headers (supports multipart).
Warning
|
--post and --raw-post will double-post by default. Unless -I/--no-remote-info is also passed. |
- -U, --no-user-agent
-
Don’t set user agent (disables default agent).
- -u agent, --user-agent=agent
-
set user agent.
The following settings should not be generally used, but might help in rare cases:
- -H, --custom-headers=header(s)
-
Add one or more custom headers to the request. Or suppress an auto one. Those headers would be sent to the remote server. If you need to send them to the proxy, use --proxy-custom-headers.
This option can be passed multiple times.
If only one header is passed. It must not end with \r\n. libcurl will add this for us. Multiple headers must be separated by \r\n. And the last on must also not end with an \r\n.
- Three Header formats are acceptable:
-
- 1- Header with info
-
Name: info
- 2- Header without info
-
Name;
- 3- Suppress default header
-
Name:
- --proxy-custom-headers=header(s)
-
Works like --custom-headers. But the headers are passed to the proxy instead of the remote server.
- --use-HEAD
-
Use HEAD instead of GET to retrieve download information from servers.
- -I, --no-remote-info
-
Just GET. Don’t do anything else. (-S/--signle is forced, -r/--resume is disabled)
- --no-http2
-
Don’t try HTTP/2 requests.
- --http2-upgrade
-
Try HTTP/2 requests with plain HTTP.
- --no-tcp-keep-alive
-
Don’t send TCP keep-alive probes.
- --assume-range-support
-
Don’t use this. The wrong data will probably be downloaded.
ENVIRONMENT
In addition to all environment variables affecting libcurl(3). The following variable(s) can affect saldl:
- SALDL_EXTRA_ARGS
-
Append these arguments to argv.
Unit Suffixes
OPTIONS that take an argument representing a byte value can be suffixed with one of the following letters:
-
- K or k
-
multiply size by 1024 (KiB).
- M or m
-
multiply size by 1024*1024 (MiB).
- G or g
-
multiply size by 1024*1024*1024 (GiB).
Verbosity Levels
saldl only shows essential information with the default verbosity level. You can pass -V/--verbosity multiple times to increase verbosity.
Note
|
Verbosity levels are incremental. Each level includes all messages from lower levels. |
- -V
-
Show non-fatal errors.
- -VV
-
Show warnings.
- -VVV
-
Show informational messages.
Warning
|
The following levels are noisy. They should only be used for debugging/tracing purposes. |
- -VVVV
-
Show debug messages.
- -VVVVV
-
Enable verbose output in libcurl.
Note
|
This can be enabled independently with the --verbose-libcurl option. |
- -VVVVVV
-
Show debug messages from event loops.
EXIT STATUS
- 0
-
Success.
- 1
-
A fatal error happened. A helpful message should explain the error.
AUTHOR
Mohammad AlSaleh (MoSal@Github).
COPYING
Copyright (C) 2014-2015 Mohammad AlSaleh. Free use of this software is granted under the terms of the GNU Affero General Public License (AGPL).