Hwp_HTTPStreamer

HTTP Streamer 2.0
=================

This plugin enables Hollywood to open and stream files from HTTP sources as if
they were
stored on a local drive. Once this plugin has been activated, all Hollywood
functions
that deal with files will "automagically" be able to open files from HTTP
sources as
well. Starting with version 2.0, HTTP Streamer also supports the hURL plugin
which makes
it possible to stream data using lots of other protocols such as HTTPS or FTP.

HTTP Streamer uses a sophisticated multi-threaded design for highly efficient
streaming.
Each connection is managed by a dedicated thread for optimal performance. The
plugin also
supports Hollywood 6.0's new streaming APIs which means that you will be able to
stream
audio and video files from HTTP sources with plugins like avcodec.hwp.


Requirements
============

This plugin requires at least Hollywood 6.0 since it uses the file adapter and
streaming
APIs introduced with Hollywood 6.0. If you want to pass options through the
"UserTags"
table argument, you need at least Hollywood 10. 

For video and audio streaming through avcodec.hwp, you need at least version 1.6
of
the avcodec.hwp plugin.

If you want to use the hURL interface of the plugin, you need at least version
2.1 of
hURL.


Usage
=====

The easiest way to use the plugin is to set the "InstallAdapter" tag to TRUE
when calling
()REQUIRE on the plugin, e.g. like this:

    ()REQUIRE "httpstreamer", {InstallAdapter = True}

If you activate the plugin like this, it will become globally available and all
Hollywood commands that deal with files will support the opening of files from
HTTP
sources. For example, you could do something like this then:

    LoadBrush(1, "http://www.example.com/testpicture.jpg")

If you don't want to activate the HTTP Streamer plugin globally, just leave out
the
"InstallAdapter" tag (or set it to FALSE) when calling ()REQUIRE on the plugin.
If you do
that, you have to manually tell Hollywood commands if you want them to load a
file using
the HTTP Streamer plugin. This can be done by setting the "Adapter" tag, e.g.
like this:

    LoadBrush(1, "http://www.example.com/testpicture.jpg", {Adapter =
"httpstreamer"})

By using the "Adapter" tag, LoadBrush() will be instructed to open the specified
file using
the specified adapter, which is "httpstreamer" in our case. 

Note that by default, the HTTP Streamer plugin only supports streaming from
plain HTTP
sources. If you need support for more advanced protocols like HTTPS, you need to
install
the hURL plugin and set the "UsehURL" tag to TRUE. In that case, all streaming
is done
through the hURL plugin which supports a wide variety of protocols. This is how
you could
load a file from a HTTPS source using HTTP Streamer and hURL:

    ()REQUIRE "httpstreamer", {InstallAdapter = True, UsehURL = True}
    LoadBrush(1, "https://www.example.com/testpicture.jpg")

Note that it's also possible to specify options locally instead of globally in
()REQUIRE.
Suppose you want to load some files using hURL and some using HTTP Streamer's
inbuilt
handler. In that case, don't set "UsehURL" globally in ()REQUIRE but only pass
it when
needed to the respective commands, e.g. like this:

    LoadBrush(1, "http://www.example.com/testpicture.jpg")   
    LoadBrush(1, "https://www.example.com/testpicture.jpg", {UserTags = {UsehURL
= True}})    


File mode vs. stream mode
=========================

By default, HTTP Streamer operates in file mode. This means that all files
opened using
the plugin will behave like normal files so it's also possible to query the file
size,
seek the file to a specific position and so on. To support things like seeking
the file
read cursor to a specific position, however, HTTP Streamer needs to buffer the
whole
file in memory or a temporary file which is why using HTTP Streamer in file mode
will
potentially use lots of resources, depending on how big the file you open with
the
plugin is. If you're just dealing with small files, file mode is usually fine
but when
dealing with big files or if you need real streaming, stream mode is usually a
better
choice.

To activate stream mode, you have to set the "FileMode" tag to FALSE (see
below). Once
you have done that, you'll only be able read sequentially from a file. It won't
be
possible to seek backwards or forwards, get the filesize and potentially other
things
that work in file mode. The only thing that's possible in stream mode is reading
sequentially from the source. This makes stream mode very memory efficient as
only small
amounts of data have to be kept in memory but, as mentioned above, this means
that some
things that work in file mode won't work. For example, when file mode is
disabled, it
won't be possible to seek the file to a specific position. It also often won't
be possible
to query the file size because this is often unknown because data is constantly
streamed. For example, consider an MP3 livestream from an internet radio source:
This source will deliver data infinitely so there is no such thing as a "file
size"
from such a source. Instead, data will be delivered continuously.

Here's how you could open and play an MP3 in stream mode:

    OpenMusic(1, "http://www.example.com/stream.mp3", {UserTags = {FileMode =
False}}
    PlayMusic(1)
       

Linking considerations
======================

Keep in mind that all files declared in the preprocessor commands are linked
automatically
into your applet or executable when Hollywood is in compile mode. Thus, if you
do the
following and HTTP Streamer's file adapter has been globally activated, the file
will be
downloaded and linked to your applet or executable:

    ()BRUSH 1, "http://www.example.com/testpicture.jpg"
    
This means that your applet or executable will not download the file any more
but they
will use a copy of the file that has been linked into the applet or executable!
If
you want the applet or executable to always use the copy on the HTTP server, you
have
to make sure that the file is never linked to the applet or executable. This can
be
achieved by setting the "Link" tag to FALSE:

    ()BRUSH 1, "http://www.example.com/testpicture.jpg", {Link = False}
    
When done like this, Hollywood will never link the file into your applet or
executable.
Instead, it will always be downloaded from the given URL.    
    

Options
=======

HTTP Streamer supports several options which can either be passed to ()REQUIRE,
to the
httpstreamer.SetConfig() command or through the "UserTags" tag which is accepted
by many
Hollywood commands. Note that ()REQUIRE and "UserTags" expect a table which can
contain
multiple options at once whereas httpstreamer.SetConfig() only accepts a single
option
at a time. See below for an example. Also note that the "InstallAdapter" tag is
only
available with ()REQUIRE, it's not possible to pass it to
httpstreamer.SetConfig() or
to the "UserTags" table. 

The following options are currently available:

FileMode:
This tag controls whether or not HTTP Streamer should operate in file mode. See
above
for a detailed explanation on the differences between file mode and stream mode.
By
default, this tag is set to TRUE. If you want HTTP Streamer to operate in stream
mode,
set this tag to FALSE. (V2.0)

UsehURL:
If this tag is set to TRUE, HTTP Streamer will use the hURL plugin instead of
its inbuilt
network handler. The inbuilt network handler only supports basic HTTP whereas
the hURL plugin
also supports HTTP through SSL/TLS and lots of other protocols. All these
protocols supported
by hURL can be easily accessed through HTTP Streamer as well by simply setting
the "UsehURL"
tag to TRUE. Note that this feature requires at least version 2.1 of hURL. The
"UsehURL" tag
defaults to FALSE which means that by default, the inbuilt HTTP handler will be
used. Note
that on AmigaOS 3, AmigaOS 4 and AROS (i.e. all AmiSSL-based Amiga platforms)
only one connection
may be established at a time when using hURL. Due to AmiSSL and
bsdsocket.library limitations
on those platforms it's currently not possible to use multiple connections at
the same time
so before a new connection is opened, all existing ones must be closed first.
Also, you must not
use any hURL functions while there's still an open HTTP Streamer connection on
AmigaOS 3,
AmigaOS 4 and AROS. All other platforms don't have this limitation. (V2.0)

InstallAdapter:
If this tag is set to TRUE, HTTP Streamer will hook itself into Hollywood's file
handler
which means that it will automatically be available to all Hollywood commands
dealing with
files. If you don't set this to TRUE, you need to explicitly pass "httpstreamer"
in the
"Adapter" tag of Hollywood commands if you want them to load files using HTTP
Streamer.
See above for more details. Note that this tag is only recognized by ()REQUIRE.
You cannot
pass it to httpstreamer.SetConfig() or through the "UserTags" table. (V2.0)

Fail404:
This tag specifies whether or not HTTP Streamer should fail with a "file not
found"
error when you pass a URL that points to a non-existent file. Normally, when you
request
a non-existent file, HTTP servers will generate a special HTML page with a "404
- file
not found" error, and send that to you instead. So you will always be getting a
file
even if you are requesting a non-existent file which can be confusing. If you
really want
this behaviour, set this tag to FALSE. Then HTTP Streamer will never fail for
any
HTTP files and will always deliver an error page in case of a non-existent file.
By default,
this tag is set to TRUE which means that no error page is generated and HTTP
Streamer
will fail to open non-existent files.

Encoded:
If this tag is set to TRUE, HTTP Streamer will expect encoded URLs instead of
unencoded URLs.
Defaults to FALSE. (V1.1)
  
Redir:
Specifies whether or not the web server is allowed to redirect you to a new URL.
This
defaults to TRUE which means that redirection is allowed.

Timeout:
This tag allows you to set a connection timeout in milliseconds. The default is
10000
which means that HTTP Streamer will timeout in case the server doesn't reply
within
an interval of 10 seconds.

Proxy:
You can specify a proxy server here that should be used when making connections.
By
default, HTTP Streamer doesn't use any proxy server.

UserAgent:
This tag allows you to change the user agent that HTTP Streamer sends to the
target
server. This is useful with servers that refuse to cooperate with unknown user
agents.
By default, HTTP Streamer will send "HTTPStreamer.hwp" in the user agent field
of
HTTP requests.

TmpFileThreshold:
When requesting larger files, HTTP Streamer will use a temporary file for its
buffer
instead of memory. This tag allows you to specify a threshold value in bytes
that
defines when HTTP Streamer should use a temporary file instead of a memory
buffer.
This defaults to 16277216 bytes which means that all files larger than 16 MB
will be
buffered on disk whereas all files smaller than 16 MB will be buffered in
memory.

RamFileThreshold:
This is only supported on AmigaOS and compatibles. When using temporary file
buffering
(see the documentation of "TmpFileThreshold" above), this tag allows you to set
a
threshold value that defines whether the temporary file should be created in RAM
disk
or on your hard drive. Whenever the temporary file is bigger than the threshold
value
specified here, it will be created on your hard drive. Otherwise it will be
created
in T: on your RAM disk. Obviously, this tag must be set to at least the value of
"TmpFileThreshold". It defaults 33554432 bytes on AmigaOS 3 and 104857600 bytes
on
all other AmigaOS compatibles. This means that on AmigaOS 3 all temporary files
smaller than 32 MB will be written to T: whereas on all other systems all
temporary
files smaller than 100 MB will be written to T:.

IOBufSize:
This tag allows you to set the IO buffering size. It's normally not necessary to
tinker
with this. Defaults to 16384 bytes.

StreamBufSize:
This tag can be used to set the size of the ring buffer that is used when the
"FileMode"
tag is set to FALSE to enable real streaming (see above). It's normally not
necessary
to tinker with this. Defaults to 524288 bytes. (V2.0)

SetStreaming:
This tag allows you to set whether HTTP Streamer should inform Hollywood that
its file handles are actually streams. By default, HTTP Streamer does so. This
allows Hollywood to try to avoid operations that are inefficient on streaming
sources
like excessive seeking operations. If you set this tag to FALSE, Hollywood
won't be made aware that the file handles managed by HTTP Streamer are actually
streams. This might entail undesirable consequences because Hollywood might try
to
seek the file handle to the end and since the HTTP protocol doesn't easily
support
direct byte seeking, the seek operation has to be emulated which can be really
expensive.
Seeking to the end of a HTTP stream for instance means that HTTP Streamer will
have to download the entire file before it can continue! Thus, you normally
won't
want to set this tag to FALSE but keep it to TRUE, which is the default.

SetNoSeek:
If this tag is set to TRUE, HTTP Streamer won't support seeking on its file
handles. Note that if you set this tag to TRUE, several file format handlers
which
depend on the seek functionality might stop working. That's why it's recommended
to leave this tag at its default value, which is FALSE. Note that if you set the
"FileMode" tag to FALSE (see above), the "SetNoSeek" tag will automatically be
set
to TRUE.

Verbose:
If this is set to TRUE, HTTP Streamer will print some output that can be useful
for
debugging network issues (currently only supported if "UsehURL" is set to TRUE).
This
tag defaults to FALSE. (V2.0)

Here is an example of how to use these tags:

    ()REQUIRE "httpstreamer", {UserAgent = "Firefox", Timeout = 20000}

                -OR-

    httpstreamer.SetConfig("UserAgent", "Firefox")
    httpstreamer.SetConfig("Timeout", 20000)
    
                -OR-
                    
    OpenFile(1, "http://www.example.com/test.html", {UserTags = {UserAgent =
"Firefox", Timeout = 20000}}

Note that when using httpstreamer.SetConfig() you can only set one tag per call.


Troubleshooting
===============

You have to be careful when using HTTP URLs in preprocessor commands because
when
Hollywood is in compile mode, it will link all files declared in the
preprocessor
commands to the applet or executable by default. This means that Hollywood will
download all files declared in the preprocessor commands and link them which is
probably not what you want in that case. You probably want Hollywood to always
use
the copy from the server instead of a linked copy. See the section "Usage" above
for information on how you can achieve this.

Note that when setting "UsehURL" to TRUE on AmigaOS 3, AmigaOS 4 and AROS (i.e.
all AmiSSL-based Amiga platforms) only one connection may be established at a
time.
Due to AmiSSL and bsdsocket.library limitations on those platforms it's
currently
not possible to use multiple connections at the same time so before a new
connection
is opened, all existing ones must be closed first. Also, you must not use any
hURL
functions while there's still an open HTTP Streamer connection on AmigaOS 3,
AmigaOS 4
and AROS. All other platforms don't have this limitation.


History
=======

Version 2.0:    (25-Apr-26)
- New [Linux]: Added build for the Linux arm64 platform
- New [iOS]: Added builds for the iOS arm64 simulator platform and removed the
iOS 32-bit x86 simulator
  platform because that's really not needed anymore
- New [macOS]: Added build for macOS arm64 (Apple Silicon) (requested by Tomasz
Paul)
- New: Version string in the binary contains the precise CPU architecture
specification now, e.g. 
  "x64-linux" instead of just "Linux"
- Change: Since it's always necessary to call ()REQUIRE on plugins with
Hollywood 11, HTTP Streamer will
  no longer automatically hook into Hollywood's file handler when you ()REQUIRE
it; if you want the
  plugin to hook into Hollywood's file handler, you have to set the
"InstallAdapter" tag to TRUE when
  calling ()REQUIRE on the plugin; this is also consistent with the behaviour of
other plugins like the
  ZIP plugin; if you don't set "InstallAdapter" to TRUE, you can still use the
plugin by passing the
  string "httpstreamer" to the "Adapter" tag of functions like OpenFile(),
LoadBrush() etc.
- New: Added "Verbose" tag; if this is set to TRUE, HTTP Streamer will print
some output that can be
  useful for debugging network issues (currently only supported by the hURL
backend); this tag defaults
  to FALSE
- New: Added "StreamBufSize" tag; this tag can be used to set the size of the
ring buffer that is used
  when setting "FileMode" to FALSE to enable real streaming (see below); the
buffer size needs to be
  specified in bytes and defaults to 524288, i.e. 512kb
- New: Added "FileMode" tag; if this is set to FALSE, HTTP Streamer won't
activate its file emulation
  layer; if you want to use HTTP Streamer for real streaming (e.g. audio or
video), you should set this
  tag to FALSE because otherwise HTTP Streamer will always emulate a file which
means that it will keep
  all data it has downloaded, just in case scripts want to seek back to the
beginning of the file or to
  an arbitrary position within the file; so for bigger files like audio or video
streams this will need
  a lot of memory, either as a memory buffer or as a temporary file on your HD
because all downloaded data
  has to be preserved until the file is closed; so if you set "FileMode" to
FALSE, memory consumption
  will be much lower but of course typical file operations like seeking or
getting the file size won't be
  possible then; when "FileMode" is FALSE, scripts and plugins can only read
sequentially from the stream;
  seeking or querying the total size won't be possible; note that when setting
"FileMode" to FALSE, the
  "SetNoSeek" tag will automatically be set to TRUE; for compatibility reasons,
the "FileMode" tag defaults
  to TRUE
- New: Added "UsehURL" tag; if this is set to TRUE, HTTP Streamer will use the
hURL plugin instead of its
  inbuilt network handler; the inbuilt network handler only supports basic HTTP
whereas the hURL plugin
  also supports HTTP through SSL/TLS and lots of other protocols; all these
protocols supported by hURL
  can now be easily accessed through HTTP Streamer as well by simply setting the
"UsehURL" tag to TRUE;
  this makes the HTTP Streamer plugin much more powerful than its name suggests
because you can also
  stream other protocols now, not just HTTP; note that this feature requires at
least version 2.1 of hURL;
  the "UsehURL" tag defaults to FALSE which means that by default, the inbuilt
HTTP handler will be used;
  note that on AmigaOS 3, AmigaOS 4 and AROS (i.e. all AmiSSL-based Amiga
platforms) only one connection
  may be established at a time when using hURL; due to AmiSSL and
bsdsocket.library limitations on those
  platforms it's currently not possible to use multiple connections at the same
time; before a new
  connection is opened, all existing ones must be closed first; also, you must
not use any hURL functions
  while there's still an open HTTP streamer connection
- New: All tags accepted by httpstreamer.SetConfig() and ()REQUIRE can now also
be passed to the plugin
  through the "UserTags" table tag that is accepted by almost all Hollywood
commands that deal with files;
  they will default to the values set during ()REQUIRE or
httpstreamer.SetConfig(); this feature requires
  Hollywood 10 or better
- Fix: The "Encoded" tag wasn't documented properly  

Version 1.1:    (25-Mar-17)
- New: Added 64-bit builds for Windows, Linux, and Mac OS
- New: Added support for Hollywood 7.0's help string feature
- Fix: Documentation wrongly documented "http.SetConfig" when it had to be
"httpstreamer.SetConfig"
- New: Added "Encoded" option; when this is set to TRUE using SetConfig(), HTTP
Streamer will expect
  encoded URLs instead of unencoded URLs (requested by Lazar Zoltan)
- Fix: HTTP Streamer didn't handle HTTP redirect requests correctly (reported by
Dominique Vanderveken)

Version 1.0:    (26-Sep-15)
- First release


Future
======

Direct seeking using HTTP range requests would be nice but not all servers
support it.


Bugs
====

Please report any bugs or issues via the Hollywood forums at
http://forums.hollywood-mal.com/


Copyright
=========

This plugin is (C) Copyright 2014-2026 by Andreas Falkenhahn
<andreas()airsoftsoftwair.de>
Refer to the COPYING file in this package for conditions concerning distribution
of this plugin. Visit http://www.hollywood-mal.com/ for more information
on Hollywood and more plugins.