packages
The Louise packages, each released on its own
Clone avec HTTPS, un jeton personnel en mot de passe, ou avec SSH, une clé ajoutée dans vos paramètres.
fetch
An HTTP/1.1 client in pure Louise for fetching what a URL points to: one
GET, over TCP or TLS, guarded against server-side request forgery (SSRF).
Built for link previews; depends on packages/url (ADR 0079).
Needs engineering review before production use: it is an SSRF surface. ADR 0079 lists the open questions.
import fetch.{Options, Response}
use config <- result.try (tls.config (tls.defaults ())) // once, and share it
use page <- result.try (url.parse "https://example.com/post")
let opts = Options(..fetch.options config, max_body: 2097152)
match fetch.get page [("User-Agent", "toki/1")] opts
{ Ok(Response(200, headers, body, truncated)) -> ...
, Ok(Response(status, ..)) -> ... // 3xx included: redirects are not followed
, Error(e) -> ...
}
API
pub type Policy { PublicOnly | AnyAddress }
pub type Options { Options(tls: Resource<TlsConfig>, policy: Policy,
max_body: Int, max_head: Int, deadline: Int) }
pub fn options :: config: Resource<TlsConfig> -> Options / <clock>
pub fn deadline_in :: ms: Int -> Int / <clock>
pub type Response { Response(status: Int, headers: List<(String, String)>,
body: Bytes, truncated: Bool) }
pub fn header :: resp: Response name: String -> Option<String>
pub type Error
{ BadUrl(String) | BadHeader(String) | Refused(host: String, ip: String)
| NoAddress(String) | Connect(String) | Tls(TlsError) | Malformed(String) | TimedOut }
pub fn get :: target: Url headers: List<(String, String)> opts: Options
-> Result<Response, Error> / <proc, clock>
pub fn request :: method: String target: Url headers: List<(String, String)>
body: Bytes opts: Options -> Result<Response, Error> / <proc, clock>
pub type PendingRequest
pub type RequestOutcome { Completed(Result<Response, Error>) | Cancelled }
pub fn start_request :: method: String target: Url headers: List<(String, String)>
body: Bytes opts: Options -> Result<PendingRequest, Error> / <proc, clock>
pub fn await_request :: PendingRequest -> RequestOutcome / <proc, clock>
pub fn cancel_request :: PendingRequest -> () / <proc>
requestisgetwith any method and a body (ADR 0079, Revision 1):fetch.request "POST" api [("Content-Type", "application/json")] body opts. The body is sent with itsContent-Length(a POST or PUT without one says0); the caller's ownContent-Lengthis dropped with the other framing headers. The method must be a token.getisrequest "GET" … (bytes.empty ()).options configisPublicOnly, a 1 MiB body, a 64 KiB head and a deadline 10 seconds from now. Change it with a record update.deadlineis an instant oftime.monotonic, in milliseconds:deadline_in 10000. One deadline can cover a whole redirect chain.headerignores case and gives the first value.
Cancellable collected requests
start_request starts the same checked exchange as request and returns its
handle instead of waiting. await_request collects the head and body, returning
Completed with the ordinary Result, or Cancelled. request and get keep
their blocking behavior.
The starter owns the handle's reply and monitor subjects and must call
await_request exactly once, including after cancellation. Any actor holding
the handle may call cancel_request: it stops only this exchange, closes its
connection as the exchange actor unwinds, and is harmless before, during, or
after completion. The starting actor's await_request removes the monitor and
discards a reply racing with the kill. A response completed before cancellation
can still be Completed; work killed before its reply cannot become a late
success.
The deadline covers time between starting and awaiting as well as the wait, and
still returns Completed(Error(TimedOut)). As with the deadline of request, a
kill that lands inside net.resolve takes effect when the lookup returns; the
caller does not wait for it.
For example, a coordinator can share the handle with a stop actor:
use pending <- result.try (fetch.start_request "POST" api headers body opts)
// Send pending to the actor responsible for stop/navigation too.
match fetch.await_request pending
{ Completed(answer) -> use_answer answer
, Cancelled -> reader_left ()
}
The stop actor calls fetch.cancel_request pending. This does not cancel other
requests to the same URL or host.
What get does
- Checks before any connection. Only
httpandhttps. The URL is parsed again from its text, so a hand-builtUrlgets the same checks as a parsed one. A header name must be a token and a value must not hold a CR, LF or NUL (BadHeader). - Resolves once, checks every address.
net.resolvegives the host's addresses. UnderPublicOnly, if any of them is not public (fetch/address), the fetch isRefused(host, ip). Then it connects to those same addresses, in order, until one accepts: the name is not resolved again, so a second lookup cannot answer differently (DNS rebinding). - TLS for
https, with the caller's configuration. The host name is the server name sent (SNI) and the name the certificate must have. - Sends
GET <path?query> HTTP/1.1,Host(with the port when it is not the scheme's),Connection: close,Accept-Encoding: identity(there is no decompression), then the caller's headers. The caller'sHost,Connection,Accept-Encoding,Content-Length,Transfer-Encoding,TEandUpgradeare dropped. - Reads the head (at most
max_headbytes), skipping1xxresponses, then the body byContent-Length,chunked, or until the server closes. AnotherTransfer-Encoding, disagreeingContent-Lengths, or a connection that ends before the body does areMalformed. A TLS stream that ends withoutclose_notifyends a body read until close. - Keeps at most
max_bodybytes and stops reading there, withtruncated: True. - Never follows redirects. A 3xx is returned as it is. To follow one,
resolve
Locationwithurl.resolveand callgetagain with the same deadline: the new host is checked like the first. - Deadline. The exchange runs in an actor of its own. At the
deadline the caller kills it and answers
TimedOut, wherever it was waiting. A kill that lands while the actor waits innet.resolve(on theioblocking pool) takes effect when the lookup returns; the caller does not wait for that.
Streamed responses
For server-sent events and other long answers, request_streaming returns
once the response head has arrived. You then read the body as it comes
(ADR 0079, Revision 2):
pub type Streamed { Streamed(status: Int, headers: List<(String, String)>, body: BodyStream) }
pub type Piece { Data(Bytes) | End(truncated: Bool) }
pub fn request_streaming :: method: String target: Url headers: List<(String, String)>
body: Bytes opts: Options -> Result<Streamed, Error> / <proc, clock>
pub fn next :: stream: BodyStream -> Result<Piece, Error> / <proc, clock>
pub fn cancel :: stream: BodyStream -> () / <proc>
- Same checks as
request: the URL, the headers, the address policy and the limits. A failure before the head is theErrorofrequest_streaming. - Reading:
nextgives the body's bytes as they arrive (a chunked body already decoded), thenEnd. AfterEndor anErrorthe connection is closed. - Deadline:
opts.deadlinecovers the whole exchange, body included, so give a long answer a long deadline. At the deadlinenextanswersTimedOutand stops the exchange. - Cancelling:
cancelcloses the connection, so the server sees the client leave, and drops what arrived unread. Call it when you stop reading early. It is harmless after the end. - Memory: at most
max_bodybytes are delivered, so a reader that falls behind holds at most that much in its mailbox. - Ownership: only the actor that made the request can read the stream.
fetch/sse parses server-sent events (text/event-stream) from those
pieces:
import sse
let (parser, events) = sse.feed parser piece // [Event(name: "delta", data: "Bon", id: "")]
It follows the HTML standard's rules:
- lines end with CRLF, LF or CR, even when one is split across pieces;
- a blank line dispatches an event that has data;
event,dataandidare read, and theidpersists from one event to the next;retryis readable withsse.retry;- comments and unknown fields are ignored;
- a leading byte order mark is dropped.
Reconnecting is the caller's choice.
fetch/address
classify ip -> Result<Reach, String> with Reach { Public | Private(reason) },
and is_public ip. Refused ranges:
| IPv4 | IPv6 |
|---|---|
0.0.0.0/8 this network | :: unspecified, ::1 loopback |
10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 private | fc00::/7 unique local |
100.64.0.0/10 carrier-grade NAT | fe80::/10 link-local, fec0::/10 site-local |
127.0.0.0/8 loopback | ff00::/8 multicast |
169.254.0.0/16 link-local (cloud metadata) | ::/96 IPv4-compatible |
224.0.0.0 and up: multicast, reserved, broadcast |
::ffff:a.b.c.d (IPv4-mapped) and 64:ff9b::/96 (NAT64) are judged by
the IPv4 address they hold. Only the forms net.resolve gives are read:
anything else (leading zeros, a zone, a host name) is an Error, and
PublicOnly refuses it.
fetch/response
The pure parsers get uses, tested on their own: head data max_head
(Ok(None) while incomplete), framing head, and a chunked decoder fed
as bytes arrive (dechunker max, feed, is_done, body, and take,
which hands out what was decoded so far for a streamed body).
Tests
The parsers have property tests in both back ends. The loopback tests
(std/http, a raw TCP server, a TLS server with the fixtures of
test/fixtures/tls) need sockets, which only native code
has, so they run when LOUISE_NET_TESTS is set:
LOUISE_NET_TESTS=1 louise test --native
The streamed responses' tests (test/stream_test.lse) use a raw server that writes a response in steps the test controls. They check that:
- a piece arrives before the rest of the body is sent;
- the cap applies to a body by length and to one read until close;
- the deadline stops a stalled body;
cancelcloses the connection, as the server sees it;- failures before the head are errors.
The pending-request tests use similarly controlled loopback servers to check cancellation before the response, during a partial body and during a TLS handshake; observed connection closes; repeated cancellation and late answers; successful completion and deadlines. Repeated exchanges exercise cleanup across fresh handles and PIDs.
The SSE parser's tests, including a property that any split of a stream parses the same, run in both back ends.