README.md
1[](https://travis-ci.org/warmcat/libwebsockets) [](https://ci.appveyor.com/project/lws-team/libwebsockets) [](https://scan.coverity.com/projects/3576) [](https://bestpractices.coreinfrastructure.org/projects/2266) [](https://www.codacy.com/app/lws-team/libwebsockets?utm_source=github.com&utm_medium=referral&utm_content=warmcat/libwebsockets&utm_campaign=Badge_Grade) [](https://lgtm.com/projects/g/warmcat/libwebsockets/alerts/) [](https://lgtm.com/projects/g/warmcat/libwebsockets/context:cpp) [](https://lgtm.com/projects/g/warmcat/libwebsockets/context:javascript)
2
3# Libwebsockets
4
5Libwebsockets is a simple-to-use, pure C library providing client and server
6for **http/1**, **http/2**, **websockets**, **MQTT** and other protocols in a security-minded,
7lightweight, configurable, scalable and flexible way. It's easy to build and
8cross-build via cmake and is suitable for tasks from embedded RTOS through mass
9cloud serving.
10
11[80 independent minimal examples](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples) for
12various scenarios, CC0-licensed (public domain) for cut-and-paste, allow you to get started quickly.
13
14
15
16News
17----
18
19## v4.0 is released
20
21Users wanting a stable branch should follow v4.0-stable to get the most stable version
22at any given time.
23
24See the [changelog](https://libwebsockets.org/git/libwebsockets/tree/changelog) for
25information on the huge amount of new features in this release, and additional information
26below.
27
28```
29 - NEW: Lws is now under the MIT license, see ./LICENSE for details
30
31 - NEW: GLIB native event loop support, lws + gtk example
32
33 - NEW: native lws MQTT client... supports client stream binding like h2 when
34 multiple logical connections are going to the same endpoint over MQTT, they
35 transparently and independently share the one connection + tls tunnel
36
37 - NEW: "Secure Streams"... if you are making a device with client connections
38 to the internet or cloud, this allows separation of the communications
39 policy (endpoints, tls cert validation, protocols, etc) from the code, with
40 the goal you can combine streams, change protocols and cloud provision, and
41 reflect that in the device's JSON policy document without having to change
42 any code.
43
44 - NEW: lws_system: New lightweight and efficient Asynchronous DNS resolver
45 implementation for both A and AAAA records, supports recursive (without
46 recursion in code) lookups, caching, and getaddrinfo() compatible results
47 scheme (from cache directly without per-consumer allocation). Able to
48 perform DNS lookups without introducing latency in the event loop.
49
50 - NEW: lws_system: ntpclient implementation with interface for setting system
51 time via lws_system ops
52
53 - NEW: lws_system: dhcpclient implementation
54
55 - NEW: Connection validity tracking, autoproduce PING/PONG for protocols that
56 support it if not informed that the connection has passed data in both
57 directions recently enough
58
59 - NEW: lws_retry: standardized exponential backoff and retry timing based
60 around backoff table and lws_sul
61
62 - NEW: there are official public helpers for unaligned de/serialization of all
63 common types, see eh, lws_ser_wu16be() in include/libwebsockets/lws-misc.h
64
65 - NEW: lws_tls_client_vhost_extra_cert_mem() api allows attaching extra certs
66 to a client vhost from DER in memory
67
68 - NEW: lws_system: generic blobs support passing auth tokens, per-connection
69 client certs etc from platform into lws
70
71 - NEW: public helpers to consume and produce ipv4/6 addresses in a clean way,
72 along with lws_sockaddr46 type now public. See eg, lws_sockaddr46-based
73 lws_sa46_parse_numeric_address(), lws_write_numeric_address()
74 in include/libwebsockets/lws-network-helper.h
75
76 - Improved client redirect handling, h2 compatibility
77
78 - NEW: lwsac: additional features for constant folding support (strings that
79 already are in the lwsac can be pointed to without copying again), backfill
80 (look for gaps in previous chunks that could take a new use size), and
81 lwsac_extend() so last use() can attempt to use more unallocated chunk space
82
83 - NEW: lws_humanize: apis for reporting scalar quanties like 1234 as "1.234KB"
84 with the scaled symbol strings passed in by caller
85
86 - NEW: freertos: support lws_cancel_service() by using UDP pair bound to lo,
87 since it doesn't have logical pipes
88
89 - NEW: "esp32" plat, which implemented freertos plat compatibility on esp32, is
90 renamed to "freertos" plat, targeting esp32 and other freertos platforms
91
92 - NEW: base64 has an additional api supporting stateful decode, where the input
93 is not all in the same place at the same time and can be processed
94 incrementally
95
96 - NEW: lws ws proxy: support RFC8441
97
98 - NEW: lws_spawn_piped apis: generic support for vforking a process with child
99 wsis attached to its stdin, stdout and stderr via pipes. When processes are
100 reaped, a specified callback is triggered. Currently Linux + OSX.
101
102 - NEW: lws_fsmount apis: Linux-only overlayfs mount and unmount management for
103 aggregating read-only layers with disposable, changeable upper layer fs
104
105 - Improvements for RTOS / small build case bring the footprint of lws v4 below
106 that of v3.1 on ARM
107
108 - lws_tokenize: flag specifying # should mark rest of line as comment
109
110 - NEW: minimal example for integrating libasound / alsa via raw file
111
112 - lws_struct: sqlite and json / lejp translation now usable
113
114
115```
116
117## Introducing Secure Streams client support
118
119Secure Streams is an optional layer above lws (`-DLWS_WITH_SECURE_STREAMS=1`) that
120separates connectivity policy into a JSON document, which can be part of the
121firmware or fetched at boot time.
122
123Code no longer deals with details like endpoint specification or tls cert stack used
124to validate the remote server, it's all specified in JSON, eg, see
125[this example](https://warmcat.com/policy/minimal-proxy.json). Even the protocol to use to talk to the
126server, between h1, h2, ws or MQTT, is specified in the policy JSON and the code
127itself just deals with payloads and optionally metadata, making it possible to
128switch endpoints, update certs and even switch communication protocols by just
129editing the JSON policy and leaving the code alone.
130
131Logical Secure Stream connections outlive any underlying lws connection, and support
132"nailed-up" connection reacquisition and exponential backoff management.
133
134See [./lib/secure-streams/README.md](https://libwebsockets.org/git/libwebsockets/tree/lib/secure-streams/README.md) and the related minimal examples
135for more details.
136
137## mqtt client support
138
139If you enable `-DLWS_ROLE_MQTT=1`, lws can now support QoS0 and QoS1 MQTT client
140connections. See the examples at ./minimal-examples/mqtt-client
141
142## libglib native event loop support
143
144glib's event loop joins libuv, libevent and libev support in lws for both the
145`lws_context` creating and owning the loop object for its lifetime, and for
146an already-existing "foreign loop" where the `lws_context` is created, attaches,
147detaches, and is destroyed without affecting the loop.
148
149This allows direct, lock-free integration of lws functionality with, eg, a GTK app's
150existing `GMainLoop` / glib `g_main_loop`. Just select `-DLWS_WITH_GLIB=1` at cmake
151time to enable. The -eventlib minimal examples also support --glib option to
152select using the glib loop at runtime.
153
154There's also a gtk example that is built if lws cmake has `-DLWS_WITH_GTK=1`.
155
156## `lws_system` helper for attaching code to a single event loop from another thread
157
158`lws_system` ops struct now has a member that enables other threads (in the
159same process) to request a callback they define from the lws event loop thread
160context as soon as possible. From here, in the event loop thread context,
161they can set up their lws functionality before returning and letting it
162operate wholly from the lws event loop. The original thread calling the
163api to request the callback returns immediately.
164
165## Improvements on tx credit
166
167H2 clients and servers can now modulate RX flow control on streams precisely,
168ie, define the size of the first incoming data and hand out more tx credit
169at timing of its choosing to throttle or completely quench the remote server
170sending as it likes.
171
172The only RFC-compatible way to acheive this is set the initial tx credit to
1730 and set it explicitly when sending the headers... client code can elect to
174do this rather than automatically manage the credit by setting a new flag
175LCCSCF_H2_MANUAL_RXFLOW and indicating the initial tx credit for that stream
176in client connection info member manual_initial_tx_credit. A new public api
177lws_wsi_tx_credit() allows dynamic get and add to local and estimated remote
178peer credit for a connection. This api can be used without knowing if the
179underlying connection is h2 or not.
180
181## `lws_system`: DHCP client
182
183DHCP client is now another network service that can be integrated into lws, with
184`LWS_WITH_SYS_DHCP_CLIENT` at CMake. When enabled, the `lws_system` state
185is held at `DHCP` until at least one registered network interface acquires a
186usable set of DHCP information including ip, subnet mask, router / gateway
187address and at least one DNS server.
188
189See the [api-test-dhcp](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-dhcpc) Minimal Example for how to use.
190
191## UDP integration with `lws_retry`
192
193UDP support in lws has new helper that allow `lws_retry` to be applied for retry,
194and the ability to synthesize rx and tx udp packetloss systemwide to confirm
195retry strategies. Since multiple transactions may be in flight on one UDP
196socket, the support relies on an `lws_sul` in the transaction object to manage
197the transaction retries individually.
198
199See `READMEs/README.udp.md` for details.
200
201## `lws_system`: system state and notification handlers
202
203Lws now has the concept of systemwide state held in the context... this is to
204manage that there may be multiple steps that need the network before it's possible
205for the user code to operate normally. The steps defined are
206
207`CONTEXT_CREATED`, `INITIALIZED`, `IFACE_COLDPLUG`, `DHCP`, `TIME_VALID`, `POLICY_VALID`,
208`REGISTERED`, `AUTH1`, `AUTH2`, `OPERATIONAL` and `POLICY_INVALID`. OPERATIONAL is the
209state where user code can run normally.
210
211User and other parts of lws can hook notifier callbacks to receive and be able to
212veto system state changes, either definitively or because they have been triggered
213to perform a step asynchronously and will move the state on themselves when it
214completes.
215
216By default just after context creation, lws attempts to move straight to OPERATIONAL.
217If no notifier interecepts it, it will succeed to do that and operate in a
218backwards-compatible way. Enabling various features like lws ntpclient also enable
219notifiers that hold progress at the related state until their operation completes
220successfully, eg, not able to enter `TIME_VALID` until ntpclient has the time.
221
222See `READMEs/README.lws_system.md` for details.
223
224## `lws_system`: HAL ops struct
225
226Lws allows you to define a standardized ops struct at context creation time so your
227user code can get various information like device serial number without embedding
228system-specific code throughout the user code. It can also perform some generic
229functions like requesting a device reboot.
230
231See `READMEs/README.lws_system.md` for details.
232
233## `lws_system`: ntpclient
234
235Optional lws system service enabled by cmake `-DLWS_WITH_SYS_NTPCLIENT` intercepts
236the `lws_system` `TIME_VALID` state and performs ntpclient to get the date and time
237before entering `TIME_VALID`. This allows user code to validate tls certificates
238correctly knowing the current date and time by the time it reached OPERATIONAL.
239
240## Connection Validity tracking
241
242Lws now allows you to apply a policy for how long a network connection may go
243without seeing something on it that confirms it's still valid in the sense of
244passing traffic cohernetly both ways. There's a global policy in the context
245which defaults to 5m before it produces a PING if possible, and 5m10 before
246the connection will be hung up, user code can override this in the context,
247vhost (for server) and client connection info (for client).
248
249An api `lws_validity_confirmed(wsi)` is provided so user code can indicate
250that it observed traffic that must mean the connection is passing traffic in
251both directions to and from the peer. In the absence of these confirmations
252lws will generate PINGs and take PONGs as the indication of validity.
253
254## `lws_system`: Async DNS support
255
256Master now provides optional Asynchronous (ie, nonblocking) recursive DNS resolving.
257Enable with `-DLWS_WITH_SYS_ASYNC_DNS=1` at cmake. This provides a quite
258sophisticated ipv4 + ipv6 capable resolver that autodetects the dns server on
259several platforms and operates a UDP socket to its port 53 to produce and parse DNS
260packets from the event loop. And of course, it's extremely compact.
261
262It broadly follows the getaddrinfo style api, but instead of creating the results
263on the heap for each caller, it caches a single result according to the TTL and
264then provides refcounted const pointers to the cached result to callers. While
265there are references on the cached result it can't be reaped.
266
267See `READMEs/README.async-dns.md` for detailed information on how it works, along
268with `api-tests/api-test-async-dns` minimal example.
269
270## Detailed Latency
271
272You can now opt to measure and store us-resolution statistics on effective
273latencies for client operations, and easily spool them to a file in a
274format suitable for gnuplot, or handle in your own callback. Enable
275`-DLWS_WITH_DETAILED_LATENCY=1` in cmake to build it into lws.
276
277If you are concerned about operation latency or potential blocking from
278user code, or behaviour under load, or latency variability on specific
279platforms, you can get real numbers on your platform using this.
280
281Timings for all aspects of events on connections are recorded, including
282the time needed for name resolution, setting up the connection, tls
283negotiation on both client and server sides, and each read and write.
284
285See `READMEs/README.detailed-latency.md` for how to use it.
286
287## Client connection logic rewrite
288
289Lws master now makes much better use of the DNS results for ipv4 and ipv6... it
290will iterate through them automatically making the best use it can of what's
291provided and attempting new connections for each potentially usable one in turn
292before giving up on the whole client connection attempt.
293
294If ipv6 is disabled at cmake it can only use A / ipv4 records, but if ipv6 is
295enabled, it tries both; if only ipv6 is enabled it promotes ipv4 to
296::ffff:1.2.3.4 IPv4-in-IPv6 addresses.
297
298## New network helpers for ipv4 and ipv6
299
300An internal union `lws_sockaddr46` that combines `struct sockaddr_in` and
301`struct sockaddr_in6` is now public, and there are helpers that can parse (using
302`lws_tokenize`) any valid numeric representation for ipv4 and ipv6 either
303into byte arrays and lengths, or directly to and from `lws_sockaddr46`.
304
305## h2 long poll support
306
307Lws now supports the convention that half-closing an h2 http stream may make
308the stream 'immortal', in terms of not being bound by normal timeouts. For
309the client side, there's an api that can be applied to the client stream to
310make it transition to this "read-only" long poll mode.
311
312See `READMEs/README.h2-long-poll.md` for full details, including how to test
313it with the minimal examples.
314
315## h1 client parser improvements
316
317H1 is not so simple to parse because the header length is not known until it
318has been fully parsed. The next header, or http body may be directly coalesced
319with the header as well. Lws has supported bulk h1 parsing from a buffer for a
320long time, but on clientside due to interactions with http proxying it had
321been stuck parsing the header bytewise out of the tls buffer. In master,
322everything now bulk parses from a buffer and uses a buflist to pass leftovers
323through the event loop cleanly.
324
325## `lws_sul` time refactor
326
327Just before v3.2 there was a big refactor about how lws handles time. It now
328explicitly schedules anything that may happen in the future on a single, sorted
329linked-list, at us resolution. When entering a poll wait (or returning to an
330event lib loop) it checks the interval between now and the earliest event on the
331list to figure out how long to wait if there are no network events. For the
332event loop case, it sets a native event lib timer to enforce it.
333
334See `READMEs/README.lws_sul.md` for more details and a handy api where you can
335schedule your own arbitrary callbacks using this system.
336
337## Master is now MIT-licensed
338
339Libwebsockets master is now under the MIT license. See ./LICENSE.
340
341## Support
342
343This is the libwebsockets C library for lightweight websocket clients and
344servers. For support, visit
345
346 https://libwebsockets.org
347
348and consider joining the project mailing list at
349
350 https://libwebsockets.org/mailman/listinfo/libwebsockets
351
352You can get the latest version of the library from git:
353
354- https://libwebsockets.org/git
355
356Doxygen API docs for master: https://libwebsockets.org/lws-api-doc-master/html/index.html
357
358
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
