Mirage_crypto_rng_eio.run now.Restorer.Table.create now takes a switch (@talex5 #306). The table is now cleared when the switch finishes. Avoids having to remember to set this up manually.Build and opam metadata:
x-maintenance-intent to opam files (@talex5 #315, requested by @hannesm).capnp-rpc 2.0 switches from using Lwt to Eio. The capnp-rpc-lwt package is now just Lwt wrappers around the Eio functions in capnp-rpc. This allows libraries using the old Lwt API to be used with applications and libraries using the new Eio one, in the same binary. Because Lwt libraries (using capnp-rpc-lwt) can be used with Eio applications (using capnp-rpc-unix), you should first upgrade your application and then upgrade the libraries afterwards.
It is recommended to upgrade in stages, checking your application still works after each step:
opam install capnp-rpc-unix.1.2.4). This uses the new version of mirage-crypto, tls, etc, which may be incompatible with other libraries you are using. Get that sorted out before switching to capnp-rpc 2.0.Lwt_main.run with code to run Eio and Lwt together, as explained at the start of the lwt_eio README.Capnp_rpc_unix and Capnp_rpc_net modules will need minor updates. They were previously called from Lwt context, so you'll need to wrap them with Lwt_eio.run_eio (or remove a Lwt_eio.run_lwt if you already updated the surrounding code).Upgrade code and libraries using Capnp_rpc_lwt:
open Capnp_rpc_lwt with open Capnp_rpc.Std.Capnp_rpc_lwt with just Capnp_rpc.dune and opam files, replace capnp-rpc-lwt with capnp-rpc.Capnp_rpc but not the Capnp_rpc.Std subset. Those should now be fully qualified (e.g. replace Persistence with Capnp_rpc.Persistence).Service.return_lwt with Lwt_eio.run_lwt. Replace Lwt.return (Ok x) with Service.return x.Once all Lwt code is gone, you can remove the dependencies on lwt and lwt_eio.
New features:
API changes:
Eio port (@talex5 #280 #284 #298 #292 #297 #300 #304).
This switches capnp-rpc from Lwt to Eio. One particularly nice side effect of this is that Service.return_lwt has gone, as there is no distinction now between concurrent and non-concurrent service methods.
Uses of Capnp_rpc_lwt should be replaced by uses of Capnp_rpc (and the old Capnp_rpc is now an internal module, Capnp_rpc_proto). The new Capnp_rpc now provides Error, Exception and Debug aliases to the same modules in Capnp_rpc_proto, so that Capnp_rpc_proto doesn't need to be used directly.
Capnp_rpc_lwt now provides (deprecated) compatibility wrappers, using lwt_eio.
This also adds Capnp_rpc.Std with some common module aliases, to reduce the need to open Capnp_rpc (which is rather large).
Debug.failf (@talex5 #291).Performance and bug fixes:
Capability.call (@talex5 #293). It was previously unusable.close problem (@talex5 #302).Build:
String.starts_with instead of Astring (@talex5 #290).tls-mirage dependency (@talex5 #289).asetmap dependency (@talex5 #288).Update dependencies and remove capnp-rpc-mirage (@talex5 #276).
Unix.ENOTCONN under macos (@talex5 #252).Capnp_rpc_unix.Cap_file.save_uri (@talex5 #253).wait_until_settled with safer await_settled (@talex5 #233). wait_until_settled makes it too easy to ignore errors.kstr and set OCaml 4.08 as the minimum version everywhere (@talex5 #232). Older versions aren't tested in CI because the unix and mirage packages require at least OCaml 4.08.Service.fail_lwt convenience function (@talex5 #229).call_for_value_exn errors (@talex5 #230). The hidden information is now logged (at debug level) instead.object_id if it is empty (@LasseBlaauwbroek #224). This improves interoperability with the C++ implementation.open_in_bin instead of open_in for Windows compatibility (@MisterDA #222).Capnp_rpc_unix.with_cap_exn to show a progress indicator while waiting to connect. It will use the log if enabled. If not, but stderr is a tty, it shows a message while connecting and then erases it (if connecting takes longer than 0.5s). If stderr is not a tty, it just prints a message there.Mirage_crypto_rng_lwt.initialize instead of Mirage_crypto_rng_unix.initialize (@hannesm #217). The former periodically feeds entropy to the RNG, while the latter seeds the RNG only once.SO_KEEPALIVE option. However, the default keepalive timeout of 2 hours is much too long for some networks. In particular, Docker's libnetwork silently drops idle connections after about 10 minutes.Windows support:
have_sockopt_int isn't available and an exception is raised. This may be the case on some environments (e.g., ExtUnix built with mingw-w64). It is a workaround until support for TCP_KEEPIDLE is added for these environments (e.g., using Winsock instead).capnpc seems to be the old name, and isn't present on Windows.Build changes:
tcpip.unix specifically to get the checksum stubs linked (@hannesm #216).Bug fixes and diagnostics:
Network.connect fails (@talex5, #206). Also, use Lwt to open connections so we don't block the whole process.New functions:
Sturdy_ref.with_cap and with_cap_exn convenience functions (@talex5, #207). Using the plain connect functions, it's easy to forget to release the live-ref at the end.Build changes:
Documentation:
test-bin/calc_direct.ml example shows how a parent process can spawn a child and communicate with it directly over a socketpair.Logging:
-lwt package merge (@avsm #190, review by @talex5 @hannesm)Breaking changes:
capnp-rpc-lwt have been split off into a new capnp-rpc-net package (@talex5, #182). This means that libraries that provide or consume capnp-rpc services can just depend on capnp-rpc-lwt, without pulling in a full TLS stack. Only applications should need to depend on capnp-rpc-net (which will probably happen automatically via capnp-rpc-unix or capnp-rpc-mirage). If you get compile errors in your code due to this change, you probably just need to open Capnp_rpc_net.Other changes:
capnp-rpc-net now reports the current number of active connections, the total number of messages received, and the total number of messages enqueued, sent and dropped (from which you can work out the current number of queued messages).Callback module (@talex5, #177).mirage-flow-unix from opam file (@talex5, #179).Breaking changes:
`Capnp tag to make it easier to compose with other types of error (#172, #173).capnp- (#163). e.g. --listen-address is now --capnp-listen-address. The old names were confusing for applications that supported other protocols too (e.g. a web server).New features:
Capability.with_ref convenience function (#170). This automatically calls dec_ref when done.Cap_file module to load and save Sturdy_refs (#165). In particular, this ensures that saved cap files get a mode of 0o600, since they contain secrets.uint (which is deprecated) to the newer stdint (#166, #168). The latest version of uint is just a wrapper around stdint, so this shouldn't break anything if you are using the latest version.Capnp_rpc_unix.manpage_capnp_options to control where in your man-page they appear.Enable SO_KEEPALIVE for TCP connections (#167). For use with Docker's libnetwork, try something like this in your stack.yml:
sysctls:
- 'net.ipv4.tcp_keepalive_time=60'Bug fixes:
Build changes:
dune files to allow duniverse / vendored builds (#165).Adjust to mirage-stack / mirage-protocols changes (Nick Betteridge, #151).
x509 API and for OCaml 4.06 (#143).Build updates:
capnp to pull it in.unix/network.ml uses Fmt.failwith, which requires fmt.0.8.4 (#139).capnp-rpc-lwt requires Uri.with_userinfo, which is only in uri >= 1.6.0 (#138).This release adds a new capnp-rpc-mirage package, which provides support for using the library within a MirageOS unikernel. See https://github.com/mirage/capnp-rpc#how-can-i-use-this-with-mirage for details.
There are a few minor API changes:
Capnp_rpc_unix.Vat_config.derived_id ?name config is now Capnp_rpc_unix.Vat_config.derived_id config name. If you weren't passing a ~name argument before, use "main" to get the same ID.Capnp_rpc_unix.Network's Socket_address module is now called Location.Network.connect, etc. This is needed to support Mirage, where the network isn't a global.Bug fixes:
Documentation and examples:
Fuzzing:
Code cleanups:
Alcotest_lwt for unit-tests.capnp:// URI handling to Capnp_rpc_lwt.Capnp_address. This allows it to be shared with the Mirage code.Capnp_rpc_lwt.VAT_NETWORK with simpler signature than S.VAT_NETWORK.S.NETWORK is now available separately as S.ADDRESS.This release brings support for RPC Level 2.
The API for implementing services and clients is mostly unchanged, but the APIs for setting up networking are very different. If you read the tutorial with the 0.1 release, you will probably want to read the new version again from the "Networking" point onwards.
The main change is that when connecting to a service you now give a URI of the form:
capnp://hash:digest@address/service
The client will connect to address, check the server's public key matches hash:digest, and then pass the (secret) service ID to get access to a particular service. The server will typically display the URI to use on start-up, or write it to a file.
The communications are encrypted using TLS. If you want to disable TLS, use the form capnp://insecure@address. This should only be needed for interoperability with non-TLS services, as the system will generate keys and certificates automatically, making secure use just as easy as the non-secure case.
The other major new feature is support for persistent services. In version 0.1 you could specify an offer argument when creating a vat, telling it a service to provide in response to bootstrap requests. Now, you pass a restore argument, which can restore different services depending on the service ID provided by the client.
The new Restorer.Table module provides a table-based lookup restorer, to which services can be added dynamically. If you have a lot of services and don't want to add them all at startup, you can use Restorer.Table.of_loader and provide your own function for loading services.
Documentation changes:
Client sub-module is gone, and service becomes local..mli files and there is a new store.ml example demonstrating persistence. The examples have been updated to the new layout convention.Main API changes:
The Capnp_rpc_lwt.Capability module adds some useful functions:
broken creates a broken capability.when_broken allows you to be notified when a capability breaks (e.g. because of a network failure).wait_until_settled waits until a promise has resolved, if you don't want to pipeline (e.g. you want to send a large amount of data, so prefer to find out where the service is and avoid any forwarding).equal tests if two capabilities designate the same service.Capnp_rpc_lwt.Sturdy_ref module provides an abstraction for off-line capabilities. Sturdy_ref.connect can be used to get a live connection. If you try to connect to multiple services in the same vat, it will share a single connection automatically. Sturdy_ref.reader and Sturdy_ref.builder can be used for passing sturdy refs in messages.Capnp_rpc_lwt.Restorer module is used to implement sturdy-refs at the hosting side.Capnp_rpc_lwt.Persistence module provides support for the Cap'n Proto persistence protocol. Clients use Persistence.save to request a sturdy ref from a service, and services can use Persistence.with_sturdy_ref to answer such requests automatically.Capnp_rpc_unix.Vat_config collects together all vat configuration in one place.Capnp_rpc_unix.File_store can store Cap'n Proto structs in a directory. It can be useful when implementing persistence.Capnp_rpc_lwt.Auth module provides support for generating and handling secret keys and fingerprints.Capnp_rpc_lwt.Tls_wrapper provides support for doing TLS handshakes, with authentication and encryption.In the core capnp-rpc package (which applications do not normally use directly):
cap#when_released method can be used to find out when a capability is no longer needed. The restorer uses this to remove dynamically loaded services from its table automatically.when_broken helper adds a callback to call when a promise or far-ref becomes broken.NETWORK_TYPES is now separate from CORE_TYPES.