This is a point release to fix a regression in 5.1.0 on compilers earlier than 4.07.
The release also fixes the test suite and CI on compilers greater than 4.07.
Pervasives (#269 @cypheon @hannesm)[@len] attribute to ensure it is a valid, positive integer (#265 @emillon)Js_of_ocaml 3.5.0 interfaces (@hhugo #268)Security: This release tightens bounds checks to ensure that data outside a given view (but still inside the underlying buffer) cannot be accessed.
sub does more checks (#244 #245 @hannesm @talex5 review by @dinosaure)add_len and set_len are now deprecated and will be removed in a future release. (#251 @hannesm)Remove Unix dependency: cstruct now uses the new bigarray-compat library instead of Bigarray directly, to avoid a dependency on Unix when using OCaml compilers less than 4.06.0. This will break downstream libraries that do not have a direct dependency on Bigarray. Simply fix it in your library by adding a bigarray dependency in your dune file. (#247 @TheLortex)
Capability module: To improve the safety of future code with stronger type checking, this release introduces a new Cstruct_cap module which makes the underlying Cstruct an abstract type instead of a record. In return for this extra abstraction, the module can enforce read-only, write only, and read/write buffers by tracking them as phantom type variables. Although this library shares an implementation internally with classic Cstruct, it is a significant revision and so we will be gradually migrating to it. Feedback on it is welcome! (#237 @dinosaure and many excited reviewers)
Ppx compare functions: A new compare_X function is generated for cenum declarations. This respects custom ids supplied in the cenum declaration and so is more robust than polymorphic compare (#248 @emillon)
The CI has also been switched over to both Azure Pipelines and Drone in addition to Travis, and as a result the tests all run on Windows, macOS, various Linux distributions, on x86 and arm64 machines, and runs AFL fuzz tests on the Drone cloud (#255 @avsm).
Sexplib is now an optional library for the base Cstruct module. A new Cstruct_sexp module has been introduced with the serialiser functions, contained within the cstruct-sexp opam package.
To convert old code, simply use Cstruct_sexp.t instead of Cstruct.t in a record type for which you are using [@@deriving sexp]. This is a type alias to Cstruct.t but also has the right sexp-conversion functions in scope. There is an example of this in the ppx_test/with-sexp directory in the source repo.
When you have converted and released your library, add an opam constraint of cstruct {>="4.0.0"} to your own opam packages to ensure that they pick up this version of the library. (fixes #222, @avsm)
_ by skipping code generation but still respecting the space usage of that field. This was a convention before but is now enforced by the code generator to save space in the output (#233 @emillon)[%%cstruct type ...] declaration generates many values that are potentially unused. The code generator in ppx_cstruct now guarantees that there will be no more "unused value" (warning 32) statements from use of the ppx form. (#228 @emillon)Cstruct.rev to allocate a reversed cstruct (#221 @emillon)Cstruct_unix now uses the post-OCaml 4.06 Unix.map_file instead of the deprecated Bigarray map_file that was removed in OCaml 4.08 (@avsm, see ocaml/ocaml#2263)(wrapped false) in the build system (@avsm)cstruct-ppx package pointer (@avsm)cstruct.lwt, cstruct.async, cstruct.ppx and cstruct.unix. These were deprecated in cstruct.3.0.0 in favour of counter part libraries with a dash in the name (cstruct-lwt, cstruct-async, cstruct.unix) or ppx_cstruct for the PPX extension. (@avsm)BE and LE modules to use as needed.of_string and of_bytes to have an option ?off offset argument into the source bytes. (#208 by @XVilka)ppx_driver. This was primarily there for older compilers, and new uses should be based around ppxlib. (#201 by @edwintorok).hexdump_pp output in a box (#175 by @cfcs)ocplib-endian (#177 by @hannesm)of_hex: string -> t (#179 by @hannesm and @pqwy)to_bytes: t -> Bytes.t (#183 by @hannesm)empty: t (#184 by @hannesm)check_alignment now treats a large alignment as an unsigned value and so doesnt raise a signal (#171 by @yallop)void * pointer arithmetic and have more portable headers in the C stubs (#170 by @fdopen)Cstruct.lenv and copyv (#159 by @yallop)blit (#160 by @yallop)of_bigarray and sub, now bith fixed (#164 by @talex5)cstruct-async package build, and depend on the latest Async packages (>="v0.9.0") as part of this. (#152 @jnfoster)ppx_driver based ppx converters (most notably ppx_sexp_conv. If you are having trouble with using ppx_cstruct with other drivers, put a constraint on ppx_cstruct>=3.0.1. (#151 #150 #149 #148 via @djs55 @g2p @avsm @diml).cstruct and lwt working together, but this will not work until a lwt>3.0 release happens.jbuilder subst for version information in the distribution.check_alignment expects a negative result.cstruct, and then separate cstruct-lwt, cstruct-async, cstruct-unix packages, and a ppx_cstruct package for the syntax extension. Transitional findlib packages with the old scheme are available, but now packages should migrate to using cstruct-async instead of cstruct.async for example. This has the added benefit of the OPAM package names now matching the findlib names. (#138 by @avsm @rgrinberg).check_alignment only takes a non-zero argument for alignment (#143 #145 by @cfcs @avsm).ppx_tools_versioned dependency (#136, @let-def)Distribute the PPX extension so that it is compatible with Jbuilder. ppx_cstruct is now distributed as both a library and a binary. Findlib predicates are used to distinguish usage:
-package cstruct.ppx alone uses the binary for rewriting.-package cstruct.ppx -predicates custom_ppx,ppx_driver" is used to link the rewriter.To use the PPX extension in jbuilder, just add:
(libraries (cstruct))
(preprocess (pps (cstruct.ppx)))to your jbuild file. This may be renamed to ppx_cstruct in a future release so that the PPX dependency is decoupled from the main library, so this cstruct.ppx is intended to be transitional as it is what was originally used.
ocaml-migrate-parsetree so it should also compile on future revisions of OCaml (#127 via @let-def).Cstruct.of_bytes/to_bytes. In common with the existing implementation, this relies on the representation of bytes and string being the same, which is true as of OCaml 4.04 and lower (#105 via @yallop).create zero out the new buffer. The new create_unsafe function can be used if you want to trade safety for speed.hexdump_pp that uses the Format module. This works better with the Logs library than using hexdump_to_buffer, and also makes it easy to indent the hexdump (#100 via @talex5).@@enum values.Cstruct.check_alignment into the stubs. Before this patch we returned the buffer address from C and then calculated using OCaml's boxed Int64.t. This patch reduces minor allocations by performing the calculation in the C stubs. This makes the function suitable for use in an assert in a performance sensitive path.-safe-string in OCaml 4.02 upwards. The main change is to rename blit_to_string to blit_to_bytes and change its type so that it writes to bytes rather than string (#74 by @yallop).camlp4 in the base library. The sexplib functions were only used in the interface, so replace them with manually written ones. This also enables compatibility with latest Core that has switched to ppx.LE.get_uint16 on invalid bounds (#75)fillv. If the source didn't fit in the buffer then we skipped the amount we wanted to copy, not the amount actually copied (#77).Cstruct.concat and Cstruct.append (#57, @pqwy)js_of_ocaml stubs (#63, #64, @djs55)memset to set all the bytes of a cstruct value efficiently (#49)Invalid_argument parameters (#48).to_sexp to expose only the current view (#44 from David Kaloper).compare and equal (#23, #24 and #45 from David Kaloper).fillv to copy over a list of buffers (from Thomas Leonard).camlp4 an optional build-time dependency (#35).ounit as a dependency in the opam file.opam description file for OPAM 1.2 workflow (#36).Comprehensive addition of bounds checking to all cstruct operations (from @pqwy in #33). The major changes are:
sub and shift operations.of_bigarray cannot create invalid cstruct values.Cstruct.BE/LE functions that violate a view. Previously, only bounds errors on the underlying buffers would raise. Bug #25, reported by Mindy Preston in mirage/mirage-tcpip#56.read/write operatiosn run to completion.Sexplib conversion functions to Cstruct.t values (#27 #22).Add a sexp optional decorator to cenum to output the values as s-expressions. This is compatible with the sexplib convention. The syntax is;
cenum foo64 {
ONE64;
TWO64;
THREE64
} as uint64_t(sexp)And sexp_of_foo64 and foo64_of_sexp functions will also be available. The representation of the Sexp is the string representation of the enum.
to_bigarray to convert back into a Bigarray slice.ocaml-ipaddr).-Wall.uint8.test.sh script compilation.VAL = 0xffffffffl, useful for 32-bit hosts.cenum definition.sizeof_<field> binding for every get/set field (should be no externally observable change).Cstruct.hexdump_to_buffer to make spooling hexdump output easier.hexdump_foo and hexdump_foo_to_buffer prettyprinting functions for a cstruct foo.Async_cstruct.Pipe to map pipes of Cstruct buffers to strings or Bigsubstring.Bigsubstring.of_string function to simplify the construction from OCaml values.cstruct.obuild for the obuild build tool.Cstruct.debug to dump internal state of a buffer to a string.set_len and add_len to manipulate the total-length field directly.sendto, read and recvfrom functions to the Lwt subpackage.Cstruct.t a record type that doesn't use Bigarray slicing to provide views onto buffers. This lets views be allocated directly on the minor heap rather than forcing a major heap allocation. It does alter the external API, so previous users of cstruct wont work.xen and unix subdirectories, as the portable Bigarray is now provided by the xenbigarray package.string_to_<cenum> function to match the <cenum>_to_string, primarily to help with command-line parsing of enum arguments..mli files.xen/ version over to using OASIS also.