add_validate_parsing_flag parameter to Command.run, enabling a -validate-parsing flag in all subcommands. The command immediately exits with a return code of 0 if argument parsing is successful.Sexp_grammar support to Interval.In Linux_ext:
Epoll implementation.memfd_create syscall.?follow_symlinks:bool parameter to Extended_file_attributes.getxattr and Extended_file_attributes.setxattr functions.Diffable support to Time_float_unix and Time_ns_unix.Time_float_unix and Time_ns_unix.Changes to Bigstring_unix:
pread and really_preadChanges to Command_test_helpers:
parse_command_line_or_error functionChanges to Core_unix:
fork_exec now accepts an optional preexec_fn parameter
proggetpgid return type changed to Pid.t option
Nonegetlogin to usernameNPROCESSORS_CONF and NPROCESSORS_ONLN variants to sysconfget_all_ifnames functionChanges to Filename_unix:
close_on_exec parameter (default: false) to open_temp_file_fdChanges to Interval_lib:
Add S.Set.to_list function
Add Interval_unix, a library that defines interval types and functions for Unix specific types, like Time_unix and Time_ns_unix.
Changes to Iobuf_unix:
Add In_channel_optimized module
In_channel with better performance using an intermediate Iobuffold_lines, iter_lines, input_lines, fold_lines_rawChanges to Linux_ext:
Add tcp_string_option type
TCP_CONGESTION variant for getting/setting the congestion-control algorithm for a socketgettcpopt_string and settcpopt_string helper functionsAdd optional pid parameter to setpriority and getpriority
Changes to Lock_file_blocking:
Update blocking_create function:
max_retry_delay parameter with a default value of min(300ms, timeout / 3)random parameter with a default value of a system-dependent low-entropy seedUpdate Flock.lock_exn function:
exclusive parameter (default: true)close_on_exec parameter (default: true)Changes to Sys_unix:
Add with_async_exns function
Break or Stack_overflow exceptions from finalizers, signal handlers, or the GC to be raised from the call site of with_async_exnsRename Time_unix to Time_float_unix
Changes to Time_ns_unix:
format function, following man strftime formatting rulesparse function, following man strptime formatting rulesTimezone.Stable module as ZoneAs Core is built on top of Base and Core_kernel you might want to have a look at Base's changelog and Core_kernel's changelog.
Unix.map_file.Iobuf.{of,to}_bytes.Md5.digest_bigstring and used it to implement a more efficient version of Md5.digest_bin_prot.Piecewise_linear and Crc.Weak_hashtbl was moved out into its own library (which is still part of the core package)Syslog out of Core.Unix into its own library core.syslog.get_bind_to_interface function to read current status of SO_BINDTODEVICE socket option. Make both bind_to_interface and get_bind_to_interface use new type Bound_interface_name.t for its argument/return value.Time. (fixes #102)Float.to_string as to_string_12, to reflect its 12-digit precision. And introduce a new Float.to_string with the behavior of Float.to_string_round_trippable.Command.exec support for specifying another executable via argv.(0) rather than Sys.executable_name.Lock_file to not delete lock files in forked processes.Linux_ext.Epoll, reduced closure allocationPiecewise_linear for Time_nsIobuf.Expert.reinitialize_of_bigstring, which allows users to use an Iobuf while being passed a bigstring, without allocatingCommand to no longer print the entire help on parse errors.[@@deriving hash] to Identifiable.S* and Hashable.S*. Added functors Identifiable.Make*_and_derive_hash_fold_t, which generate hash_fold_t from hash.Iobuf inlining annotations for Flambda.Schedule a Zoned_between constructor, which allows the span to cross the midnight boundary, and allows a different time zone for the start and end ofdays. Because of the new expressive power, renamed the existing Schedule to Schedule_v4_deprecated and added the new functionality in Schedule_v5.Schedule_v5.Between to support crossing the midnight boundary.[@@deriving hash] in Uuid.Command to display build time in the local zone.Command.exec to allow executing a subcommand of the child executable.Command.Param.flag_optional_with_default_doc, which takes a default value and formats that value into the doc.Command.lazy_group and Command.Sexpable.Lazy, so that help for lazy commands and groups of commands doesn't have to force the lazy unless necessary.Filename and Unix functions that generate temporary filenames and directories to include .tmp. in the name, so one can reliably recognize them.Daemon.daemonize_wait to obey stdout/stderr redirection even when they are already pointing at a regular file.Time.Format's handling of zones.Linux_ext, added support for the TCP_QUICKACK socket option, for use on TCP connections where latency is importantSys.quote function for quoting bash input, like Filename.quote, but with more readable output.Signal for signals that were added in OCaml 4.03: bus, poll, sys, trap, urg, xcpu, xfsz.Udp.sendto for MacOS.Linux_ext.Eventfd module, with bindings for eventfd(2).Iobuf.unsafe_blit.Inet_addr.inet4_addr_to_int32_exn, removing an intermediate string allocation.Time_ns comparison functions to use the Int63 primitives; they had been shadowed to use slower generic operations exported by Identifiable.Make.Unix.Rlimit a Limit submodule, with min and max functions.Unix.Inet_addr.inet4_addr_of_int63, for efficient conversion from an Int63.t to an Inet_addr.t.Unix.create_process_env's execvp emulation w.r.t environment handling, PATH, and allowed nonfatal errors.Unix.sysconf to return an option, and to correctly detect when there is no value.Unix.IOVec.max_iovecs lazy, to avoid calling sysconf at top-level.Time.Ofday.Zoned.compare, which was misleading because it does not correspond to a temporal ordering. The comparison is now available as Time.Ofday.Zoned.With_nonchronolgical_compare.Time_ns functions: of_time, local_now, of_local_time. These functions do not exist in Time.Ofday and are simple wrappers around existing functions.Command.basic as basic_spec. Renamed Command.basic' as basic. We want to encourage the use of Command.Param and discourage the use of Command.Spec.Uuid.t_of_sexp.-safe-string.Uuid.sexp_of_t to show the nil UUID when am_running_inline_test.Interval.Time_ns.Ofday's to_string and sexp_of_t functions to output nine decimal places rather than six.Uuid.t_of_sexp, in favor of Unstable.t_of_sexp and Stable.V1.t_of_sexp.Unix.Iovec.of_string and of_bigstring so that the default len includes the suffix of the input starting at pos, as usual.Unix.create_process's support for the old ml_create_process backend, leaving it with only the spawn_vfork backend.Time.Ofday.of_string supports "AM and "PM" suffixes.Command.Flag so it no longer uses mutable state. It now updates a Univ_map.t instead.Renamed:
core/test --> core/test-bin
since these directories contain executables to run rather than libraries with standard unit tests. This is in preparation for moving the standard unit tests to a more "normal" test directory.
Core.Std.Lock_file doesn't work properly on OSX, see this:
https://github.com/janestreet/jenga/issues/4#issuecomment-205176593
This fix the issue by requiring both flock and lockf only on Linux.
Closes janestreet/jenga#4
Core.Core_list, which was not in Core.Std and had only an unused to_sequence.Shift operations are unspecified outside of the range 0 <= x < bitsize.
This fixes unit tests for core/src/core_unix.ml in 32bit.
Replace all occurences (in Core) of Time_ns.Span.to_int_ns and Time_ns.to_int_ns_since_epoch with their 63bit counterpart.
Time_ns.Span.to_int_ns and Time_ns.to_int_ns_since_epoch are not implemented in 32bit.
The is one step toward have unit tests to pass in 32bit.
New arg types in Command.Param:
sexp = Arg_type.create Sexp.of_string
sexp_conv a_of_sexp = Arg_type.create (fun s -> a_of_sexp (Sexp.of_string s))
Time_ns.Ofday.create presently makes 2 function calls to caml_int_compare. Changed to use the native int comparisons and avoid the calls in create (and other comparisons).endian.h.Command.Command.Param.choice represents a sum type.Command.Spec.of_param for conversion from new-style to old style command line specifications. This is expected to be useful for gradually converting Spec-based commands into Param-based ones.flag_names and anon_names gives the flags and anons used by a Command.Param.t, while mnemonic combines them to give a string that's good for referring to that param in error messages.
This feature improves the interface of Command.Param.one_of by saving the caller from passing the flag names separately.
Date0.create_exnAdd Ofday.Option to time_ns.ml.
Exports Time_ns.Span.Option.unchecked_value and .Ofday.Option.unchecked_value!
Finally fix Time_ns.Of_day.of_local_time w.r.t. DST transitions after regression test failures at the DST transition.
As a result, clarified slightly the role of Time_ns.Ofday, like Time.Ofday, as 24h wall-clock times, not linear time offsets, which latter would have to be up to 25h for DST.
Command.Spec.Arg_type.percent.Stable.V1 to Time_ns.Ofday.Option.t, add it to Core.Stable, and standardize the interface to Option modules in Time_ns.Time_ns.to_ofday.coretop to use ppx_janeUpdated corebuild to use ppx-jane instead of camlp4
Closes #70
N.B. Some interface change were made which are not listed here, as they are only cascading from core_kernel. Look at core_kernel's CHANGES.md file to get a complete history of the changes made for this release.
Core.Command.Arg_type.Export.time_zoneFix Command.shape to run external programs only once to get their sexp.
Introduces a new variant of Command.t called Proxy. The Exec variant represents the top-level command of an external executable that has not yet been run; the Proxy variant represents an arbitrary subcommand that has been extracted by running an external executable. Command.exec constructs an Exec variant; Command.shape of an Exec variant runs the executable and generates a tree of Proxy variants representing all the information from the generated sexp, so the executable will not need to be re-run.
A version of recvmmsg that pre-allocates and reuses the iovec record. Profiling indicates this is a non-trivial amount of our I/O loop (under very heavy load).
Since nobody is using the heavyweight features of the existing recvmmsg, replace it with the lightweight one. This leads to minor but important changes to the interfaces of Iobuf.recvmmsg_assume_fd_nonblocking and Udp.recvmmsg_loop.
Time.set_sexp_zone affects t_of_sexp
If we're willing to read sexps without zones, we should be willing to let you control what timezone they're read with.
Llimiter.Interval.Int implement Container and Binary_searchable.Identifiable.S to Unix.Cidr so that it supports hash tables. Update Unix.Cidr.t to normalize values, e.g. "192.168.1.101/24" ==> "192.168.1.0/24".Linux_ext.unsafe_timerfd_settime.In Iobuf, made some functions take:
(> write, _) Iobuf.t
rather than:
(read_write, _) Iobuf.t
if they only need to write to the iobuf and don't need to read it.
Time.of_string_abs didn't support ISO 8601 time zone strings without colons, or those specified as locations. This version adds support for time zones without colonsUnix.Passwd.getpwents takes the lock, partially applies Exn.protect, then releases the lock, then completes the application and actually runs stuff.Command.file-completion
Files are now completed correctly when paths contain a directory.
Previously, completion when pointed at a directory would put a space at the end. This would cause the user to hit backspace every time a directory was in the path.
diff_weekdays and diff_weekend_days functions to date module.Time.to_date_ofday_precise implements a complete inverse for of_date_ofday. This is needed to give the DWIM-est semantics to Schedule.t that we can think of.Reduce allocation in Linux_ext.EpollTime_ns.Of_day.(add_exn, sub_exn, diff).head_padded_fixed_string to Iobuf.Core_extended.Std.Sys.home to Core.Std.Sys.home_directory.Iobuf.{read,write,input,output} akin to the bigstring versions.Int63.t conversions for types in Time_ns.Inet_addr sexp conversions to expose their blocking nature.Unix.Inet_addr.Stable.Fixed Iobuf_packet.iter, which behaved incorrectly if the packet didn't start at the lo_min of the iobuf. It used Iobuf.rewind when it should have used Iobuf.Lo_bound.restore.
Added @@deriving compare to Iobuf.Bound.
Time.format for a specific time zoneMake more information available via Command.Shape.t
Expose machine-readable info on anonymous arguments.
(^/) in core_filename.ml. It had one call site and wasn't exposed. The (^/) everyone uses comes from std.mlAdd getifaddrs to Core.Std.Unix.
Handles Packet (for interfaces that do not have an address on Linux systems only), IPv4 and IPv6 address families.
Time_ns.Span.to_string_hum by analogy to Time.Span.to_string_hum. The code and tests are essentially copied from "lib/core/src/span.ml".Remove our stubs for Unix.stat.
They were upstreamed in 4.02.2.
Tweaked Unix.stat's C code to reduce float rounding error, by using double-precision rather than single-precision floats.
Following Xavier's comment: http://caml.inria.fr/mantis/view.php?id=6285
Date.O module.Interval, exposed compare in stable types by having the appropriate modules match the Stable signature.Iobuf.Fill.decimal and Poke.decimal, for efficiently writing integers in decimal format.Filename.O (and /^), since it's not used that much, and is inconsistent with the older operator for this that's exposed in Core.Std: ^/.Improved Command autocompletion to work even if some arguments can't be parsed.
This is useful because the completion mode does not get fed precisely the same arguments that it would get if you hit RETURN.
As a simple example, if completion is set up for my-exe which takes a sexp as its first argument, then:
my-exe '(a b)' <TAB>will run my-exe in completion mode with:
'(a b)'as its first argument, rather than:
(a b)as would be passed if RETURN had been pressed instead. Since Sexp.of_string "'(a b)'" fails, in this example tab completion won't work.
Changed the internals a bit to make this possible. Most notably, the Parser module is now applicative rather than monadic, which required only a few simple changes to support.
Made Time_ns.to_time and Time_ns.Span.to_span round to the nearest microsecond in all cases.
Previously, Time_ns.Span.to_span sometimes rounded incorrectly for negative spans.
Added Time.Zone.prev_clock_shift, the analog of next_clock_shift.
val prev_clock_shift
: t
-> before:Time_internal.T.t
-> (Time_internal.T.t * Span.t) optionImplemented next_clock_shift and prev_clock_shift using Array.binary_search_segmented.
Lock_file.get_pid : string -> Pid.t option.val random: unit -> int to Time_ns and Time_ns.Span.Renamed Iobuf.sub as Iobuf.sub_shared.
This more closely matches Bigstring, and clarifies the semantics of Iobuf.sub vs sub in Blit_intf.S.
Iobuf blit modules: Blit, Blit_consume, Blit_fill, Blit_consume_and_fill.Added Piecewise_linear.first_knot and last_knot.
val first_knot : t -> (key * value) option
val last_knot : t -> (key * value) optionUnix.Cidr match Comparable.S_binable, and added Cidr.create and Cidr.netmask_of_bits.Unix.tm and Unix.strftime from Core_kernel to Core.Made Crc.crc32 return Int63.t rather than int64, and added Crc.bigstring_crc32 and Iobuf.crc32.
Cleaned up old cruft in the C stubs for CRC checking.
Iobuf.narrow_lo and narrow_hi, which comprise Iobuf.narrow.Changed Linux_ext.Timerfd, Epoll.wait, and Unix.select to use (int) Time_ns rather than (float) Time.
This avoids spurious float conversions and rounding problems.
Made all timeouts consistently treat negative timeouts as "timeout immediately".
This fixes an incorrect behavior of Linux_ext.Timerfd.set_after and set, which had been rounding to the nearest microsecond, which was particularly bad for time spans smaller than 500ns, which would be rounded to zero, and then would cause the timerfd to never fire. Now, the small span is directly fed to timerfd_settime. We also changed a span of zero to be treated as 1ns, to avoid the behavior of timerfd_settime 0, which causes the timerfd to be cleared and never fire.
Or_error match Applicative.S.Added Command.Param, with the intention of one day replacing Command.Spec and providing an applicative interface for command-line parsing.
This change required lots of rearrangement of command.mli so that Command.Param and Command.Spec could share large portions of their interface. As a side effect, the interface is more sweeksy than before.
Command.shape, for exposing the shape of a command, including what subcommands its has.Changed Syscall_result.to_result to return a preallocated object, and changed many uses to take advantage of this property.
Pattern matching on results is much clearer than if-analysis and avoids double-checking errors in many cases.
Syscall_result.to_result can only return preallocated results for a few Ok values from Syscall_result.Int, of course, and likewise for other large ok_value types. We have initially, and arbitrarily, limited preallocation to 64 errno's and 2048 ok_value's.
Sys.big_endian : bool, from Caml.Sys.Disabled unit tests in Time_ns that started failing around 10:40pm NYC 2015-05-15.
The tests indicate an off-by-one-microsecond error in round tripping between Time.Span.t and Time_ns.Span.t.
Dequeue as Deque.Fdeque, a functional deque (a double-ended Fqueue, or a functional Deque).Changed Fqueue's bin-io format, and added a stable type.
Deprecated deque-like functions in favor of Fdeque.
Fheap, a functional heap implementation based on pairing heaps.Or_error's bin-io format made in 112.17, going back to the format in 112.16 and before.Unix.env type a ``Replace_rawvariant, as used inexecandfork_exec`.Array.Permissioned module, which has a permissioned array type and permissioned versions of all the regular array functions.Date.week_number : t -> int.Added values to Day_of_week: of_int_exn, iso_8601_weekday_number, weekdays.
val of_int_exn : int -> t
val iso_8601_weekday_number : t -> int
val weekdays : t list (- [ Mon; Tue; Wed; Thu; Fri ] *)Float IEEE functions to use Int63.t for the mantissa rather than int, so they work on 32-bit platforms.length field to the Map.t record, making Map.length O(1) rather than O(n).Time_ns from Core to Core_kernel, enough so that Async_kernel can use Core_kernel.Time_ns and ultimately only depend on Core_kernel.Time_ns 32-bit Linux.Bounded_int_table.clear.Fixed the module_name passed to Identifiable.Make for a number of modules.
The module name must be an absolute module path.
Reported here: https://github.com/janestreet/core/issues/52
Tuple.Binable functor, for making binable tuples.Sped up a Time_stamp_counter unit test.
Time_stamp_counter unit test has an 18s unit test, which seems excessive. Take a couple of orders of magnitude off the number of iterations.
Added Time_ns.pause, whose implementation is the same as Time.pause.
This involved moving the nanosleep C code from Core to Core_kernel.
This was necessary so that Async_kernel can pause without introducing a dependence of Async on Core.
Made Core_kernel.Time_ns.Alternate_sexp use a similar format to Core.Time_ns.
This was needed so that Async_kernel can use a nice sexp format for time spans.
Changed Timing_wheel implementation to use Time_ns, and moved to Core_kernel.Timing_wheel_ns; made Core.Timing_wheel a wrapper around Timing_wheel_ns.
Generalized the timing-wheel interface to be parametric in Time, so that one interface applies to both Timing_wheel and Timing_wheel_ns.
Generalized the timing-wheel unit tests to a functor, Timing_wheel_unit_tests.Make, that is used to test both Timing_wheel_ns and Timing_wheel_float. Moved a few tests that depend on Time and Date from the functor into timing_wheel_float_unit_tests.ml.
Split out Timing_wheel.Debug into a separate functor, Timing_wheel_debug.Make.
This was done in so that Async_kernel can depend only on Core_kernel and not Core.
Added optional arguments to Command.group: ?body and ?preserve_subcommand_order.
preserve_subcommand_order : unit causes subcommands to be in the order they are specified, rather than sorted.
body : (path:string list -> unit) is called when no additional arguments are passed.
Command.summary : t -> string.Time.Span robust comparison.Changed Command's tab-completion bash code so that it is possible for programs to return completions containing spaces.
Actually knowing when and how to do so is difficult, because of course there's escaping to worry about. Adding helper functions to make that sort of thing manageable is left for future work.
Command, made the -version and -build-info flags work at the top level when there are subcommands.Sequence.interleaved_cartesian_product, which implements cartesian product of potentially infinite sequences.Deprecated the single-line files that simply include the corresponding Core_kernel module. Those are unnecessary, because people should use Core.Std.
We will keep these aliases around for a version before deleted them entirely.
Removed Time.Zone.find_office.
Replaced uses with the still-blocking Time.Zone.find_exn
Made many changes to Time to make the time zone explicit instead of implicitly using the local timezone.
Added zone:Time.Zone.t parameters to many functions. In almost all cases, used ~zone:Time.Zone.local where previously it was implicit.
Removed of_local_ofday and to_local_ofday in favor of the explicit versions (with Time.Zone.local).
Removed Time.Zone.machine_zone () in favor of local.
Core.Std.With_return.Core.Std.with_return_option.Time_ns.Ofday.of_span_since_start_of_day to check its input.Time_ns.to_span and of_span to round to microseconds, for round trippability.Unix.Error module, for the Unix.error type.Added Unix.Syscall_result, a new abstract type representing the result of a Unix system call as an int, to avoid allocation.
A lot of Unix system calls return an integer on success, so for ones that are called a lot, we can encode errors as -errno. This module abstracts this concept.
Iobuf.recvmmsg functions to return the new Unix.Syscall_result.Unix.exec's ?env argument to support extending the environment in addition to replacing it.with compare to Unix.Exit.t and Unix.Exit_or_signal.t.Moved Backtrace to Core_kernel.
Deleted backtrace_stubs.c, now that we have Printexc.get_callstack.
Bigstring.read_assume_fd_is_nonblocking and send_nonblocking_no_sigpipe to return Unix.Syscall_result.t, to reduce allocation.Iobuf.send_nonblocking_no_sigpipe to handle EINTR like EAGAIN, instead of raising.Command.Spec.char.Process_env.parse_ssh_client to accept an SSH_CLIENT that is just IP address without ports.Renamed Linux_ext.gettid as Unix.gettid, and added OpenBSD support.
SYS_gettid is not available on OpenBSD, but is used in Core_extended. See the mailing list discussion about this here:
https://groups.google.com/forum/#!topic/ocaml-core/51knlnuJ8MM
Seems like the OpenBSD alternative is:
pid_t getthrid(void);although it's not defined in any header file, which is a bit unfortunate.
Piecewise_linear.precache, which computes a lookup table that speeds up subsequent calls to Piecewise_linear.get.Time_ns module, representing times as 63-bit integers of nanoseconds since the epoch.unix_stubs.c on OpenBSD.Daemon, fixed an error message regarding WSTOPPED (fixes #47).Added Time.Span.Stable.V2, with sexps that use new suffixes for microseconds (us) and nanoseconds (ns).
Time.Span.of_string supports the new format, but Time.Span.to_string doesn't yet produce it -- we plan to change that later, after the new of_string has made it out more widely.
Time.Span.to_string_hum, which gives more options for rendering time spans.Merged the recvmmsg stubs in Bigstring and Iobuf.
Factored out a shared underlying recvmmsg call that both stubs use.
Restored -pedantic by avoiding a C99 feature (variable-length stack arrays).
Date.t abstract, and changed its representation from a 4-word record to an immediate int (packing year, month, day).Daemon, changed the permissions of the std{err,out} files generated during daemonization from 0o777 to 0o644.Moved Thread_safe_queue from core to core_kernel.
This was done so that Async_kernel can use it, eliminating one of Async_kernel's dependencies on Core.
Thread_safe_queue_unit_tests remains Core, at least for now, because it has some dependencies on other stuff in Core.
Added Command support for flags that are passed one or more times.
Added Command.Spec.one_or_more and Command.Spec.non_empty_sequence to deal with the cases where you expect a flag or anonymous argument (respectively) to be passed one or (optionally) more times. This is common enough and distinct from the case where you want the argument passed zero or more times that it seems like we should canonize it in the library.
In Lock_file, made stale lock detection more robust.
Made Lock_file.create foo succeed if foo is absent and foo.nfs_lock file is present and stale. Previously, it would fail.
Syslog.syslog's add_stderr argument; use the PERROR option instead.Fixed unix_stubs.c compilation on NetBSD
Closes #45
Filename operators /^ and /@, and of_parts, like the same functions for Catalog paths.Changed Iobuf functions that advance the iobuf to not also return a redundant number of bytes processed.
This avoids a small allocation (in the case of the int option functions) and normalizes the result (so the same information isn't returned two ways). Actually, it doesn't yet avoid the allocation in the implementation, as the corresponding Bigstring functions must still return the number of bytes processed, and currently do so as an option. We hope to eventually change that.
In the future I expect we will change unit to some error variant to also avoid the exception construction for EWOULDBLOCK/EAGAIN. We can even make Unix syscalls noalloc if we're careful.
Unix module, added unit tests for Cidr.does_match.Added Piecewise_linear.create_from_linear_combination.
val create_from_linear_combination : (t * float) list -> t Or_error.tTime.is_{earlier,later} : Time.t -> than:Time.t -> bool, which are easier to read than Time.(<) and friends.Added Command.exec, which allows one to include the Command hierarchy from one executable in another.
Command.exec takes the file path to an executable that uses the Command module and returns a Command.t that integrates the executable (by exec'ing it), including providing recursive help and autocompletion as if it were a standard Command.t.
Hashtbl.replace with Hashtbl.set.Float.epsilon to robust_comparison_tolerance, to avoid confusion with epsilon_float.Gc.disable_compaction function.Time.to_string_abs_trimmed, which prints a trimmed time and takes a zone argument.unix_stubs.c to suppress a warning when building with some versions of gcc.Changed Time.Zone to allow the zoneinfo location to be specified by an environment variable.
Closes #40
Time.Zone.init would not properly traverse the directory containing timezone information.Time.With_utc_sexp, which has stable serialization of Time.t that is byte-for-byte equal across timezones.Uuid stable.Piecewise_linear stable.Fixed a bug in Bigstring.really_recv if recv doesn't receive all the data it wants.
This bug has been around forever; it may not have caused trouble because Bigstring.really_recv (1) is barely used (the only use is in Bigstring.unmarshal_from_sock) and (2) passes recv the MSG_WAITALL flag, so it will read the full amount unless it gets interrupted by a signal.
Fixed Bigstring.read's handling of EINTR so that it retries rather than returning zero.
This fixes a bug introduced in 111.09 in the interaction between Bigstring.read and Async.Reader. Prior to 111.09, Bigstring.read would raise on EINTR, and Async.Reader would propagate the exception. From 111.09 to 111.16, Bigstring.read would return zero, which would confuse Async.Reader into thinking it reached EOF when it hadn't. From 111.17, Bigstring.read will retry and not return zero when not at EOF.
We believe the bug was rare, because otherwise we would have frequently seen EINTR exceptions prior to 111.09.
Added Command.Spec.apply and pair, which allow one to program more with Spec.param rather than Spec.t.
val apply : ('a -> 'b) param -> 'a param -> 'b param
val pair : 'a param -> 'b param -> ('a * 'b) paramAdded Command.Spec.file, which builds an Arg_type value with the same autocompletion as Spec.file.
(** [file] defines an [Arg_type.t] that completes in the same way as
[Command.Spec.file], but perhaps with a different type than [string] or with an
autocompletion key. *)
val file
: ?key:'a Univ_map.Multi.Key.t
-> (string -> 'a)
-> 'a tChange some Bigstring functions to retry on EINTR rather than raise.
The following functions (and their unsafe versions) were affected:
readreally_readreally_recvreally_writereally_send_no_sigpipeSome other Bigstring functions, like input and output, already retried on EINTR, so this change has precedent.
All of the affected stubs raise Bigstring.IOError on failure, rather than Unix_error, which means the normal method for retrying on EINTR doesn't work. In particular Async.Reader didn't retry them, even though it was supposed to.
Additionally, the documentation for the following functions was corrected to say that they raise =Unix_error= rather than =IOError=:
pread_assume_fd_is_nonblockingpwrite_assume_fd_is_nonblockingEliminated global binary-to-decimal tables computed at startup for converting Date and Of_day to string.
Used an efficient implementation of division by 10, rather than the sprintf tables in time_internal.ml. This result in much less allocation at startup and it is a bit faster:
Name | Time/Run | mWd/Run | Percentage |
|---|---|---|---|
Date.to_string | 69.39ns | 3.00w | 100.00% |
Name | Time/Run | mWd/Run | Percentage |
|---|---|---|---|
Date.to_string | 53.38ns | 3.00w | 100.00% |
Time.Zone tests so that they are deterministic.Iobuf.to_string_hum, which produces a readable, multi-line representation of an iobuf, intended for debugging.Command.Improved Command to print a good error message if command-line parsing raises.
Command's Exn.handle_uncaught now protects the phase of parsing command-line arguments in addition to protecting the phase of running the main function as it did already.
Added inline benchmarks for =Iobuf= and =Time=.
Hera are some of the results from the new benchmarks, with some indexed tests dropped.
Name | Time/Run | mWd/Run | Percentage |
|---|---|---|---|
[time.ml:Time] Time.to_string | 848.74ns | 249.98w | 100.00% |
[time.ml:Time] Time.to_ofday | 59.66ns | 38.00w | 7.03% |
[time.ml:Time] Time.now | 39.78ns | 2.00w | 4.69% |
[time.ml:Time] Time.Zone.find_office | 83.64ns | 4.00w | 9.85% |
[time.ml:Time] Time.Span.of_hr | 3.71ns | 2.00w | 0.44% |
[time.ml:Time] Time.Span.of_min | 3.69ns | 2.00w | 0.44% |
[time.ml:Time] Time.Span.of_sec | 2.72ns | 0.32% | |
[time.ml:Time] Time.Span.of_ms | 6.02ns | 2.00w | 0.71% |
[time.ml:Time] Time.Span.of_ns | 5.98ns | 2.00w | 0.71% |
Name | Time/Run | Percentage |
|---|---|---|
[iobuf.ml:Blit tests] functor blit:5 | 15.53ns | 7.66% |
[iobuf.ml:Poke tests] char:0 | 4.11ns | 2.03% |
[iobuf.ml:Poke tests] uint8:0 | 5.35ns | 2.64% |
[iobuf.ml:Poke tests] int8:0 | 4.59ns | 2.26% |
[iobuf.ml:Poke tests] int16_be:0 | 5.19ns | 2.56% |
[iobuf.ml:Poke tests] int16_le:0 | 5.14ns | 2.53% |
[iobuf.ml:Poke tests] uint16_be:0 | 5.11ns | 2.52% |
[iobuf.ml:Poke tests] uint16_le:0 | 5.12ns | 2.53% |
[iobuf.ml:Poke tests] int32_be:0 | 5.17ns | 2.55% |
[iobuf.ml:Poke tests] int32_le:0 | 4.91ns | 2.42% |
[iobuf.ml:Poke tests] uint32_be:0 | 5.73ns | 2.83% |
[iobuf.ml:Poke tests] uint32_le:0 | 5.74ns | 2.83% |
[iobuf.ml:Poke tests] int64_be:0 | 5.33ns | 2.63% |
[iobuf.ml:Poke tests] int64_le:0 | 4.93ns | 2.43% |
[iobuf.ml:Peek tests] char:0 | 5.50ns | 2.71% |
[iobuf.ml:Peek tests] uint8:0 | 4.68ns | 2.31% |
[iobuf.ml:Peek tests] int8:0 | 4.91ns | 2.42% |
[iobuf.ml:Peek tests] int16_be:0 | 5.19ns | 2.56% |
[iobuf.ml:Peek tests] int16_le:0 | 4.90ns | 2.42% |
[iobuf.ml:Peek tests] uint16_be:0 | 5.17ns | 2.55% |
[iobuf.ml:Peek tests] uint16_le:0 | 5.10ns | 2.51% |
[iobuf.ml:Peek tests] int32_be:0 | 5.17ns | 2.55% |
[iobuf.ml:Peek tests] int32_le:0 | 4.92ns | 2.42% |
[iobuf.ml:Peek tests] uint32_be:0 | 5.45ns | 2.69% |
[iobuf.ml:Peek tests] uint32_le:0 | 5.46ns | 2.69% |
[iobuf.ml:Peek tests] int64_be:0 | 6.61ns | 3.26% |
[iobuf.ml:Peek tests] int64_le:0 | 6.31ns | 3.11% |
Re-implemented Thread_safe_queue to improve performance and reduce allocation.
The new implementation requires 3 words per element, down from the 7 words required by the old implementation.
The new implementation pools elements so that they can be reused, so there is no allocation in steady-state use.
The new implementation has dequeue_exn rather than dequeue, so that one can dequeue without allocating 2 words.
Eliminated create'. One should just use create and explicit calls to enqueue and dequeue_exn.
Eliminated dequeue_until_empty. One should use an explicit while loop guarded by length and using dequeue_exn.
Moved Thread_safe_queue from Core_kernel to Core, since it's thread related.
All in, there is now no allocation in a steady-state usage of enqueueing and dequeueing elements, as opposed to 9 words per enqueue+dequeue in the old implementation. This reduces the cost from enqueue+dequeue taking 166-216ns to enqueue+dequeue_exn taking 48-82ns (plus eliminating gc impacts). Here are some BENCH results, the first table being the old implementation, and the second table the new.
Name | Time/Run | mWd/Run | mjWd/Run |
|---|---|---|---|
[thread_safe_queue.ml] enqueue + dequeue of immediate | 183.89ns | 9.00w | 7.02w |
[thread_safe_queue.ml] enqueue + dequeue of young object | 216.69ns | 11.00w | 9.01w |
[thread_safe_queue.ml] enqueue + dequeue_exn of old object | 166.75ns | 9.00w | 7.02w |
Name | Time/Run | mWd/Run |
|---|---|---|
[thread_safe_queue.ml] enqueue + dequeue_exn of immediate | 48.20ns | |
[thread_safe_queue.ml] enqueue + dequeue_exn of young object | 81.96ns | 2.00w |
[thread_safe_queue.ml] enqueue + dequeue_exn of old object | 48.30ns |
Changed {Bigstring,Iobuf}.recvmmsg_assume_fd_is_nonblocking, when no message is available, to return a negative number rather than raise.
This was done for performance reasons, because raising an exception is expensive, due to the stashing of the backtrace and the string creation.
Iobuf.unsafe_resize.Changed Bigstring.blit so that it doesn't release the OCaml lock on map_file bigstrings.
The old behavior of releasing the lock for blits of (small) bigstrings involving mmapped files was problematic and inconsistent. Its cost is high, and fundamentally any access to a mapped bigstring could cause some level of blocking.
Arg_type.t values to Command.Spec.Added module Type_immediacy, which has witnesses that express whether a type's values are always, sometimes, or never immediate.
This code used to be in the Typerep_immediate library in typerep.
Unix.Syslog module.Command.run to no longer ignore the first element of its ~argv parameter.Time.Span.to_short_string show microsecond precision.Time unit tests that failed in London because of timezone dependence.Iobuf.protect_window_and_bounds, which calls a user function and restores the iobuf's bounds afterwards.Unix.mcast_join's ?source : Inet_addr.t argument.Added Iobuf.unsafe_advance.
This can be used to benchmark inner loops that have redundant bounds checking, to see if the checks matter. For example, see the following two advance calls:
let rec process_buffer buf ~f =
let len = Iobuf.length buf in
if header_len <= len then
let msg_len = header_len + Iobuf.Unsafe.Peek.uint16_be buf ~pos:0 in
if msg_len <= len then begin
let len = msg_len - header_len in
Iobuf.advance buf header_len;
f (Protocol.packed_of_iobuf buf);
Iobuf.advance buf len;
process_buffer buf ~f
endWeak_hashtbl.add_exn and sexp_of_t.Fixed Lock_file.create to behave correctly if the target mountpoint is out of space.
Previously in this situation, Lock_file.create would create an empty lock and exit with exception trying to write pid/message there. Subsequent runs would not able to read pid out of empty pid file and blocking_create would block instead of removing defective lock.
Debug.should_print_backtrace : bool ref, to control whether Debug.am* functions print backtraces.Float inline benchmarks.Moved all of the Gc module into Core_kernel.
Part of the Gc module used to be in Core because it used threads. But it doesn't use threads anymore, so can be all in Core_kernel.
Improved Iobuf support for copying to/from strings and bigstrings.
The new modules are Iobuf.{Consume,Peek}.To_{bigstring,string}. They match a Blit-like interface. We don't yet implement the Blit interface in all applicable contexts, but do use Blit.Make and expose some of the operations we want in the forms we expect them to take under a Blit interface.
Linux_ext.Timerfd.to_file_descr.Time.next_multiple an optional argument to control whether the inequality with after is strict.Added Time.Zone.local, a lazily cached Time.Zone.machine_zone ().
This is the first stage in a plan to make explicit timezones more pervasive. First, they are made more convenient, by replacing the relatively wordy Time.Zone.machine_zone () with Time.Zone.local. This involves making the underlying timezone type lazy.
The next stage will be to remove machine_zone and use Time.Zone.local everywhere instead. Then (it is hoped) instead of of_local_whatever, we just say e.g. of_date_ofday Time.Zone.local and currently-implicitly-local functions will be able to switch over to explicit timezones without becoming too verbose.
Timing_wheel.Alarm.null.Made Unix.File_descr.t have with sexp.
Closes janestreet/async_unix#3
Lock_file.is_locked to require read permission, not write permission, on the lock file.Added to Unix.mcast_join an optional ?source:Inet_addr.t argument.
From pull-request on bitbucket: https://bitbucket.org/janestreet/core/pull-request/1/receive-source-specific-multicast/diff
with typerep to many Core types.open Core.Std support with typerep.Added Iobuf.recvmmsg_assume_fd_is_nonblocking_no_options, a specialization of recvmmsg_assume_fd_is_nonblocking for improved performance.
This improvement was driven by profiling at high message rates.
Unix.Rlimit.virtual_memory be an Or_error.t, for platforms where it is undefined.Linux_ext.Epoll.close.Added Weak_hashtbl module, moved from Async.
It had only been in Async to use Async's finalizers. The move to Core exposes a bit more with respect to finalization so that one can still implement Async.Weak_hashtbl, as well as do other things (e.g. use Weak_hashtbl in Incremental, which does not use Async).
Simplified the implementation of Weak_hashtbl to eliminate "entry ids". They were only used to avoid removing a table entry that was in use. But there is a more direct way to test for that -- just check if the weak is None or Some.
Unix.File_descr.equal.Lock_file.Nfs.unlock, the Or_error version of unlock_exn.Lock_file.Nfs.create{,_exn}.Added Unix.set_mcast_ifname, to control the interface used for UDP multicast traffic.
Added bindings for setsockopt IP_MULTICAST_IF.
See 6.3 in: http://www.tldp.org/HOWTO/Multicast-HOWTO-6.html
Changed Command argument processing to treat a single dash (-) as an anonymous argument rather than a flag.
This change follows the unix convention of passing - as an anonymous argument meaning stdin.
Added more bin-prot support to Iobuf: Consume.bin_prot, Fill.bin_prot, Peek.bin_prot, Poke.bin_prot.
Framing doesn't do much for Iobuf, so these are to be more standard, unframed accessors, as opposed to fill_bin_prot.
Added Core.Debug.am, amf, and ams, for outputting debugging messages showing the current source-code position.
Unfortunately, these aren't available in Core.Std.Debug, but only in Core.Debug. That will be fixed in 109.49.
Time_stamp_counter compile on non x86-64 platforms.Made Core.Std.Debug be Core.Debug rather than Core_kernel.Debug.
This exposes the Debug.am* functions added in 109.48.
Added Time_stamp_counter module, which has fast (< 10 nanos) access to the hardware time-stamp counter.
This module provides the fast function Time_stamp_counter.now () which is our best effort high-performance cycle counter for a given platform. For x86 systems this retrieves the CPU's internal time stamp counter using the RDTSC instruction. For systems that do not have a RDTSC instruction, we fallback to using clock_gettime(CLOCK_MONOTONIC).
Here is a benchmark of execution time in nanos and allocations in words:
Name Time/Run Minor
------------------------------- ---------- -------
Time.now 39.02 2.00
TSC.now 7.54
TSC.to_time 4.88 2.00
TSC.to_time (TSC.now ()) 8.54 2.00
TSC.to_time_nanos 4.49
TSC.to_time_nanos(TSC.now ()) 8.95
Calibrator.calibrate 279 34.00Historically, the rate of increment of the TSC (sometimes referred to as the TSC frequency) varied based of CPU overclocking, temperature, load etc. On modern Intel CPU's the TSC is expected to be stable. On Linux systems, the "constant_tsc" in /proc/cpuinfo indicates that the machine has a stable TSC rate. While this module assumes that the TSC is relatively stable, it can adapt to small variations in the TSC frequency.
Changed Daemon.daemonize and daemonize_wait to leave the umask alone by default.
Previously, these had alwasy set the umask to 0, which means that all app-harness programs and all binaries run from grass were creating world-writeable (0666) files by default.
Added Core.Std.phys_same, which is like phys_equal, except has a more general type.
val phys_equal : 'a -> 'a -> bool
val phys_same : _ -> _ -> boolphys_same is useful when dealing with existential types, and one has a packed value and an unpacked value that one wants to check are physically equal. One can't use phys_equal in such a situation because the types are different.
Added Iobuf.set_bounds_and_buffer and set_bounds_and_buffer_sub, which make it easier to use with zero allocation.
(** [set_bounds_and_buffer ~src ~dst] copies bounds (ie limits + window) and shallowly
copies the buffer from [src] to [dst]. [read_write] access is required on [src]
because the caller might have [read_write] access to [dst], and would after the call
then effectively have [read_write] access to [src]. *)
val set_bounds_and_buffer : src:(read_write, _) t -> dst:(_, seek) t -> unit
(** [set_bounds_and_buffer_sub ?pos ?len ~src ~dst ()] is a more efficient version of:
[set_bounds_and_buffer ~src:(Iobuf.sub ?pos ?len src) ~dst].
[set_bounds_and_buffer ~src ~dst] is not the same as
[set_bounds_and_buffer_sub ~dst ~src ()], because the limits are narrowed in the
latter case. *)
val set_bounds_and_buffer_sub
: ?pos:int
-> ?len:int
-> src:(read_write, _) t
-> dst:(_, seek) t
-> unit -> unit?timeout:Time.Span.t argument to Lock_file.blocking_create, Lock_file.Nfs.blocking_create and critical_section.val Day_of_week.num_days : from:t -> to_:t -> int.Added Time.of_date_ofday_precise and Time.Zone.next_clock_shift, to deal with clocks going forward and backward.
Due to clocks going forward/backward, some local times occur twice, and some never occur at all. Time.of_date_ofday_precise identifies these cases and returns all of the relevant information.
Added accessors for Unix.Cidr: base_address and bits.
(** Accessors.
- [base_address 192.168.0.0/24 ` 192.168.0.0]
- [bits 192.168.0.0/24 ` 24]. *)
val base_address : t -> Inet_addr.t
val bits : t -> intZone from Core.Std; use Time.Zone instead.Time.Date; use Date instead.Improved the performance of Piecewise_linear by using arrays with binary search on indices.
The previous implementation Piecewise_linear used (float * float) list (a list of (x,y) pairs) to represent piecewise linear functions, with a linear search through the knots to evaluate the function at a point. This type is now:
{ x : float array
; y : float array
}and the implementation uses binary search to identify the correct array index.
Here are the costs of the get function under the old (list) and new (array) implementations for various numbers of knots:
knots | old | new
------+------+-----
1 | 11ns| 12ns
2 | 18ns| 14ns
5 | 27ns| 19ns
100 | 182ns| 38ns
1000 |1974ns| 52nsTime.Ofday.Zoned, which is a pair of an Time.Ofday.t and a Time.Zone.t.with compare to Time.Zone.Stable.t.Added Timing_wheel functionality.
Config module, which combines alarm_precision and timing_wheel_level_bits.create.mem and clear functions.interval_num, now_interval_num, interval_num_start, add_at_interval_num.This makes it easier to use a timing wheel with int interval numbers, which are more efficient than than float times.
Added Command.Spec.map_anon and map_flag.
(** [map_flag flag ~f] transforms the parsed result of [flag] by applying [f] *)
val map_flag : 'a flag -> f:('a -> 'b) -> 'b flag
(** [map_anons anons ~f] transforms the parsed result of [anons] by applying [f] *)
val map_anons : 'a anons -> f:('a -> 'b) -> 'b anonsFixed Unix.open_flag to compile with OCaml 4.01.
It needed the additional constructor O_CLOEXEC.
Fixed Blang to build with OCaml 4.01.
In blang.mli:
Blang.t is a private variant type, Blang.Stable.V1.t is a private variant type, and client code knows Blang.t = Blang.Stable.V1.t. Previously, this was done in a strange way, using with type 'a t = private 'a t on the signature of Blang.Stable.V1. In addition to being strange, this line no longer builds in OCaml 4.01, which caused problems for building Real World OCaml.
This patch changed the code to something much more straightforward, although not quite so straightforward as we expect to be able to achieve once a nonrec bug is fixed.
Added or_error functions in Unix.Exit_* types to unit Or_error.t.
This makes it easier to deal with combining with infix operators >>=? and >>|?
Added val Backtrace.get_opt : unit -> t option.
This is more convenient to use than Backtrace.get, which is an Or_error.t.
Moved functions for dealing with finalizers into the Gc.Expert module.
This was done to make people be very explicit and sure that they want finalizers, which are very hard to use because they essentially introduce multithreading semantics.
One should typically use async finalizers.
Normalized Command's help messages.
Made anonymous argument names uppercase and subcommand names lowercase.
In Iobuf, added duals to flip and snapshot to work on the high end of the window.
flip has been renamed to flip_lo. The dual of flip_lo is the newly added flip_hi, and shifts the window forward to continue reading, rather than back to switch from writing to reading, as flip_lo does.
flip_hi, in practice, needs snapshots of the upper bound of the window, we split Snapshot into Lo_bound and Hi_bound and introduced bounded versions of flip_lo, compact, and flip_hi to support buffers which are only partially filled, but have substructure, like packet buffers.
Here's the new API.
module type Bound = sig
type ('d, 'w) iobuf
(** Expose =t = private int= only if a =t= is stored in a mutable data structure
somewhere and leads to a measurable =caml_modify= performance problem. *)
type t with sexp_of
val window : (_, _) iobuf -> t
val limit : (_, _) iobuf -> t
val restore : t -> (_, seek) iobuf -> unit
end
module Lo_bound : Bound
module Hi_bound : Bound
val flip_lo : (_, seek) t -> unit
val bounded_flip_lo : (_, seek) t -> Lo_bound.t -> unit
val flip_hi : (_, seek) t -> unit
val bounded_flip_hi : (_, seek) t -> Hi_bound.t -> unitCreated submodule Core.Signal.Expert module.
This is for functions previously in Core.Signal that introduce multithreading semantics, and are hence hard to reason about and should only be used by experts.
Timing_wheel from Zero.recvmmssg, which isn't available on our CentOS 5 machines.Option.compare using with compare so that their comparisons are consistent.Cleaned up the Dequeue module's interface and implementation.
The interface now matches the conventions used elsewhere in Core. The new implementation is also cleaner and more efficient.
Reimplemented the Stack module to improve performance, and renamed the old implementation as Linked_stack.
The new Stack is implemented with this type:
type 'a t `
{ dummy : 'a;
mutable length : int;
mutable elts : 'a array;
}Linked_stack is implemented with this type:
type 'a t `
{ mutable length : int;
mutable elts : 'a list;
}Using an array rather than a linked list is a more efficient and traditional implementation. Pushing to the stack now does not require allocation, except in the rare case when the stack grows.
One downside is that Stack.clear is now O(n) rather than O(1).
This change also eliminates the Stack.Empty exception, so any code matching on that exception should fail to compile, and should be changed to depend on option-returning top and pop operations.
Improved Lock_file.Nfs.
create might throw an exception.Core into Core_kernel and Core.Core_kernel is composed of all modules of Core that do not depend on unix or threads, and Core contains the rest and depends on Core_kernel.
The goal is to have a very portable standard library that can especially run on exotic environment such as Javascript.
So that code that directly refers to Core (rather than Core.Std) for modules that have moved to Core_kernel, we included "proxy" modules in Core that simply include the corresponding module from Core_kernel.
Fixed Core.Flags to build on 32-bit machines.
It had contained a unit test with an integer literal too large to be accepted by the compiler when building on a 32-bit machine.
Added module Core.Iobuf, a module aimed at zero-copy I/O.
An iobuf is a bigstring combined with a position and length, that defines a contiguous region of bytes in the bigstring. Operations on an iobuf operate relative to start of the region and cannot look outside the region.
Core.Validated for creating an abstract type that ensures a validation function has been run.Bigstring.recvmmsg_assume_fd_is_nonblocking, which allows one to read a number of UDP packets with a single system call.Unix.create_process on OSX.Core.Std.Flags module.Heap module implement Container.S1.Ref.Permissioned, which is a ref with read_only / read_write access control.Exposed the unique id in Type_equal.Id.
This allows, e.g. hash tables indexed by unique ids.
Removed the use of Obj.magic from the implementation of Type_equal.Id.same.
It is not needed because the Id.t contains a Uid.t and we can just use Uid.equal.
Massively improved the signatures of Map and Set, both for readability and ocamldoc, as well as improved type error messages.
For instance the type of Int.Set.singleton was:
('a, 'comparator, 'a Core.Std.Int.Set.elt_ -> ('a, 'comparator) Core.Std.Int.Set.t_) Core.Core_set_intf.without_comparatorNow it is simply:
int -> Int.Set.tAdded an optional argument to Command.run that can be used to specify default flags from a user config file.
The optional argument can extend the command line based on the path to the command.
Rename module Weekday as Day_of_week.
The name Weekday conflicted with ordinary usage of "weekday" to mean Monday through Friday.
Changed sexp_of_t for {Month,Ofday,Time,Time.Span}.{Set,Map} to use the nice sexp format of the underlying atomic type.
Previously, the converter had used thes raw type (float, int, etc.). t_of_sexp still accepts both formats; we will remove the ability to accept the raw format in the distant future.
This output-format change was planned when we originally in 108.06b improved those t_of_sexp functions to accept both formats.
Unix.remove.IFDEF's connected to OCaml <4 support.Unix.wordexp in an Or_error.t since it is not available on all systems.Process_env.parse_ssh_client. This gets the address from which you're currently ssh'd in.Unix module the ability to get and set IP_MULTICAST_LOOP and IP_MULTICAST_TTL.Core.Std.Ref, which was previously only available via Core.Ref.Remove Mutex.am_holding_mutex and changed the type of Mutex.try_lock.
With NPTL it is impossible to determine which thread is holding the lock. So, Mutex.am_holding_mutex is unimplementable. Also, Mutex.try_lock was incorrect because it claimed to raise if one was attempting to recursively lock. Since it's not possible to distinguish between recursive locking and the lock being held by another thread, we changed the type to make this clear:
val try_lock : t -> [ `Already_held_by_me_or_other | `Acquired ]Removed our custom version of the OCaml runtime's core_sys_open function.
There used to be a bug in the OCaml runtime, PR#5069, in which open_{in,out}_gen could block while holding the OCaml lock, because they made a call to fcntl outside the blocking section. We had our own C code with the bug fix and re-exposed the fixed versions of the functions in Core.
The bug in OCaml has been fixed, so we have removed our patched function from Core.
In unix_stubs.c, switched from using FNM_FILE_NAME to FNM_PATHNAME.
The GNU project introduced FNM_FILE_NAME as a non-portable synonym for FNM_PATHNAME.
We were using pre-processor macros to define FNM_FILE_NAME as FNM_PATHNAME if unavailable, but it is simpler to just use the more portable FNM_PATHNAME everywhere.
Changed Time.to_string and Time.sexp_of_t to include the timezone.
This is an incompatible change with very old programs in which Time.of_string and Time.t_of_sexp did not support the timezone.
If you have programs that are:
Time.of_string and Time.t_of_sexp.you should recompile/review your code to make sure you won't have issues.
Added function List.remove_consecutive_duplicates : 'a t -> equal:('a -> 'a -> bool) -> 'a t.
This returns the input list with consecutive duplicates removed, and doesn't change the order of the remaining elements.
Added module User_and_group, which is a pair of a unix username and primary unix group.
The string/sexp converters follow the usual unix convention of <user>:<group>.
Added function Date.first_strictly_after : t -> on:Weekday.t -> t.
first_strictly_after t ~on:day_of_week returns the first occurrence of day_of_week strictly after t.
Added functor Type_equal.Lift.
It is always safe to conclude that if type a equals b, then type a X.t equals b X.t, for any type X.t. The OCaml type checker uses this fact when it can. However, sometimes, e.g. when using Type_equal.conv, one needs to explicitly use this fact to construct an appropriate Type_equal.t. The Type_equal.Lift* functors do this.
module Type_equal : sig
type ('a, 'b) t
...
module Lift (X : T1) : sig
val lift : ('a, 'b) t -> ('a X.t, 'b X.t) t
end
endchanged implementation of Array.sort to use introsort.
See http://en.wikipedia.org/wiki/Introsort.
Core.Flags to not print a message to stderr.Fixed Random.self_init, which was broken since 109.00.00 with the upgrade to OCaml 4.0
The fix changed the type signature expressed in core_random.ml of the standard OCaml caml_sys_random_seed C function from unit -> int from unit -> int array. That C function changed between OCaml 3.12 and 4.0.
Core_extended.Unix.Cidr into Core.Unix.Unix.wordexp into an Or_error.t to handle systems that does not implement it in the libc.Added Array.int_blit and Array.float_blit, which are specialized fast blits for int array and float array.
For motivation underlying this change and other design alternatives please see Section 3 "Fast, Slow and Incorrect Array blits" of http://janestreet.github.com/ocaml-perf-notes.html
Unpack_buffer.Unpack_one.sexp for parsing sexps using the Unpack_buffer interface.Time.Robustly_compare functions from 1E-7 to 1E-6.Fixed the names of some toplevel pretty-printers, which referred to nonexistent modules.
Fix some of the pp's for Core which are used to install printers in the top-level. Some of the toplevel printers refer to non-existent modules like Core.Nativeint.pp; this feature changed to the correct name, like Core.Std.Nativeint.pp.
Added to module Unix functionality for getting and setting flags in the open-file-descriptor table.
module Open_flags : sig type t include Flags.S with type t :` t ... end
val fcntl_getfl : File_descr.t -> Open_flags.t
val fcntl_setfl : File_descr.t -> Open_flags.t -> unitAdded module Linux_ext.Timerfd.
This allows one to create a file descriptor that can be monitored by epoll or select and notify them at a certain time. It makes it possible to use epoll with sub-millisecond timeouts.
Version_util.application_specific_fields, which allows custom build-time information to be included in an executable.Fixed major performance problem with hashing in Int.Table.
Our Int.Table.replace was 3 times slower than polymorphic hash table and find was 8 times slower.
This was caused by using:
external seeded_hash_param : int -> int -> int -> 'a -> int = "caml_hash" "noalloc"in Int.Table but:
external old_hash_param : int -> int -> 'a -> int = "caml_hash_univ_param" "noalloc"everywhere else.
The seeded_hash_param was introduced in Caml 4.
We fixed this problem by changing Int.hash from:
let hash (x : t) = Hashtbl.hash xto:
let hash (x : t) = if x >= 0 then x else ~-xBigstring.{pread,pwrite}, which allow reading and writing at a specific file offset.Added module Nothing, which is a type with no values.
This is useful when an interface requires you to specify a type that you know will never be used in your implementation.
Changed Identifiable.Make so that it registers a pretty printer.
Identifiable.Make now uses Pretty_printer.Register. This requires all calls to Identifiable.Make to supply a val module_name : string.
Core.Zone match the Identifiable signature.Made polymorphic equality always fail on Core.Map.t and Core.Set.t.
Before this change, polymorphic equality on a Core.Map or a Core.Set could either raise or return false. It returnd false if the data structures were unequal, and raised if the data structures were equal.
This is because their type definitions looked like:
type ('k, 'v, 'comparator) t =
{ tree : ('k, 'v) Tree0.t;
comparator : ('k, 'comparator) Comparator.t;
}and polymorphic equality visits a block's fields in order. So, it will detect unequal trees and return false, but if the trees are equal, it will compare the comparators and raise because of the functional value.
This change reversed the order of the fields so polymorphic equality always fails.
Added Command.Spec.flags_of_args_exn, for compatibility with OCaml's standard library.
This function converts a Core.Std.Arg.t into a Command.Spec.t.
Made various modules Identifiable: Char, String, and the various Int modules.
In particular, Int being identifiable is useful, because one can now write:
module My_numeric_identifier : Identifiable ` IntYou might think that we could now delete String_id, and just write:
module My_string_identifier : Identifiable ` StringBut this is not quite equivalent to using String_id, because String_id.of_string enforces that its argument is nonempty.
Removed module Space_safe_tuple, which became unnecessary in OCaml 4.00.0.
OCaml 4.00.0 included Fabrice's patch to fix the space leak that Space_safe_tuple was circumventing (PR#5288, commit SVN 11085).
Exn.to_string_mach, for single-line output.Added Linux_ext.bind_to_interface, to improve security of UDP applications.
val bind_to_interface : (File_descr.t -> string -> unit) Or_error.tThis uses the linux-specifc socket option BINDTODEVICE to prevent packets being received from any interface other than one named.
Unix.mkdir_p on Mac OS X.Add some functions to Byte_units.
to_string_hum, scale, Infix.//.Byte_units.t is just a float.Improved the performance of Array.of_list_rev.
The new implementation puts the list elements directly in the right place in the resulting array, rather that putting them in order and then reversing the array in place.
Benchmarking shows that the new implementation runs in 2/3 the time of the old one.
Fixed Fqueue.t_of_sexp, which didn't work with sexp_of_t.
There was a custom sexp_of_t to abstract away the internal record structure and make the sexp look like a list, but there wasn't a custom t_of_sexp defined, so it didn't work.
Stable.V1 types for Host_and_port.Removed Identifiable.Of_sexpable and Identifiable.Of_stringable, in favor of Identifiable.Make
Identifiable.Of_sexpable encouraged a terrible implementation of Identifiable.S. In particular, hash, compare, and bin_io were all built by converting the type to a sexp, and then to a string.
Identifiable.Of_stringable wasn't as obviously bad as Of_sexpable. But it still used the string as an intermediate, which is often the wrong choice -- especially for compare and bin_io, which can be generated by preprocessors.
Added Identifiable.Make as the replacement. It avoids using sexp conversion for any of the other operations.
Added List.intersperse and List.split_while.
These came from Core_extended.List.
val intersperse : 'a list -> sep:'a -> 'a list
val split_while : 'a list -> f:('a -> bool) -> 'a list ** 'a listAdded a functor, Pretty_printer.Register, for registering pretty printers. The codifies the idiom that was duplicated in lots of places:
let pp formatter t = Format.pp_print_string formatter (to_string t)
let () = Pretty_printer.register "Some_module.pp")Interned_string This has a functor for creating modules of interned strings. It uses the very simple mechanism of mapping distinct strings to consecutive ints.Hashtbl.find_and_remove.|>, which means the same as |!, but is likely to replace it someday. This is mostly because |> is an accepted notation elsewhere, particularly in F#. In the future, we will consider eliminating |! in favor of |>, so as to avoid the duplication.module Lazy into a monad.List.stable_dedup_involving_an_application_of_the_set_functor as List.stable_dedup_staged. Made it use Staged.t to make explicit the expectation of partial application.Error and Info.Core.Std, exposed Or_error.ok_exn and Or_error.errorRemoved some values exported by Core.Std.
Removed some values from Core.Std that weren't widely used, or we didn't think should be exposed, including ascending, descending, and equal, which use polymorphic comparison, and we want to discourage.
Here's a guide to some of what was removed, and what one should now use instead.
removed | replace with |
-----------------------------------+--------------------------------------- | |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<:sexp_of< with two underscores; using a single underscore instead.Command.Spec.Arg_type.of_alist_exn as an alternative for of_map. This captures the common pattern to create the map from an alist.Hashtbl. Constrained hashtbl size to a power of two and used a bitmask rather than mod operation for finding hash buckets.Improved the performance of Univ, using the Type_equal GADT. The new implementation improves the run-time and space usage over the old one. In the old implementation, a Univ.t was represented as record with three fields: an exception, a string, and a closure. Creating a univ required allocating three heap blocks, the exception (3 words), the closure (3 words), and the three-field record (4 words). In the new implementation, a Univ.t is represented as a 2-field heap block containing the Constr.t and the value. Creating a univ allocates that single 3-word block, improving on the 10 words needed previously.
Matching on univs is also faster. In the old implementation, matching on a univ required making a function call, testing exception equality, and allocating a Some block. Now, it does just the test and allocation. Furthermore, it is possible to use does_match and match_exn to avoid the allocation.
Version_util.build_info_as_sexp._squelch_unused_module_warning_ to Comparable.S.Replace_polymorphic_compare.README.OCAMLRUNPARAM is not set.Changed Command so that executables show build info and version info This happens when an executatble is called as:
foo.exe version
Before this change, rather than display build info, executables would display the not-so-helpful:
(no option given - printing version)
Float rounding functions with a hardcoded direction.with bin_io and with compare for the =sexp_bool= type.Core.Never_returns.sexp_of_t.Or_error.tag{,_arg} These are analogous to Error functions of the same name.Sexpable.Of_sexpable This is for serializing values of one type as though it were some other isomorphic type.Backtrace.Exn This exposes OCaml stdlib's Printexc functions for backtraces.Flags This implements Unix-style sets of flags that are represented as an int with various bits set, one bit for each flag, e.g., Linux_ext.Epoll.Flag.Uuid This module implements universally unique identifiers based on version 3 of the UUID specification. It used to be in Core_extended=Type_equal, which defines the "equality" GADT.