CHANGELOG

# Changelog

## 2024-09-26 - Version 0.4.5
Point release addressing some issues in the Python and Rust wrappers.

**New Features**
- The Python wrapper now packages the tzdata database inside the wheel to ensure consistent performance across platforms.
- The Rust wrapper now has the ability to download a fresh copy of the tzdata database if needed
- Added a `regoSetTZDataPath` method to the C API and exposed it for the Python and Rust wrappers.
- The `regoNew` C API method now supports the `v1_compatible` flag for interpreter creation
- The library embeds the `windowsZones.xml` mapping file so it can provide it where needed
- The Python wrapper provides a more natural interface for sets and objects
- The CMake system will now look for a `REGOCPP_TZDATA_PATH` environment variable to use for setting the default path

**Bug Fix**
- Fixed a bug where builtins would not be available if an interpreter was re-used
- Fixed a bug with the Rust wrapper where it was aggressively trimming strings


## 2024-09-23 - Version 0.4.4
Point release adding the `uuid`, `time`, and `walk` builtins.

In order to add support for the `time` built-ins that worked cross-platform and with the widest
range of C++ compilers, we had to introduce a dependency on the [`date`](https://github.com/HowardHinnant/date)
library. This contains a full implementations of the `std::chrono` functionality which will eventually be
supported across most c++ compilers as part of the STL. In particular, it provides the crucial Time Zone
lookup functionalities required by the Rego `time` built-ins. The Time Zone lookup requires a valid
[`tzdata`](https://www.iana.org/time-zones) database from the IANA. This can either be configured to use the local
system TZ data (default for Linux variants) or a manual database (default for Windows). If the `REGOCPP_USE_MANUAL_TZDATA` flag
is set, then the project will download the latest database and install it as part of its build process.

**New Features**
- Added the `uuid` built-ins
- Added the `time` built-ins
- Added the `walk` built-in
- It is now possible for `BuiltInDef` implementations to cache values or otherwise maintain state. To facilitate this,
  a new virtual `clear()` method has been added which will be called at the start of each query evaluation.

**Bug Fixes**
- Fixed a bug with adding zero to negative integers.


## 2024-09-10 - Version 0.4.3
Point release updating to Rego v0.68.0

**New Features**
- Updated the support version of Rego to v0.68.0
- Updated to the latest build of Trieste.
- Added a `v1-compatible` flag to the test driver and interpreter which forces rego-v1 compliance
- Added code to run both the v0 and v1 OPA test suites
- Added the new `strings.count` built-in
- Improved the tool messaging around debug options

**Bug Fixes**
- Fixed an issue with recursion errors being swallowed by `not`
- Fixed issues where undefined values were not handled correctly during unification
- Fixed issue with whitespace after package definitions
- Fixed issue where internal values were lingering in rule evaluations

## 2024-06-29 - Version 0.4.2
Point release updating the Trieste version.

This change should address build instability due to snmalloc updates.

## 2024-06-23 - Version 0.4.1
Point release containing doc updates and bug fixes.

- Updated the docs to reflect the new interfaces and outputs

**Bug Fixes**
- Fixed an issue with actions attached to an `In` statement for some builds/environments
- Fixed a bug with modules that start with comments
- Fixed a bug with comprehension contexts when an assigned variable is shadowed in the body of the comprehension

## 2024-06-14 - Hardening
Hardening for the 0.4.0 release.

- Errors with individual input and data files are handled properly
- Issues with binary inputs have been dealt with upstream in Trieste
- Fixed an error with nested enumerations where results were incorrectly handled in some circumstances.

## 2024-06-12 - Update to rego v0.65.0
Update to Rego v0.65.0

**New Features**
- Updated the supported version of Rego to v0.65.0.
- `json.marshal_with_options`
- `numbers.range_step`
- `rego.v1` import and behaviour, plus a set of tests in `regov1.yaml`
- The interpreter now returns more than one `Result` when appropriate, and separates out the `Bindings` and `Terms`.
- The output of `rego` is now JSON instead of plain text
- Added `regoOutputSize`, `regoExpressions`, `regoBindingsAtIndex`, and `regoExpressionsAtIndex` to the C API
- Updated the Python and Rust wrappers to use the new API

**Breaking Changes**
- Output of `rego` tool has changed. Any downstream tooling that was parsing that will need to be updated to parse the JSON.
- `rego::wf_result` has changed. Any tooling that was using the output node directly will need to be updated
- the `add_*` methods on `Interpreter`, as they now perform parsing, now produce either a `nullptr` if successful or an error `Node`.
  Downstream consumers that were checking the boolean return value will need to be updated. Behavior in the C API is unchanged, but
  there will now be detailed parse/compile errors when an individual module etc. fails to compile.

## 2024-06-04 - Version 0.4.0
Major refactor of the code to use the new version of Trieste.

**New Features**
- Updated to latest version of Trieste
- Uses Trieste's YAML and JSON parsers and emitters
- Uses Trieste's more robust Unicode support
- Passes broken up into a `Reader`, which parses Rego files, and a `Rewriter` which performs unification
- Added the `encoding` and `graphs` builtins
- Added `set_log_level_from_string` and `setLogLevelFromString` to make it easier to set the log level. All executables now use this option.

**Improvements**
- The Interpreter caches intermediate versions of all modules, reducing query execution time
- All of the early passes have been rewritten and are more efficient, resulting in reduced parsing time
- The reduction of the input passes (and their restriction to a single file) along with other optimizations to header files has resulted in reduced compile times
- The `rego_fuzzer` xecutable allows for more fine-grained fuzzing and testing of the Rego toolchain
- Multiple bug fixes

**Deprecation**
- The `regoSetInputJSON` method is a misnomer, as the input does not strictly need to be JSON. This has been replaced with an (otherwise identical) function called
  `regoSetInputTerm` which does the same thing, and the `JSON` version has been deprecated.

**Breaking Changes**
- The `rego_trieste` executable has been replaced by the new `rego_fuzzer` executable

**New Test Coverage**
- `base64builtins`
- `base64urlbuiltins`
- `hexbuiltins`
- `jsonbuiltins`
- `reachable`
- `urlbuiltins`

## 2024-01-19 - Version 0.3.11
Minor improvements and bug fixes.

**New Features**
- Updated to more recent Trieste version
- More sophisticated logging

**Bug fixes**
- Comprehensions over local variables were not properly capturing the local (regression due to optimization)
- Local variable initializations were order-dependent (regression due to optimization)
- In some circumstances, indexing the data object with an undefined key caused a segfault.

**Other**
- Various CI changes due to issues with Github actions.

## 2023-09-21 - Version 0.3.10
Instrumentation and optimization. 

**New Features**
- A new `REGOCPP_ACTION_METRICS` option is available. When turned on, it will compile rego-cpp with
  instrumentation to measure the number of times a Trieste action is executed and how long is spent
  in each.
- A new internal `join` method.

**Improvements**
- `ValueDef` now caches its `str` and `json` representations instead of computing them each time.
  This has had a significant impact on unification (~5x speedup)
- Use of the new `join` method reduces allocations and copies for common string operations
- The unifier can now act over `Object`, `Set`, and `Array` nodes directly.
- The `functions` pass has been refactored to produce fewer temporary variables and to perform
  fewer rewrites, reducing its runtime by half.

## 2023-09-20 - Version 0.3.9
Fixing several impactful bugs.

**Bug fixes**
- The ACI tests were incorrectly written in such a way that they would always pass. This has been addressed,
  and safeguards put in place in the test driver to detect this in the future.
- Some statements nested in an every block would break the conditions if the item sequence was an array for
  which multiple elements could be identified by the some that were valid (see the new `every_some` test for
  a minimal reproduction). This has been fixed.
- Dead links were being incorrectly identified in some cases. This has been addressed by making the check
  more precise and moving it to the functions pass.
- Var lookups into a virtual or base document would resolve the entire document, causing needless recursion.
  This has been fixed. 

## 2023-09-18 - Version 0.3.8
Adding new examples for Python and Rust usage.

**Python API**
- `regopy.build_info()` method added to return useful information about the `rego-cpp` build.
- `__repr__` implementations added to `Interpreter`, `Output`, and `Node` for easier repl usage

**Rust API**
- `Display` implementations added to `Node` and `Output`

## 2023-09-17 - Version 0.3.7
Adding the ability to build the library targeting the C++ 17 standard (as opposed to
c++ 20). Also a series of fixes to make the `cibuildwheel` pipeline work.

## 2023-09-14 - Version 0.3.6
Adding a Python wrapper.

**New Features**
- `wrappers/python/` contains a Python package, `regopy`, which wraps the library
  and exposes a Python API.

**Bug Fixes**
- Fixed a bug where `to_json()` was incorrectly sorting arrays when the `sort`
  flag was set to `true`.
- Removes a memory leak caused by a cyclic graph of Unifiers
- Fixes various use-after-free violations caused by missing `clone()` invocations.

## 2023-09-12 - Version 0.3.5
Adding C++ documentation.

**New Features**
- The C++ API is now documented using Doxygen. The documentation can be found
  [here](https://microsoft.github.io/regocpp/).
- Logging levels have been added and can be set via the API (instead of just
  binary enabled/disabled)

**Breaking changes**
- _C API_: The `regoSetLoggingEnabled` function has been replaced with
  `regoSetLoggingLevel`. This is also reflected in the C++ and Rust APIs.h
- _C API_: The `regoSetExecutablePath` function has been removed due to the
  `Interpreter::executable()` property being removed from the C++ API.
- The `to_json` method flag `rego_set` has been changed to `set_as_array` and
  the order of its flag inverted (it is now `false` by default).


## 2023-09-10 - Version 0.3.4
Adding a Rust wrapper.

**New Features**
- `wrappers/rust/` contains a Rust crate, `regorust`, which wraps the library
  and exposes a Rust API.

**Breaking changes**
- _C API_: The `regoAddInput*` functions are now named `regoSetInput*` and can
  be used multiple times on the same interpreter.
- _C API_: `regoGet/SetStrictBuiltInErrors` has been added.
- The public headers have been reduced to just two files:
  * `rego.hh` which contains the C++ API
  * `rego_c.h` which contains the C API
- The `rego.hh` header is now a C++ header and will not compile in C.
- Many definitions have been moved to the (non-public) `internal.hh` header,
  and code which relied on having access to those definitions will break.

## 2023-09-07 - Version 0.3.3
Adding a C API.

**New Features**
- `rego_c.h` contains a C API for the library
- `examples/c` contains an example that builds an interpreter using the C API

**Breaking Change**
The project was incorrectly configured to set the PUBLIC CMake build directory
to `include/rego` instead of `include`. This meant that projects consuming the
library using CMake FetchContent would include header files without a `rego/`
prefix, which would cause problems long term. This has been fixed, but as a result
any projects using the library via CMake FetchContent will need to update their
`#include` statements to include the `rego/` prefix.

## 2023-09-03 - Version 0.3.2
Adding tests based upon the Confidential ACI Rego framework.

**New Features**
- `aci.yaml` test suite
- Test cases can now specify modules using filenames

**Bug fixes**
- Long arithmetic expressions which are split over multiple lines are now
  correctly parsed.
- A few fuzzer errors have been resolved

## 2023-09-03 - Version 0.3.1
Adding further builtins and test coverage, plus some minor
refactoring.

Adds the `bits`, `casts`, `types` builtins. Also adds the `regex`
builtins with the exception of `regex.globs_match`, which we will
add later along with the `globs` builtins.

**New Features**
- `bits` builtins
- `casts` builtins
- `types` builtins
- `regex` builtins (w/out `regex.globs_match`)
- Miscellaneous builtins
  * `any`
  * `all`

**New Test Coverage**
- `any`
- `all`
- `bitsand`
- `bitsnegate`
- `bitsor`
- `bitsshiftleft`
- `bitsshiftright`
- `bitsxor`
- `casts`
- `regexfind`
- `regexfindallstringsubmatch`
- `regexisvalid`
- `regexmatch`
- `regexreplace`
- `regexsplit`
- `regexmatchtemplate`

**Improvements**
- `unwrap_arg` and the associated `UnwrapOpt` struct provide a
  straightforward way to unwrap arguments for built-ins while producing
  the expected error messages for type mismatch.
- `unwrap` provides a set-based way to do optional unwrapping of terms.

## 2023-09-02 - Version 0.3.0
Minor release. From this point the implementation passes the Rego
test suite and can be considered a compatible Rego implementation.

Adds the `semver`, `objects`, and `units` builtins.

**New Features**
- `semver` builtins
- `objects` builtins
- `units` builtins
- Miscellaneous builtins
  * `cast_array`
  * `cast_set`
  * `opa.runtime`
  * `json.marshal`
  * `time.now_ns`

**New Test Coverage**
- `assignments`
- `baseandvirtualdocs`
- `completedoc`
- `compositereferences`
- `comprehensions`
- `containskeyword`
- `dataderef`
- `defaultkeyword`
- `disjunction`
- `elsekeyword`
- `embeddedvirtualdoc`
- `eqexpr`
- `evaltermexpr`
- `every`
- `example`
- `fix1863`
- `functionerrors`
- `functions`
- `indirectreferences`
- `inputvalues`
- `negation`
- `nestedreferences`
- `object*`
- `partialdocconstants`
- `partialiter`
- `partialobjectdoc`
- `partialsetdoc`
- `refheads`
- `semver*`
- `subset`
- `toarray`
- `topdowndynamicdispatch`
- `toset`
- `type`
- `undos`
- `units`
- `varreferences`
- `virtualdocs`
- `withkeyword`


**Bug Fixes**
- A great many implementation details exposed by testing

## 2023-08-12 - Version 0.2.6
Adding the `aggregates`, `arrays`, and `strings` builtins.

**New Features**
- `aggregates` builtins
- `arrays` builtins
- `strings` builtins
- The library now supports arbitrary precision integers

**New Test Coverage**
- `aggregates`
- `array`
- `comparisonexpr`
- `compositebasedereference`
- `helloworld`
- `indexing`
- `intersection`
- `replacen`
- `sprintf`
- `strings`
- `trim`
- `trimleft`
- `trimprefix`
- `trimright`
- `trimspace`
- `trimsuffix`
- `union`

**Bug Fixes**
- Multiple implementation details exposed by testing


## 2023-08-12 - Version 0.2.5
Adding the `numbers` built-ins.

**New Features**
- `numbers` built-ins have been added
- The library now supports arbitrary precision integers

**Test Coverage**
- `arithmetic`
- `numbersrange`
- `rand`
- `sets`

**Bug Fixes**
- Multiple implementation details exposed by testing


## 2023-08-12 - Version 0.2.4
Adding a test driver that can execute OPA Rego-style YAML test files against the
`regocpp` interpreter.

**New Features**
- The `rego_test` driver is a tool that can process OPA Rego-style test case YAML files.

**Bug Fixes**
- Fixed an issue with assignment of arrays and objects to local variables.

**Improvements**
- All existing tests have been converted to use the test driver

## 2023-08-08 - Version 0.2.3
Adding the remainder of the missing language features, bringing the implementation fully
compliant with the published v0.55.0 grammar.

**New Features**
- Array, Set, and Object comprehensions
- `every` keyword
- `import` support
- `with` keyword
- Basic built-in framework in place. Added some builtins as well:
  * `print`
  * `startswith`
  * `endswith`
  * `count`
  * `to_number`
  * `union`
  * `intersection`
- `bin-infix` support
- `ref` package names

**Bug fixes**
- Nested enumeration was not working properly. This has been fixed.

**Improvements**
- Turning on unification logging can be done on the command line (no recompile needed)
- Passes have been broken out for better comprehensibility
- Build time has been reduced

## 2023-07-11 - Version 0.2.2
Adding the `in`, `if`, `contains`, and `else` keywords.

**New Features**
- Added future keyword support to the runtime
- Added `in`, `if`, `contains`, and `else` keywords to the runtime
- Added partial set rule construction
- Added partial object rule construction

**Bug fixes**
- Else rule bodies were not previously handled correctly when the else keyword
  was omitted. This has been fixed.
- `ExprCall` nodes can now (correctly) be a `RefHead`.

**Improvements**
- Function calls are now represented interally via a `ExprCall` node as opposed to
  a `RefArgCall` node, to better reflect the target grammar.

## 2023-06-27 - Version 0.2.1
Addressing various minor improvements, bugs, and documentation issues.

**Bug fixes**
- Fixed a segfault when an input, data, or module file did not exist
- Fixed a segfault when a lookdown returns zero definitions
- Fixed various typos and errors in the documentation
- Fixed a potential infinite rewrite loop in the RuleBody pass

**Improvements**
- Rule values will no longer always be `UnifyBody` nodes but will be `Term` nodes
  in cases when the rule returns a constant value.
- The code now uses the well-formed child indexing wherever possible for increased
  robustness.


## 2023-06-21 - Version 0.2.0
Initial Release.