DP Lib v3.0.0

This is a major release with new features, improvements, deprecations, and bug fixes across the libraries. It includes breaking changes, which are listed below.

Note: All features not mentioned in the release notes are considered experimental for this release. In particular, the accounting library, the stochastic testing API in C++, and the PostgreSQL extension are experimental and likely to change or be removed in the future.

Changes since the 2.1.0 release

C++ DP Lib:

Breaking changes:

• Remove default L1 sensitivity for Laplace mechanisms. L1 sensitivity has to be set explicitly.
• Remove GetOutputConfidenceInterval from algorithms and clarify semantics of NoiseConfidenceInterval.
• Remove base::Status and migrate remaining API to absl::Status
• Remove logging soft-fork and use logging from absl instead.
• Remove deprecated protocol buffer fields.

New features:

• Add a payload to the Status output when approximate bounds could not find appropriate bounds.

Java DP Lib:

Breaking change:

• Noise interface in Java now has 2 additional addNoise() methods that accept delta of type double. Clients with their own implementation of the interface must implement the new methods to avoid compilation errors.

Deprecated:

• Deprecated the methods that accept delta of type Double in the Noise interface and aggregation primitives (e.g., Count, BoundedSum, etc). We will delete the deprecated methods in the next release. Please migrate to their overloaded versions that accept delta of type double in the meantime.

New feature: Long bounded sum.

Go DP Lib:

New features:

• Pre-thresholding for PreAggSelectPartitions and Count aggregation primitives.
• Add IncrementBy for the PreAggSelectPartitions aggregation primitive.
• Negative counts are now explicitly allowed when incrementing via IncrementBy in Count & PreAggSelectPartitions, meaning it is now possible to decrement privacy ID counts.

Privacy on Beam:

Breaking changes:

• New API for PrivacySpec and AggregationParams to enable new features.
• Test Mode is now a field in PrivacySpecParams, no need to use pbeamtest for enabling test mode.

Deprecated:

• Merge SelectPartitionsParams & PartitionSelectionParams. Both SelectPartitions and private partition selection of other aggregations now use PartitionSelectionParams. SelectPartitionsParams is deprecated and might be deleted in future releases.

New features:

• Pre-thresholding on top of DP thresholding, available for all aggregations. See pre-thresholding documentation for more details.
• Aggregation and partition selection budgets can be specified separately instead of being split automatically. This allows for granular budget allocation.
• Test mode can now also be used in non-test runs, e.g. in order to compare differentially private results with raw results.
• Scalable public partitions for <K, V> types. Previously, due to an issue with how we processed public partitions, only in-memory public partitions or small PCollections as public partitions worked with KV type aggregations. Now, it should be possible to use arbitrarily large public partitions. Note that this issue did not affect V-type aggregations (i.e. DistinctPrivacyID, Count).
• Add option to CountParams to allow negative outputs to allow more accurate statistical analysis of the output.

DP Lib v2.1.0

C++ DP Lib:

• New feature: Pre-thresholding Partition Selection

Java DP Lib:

• New feature: Pre-thresholding Partition Selection
• New feature: Truncated Geometric Partition Selection

Go / Privacy on Beam:

• Internal refactorings only

DP Lib v2.0.0

This is a major release with new features, improvements and bug fixes across the libraries. It includes breaking changes, which are listed below.

Overview Table

Algorithm	C++	Go	Java
Laplace mechanism	✅	✅	✅
Gaussian mechanism	✅	✅	✅
Count	✅	✅	✅
Sum	✅	✅	✅
Mean	✅	✅	✅
Variance	✅	✅	✅
Quantiles	✅	✅	✅
Automatic bounds approximation	✅	❌	✅
Truncated geometric thresholding	✅	✅	✅
Laplace thresholding	✅	✅	✅
Gaussian thresholding	✅	✅	✅

Note: All features not mentioned in the release notes are considered experimental for this release. In particular, the accounting library, the stochastic testing API in C++, and the PostgreSQL extension are experimental and likely to change or be removed in the future.

Changes since the 1.1.2 release

C++

New features:

• Add confidence intervals for mean.

Breaking changes:

• Removed budget fraction for DP algorithms; use absolute privacy budgets during initialization instead.
• The semantics of ApproxBounds budget has changed. A DP algorithm now consumes at most the given epsilon and delta. If the DP algorithm uses ApproxBounds internally, the algorithm splits the budget.

New deprecations (not yet removed, but users should migrate):

• Deprecated BoundedStandardDeviation; use BoundedVariance instead.
• Use absl::Status instead of our own soft fork.

Java

New features:

• Implement discrete Laplace noise generator.
• Implement automatic approximate DP bounds calculation.

Go

New features:

• Allow setting equal lower and upper bounds for BoundedSum contributions.
• Improved error reporting.

Breaking changes:

• Return errors instead of log.Fatal/Exit'ing. This changes the function signatures, so the errors now have to be handled or ignored.
• Rename BoundedMeanFloat64 to BoundedMean.
• Disable defaults for MaxPartitionsContributed.
• Disallow using 0 or very small epsilon for Gaussian noise.

Privacy on Beam:

New features:

• Public partitions improvements:
  • Support public partitions for DistinctPerKey.
  • Support in-memory public partitions for all aggregations.
  • Faster public partitions for Count & DistinctPrivacyID when public partitions are a PCollection (i.e. not in-memory).
• Clamp negative counts to 0 for DistinctPrivacyID.
• Improve error reporting.
• Depends on GitHub for the Go DP Library dependency instead of the local version.

Breaking changes:

• Disallow equal bounds for MeanPerKey.

DP Lib 1.1.2

This patch release only affects the Go DP Library and Privacy on Beam.

This fixes a privacy-impacting bug that only affects ThresholdedResult() function of dpagg.Count and pbeam.DistinctPrivacyId() without public partitions in Privacy on Beam where a conversion of a floating point threshold to an integer threshold caused the mechanism to exhibit a larger delta than specified. For example:

• Calling ThresholdedResult() on a Count with (Epsilon: 1.0, Noise: Laplace, MaxPartitionsContributed=1.0) and thresholdDelta=1e-6 should use a threshold of 14.122363 but instead it used 14.0, which increased the thresholdDelta to 1.130165e-6.
• Similarly, calling ThresholdedResult() on a Count with (Epsilon: 2.0, Noise: Laplace, MaxPartitionsContributed=1.0) and thresholdDelta=1e-4 should use a threshold of 5.258597 but instead it used 5.0, which increased the thresholdDelta to 1.677313e-4.

See the single commit for more details on the bug & the fix.

DP Lib 1.1.1

This patch release only affects the C++ DP Lib.

We fixed the implicit approximate bounds use case for multiple contributions for sum, mean. variance, and stddev, i.e., when neither approximate bounds, nor upper/lower limits have been set in the builder. Since the approx bounds algorithm just provides the sensitivity for a subsequently executed algorithm, this might not be catastrophic and still provides differentially private outputs with a slightly increased privacy loss.

DP Lib 1.1.0

This is a minor release with new features, improvements and bug fixes across the libraries. There should be no breaking changes.

Overview Table

Algorithm	C++	Go	Java
Laplace mechanism	✅	✅	✅
Gaussian mechanism	✅	✅	✅
Laplace mechanism	✅	✅	✅
Count	✅	✅	✅
Sum	✅	✅	✅
Mean	✅	✅	✅
Variance	✅	✅	❌
Standard deviation	✅	✅	❌
Quantiles	✅	✅	✅
Automatic bounds approximation	✅	❌	❌
Truncated geometric thresholding	✅	✅	✅
Laplace thresholding	✅	✅	✅
Gaussian thresholding	✅	✅	✅

✅ => supported ; ❌ => not supported yet

New features since the 1.0.0 release

C++

• Support for Gaussian Partition Selection
• NumericalMechanism supports GetVariance
• Users can have the library automatically select the numerical mechanism (Laplace or Gaussian) with the smaller variance

Java

• Confidence intervals for Quantiles

Go

• Support for variance and standard deviation

Privacy on Beam

• Support for multiple quantiles using quantile trees

Bug Fixes

Privacy on Beam

• Fix a privacy bug in DistinctPerKey where contributions might not be bound correctly in some rare cases
• Fix a bug in codelab in sum.go and multiple.go where instead of summing up revenue we sum up time spent

Other

Privacy on Beam

• Refactor error reporting, errors are propagated up to top-level functions as much as possible

Usage

Java via Maven

<dependency>
  <groupId>com.google.privacy.differentialprivacy</groupId>
  <artifactId>differentialprivacy</artifactId>
  <version>1.1.0</version>
</dependency>

Or use the Java artifact with other build systems.

Via the Go command

For the go building blocks library:

go get github.com/google/differential-privacy/[email protected]

For Privacy on Beam:

go get github.com/google/differential-privacy/[email protected]

DP Lib 1.0.1

Only affects Privacy-on-Beam.

This is a patch release for v1.0.0 that includes a fix for the rare privacy bug in the DistinctPerKey function of Privacy-on-Beam.

The bug occurred when there are outlier users in the input that contribute to many partitions or to many values AND the values contributed are the same as values from other users (the second part is critical, if the contributed values only come from a single user then the bug does not occur). Then, the output might not have been differentially private due to incorrect contribution bounding. See the single commit for more details on the bug & the fix.

DP Lib 1.0.0

This is the initial release of Google’s differential privacy libraries. We are using semantic versioning. The initial version number is 1.0.0, as this library is already used for production use cases and we consider our API as stable.

We are supporting C++, Java, and Go.  This release also includes Privacy-on-Beam, a framework for differential privacy build on top of Apache Beam Go.

Note:  All features not mentioned in the release notes are considered experimental for this release.  In particular, the accounting library, the stochastic testing API in C++, and the PostgreSQL extension are experimental and likely to change or be removed in the future.

DP building blocks libraries

Overview table

Algorithm	C++	Go	Java
Laplace mechanism	✅	✅	✅
Gaussian mechanism	✅	✅	✅
Count	✅	✅	✅
Sum	✅	✅	✅
Mean	✅	✅	✅
Variance	✅	❌	❌
Standard deviation	✅	❌	❌
Quantiles	✅	✅	✅
Automatic bounds approximation	✅	❌	❌
Truncated geometric thresholding	✅	✅	✅
Laplace thresholding	✅	✅	✅
Gaussian thresholding	❌	✅	✅

✅ => supported ; ❌ => not supported yet

Base features in C++/Java/Go

Aggregations: count, sum, mean, quantiles
Partition selection mechanisms: truncated geometric thresholding, Laplace thresholding
Numerical mechanisms for providing secure Laplace and Gaussian noise

Additional features in C++

Additional aggregations: variance, standard deviation, max, min
Automatic per-partition bounds approximation

Additional features in Java

Additional partition selection mechanisms: Gaussian thresholding

Additional features in Go

Additional partition selection mechanisms: Gaussian thresholding

Privacy-on-Beam (based on Apache Beam Go)

Aggregations: count distinct privacy IDs, count distinct values per key, count per key, sum per key, mean per key, partition selection
Partition selection mechanisms: truncated geometric thresholding, Laplace thresholding, Gaussian thresholding
Numerical mechanisms: Laplace noise, secure Gaussian noise
In-memory public partitions in aggregations when the list of public partitions is small enough to fit in memory
Two test modes: no noise with contribution bounding, no noise without contribution bounding
Support to modify private collections using functional DoFns.