Skip to content

Commit

Permalink
Prepare 0.3.0 release
Browse files Browse the repository at this point in the history
  • Loading branch information
squidfunk committed Nov 30, 2015
1 parent 8bc12b5 commit fe37a29
Show file tree
Hide file tree
Showing 172 changed files with 16,073 additions and 10,506 deletions.
19 changes: 12 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.

# Mac OS X internals
.DS_Store

# Files generated by autoconf
aclocal.m4
autom4te.cache/
Expand Down Expand Up @@ -49,13 +46,18 @@ Makefile.in
# Files generated by dsymutil
*.dSYM

# Files generated by build process
# Files generated by protobluff
*.pb.c
*.pb.h

# Files generated by the build process
*.la
*.lo
*.a
*.o

# Executables
examples/**/example
src/bin/protoc-gen-protobluff
tests/**/test

Expand All @@ -65,7 +67,10 @@ tests/**/*.log
tests/*.log

# Files generated by lcov and genhtml
tools/coverage/coverage.info
tools/coverage/**/
tests/coverage.info
tests/coverage/**/
*.gcno
*.gcda
*.gcda

# Mac OS X internals
.DS_Store
5 changes: 2 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,8 @@ before_install:
- ./autogen.sh

# Perform build and tests
script: ./configure --enable-coverage && make && make test
script: ./configure --enable-coverage && make && make test coverage

# If successful, create and push coverage report
after_success:
- cd tools/coverage && make
- coveralls-lcov coverage.info
- coveralls-lcov tests/coverage.info
54 changes: 51 additions & 3 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,6 +1,54 @@
protobluff-0.3.0 (2015-11-30)

Complete overhaul of the internal structure: an encoder and decoder have
been implemented which now form the core of protobluff, together with the
buffer, descriptor and varint-packing logic. protobluff is turning into a
full-featured multi-purpose library: messages can be streamed using the
encoding and/or decoding functions, fields and messages can be read and
written without decoding/encoding the entire message using the message
and cursor functions (as before).

In addition protobluff has been split into a lite and full runtime. The lite
runtime only contains the core functionality and currently weighs 14kb
(stripped) on x86_64, the full runtime weighs 36kb (stripped). Though not
being the highest priority task on the list, compatibility with embedded
environments is planned, as the groundwork has now been done. Furthermore
the project structure has been revamped and is now completely modular, and
some examples on how to use protobluff for encoding/decoding were added.

Regardless of the addition of a whole bunch of new functions, this release is
mostly compatible with prior versions. Nevertheless, the libtool version has
been updated to 2.0.0 to reflect some breaking changes. See the MIGRATION
file for more information on how to migrate from older versions.

Runtime:

* Complete overhaul of runtime
* Fixed improper encoding and decoding of zig-zag encoded varints
* Added functions for type-specific packing and unpacking of varints
* Added function to efficiently grow buffers
* Added decoder and encoder to stream messages
* Added tests for all new structures
* Added validator as a replacement for pb_message_check()
* Added some usage examples for encoding and decoding messages
* Changed directory structure to reflect new modular design
* Changed macros to inline functions for better type checking
* Renamed binary into buffer and fleshed out the implementation
* Renamed binary stream into stream and refactored implementation
* Merged journaling with binary functionality into journal
* Merged descriptors into single translation unit
* Adapted existing code to work with new internal structure

Generator:

* Fixed accessor generation for nested enumerations
* Added compile-time field, message, enum and enum value deprecations
* Added generators for interfacing with the decoder and encoder
* Changed generated code from macros to inline functions

protobluff-0.2.2 (2015-09-05)

* Fixed circular message type references in code generator
* Fixed circular message type references in generator
* Added chunk allocator implementation
* Added tests for chunk allocator
* Removed implicit default values for non-optional enums
Expand Down Expand Up @@ -30,9 +78,9 @@ protobluff-0.2.0 (2015-08-27)
* Removed double underscore prefixes in function names
* Renamed pb_message_validate() into pb_message_check()

Code generator:
Generator:

* Complete overhaul of code generator
* Complete overhaul of generator
* Fixed order of message descriptor fields in generated code
* Added generators for extensions
* Added generators for enums and enum values
Expand Down
56 changes: 56 additions & 0 deletions MIGRATION
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
0.2.x => 0.3.0

Binaries

The "binary" which holds the underlying serialized data in wire format was
renamed to be a "buffer". Additionally, the journaling functionality is now
part of a "journal" which itself contains a buffer that is tracked. The
former is necessary for encoding and decoding messages, the later for
performing random access on encoded messages.

< 0.3.0:

pb_binary_t binary = pb_binary_create(...);
pb_binary_destroy(&binary);

= 0.3.0:

pb_buffer_t buffer = pb_buffer_create(...);
pb_buffer_destroy(&buffer);

pb_journal_t journal = pb_journal_create(...);
pb_journal_destroy(&journal);

Validation

Message validation was refactored into a separate "validator" structure,
so that validation can directly take place on a binary. This step was
necessary to make validation available to the lite runtime.

< 0.3.0:

pb_message_t message = pb_message_create(...);
pb_message_check(&message);

= 0.3.0:

pb_validator_t validator = pb_validator_create(...);
pb_validator_check(&validator, &buffer);

0.1.x => 0.2.0

Validation

The function for message validation was renamed into pb_message_check() in
order to remove any ambiguities with pb_message_valid() which checks if the
message is still valid in terms of alignment.

< 0.2.0

pb_message_t message = pb_message_create(...);
pb_message_validate(&message);

= 0.2.0

pb_message_t message = pb_message_create(...);
pb_message_check(&message);
21 changes: 18 additions & 3 deletions Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ SUBDIRS = include src tests
# Create pkg-config recipe
pkgconfigdir = @libdir@/pkgconfig
pkgconfig_DATA = \
src/lib/libprotobluff.pc
src/protobluff.pc \
src/protobluff-lite.pc

# -----------------------------------------------------------------------------
# Custom build rules
Expand Down Expand Up @@ -71,6 +72,19 @@ purge: clean
rm -f stamp-h1
rm -f test-driver

# -----------------------------------------------------------------------------
# Coverage report
# -----------------------------------------------------------------------------

if HAVE_COVERAGE

# Generate and display code coverage report
coverage:
@LCOV@ -q -c --no-external -d @top_builddir@/src -o tests/coverage.info
@GENHTML@ tests/coverage.info -o tests/coverage -c tests/coverage.css

endif # HAVE_COVERAGE

# -----------------------------------------------------------------------------
# Leak check
# -----------------------------------------------------------------------------
Expand All @@ -79,9 +93,10 @@ if HAVE_VALGRIND

# Perform leak check using unit tests
memcheck: check
@for t in `find tests -maxdepth 2 -executable -type f`; do \
@for t in `find tests -maxdepth 3 -perm +111 -type f`; do \
@LIBTOOL@ --mode=execute @VALGRIND@ --dsymutil=yes --tool=memcheck \
--track-origins=yes --leak-check=full --show-reachable=yes $$t; \
--track-origins=yes --leak-check=full --show-reachable=yes \
--suppressions=valgrind.supp $$t; \
done

endif # HAVE_VALGRIND
85 changes: 48 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
[![Release Status](https://img.shields.io/github/release/squidfunk/protobluff.svg)](https://github.com/squidfunk/protobluff/releases/latest)
[![License](https://img.shields.io/github/license/squidfunk/protobluff.svg)](https://github.com/squidfunk/protobluff/blob/master/LICENSE)

protobluff is an extremely lightweight Protocol Buffers implementation for C.
protobluff is a modular Protocol Buffers implementation for C.

## Theory of Operation

Expand All @@ -19,14 +19,14 @@ number of scattered allocations which in turn is not very cache-friendly.

protobluff follows a different approach. It entirely skips the necessary
decoding and encoding steps when reading or writing values from messages,
as it directly operates on the encoded binary. New values can be incrementally
as it directly operates on the encoded data. New values can be incrementally
read or written, memory management is centralized and handled by the underlying
binary. If no alterations that change the size of the underlying binary are
expected, the binary can be used in *zero-copy mode*, omitting all dynamic
journal. If no alterations that change the size of the underlying journal are
expected, the journal can be used in *zero-copy mode*, omitting all dynamic
allocations.

Updates on fixed-sized wire types on little-endian machines can be carried out
in-place using raw-pointers to the underlying binary. These include the native
in-place using raw-pointers to the underlying data. These include the native
Protocol Buffers types `fixed(32|64)`, `sfixed(32|64)`, `float` and `double`
(see the [Protocol Buffers Encoding Guide][] for more information). Strings may
also be accessed through raw-pointers, however writing a string of different
Expand Down Expand Up @@ -100,53 +100,65 @@ Here's a usage example taken from the original description of the Google
Protocol Buffers library and adapted to protobluff:

``` c
/* Create an empty binary to assemble a new person message */
pb_binary_t binary = pb_binary_create_empty();
/* Create an empty journal to assemble a new person message */
pb_journal_t journal = pb_journal_create_empty();

/* Create a person message */
pb_message_t person = person_create(&binary);
pb_message_t person = person_create(&journal);

/* Define the values we want to set */
pb_string_t name = pb_string_init("John Doe"),
email = pb_string_init("[email protected]"),
home = pb_string_init("+1-541-754-3010"),
mobile = pb_string_init("+1-541-293-8228");
pb_string_t name = pb_string_init_from_chars("John Doe"),
email = pb_string_init_from_chars("[email protected]"),
home = pb_string_init_from_chars("+1-541-754-3010"),
mobile = pb_string_init_from_chars("+1-541-293-8228");
int32_t id = 1234;

/* Set values on person message and check return codes */
pb_error_t error = PB_ERROR_NONE;
do {
if ((error = person_put_name(&person, &name)) ||
(error = person_put_email(&person, &email)) ||
(error = person_put_id(&person, &id)))
(error = person_put_id(&person, &id)) ||
(error = person_put_email(&person, &email)))
break;

/* Set home number */
pb_message_t phone1 = person_create_phone(&person);
if ((error = person_phonenumber_put_type_home(&phone1)) ||
(error = person_phonenumber_put_number(&phone1, &home)))
break;

/* Set mobile number */
pb_message_t phone2 = person_create_phone(&person);
if ((error = person_phonenumber_put_type_mobile(&phone2)) ||
(error = person_phonenumber_put_number(&phone2, &mobile)))
break;

/* All values were set successfully, the binary is ready to be persisted,
sent or whatever - no marshalling necessary. The raw message data and size
can be directly obtained through the binary */
const uint8_t *data = pb_binary_data(&binary);
const size_t size = pb_binary_size(&binary);
if (!(error = person_phonenumber_put_number(&phone1, &home)) &&
!(error = person_phonenumber_put_type_home(&phone1))) {

/* Set mobile number */
pb_message_t phone2 = person_create_phone(&person);
if (!(error = person_phonenumber_put_number(&phone2, &mobile)) &&
!(error = person_phonenumber_put_type_mobile(&phone2))) {

/* Dump the journal */
pb_journal_dump(&journal);

/* The encoded message can be accessed as follows */
// const uint8_t *data = pb_journal_data(&journal);
// const size_t size = pb_journal_size(&journal);
}
person_phonenumber_destroy(&phone2);
}
person_phonenumber_destroy(&phone1);
} while (0);

/* Print error, if any */
if (error)
fprintf(stderr, "ERROR: %s\n", pb_error_string(error));

/* Cleanup and invalidate */
person_destroy(&person);

/* Free all allocated memory */
pb_binary_destroy(&binary);
/* Free all allocated memory and return */
pb_journal_destroy(&journal);
return error
? EXIT_FAILURE
: EXIT_SUCCESS;
```
See the examples directory for more information.
## Linking
### Manually
Expand All @@ -157,8 +169,8 @@ library. Therefore, the following compiler and linker flags must be obtained
and added to your build toolchain:
``` sh
pkg-config --cflags libprotobluff # Add output to compiler flags
pkg-config --libs libprotobluff # Add output to linker flags
pkg-config --cflags protobluff # Add output to compiler flags
pkg-config --libs protobluff # Add output to linker flags
```

### Autotools
Expand All @@ -169,7 +181,7 @@ the compiler flags into the variable `protobluff_CFLAGS` and the linker flags
into the variable `protobluff_LDFLAGS`:

``` makefile
PKG_CHECK_MODULES([protobluff], [libprotobluff])
PKG_CHECK_MODULES([protobluff], [protobluff])
```

## Features
Expand Down Expand Up @@ -198,9 +210,8 @@ PKG_CHECK_MODULES([protobluff], [libprotobluff])
### Roadmap

1. Oneofs
2. Streaming API
3. Packed fields
4. Services
2. Packed fields
3. Services

## License

Expand Down
Loading

0 comments on commit fe37a29

Please sign in to comment.