1# gRPC (Core) Compression Cookbook 2 3## Introduction 4 5This document describes compression as implemented by the gRPC C core. See [the 6full compression specification](compression.md) for details. 7 8### Intended Audience 9 10Wrapped languages developers, for the purposes of supporting compression by 11interacting with the C core. 12 13## Criteria for GA readiness 14 151. Be able to set compression at [channel](#per-channel-settings), 16 [call](#per-call-settings) and [message](#per-message-settings) level. 17 In principle this API should be based on _compression levels_ as opposed to 18 algorithms. See the discussion [below](#level-vs-algorithms). 191. Have unit tests covering [the cases from the 20 spec](https://github.com/grpc/grpc/blob/master/doc/compression.md#test-cases). 211. Interop tests implemented and passing on Jenkins. The two relevant interop 22 test cases are 23 [large_compressed_unary](https://github.com/grpc/grpc/blob/master/doc/interop-test-descriptions.md#large_compressed_unary) 24 and 25 [server_compressed_streaming](https://github.com/grpc/grpc/blob/master/doc/interop-test-descriptions.md#server_compressed_streaming). 26 27## Summary Flowcharts 28 29The following flowcharts depict the evolution of a message, both _incoming_ and 30_outgoing_, irrespective of the client/server character of the call. Aspects 31still not symmetric between clients and servers (e.g. the [use of compression 32levels](https://github.com/grpc/grpc/blob/master/doc/compression.md#compression-levels-and-algorithms)) 33are explicitly marked. The in-detail textual description for the different 34scenarios is described in subsequent sections. 35 36## Incoming Messages 37 38 39 40## Outgoing Messages 41 42 43 44## Levels vs Algorithms 45 46As mentioned in [the relevant discussion on the spec 47document](https://github.com/grpc/grpc/blob/master/doc/compression.md#compression-levels-and-algorithms), 48compression _levels_ are the primary mechanism for compression selection _at the 49server side_. In the future, it'll also be at the client side. The use of levels 50abstracts away the intricacies of selecting a concrete algorithm supported by a 51peer, on top of removing the burden of choice from the developer. 52As of this writing (Q2 2016), clients can only specify compression _algorithms_. 53Clients will support levels as soon as an automatic retry/negotiation mechanism 54is in place. 55 56## Per Channel Settings 57 58Compression may be configured at channel creation. This is a convenience to 59avoid having to repeatedly configure compression for every call. Note that any 60compression setting on individual [calls](#per-call-settings) or 61[messages](#per-message-settings) overrides channel settings. 62 63The following aspects can be configured at channel-creation time via channel arguments: 64 65#### Disable Compression _Algorithms_ 66 67Use the channel argument key 68`GRPC_COMPRESSION_CHANNEL_ENABLED_ALGORITHMS_BITSET` (from 69[`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 70takes a 32 bit bitset value. A set bit means the algorithm with that enum value 71according to `grpc_compression_algorithm` is _enabled_. 72For example, `GRPC_COMPRESS_GZIP` currently has a numeric value of 2. To 73enable/disable GZIP for a channel, one would set/clear the 3rd LSB (eg, 0b100 = 740x4). Note that setting/clearing 0th position, that corresponding to 75`GRPC_COMPRESS_NONE`, has no effect, as no-compression (a.k.a. _identity_) is 76always supported. 77Incoming messages compressed (ie, encoded) with a disabled algorithm will result 78in the call being closed with `GRPC_STATUS_UNIMPLEMENTED`. 79 80#### Default Compression _Level_ 81 82**(currently, Q2 2016, only applicable for server side channels. It's ignored 83for clients.)** 84Use the channel argument key `GRPC_COMPRESSION_CHANNEL_DEFAULT_LEVEL` (from 85[`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 86valued by an integer corresponding to a value from the `grpc_compression_level` 87enum. 88 89#### Default Compression _Algorithm_ 90 91Use the channel argument key `GRPC_COMPRESSION_CHANNEL_DEFAULT_ALGORITHM` (from 92[`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 93valued by an integer corresponding to a value from the `grpc_compression_level` 94enum. 95 96## Per Call Settings 97 98### Compression **Level** in Call Responses 99 100The server requests a compression level via initial metadata. The 101`send_initial_metadata` `grpc_op` contains a `maybe_compression_level` field 102with two fields, `is_set` and `compression_level`. The former must be set when 103actively choosing a level to disambiguate the default value of zero (no 104compression) from the proactive selection of no compression. 105 106The core will receive the request for the compression level and automatically 107choose a compression algorithm based on its knowledge about the peer 108(communicated by the client via the `grpc-accept-encoding` header. Note that the 109absence of this header means no compression is supported by the client/peer). 110 111### Compression **Algorithm** in Call Responses 112 113**Server should avoid setting the compression algorithm directly**. Prefer 114setting compression levels unless there's a _very_ compelling reason to choose 115specific algorithms (benchmarking, testing). 116 117Selection of concrete compression algorithms is performed by adding a 118`(GRPC_COMPRESS_REQUEST_ALGORITHM_KEY, <algorithm-name>)` key-value pair to the 119initial metadata, where `GRPC_COMPRESS_REQUEST_ALGORITHM_KEY` is defined in 120[`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 121and `<algorithm-name>` is the human readable name of the algorithm as given in 122[the HTTP2 spec](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) 123for `Message-Encoding` (e.g. gzip, identity, etc.). See 124[`grpc_compression_algorithm_name`](https://github.com/grpc/grpc/blob/master/src/core/lib/compression/compression.c) 125for the mapping between the `grpc_compression_algorithm` enum values and their 126textual representation. 127 128## Per Message Settings 129 130To disable compression for a specific message, the `flags` field of `grpc_op` 131instances of type `GRPC_OP_SEND_MESSAGE` must have its `GRPC_WRITE_NO_COMPRESS` 132bit set. Refer to 133[`grpc/impl/codegen/compression_types.h`](https://github.com/grpc/grpc/blob/master/include/grpc/impl/codegen/compression_types.h)), 134
