JSON vs Protobuf: same payload, different wire.

JSON vs Protobuf — what's the difference?

Same record, very different bytes.

JSON (2001, RFC 8259) and Protocol Buffers (Google, 2008, open-sourced 2008) solve the same problem — encoding a record as bytes for transmission — with opposite trade-offs. JSON is human-readable and schemaless; Protobuf is compact and schema-first. The simulator above encodes the same record both ways so you can see the size and parse-cost difference for yourself.

Imagine an order-tracking service. Every time a customer places an order, the front-end pings the back-end with a small object: an ID, a customer name, maybe a timestamp and a price. A typical record might be eighty characters of JSON. Multiply by a million orders an hour at peak — that's eighty megabytes per hour of pure structural overhead before any real data joins the wire. Across a fleet of thousands of services calling each other, the cost of carrying field names and quotes and commas around becomes a meaningful slice of the bandwidth bill, and an even bigger slice of the CPU time spent parsing.

The problem JSON solves is "I want this object to be readable by anything, including a human staring at a terminal". The problem Protobuf solves is "I have a known schema on both ends of this wire and I want the bytes to be as small and as fast to parse as possible". They are answers to different questions. JSON makes the wire self-describing — the field names travel with every message, so a consumer who has never seen the schema can still figure out what they are looking at. Protobuf strips the field names off and replaces them with small integer tags; the schema lives in a .proto file shared between both ends at compile time. Without that file, a Protobuf payload looks like noise.

Some concrete numbers anchor the trade. The same little order record above costs about 28 bytes in JSON and about 8 bytes in Protobuf — a 3.5× shrink, mostly because the field name "customer" no longer ships with every message. On parse, a high-quality JSON tokeniser hits roughly 200–400 MB/s on a single core; a Protobuf decoder hits 1–2 GB/s. On a service that handles a hundred thousand requests per second, the difference between a 12 % CPU floor and a 3 % CPU floor is real money and real headroom. On a public API where Curl and a junior engineer in another company have to read the response by eye, JSON's transparency is worth far more than the CPU savings.

The simulator above lets you flip between modes and watch the byte counts on the same record. The compact mode strips JSON whitespace; the protobuf column shows the schema-encoded equivalent. Sub-percentage differences vanish at low volumes; at scale they compound. The widely held rule of thumb in 2026 is JSON at the network edge — anywhere humans, browsers, or unknown third parties might be reading — and Protobuf inside the service mesh, where bandwidth and parse time are both real costs and the schema is under your control. The rest of this article walks through both wire formats in detail, the rules for evolving the schema without breaking deployed clients, and the wider zoo of binary formats (Avro, Thrift, Cap'n Proto, FlatBuffers, MessagePack, Arrow) that fill in the corners between the two extremes.

Origins — JSON 2001, Protobuf 2008, two formats from different decades

Two formats from different decades.

JSON was specified by Douglas Crockford in State Software circa 2001, lifted from the object-literal syntax of JavaScript with a few subtractions (no functions, no comments, no trailing commas) and a few clarifications (strict UTF-8, restricted number grammar). Crockford's original site json.org went live in 2002. The IETF standardised it in RFC 4627 (2006), updated to RFC 7159 (2014), updated again to RFC 8259 (2017). Ecma International published an essentially identical specification as ECMA-404 in 2013, revised in 2017. The two specs are interoperable; differences are limited to whether the document must be a value (RFC 8259) or may be only an object or array (older readings).

JSON's design rationale was minimalism. Crockford has said in interviews and in his 2008 book JavaScript: The Good Parts (O'Reilly) that he discovered JSON rather than designing it — every part of the syntax was already valid JavaScript, so any browser of the era could evaluate it directly via eval(). That was the original sin which forced the spec to forbid executable content; today every language ships a parser that does not eval. The format won by being the path of least resistance for AJAX, the early-2000s web technique that made browsers feel like applications.

Protocol Buffers began earlier inside Google. Internal documents date the first version (proto1) to 2001; Jeff Dean and Sanjay Ghemawat were among the original architects. Proto2 became the long-running internal lingua franca, and Google open-sourced it in July 2008 alongside an Apache 2.0 licence. Proto3 followed in 2016 with a deliberately smaller surface area: removed required fields, removed extension groups, default values are zero. Editions, the post-proto3 evolution shipped in 2023, treat individual features (such as field presence) as opt-in flags so projects can mix proto2 and proto3 semantics in the same schema.

The two formats answer different questions. JSON answers "how do I move structured data through a system that already speaks text." Protobuf answers "how do I move structured data through a system where bandwidth and parse time are non-trivial costs and a schema can be enforced." Each is excellent at its question and uncomfortable at the other's.

A revealing historical detail: every binary serialisation format in production today — Thrift, Avro, MessagePack, Cap'n Proto, FlatBuffers, BSON, CBOR, Arrow IPC — was published between 2007 and 2016. The same window saw JSON spread from "ad-hoc browser format" to RFC-blessed lingua franca. The two waves were not unrelated. Cheap CPU and cheap memory in the 2010s made text-based interchange tolerable everywhere except inside Google-scale fleets, which is where the binary alternatives kept being designed and redesigned. As fleet sizes grew further the pendulum swung back; 2026's web is a hybrid in which both layers coexist, and most engineers will write code on both sides of the boundary at some point in their careers.

Wire format — bytes that earn their keep

Bytes that earn their keep.

Protobuf encodes each field as a tag-length-value triple. The tag packs the field number and one of six wire types (varint, fixed64, length-delimited, fixed32, plus two deprecated group markers) into a varint; for field numbers 1–15, this fits in a single byte. Varints use one bit per byte as a continuation flag and seven for data, so values under 128 cost one byte, values under 16384 cost two, and full 64-bit values cost up to ten. ZigZag encoding maps signed integers to unsigned by interleaving — sint32(-1) becomes the unsigned 1, sint32(1) becomes 2 — so small negative numbers stay one byte instead of ten.

message Order { int32 id = 1; string customer = 2; }
Order { id: 8120, customer: "Ada" }
        ↓
08 b8 3f 12 03 41 64 61
│  └─┬─┘ │  │  └─┬───┘
│   id   │  │   "Ada"
│        │  └ length=3
│        └ tag(2)+wire(LEN)
└ tag(1)+wire(VARINT)

Strings, byte arrays, and nested messages all share the length-delimited wire type: a varint length, then that many bytes of payload. Packed repeated fields (proto3 default for primitives) collapse a list into one length-delimited region with all elements concatenated, eliminating the per-element tag. Fixed-width types (fixed32, fixed64, floats, doubles) skip varint encoding; useful when most values are large enough that varint would not save bytes anyway.

JSON has no equivalent compactness. Every field name appears in every message; numeric values are rendered as decimal text; booleans cost four or five bytes; the structural punctuation ({}[]":,) is per-field overhead. Compare {"id":8120,"customer":"Ada"} — 28 bytes — to its eight-byte Protobuf encoding. On chunky payloads dominated by short numeric fields, the ratio reaches 4–8×. Multiplied across millions of records per second across an entire fleet, the difference becomes infrastructure budget.

Two encoding details deserve specific call-outs. Floating-point values in Protobuf use IEEE 754 single (float, 4 bytes) or double (double, 8 bytes) directly; JSON renders them as decimal text with platform-dependent precision. The canonical .proto rule "field numbers 1–15 cost one byte" is load-bearing: organise your messages so the most frequent fields claim the low numbers. A logging-format schema with the timestamp at field 1 and the severity at field 2 saves real bytes against the same schema with those at fields 17 and 18. Google's internal style guide enforces this; Buf's lint rules surface it as a warning.

Schema philosophy — schemaless JSON vs schema-first Protobuf

Two philosophies, two wires.

JSON puts every field name in every message. The wire format is the schema. You can read a JSON document with no other context — that is the whole pitch. Protobuf splits them: a .proto file declares the schema once, the wire is bare numbered tags and packed bytes. Without the schema, the wire is opaque; in exchange, the wire is small and the parse is unambiguous.

JSON does have an optional schema layer. JSON Schema (active since 2009, with draft-2020-12 the current version, edited by Henry Andrews and Austin Wright) describes the shape of a document with another JSON document; tools like ajv (Ajv, the dominant JavaScript validator), Python's jsonschema, and Go's gojsonschema validate documents at the boundary. OpenAPI 3.1 uses JSON Schema for its request and response bodies, aligning the two specs after years of partial overlap. The schema is a runtime check, however, not a compile-time guarantee; nothing forces a producer to consult it.

The decision rule: JSON at the network edge (browsers, public APIs, debugging) and Protobuf inside the mesh (microservices, gRPC, internal queues). Most mature systems run both, with a translation layer at the boundary. gRPC-Gateway, Connect (Buf), and tRPC are different points along the same translation continuum.

Self-describing formats also handle the long tail of one-off integrations more gracefully. A partner who wants to consume your data once a week from a different language ecosystem will always reach for JSON; the moment you ask them to install protoc, set up a generated client, and import a third-party SDK, you have lost half the prospective integrators. This is the operational reason public APIs from Stripe, Twilio, GitHub, Slack, and a thousand smaller SaaS companies stay on REST + JSON even when their internal traffic has moved entirely to gRPC. The format is partly a UX choice for the consumer.

Adjacent formats — Cap'n Proto, FlatBuffers, Avro, Thrift, Arrow

Cap'n Proto, FlatBuffers, Avro, Thrift, Arrow.

Apache Thrift (Facebook 2007, donated to Apache 2008) is the older cousin of Protobuf with a near-identical IDL and a wider transport story (binary, compact, JSON, plus pluggable transports). Used in the early Cassandra protocol, Evernote, and a generation of Facebook services, Thrift remains active in the Apache ecosystem; the wire format trades one or two bytes against Protobuf in exchange for slightly different layout decisions. Apache Avro (Doug Cutting, 2009, originally for Hadoop) takes a different angle: the schema travels with the data when desirable, supports both binary and JSON encoding, and dominates the Kafka + Schema Registry world.

Cap'n Proto (Kenton Varda, 2013, designed by the same engineer who maintained protobuf-c at Google) inverts the usual trade. The wire format is the in-memory layout — no parse step, no allocation — at the cost of slightly larger payloads on average. Reads of partially-loaded buffers work directly out of an mmap. Cap'n Proto's RPC layer adds promise pipelining: the client can issue a chained call before the previous call's response has arrived, eliminating round trips on dependent operations. Used in Cloudflare Workers, Sandstorm, and any system where parse time on the receiver dominates.

FlatBuffers (Wouter van Oortmerssen at Google, 2014) targets the same zero-parse goal for game and mobile workloads, with offset tables that allow random access into a flat buffer. MessagePack (Sadayuki Furuhashi, 2008) is a JSON-isomorphic binary format used by Redis 2+ replication, fluentd, and the Vim msgpack-rpc plugin family. BSON (MongoDB's wire format, 2009) embeds typing and lengths into a JSON-like document; useful for the database, less attractive standalone. CBOR (RFC 8949, Carsten Bormann and Paul Hoffman, 2020 update of RFC 7049) is the IETF's MessagePack-flavoured binary, used in CoAP and the IoT stack.

Apache Arrow (Wes McKinney, Jacques Nadeau et al, 2016) takes a different approach again — a columnar in-memory format with a serialisable IPC envelope, optimised for analytics and zero-copy interchange between Pandas, Spark, DuckDB, and the broader data-science Python and R ecosystems. The format is not a competitor to Protobuf for RPC; it competes with Parquet for analytical interchange. Knowing the zoo exists is what saves you from picking JSON or Protobuf when one of the others would have been a better fit; for most internal RPC the answer remains Protobuf, but the boundary deserves consideration.

Schema evolution — backward and forward compatibility

Compatibility, in both directions.

In a polyglot deployment you cannot upgrade every service at once. There will be producers running v2 talking to consumers still on v1, and vice versa. The format determines what you can change without coordinated downtime.

Protobuf encodes the rules into the wire format itself. Field numbers are forever; old code reading new wire silently skips unknown numbers; new code reading old wire sees defaults for missing fields. The single biggest production hazard in proto3 was the absence of field presence — there was no way to distinguish "the field was set to its default" from "the field was not set". Proto3 added optional back in 2020 (after a years-long debate). Editions, the post-proto3 model, treats presence as a flag that can be set per file or per field. The takeaway: JSON evolves by trust, Protobuf evolves by contract. Trust scales until it does not; contract scales because tooling enforces it.

Performance — how big, how fast

How big, how fast.

A typical user record — half a dozen fields, a few small arrays, a couple of timestamps — encodes to roughly 150–250 bytes of JSON and 40–70 bytes of Protobuf. The compression ratio narrows with payload shape: long string-heavy messages converge toward 1.5–2×; chunky numeric messages diverge toward 5–8×. Add gzip at the HTTP layer and JSON closes much of the gap on the wire (text compresses well; varint-packed binary does not). The remaining gap is parse cost, which gzip cannot help with.

Encoding speed differs by larger margins. On modern x86 servers with one 500-byte message, encoding to JSON via Go's encoding/json takes around 1.2–1.8 microseconds; encoding to Protobuf via google.golang.org/protobuf takes 0.2–0.4 microseconds. The Rust ecosystem's prost and the C++ runtime are both significantly faster again. Switch to simdjson (Daniel Lemire and Geoff Langdale, 2019) and JSON parse closes by 2–4×; switch to vtprotobuf and Protobuf does the same. The relative gap stays roughly intact.

Public benchmarks worth reading: Karl Seguin's 2019 Benchmarking JSON vs MessagePack vs Protobuf, Discord's 2017 post on switching to Erlang's term_to_binary for inter-node messaging, Uber's 2019 piece on moving Kafka payloads from JSON to Protobuf for a measured 30% throughput gain, and Bufbuild's continuous benchmark suite comparing Connect-Go, grpc-go, and several JSON-over-HTTP variants on identical workloads. None of these benchmarks tell the whole story; they all tell some of it.

Cost in dollars follows from bytes and CPU. AWS egress at $0.09/GB across regions: a service moving 100 TB/month spends $9000 just to move the bits. A 3× wire-size reduction saves $6000/month on that workload alone, plus a similar fraction off receiver CPU. For internal mesh traffic at any meaningful scale, the financial argument tends to dominate the architectural one — which is why the Googles and Metas and Discords end up on Protobuf even when individual engineers might prefer the legibility of JSON.

Latency at the tail tells a similar story. p99 serialisation time for a typical Protobuf message stays in the low microseconds even under GC pressure; JSON's tail balloons during string-allocation spikes because every parse allocates per-key and per-value buffers. Stripe's 2022 internal post-mortem on a JSON-related GC pause cycle led the team to migrate hot-path serialisation to a hand-tuned binary format; Discord's 2017 Erlang post made the same point about term_to_binary stalling under load. The pattern is consistent: text formats are gentle on average, harsh at the tail; binary formats are flatter on both. For latency-sensitive systems the tail is what the engineering team measures and tunes.

// JSON · 142 bytes
{
  "id": 8120,
  "customer": "Ada Lovelace",
  "items": [
    { "sku": "BK-19", "qty": 2, "price_cents": 4500 }
  ],
  "status": "PAID"
}

// .proto · 38 bytes on the wire
syntax = "proto3";
message Order {
  int32  id          = 1;
  string customer    = 2;
  repeated LineItem items = 3;
  Status status      = 4;
  enum Status { PENDING = 0; PAID = 1; SHIPPED = 2; }
}
message LineItem { string sku = 1; int32 qty = 2; int32 price_cents = 3; }

Where each format surprises you

Where each format surprises you.

JSON's number type is a single decimal value with unspecified precision. Most parsers map it to IEEE 754 doubles (53 bits of mantissa), which silently lose precision above 2^53. Twitter's API famously serialised tweet IDs as JSON numbers and broke every JavaScript client when IDs crossed the 53-bit threshold; the workaround was a parallel id_str field with the integer rendered as a string. Anyone moving 64-bit identifiers over JSON should ship them as strings from day one.

Protobuf has corresponding hazards. The classic proto3 surprise is field presence: message.count == 0 is indistinguishable from "count was not set" without the optional keyword. The 2020 reintroduction of optional fixes this, but a decade of code shipped without it. Field number 19000–19999 is reserved for the Protobuf implementation itself; using one anyway will silently mangle the wire. Map fields are syntactic sugar over a repeated message of (key, value) pairs; ordering is not preserved across encodings. Unknown fields in proto3 used to be discarded by default; the 3.5 release in late 2017 changed that to preserve them, restoring round-trip fidelity for proxies that re-serialise messages they only partially understand.

Both formats inherit the same security pitfalls around input validation. Stripe's 2020 incident report on a parser DoS, Cloudflare's 2017 Cloudbleed (a buffer-overrun in their HTML parser, not directly JSON, but the class is identical), and a long string of OpenSSH and curl CVEs around malformed input all reinforce the same point: any parser that touches untrusted bytes is attack surface. Use vetted libraries, keep them current, and prefer explicit length limits to implicit ones. json.MarshalSize-style limits, Protobuf's SetRecursionLimit, and gRPC's MaxReceiveMessageSize are all worth setting deliberately rather than accepting defaults.

One last operational principle. Pick the format the consumer's tooling supports best. A REST API for partner developers should be JSON because every developer's tool chain understands JSON; a queue between two Go services should be Protobuf because every Go service has an importable schema. The wrong format chosen for the right reasons (small bytes, fast parse) costs an organisation more in friction than the bytes ever save.

Two related anti-patterns are worth naming. The first is Protobuf-as-string-blob, in which a service wraps a Protobuf-encoded payload inside a JSON envelope as a base64 string. The result combines the wire size of JSON with the debuggability of Protobuf — the worst of both. The second is JSON with a bespoke schema language, where a team invents its own validator instead of using JSON Schema or OpenAPI. Both fail the same test: future maintainers cannot use existing tools. Reaching for the standard, even when it is slightly less convenient, is almost always the higher-use choice. Document the boundaries between formats in the same place you document API versioning, and the next engineer will thank you. Above all, keep the schema in version control where everyone can find it.

Further reading on JSON vs Protobuf

Primary sources, in order.

• IETF · RFC 8259
  The JavaScript Object Notation (JSON) Data Interchange Format
  The current canonical JSON spec, replacing 4627 and 7159. Two pages of normative text and a clear grammar.
• protobuf.dev
  Protocol Buffers Encoding
  The wire format, byte by byte. Required reading once. Tags, varints, ZigZag, packed repeated fields.
• Kenton Varda · 2013
  Cap'n Proto
  The zero-parse alternative. The home page is also the introduction; pair with the "Time travel!" pipelining post.
• IETF · RFC 8949
  CBOR · Concise Binary Object Representation
  The IETF's binary cousin of JSON, used in IoT and CoAP stacks. Bormann and Hoffman, 2020 update of RFC 7049.
• Apache Avro
  Avro Specification
  The Hadoop-era format that runs much of the Kafka world. Schema-shipped-with-data is the distinguishing trick.
• Buf · 2022
  Connect: A better gRPC
  The runtime that speaks gRPC, gRPC-Web, and a JSON-over-HTTP variant on the same endpoint. The pragmatic translation layer.
• Uber Engineering
  How Uber Squeezed Trip Data
  A practitioner case study on switching internal pipelines from JSON to compressed Protobuf and the throughput gains.
• Martin Kleppmann · book
  Designing Data-Intensive Applications · Chapter 4
  "Encoding and Evolution" walks through JSON, Protobuf, Avro, Thrift side by side, with a long discussion of forward and backward compatibility. The accessible textbook treatment.
• Douglas Crockford
  Crockford's writings on JavaScript & JSON
  Notes from the person who discovered (his word) JSON. The page on JSON's design rationale is short, direct, and often pointed.
• Kenton Varda · talk
  Cap'n Proto: Insanely fast data interchange and RPC
  A student-friendly talk introducing zero-parse formats and promise pipelining. Worth an hour for the contrast against Protobuf.
• Semicolony simulator
  gRPC vs REST
  The transport layer that usually sits on top of the format choice. Same call, two wires.
• Semicolony guide
  HTTP, picked apart
  The substrate JSON-over-HTTP and Protobuf-over-gRPC both ride on. Versions, framing, semantics.