JSON Message Construction

JSON Message Construction

Overview

This section covers creating and parsing JSON messages on the ESP32 using the ArduinoJson library. By the end of this section, you will be able to:

• Install and configure the ArduinoJson library
• Build JSON objects for MQTT publishing
• Parse incoming JSON messages from Node-RED or other MQTT clients
• Use the ArduinoJson Assistant tool to generate correct code

Prerequisites

• PubSubClient library configured (01-10)
• Basic Sketch architecture (01-11)

Key Concepts

Why JSON for IoT

JSON (JavaScript Object Notation) is the standard data format for IoT communication because:

• Human-readable: Easy to inspect and debug
• Self-describing: Field names explain the data content
• Language-independent: Usable across ESP32, Node-RED, Python, JavaScript, etc.
• Flexible structure: Easily add or remove fields without breaking parsers
• Widely supported: Every MQTT client library handles JSON

Example JSON Payload:

{
  "device": "ESP32",
  "temperature": 25.5,
  "humidity": 65.2,
  "lux": 312,
  "rssi": -67
}

ArduinoJson Library

ArduinoJson is the most popular JSON library for Arduino/ESP32. Key features:

• Deserialization: Parse JSON strings into program-accessible objects
• Serialization: Build JSON strings from program variables
• Memory-efficient: Configurable allocation via StaticJsonDocument or DynamicJsonDocument
• Assistant Tool: Web-based code generator at arduinojson.org/v7/assistant/

ArduinoJson v7 (Current):

Class	Description
JsonDocument	Base class for JSON document storage
StaticJsonDocument<N>	Fixed-size allocation (stack) — use when size is known
DynamicJsonDocument<N>	Heap allocation — use for variable sizes
JsonObject	Named key-value pairs
JsonArray	Ordered list of values

Implementation Steps

Step 1: Install ArduinoJson

PlatformIO (platformio.ini):

lib_deps =
    bblanchon/ArduinoJson @ ^7.0

Arduino IDE: Library Manager → search “ArduinoJson” → Install

Step 2: Using the ArduinoJson Assistant

The ArduinoJson Assistant generates correct code for your specific JSON structure:

1. Go to https://arduinojson.org/v7/assistant/
2. Select your board: ESP32
3. Select operation: Deserialize (receive) or Serialize (send)
4. Paste a sample JSON document
5. The assistant generates the exact code needed
6. Copy the code into your sketch

This eliminates manual memory calculation and syntax guessing.

Step 3: Serializing JSON (ESP32 → MQTT Publish)

Building a JSON payload to publish via MQTT:

#include <ArduinoJson.h>

void sendMQTTValues() {
  // Calculate required document size
  // Use the Assistant for accuracy: https://arduinojson.org/v7/assistant/
  StaticJsonDocument<200> doc;

  // Fill the JSON document
  doc["device"] = "ESP32";
  doc["temperature"] = 25.5;
  doc["humidity"] = 65;
  doc["lux"] = 312;
  doc["online"] = true;

  // Serialize to a string buffer
  char buffer[256];
  size_t len = serializeJson(doc, buffer);

  // Publish via MQTT
  if (mqttClient.publish("esp32/data", buffer, len)) {
    Serial.print("Published JSON: ");
    Serial.println(buffer);
  }
}

Using real sensor data:

void sendSensorData(float temperature, float humidity, int lux) {
  StaticJsonDocument<200> doc;

  doc["device"] = "sensor-01";
  doc["temperature"] = temperature;
  doc["humidity"] = humidity;
  doc["lux"] = lux;
  doc["rssi"] = WiFi.RSSI();
  doc["timestamp"] = millis() / 1000;  // Uptime in seconds

  char buffer[256];
  size_t len = serializeJson(doc, buffer);

  mqttClient.publish("esp32/data", buffer, len);
}

Nested JSON objects:

void sendComplexData() {
  StaticJsonDocument<300> doc;

  // Device info
  JsonObject device = doc.createNestedObject("device");
  device["id"] = "ESP32-001";
  device["firmware"] = "2.1.0";

  // Sensor readings
  JsonObject sensors = doc.createNestedObject("sensors");
  sensors["temperature"] = 25.5;
  sensors["humidity"] = 65.2;

  // Array of values
  JsonArray history = doc.createNestedArray("history");
  history.add(25.1);
  history.add(25.3);
  history.add(25.5);

  char buffer[512];
  serializeJsonPretty(doc, Serial);  // Pretty-print for debugging
  size_t len = serializeJson(doc, buffer);
  mqttClient.publish("esp32/data", buffer, len);
}

Output:

{
  "device": {
    "id": "ESP32-001",
    "firmware": "2.1.0"
  },
  "sensors": {
    "temperature": 25.5,
    "humidity": 65.2
  },
  "history": [25.1, 25.3, 25.5]
}

Step 4: Deserializing JSON (MQTT Receive → ESP32)

Parsing JSON messages received from Node-RED:

void mqttCallback(char* topic, byte* payload, unsigned int length) {
  // Convert payload to string
  char json[length + 1];
  memcpy(json, payload, length);
  json[length] = '\0';

  Serial.print("Received JSON: ");
  Serial.println(json);

  // Parse JSON
  StaticJsonDocument<256> doc;
  DeserializationError error = deserializeJson(doc, json);

  if (error) {
    Serial.print("JSON parse failed: ");
    Serial.println(error.c_str());
    return;
  }

  // Extract values
  const char* command = doc["command"];
  int value = doc["value"];
  bool enabled = doc["enabled"];

  Serial.print("Command: ");
  Serial.println(command ? command : "null");
  Serial.print("Value: ");
  Serial.println(value);
  Serial.print("Enabled: ");
  Serial.println(enabled ? "true" : "false");

  // Act on the parsed data
  if (strcmp(command, "LED_ON") == 0) {
    digitalWrite(2, HIGH);
  } else if (strcmp(command, "LED_OFF") == 0) {
    digitalWrite(2, LOW);
  } else if (strcmp(command, "SET_THRESHOLD") == 0) {
    temperatureThreshold = value;
  }
}

Step 5: Handling JSON Arrays

// Incoming JSON: {"channels": [1, 3, 5], "mode": "auto"}
void parseArrayExample(char* json) {
  StaticJsonDocument<256> doc;
  DeserializationError error = deserializeJson(doc, json);

  if (error) return;

  // Access array
  JsonArray channels = doc["channels"];
  const char* mode = doc["mode"];

  Serial.print("Mode: ");
  Serial.println(mode);
  Serial.print("Active channels: ");

  for (int channel : channels) {
    Serial.print(channel);
    Serial.print(" ");
    // Activate each channel
  }
  Serial.println();
}

Step 6: Memory Management

The most common ArduinoJson pitfall is incorrect memory allocation:

// TOO SMALL — will overflow silently
StaticJsonDocument<64> doc;   // Too small for most payloads
doc["temperature"] = 25.5;    // Data may be corrupted
doc["humidity"] = 65.2;       // Overflow!

// CORRECT — use the Assistant to calculate
StaticJsonDocument<200> doc;  // Right size for typical sensor payload

// For unknown sizes, use DynamicJsonDocument
DynamicJsonDocument doc(2048); // Heap-allocated, 2048 bytes

Memory Estimation:

• Each key: ~ (key_length + 1) bytes
• Each string value: ~ (string_length + 1) bytes
• Each number: 8 bytes
• Each boolean: 2 bytes
• JSON overhead: ~ 30-40 bytes

Use the ArduinoJson Assistant to calculate precisely.

Verification

• JSON payloads published via MQTT are valid JSON (verify with a JSON validator)
• Node-RED can parse the published JSON payloads
• ESP32 correctly parses JSON commands from Node-RED
• Array and nested object handling works correctly
• Memory allocation is sufficient for the largest expected payload

Troubleshooting

Serialized JSON is truncated or garbled

Cause: Buffer size too small for the JSON document.

Solution:

1. Increase StaticJsonDocument<N> size
2. Increase char buffer[N] size (must be larger than the document)
3. Use serializeJson(doc, buffer, sizeof(buffer)) to prevent overflow

deserializeJson() returns “NoMemory”

Cause: JsonDocument is too small for the incoming JSON.

Solution:

1. Use the Assistant to calculate the correct size
2. Switch to DynamicJsonDocument for variable-size payloads
3. Increase the document size

Compilation error: “JsonDocument is too large for stack”

Cause: StaticJsonDocument allocates on the stack; very large documents exceed stack limits.

Solution:

1. Switch to DynamicJsonDocument (heap allocation)
2. Reduce document size by simplifying the JSON structure
3. Increase ESP32 stack size via build_flags = -DCONFIG_ARDUINO_LOOP_STACK_SIZE=16384

Best Practices

• Always check deserialization errors: if (deserializeJson(doc, json)) catches malformed JSON
• Use the Assistant: It eliminates guesswork for memory allocation
• Prefer StaticJsonDocument for known sizes: It’s faster and avoids heap fragmentation
• Use serializeJsonPretty() during development: Readable output helps debugging
• Avoid dynamic field names in critical paths: Creating strings for keys is slower than literals
• Use JsonObject and JsonArray for complex structures: They are type-safe and efficient

Summary

1. ArduinoJson handles JSON serialization (ESP32 → MQTT) and deserialization (MQTT → ESP32)
2. The ArduinoJson Assistant web tool generates correct code for any JSON structure
3. Memory management is critical: use the correct JsonDocument size
4. Always check DeserializationError when parsing incoming messages
5. Complex nested objects and arrays are fully supported
6. JSON is the standard data format for ESP32 ↔ Node-RED communication in this course

References

• ArduinoJson Documentation
• ArduinoJson v7 Assistant
• ArduinoJson GitHub