JSON Data Parsing

JSON Data Parsing

Overview

This section covers parsing JSON data on the ESP32 using the ArduinoJson library. The ESP32 receives weather data as a JSON string via MQTT and needs to extract individual values for display on the e-paper screen. After completing this section, you will be able to:

• Install and configure the ArduinoJson library
• Parse JSON strings to extract specific values
• Handle numeric, string, and nested JSON fields
• Build display-ready data structures from parsed JSON

Prerequisites

Before starting this section, please ensure:

• ArduinoJson library is installed
• Basic understanding of JSON format
• MQTT communication is configured (see Chapter 01)

Key Concepts

JSON Structure for Weather Data

The ESP32 receives JSON data via MQTT in this format:

{
    "temperature": "25.5",
    "feels_like": "24.0",
    "humidity": 60,
    "pressure": 1013,
    "description": "scattered clouds",
    "icon": "03d",
    "city": "London",
    "timestamp": "2026-05-17T10:30:00.000Z",
    "error": false
}

ArduinoJson Library

ArduinoJson is the most widely used JSON library for embedded systems:

Feature	Description
Memory model	Static (stack) or Dynamic (heap) allocation
Version 6 vs 7	Both supported; v7 has improved API
Minimal RAM	~250 bytes for basic parsing
Supported types	Object, Array, String, Number, Boolean, Null

Memory Calculation:

// JSON document size depends on complexity:
// Formula: JSON_OBJECT_SIZE(number_of_keys)
// For 8 keys: JSON_OBJECT_SIZE(8) ≈ 8 * 2 + 8 = 24 bytes (overhead) + data

// Recommended: Use ArduinoJson Assistant to calculate
// https://arduinojson.org/v6/assistant/

Implementation Steps

Step 1: Install ArduinoJson Library

For PlatformIO (platformio.ini):

lib_deps =
    bblanchon/ArduinoJson@^7.0.0

For Arduino IDE:

1. Tools → Manage Libraries
2. Search for “ArduinoJson” by Benoit Blanchon
3. Click Install

Step 2: Basic JSON Parsing

#include <ArduinoJson.h>

void setup() {
    Serial.begin(115200);

    // Sample JSON data received via MQTT
    const char* jsonData = "{\"temperature\":\"25.5\",\"humidity\":60,\"description\":\"scattered clouds\",\"city\":\"London\"}";

    // Create a JSON document with sufficient capacity
    JsonDocument doc;

    // Parse the JSON string
    DeserializationError error = deserializeJson(doc, jsonData);

    // Check for parsing errors
    if (error) {
        Serial.print("JSON parsing failed: ");
        Serial.println(error.c_str());
        return;
    }

    // Extract values
    const char* temperature = doc["temperature"];
    int humidity = doc["humidity"];
    const char* description = doc["description"];
    const char* city = doc["city"];

    // Display extracted values
    Serial.print("City: ");
    Serial.println(city);
    Serial.print("Temperature: ");
    Serial.println(temperature);
    Serial.print("Humidity: ");
    Serial.println(humidity);
    Serial.print("Description: ");
    Serial.println(description);
}

void loop() {}

Step 3: Parsing in the MQTT Callback

Integrate JSON parsing with the MQTT callback function:

#include <WiFi.h>
#include <PubSubClient.h>
#include <ArduinoJson.h>

// WiFi and MQTT configuration
const char* ssid = "YOUR_SSID";
const char* password = "YOUR_PASSWORD";
const char* mqtt_server = "192.168.1.100";
const char* mqtt_topic = "factory/weather/data";

WiFiClient espClient;
PubSubClient client(espClient);

// Display data structure
struct WeatherData {
    char temperature[8];
    int humidity;
    char description[32];
    char city[32];
    bool valid;
};

WeatherData currentWeather = { "", 0, "", "", false };

void callback(char* topic, byte* payload, unsigned int length) {
    // Convert payload to null-terminated string
    char jsonBuffer[256];
    if (length >= 256) {
        Serial.println("Payload too large!");
        return;
    }
    memcpy(jsonBuffer, payload, length);
    jsonBuffer[length] = '\0';

    // Parse JSON
    JsonDocument doc;
    DeserializationError error = deserializeJson(doc, jsonBuffer);

    if (error) {
        Serial.print("JSON parse error: ");
        Serial.println(error.c_str());
        currentWeather.valid = false;
        return;
    }

    // Extract values with default fallbacks
    strlcpy(currentWeather.temperature,
            doc["temperature"] | "--.-",
            sizeof(currentWeather.temperature));

    currentWeather.humidity = doc["humidity"] | 0;

    strlcpy(currentWeather.description,
            doc["description"] | "Unknown",
            sizeof(currentWeather.description));

    strlcpy(currentWeather.city,
            doc["city"] | "Unknown",
            sizeof(currentWeather.city));

    currentWeather.valid = true;

    // Log parsed data
    Serial.println("Weather data updated:");
    Serial.print(" - Temp: "); Serial.println(currentWeather.temperature);
    Serial.print(" - Humi: "); Serial.println(currentWeather.humidity);
    Serial.print(" - Cond: "); Serial.println(currentWeather.description);
}

void setup() {
    Serial.begin(115200);

    // Setup WiFi
    WiFi.begin(ssid, password);
    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
    }
    Serial.println("WiFi connected");

    // Setup MQTT
    client.setServer(mqtt_server, 1883);
    client.setCallback(callback);

    // Connect and subscribe
    client.connect("ESP32_Display");
    client.subscribe(mqtt_topic);
}

void loop() {
    if (!client.connected()) {
        // Reconnect logic
    }
    client.loop();
}

Step 4: Parsing with Nested Objects

Some APIs return nested JSON. Handle nested objects:

{
    "main": {
        "temp": 25.5,
        "humidity": 60,
        "pressure": 1013
    },
    "weather": [
        {
            "description": "scattered clouds",
            "icon": "03d"
        }
    ],
    "name": "London"
}

Parsing Nested Data:

void parseNestedJSON(const char* json) {
    JsonDocument doc;
    DeserializationError error = deserializeJson(doc, json);

    if (error) return;

    // Access nested objects
    float temp = doc["main"]["temp"];
    int humidity = doc["main"]["humidity"];

    // Access array element
    const char* description = doc["weather"][0]["description"];
    const char* icon = doc["weather"][0]["icon"];

    // Access top-level field
    const char* city = doc["name"];

    Serial.print(city);
    Serial.print(": ");
    Serial.print(temp);
    Serial.print("°C, ");
    Serial.println(description);
}

Step 5: Building Display Strings

Convert parsed JSON data into display-ready strings:

void updateDisplayFromJSON() {
    if (!currentWeather.valid) {
        displayError("No Data");
        return;
    }

    display.fillScreen(GxEPD_WHITE);

    // Display city name
    display.setFont(&FreeMonoBold12pt7b);
    display.setCursor(5, 25);
    display.print(currentWeather.city);

    // Display temperature (large)
    display.setFont(&FreeMonoBold24pt7b);
    display.setCursor(5, 65);

    // Build temperature string
    char tempStr[16];
    snprintf(tempStr, sizeof(tempStr), "%s°C", currentWeather.temperature);
    display.print(tempStr);

    // Display condition
    display.setFont(&FreeMono9pt7b);
    display.setCursor(5, 95);
    display.print(currentWeather.description);

    // Display humidity
    char humStr[32];
    snprintf(humStr, sizeof(humStr), "Humidity: %d%%", currentWeather.humidity);
    display.setCursor(5, 115);
    display.print(humStr);

    display.display();
}

Memory Optimization

Use the ArduinoJson Assistant to calculate required memory:

1. Go to ArduinoJson Assistant
2. Paste your JSON document
3. The tool calculates:
   • Required JsonDocument size
   • Recommended allocation method

Example:

// For the weather JSON with 8 fields:
// Recommended: JsonDocument with 256 bytes

// Static allocation (stack) — faster, limited size
StaticJsonDocument<256> doc;

// Dynamic allocation (heap) — larger, slightly slower
DynamicJsonDocument doc(1024);

Verification

• JSON string is parsed without errors
• Temperature value is extracted correctly
• Humidity integer is extracted correctly
• String fields (description, city) are not truncated
• Display updates with parsed data
• Error handling works with invalid JSON

Troubleshooting

Issue 1: Parsing Returns Empty Values

Symptom:

Temperature: (empty)

Solutions:

• Verify JSON key names match EXACTLY (case-sensitive)
• Check if JSON structure is nested differently than expected
• Print the raw JSON before parsing to verify

Issue 2: Memory Allocation Failure

Symptom:

DeserializationError::NoMemory

Solutions:

• Increase JsonDocument size
• Use DynamicJsonDocument for larger payloads
• Reduce the number of fields being extracted

Issue 3: Numeric Values Incorrect

Symptom:

• Temperature reads as 0 or garbage

Solutions:

• Check if value is a string ("25.5") or number (25.5)

// If temperature is a string in JSON
float temp = atof(doc["temperature"]);

// If temperature is a number in JSON
float temp = doc["temperature"].as<float>();

Best Practices

• ✅ Always check DeserializationError before using parsed data
• ✅ Use | operator to provide default values for missing fields
• ✅ Limit string buffer sizes to match display capabilities
• ❌ Don’t allocate large JSON documents on the stack — use DynamicJsonDocument
• ❌ Avoid parsing the entire JSON if you only need a few fields
• ❌ Don’t ignore parsing errors — they indicate data quality issues

Summary

1. ArduinoJson is the standard JSON library for ESP32 — recommended v7
2. Always check parsing errors — invalid JSON should not update the display
3. Use doc["key"] | default syntax for safe field extraction
4. Nested JSON requires chained access: doc["parent"]["child"]
5. Array access uses index: doc["array"][0]
6. Memory planning is critical — use ArduinoJson Assistant for accurate sizing

References

• ArduinoJson Documentation
• ArduinoJson v7 Migration Guide
• ArduinoJson Assistant (Memory Calculator)

---

Target Audience: Alibaba.com IoT Pre-sales Engineers
Status: ✅ Completed