Diff Checker Guide: JSON/XML Comparison & API Response Diffing

Data comparison is a frequent task in daily development and operations: verifying API changes, checking config file differences, and comparing expected vs. actual responses in regression tests. Mastering efficient JSON/XML comparison techniques can significantly improve debugging speed and data quality. This guide covers comparison strategies and real-world scenarios.

Why Compare Data?

The need for data comparison spans the entire development lifecycle:

• API version upgrades: Is the new API response backward-compatible? What fields were added?
• Config change review: What's different between production and staging configs?
• Data migration verification: Is the data consistent before and after migration?
• Regression testing: Did code changes introduce unexpected response modifications?
• Bug investigation: Compare anomalous data with normal data to quickly pinpoint the differing fields

Key Challenges in JSON Comparison

1. Key Ordering

The JSON spec (RFC 8259) explicitly states that object keys are unordered. This means {"a":1,"b":2} and {"b":2,"a":1} are semantically equivalent. A proper JSON diff tool should support an "ignore key order" mode.

2. Formatting Differences

Whitespace, indentation style, and line breaks don't affect JSON semantics, but a simple text diff generates excessive noise. The correct approach is to parse first, then compare—rather than doing a raw text diff.

3. Data Type Differences

In JSON, 1 (number) and "1" (string) are different values, and null is distinct from a missing field. You need to be clear about whether types should be distinguished during comparison.

4. Floating-Point Precision

The result of 0.1 + 0.2 is 0.30000000000000004, not 0.3. Consider using tolerance-based comparison instead of strict equality.

JSON Comparison Strategies

Shallow vs. Deep Comparison

Shallow comparison only checks top-level fields:

const shallowDiff = (obj1, obj2) => {
  const keys1 = Object.keys(obj1);
  const keys2 = Object.keys(obj2);
  return {
    added: keys2.filter(k => !keys1.includes(k)),
    removed: keys1.filter(k => !keys2.includes(k)),
    changed: keys1.filter(k => obj1[k] !== obj2[k])
  };
};

Deep comparison recursively checks all nested levels:

const deepDiff = (obj1, obj2, path = '') => {
  const diffs = [];
  const allKeys = new Set([...Object.keys(obj1), ...Object.keys(obj2)]);
  
  for (const key of allKeys) {
    const currentPath = path ? `${path}.${key}` : key;
    if (!(key in obj1)) {
      diffs.push({ path: currentPath, type: 'added', value: obj2[key] });
    } else if (!(key in obj2)) {
      diffs.push({ path: currentPath, type: 'removed', value: obj1[key] });
    } else if (typeof obj1[key] === 'object' && typeof obj2[key] === 'object') {
      diffs.push(...deepDiff(obj1[key], obj2[key], currentPath));
    } else if (obj1[key] !== obj2[key]) {
      diffs.push({ path: currentPath, type: 'changed', 
        old: obj1[key], new: obj2[key] });
    }
  }
  return diffs;
};

Comparison Result Example

+  "email": "alice@example.com"
-  "phone": "13800000000"
~  "age": 28 → 29
+  "address": { "city": "Shanghai" }

API Response Comparison in Practice

Comparing API responses is one of the most common scenarios in development. Here's a systematic approach:

Step 1: Capture Baseline Data

# Save current API response as baseline
curl -s https://api.example.com/users/1 | jq '.' > baseline.json

# Save response headers
curl -s -D headers.txt https://api.example.com/users/1 > body.json

Step 2: Run the Comparison

# Get new version response
curl -s https://api.example.com/v2/users/1 | jq '.' > current.json

# Compare using diff
diff baseline.json current.json

# Structural comparison using jq
diff <(jq -S 'keys | sort' baseline.json) <(jq -S 'keys | sort' current.json)

Step 3: Handle Dynamic Fields

API responses typically contain dynamic values like timestamps and UUIDs. Direct comparison produceslarge false positives. Filter these fields before comparing:

# Remove dynamic fields with jq before comparing
jq 'del(.timestamp, .requestId, .sessionToken)' baseline.json > clean_base.json
jq 'del(.timestamp, .requestId, .sessionToken)' current.json > clean_curr.json
diff clean_base.json clean_curr.json

XML Comparison Methods

XML comparison is more complex than JSON due to namespaces, attribute ordering, CDATA sections, and other factors.

xmldiff Tool

# Install xmldiff
pip install xmldiff

# Compare two XML files
xmldiff file1.xml file2.xml

# Formatted output
xmldiff --formatter=xml file1.xml file2.xml

Using the diff Command (After Formatting)

# Format XML first, then text diff
xmllint --format file1.xml > file1_fmt.xml
xmllint --format file2.xml > file2_fmt.xml
diff -u file1_fmt.xml file2_fmt.xml

Config File Comparison Best Practices

1. Version Your Configs

Track config files in Git version control and use git diff to monitor changes:

# View config changes
git diff config/production.json

# View changes at a specific point in time
git diff HEAD~5 config/production.json

2. Environment Comparison

# Compare dev and production configs
diff <(jq -S '.' config/dev.json) <(jq -S '.' config/prod.json)

# Focus on fields with different values
jq -S '.' config/dev.json > /tmp/dev.json
jq -S '.' config/prod.json > /tmp/prod.json
diff /tmp/dev.json /tmp/prod.json

3. Ignore Specific Fields

Certain fields will inevitably differ across environments (e.g., database connection strings). Exclude them from comparison results:

# Exclude specific paths using jq
jq 'del(.database.url, .redis.host, .logging.level)' config/dev.json
jq 'del(.database.url, .redis.host, .logging.level)' config/prod.json

Choosing a Diff Tool

FAQ