JSONPath Guide: Syntax Reference & jq Command-Line Tutorial

JSON has become the dominant data exchange format in modern development. Whether you're processing API responses, configuration files, or log data, quickly extracting specific fields from complex JSON structures is a core skill. This guide systematically covers JSONPath syntax and the jq command-line tool, taking you from beginner to advanced.

What Is JSONPath?

JSONPath is a query language for extracting specific data from JSON documents, inspired by XPath. It uses concise path expressions to locate nodes in JSON without writing complex parsing code. The most widely used specification was proposed by Stefan Goessner.

Imagine you have a deeply nested JSON object—traditionally, you'd need to traverse layer by layer to get the target value. With JSONPath, a single expression pinpoints it exactly. This is extremely useful in big data processing, API testing, and configuration management.

JSONPath Basic Syntax

Common Operators

In filter expressions, JSONPath supports the following comparison and logical operators:

• == Equal to (note: the JSONPath standard uses single =, but many implementations accept double ==)
• != Not equal to
• <, <=, >, >= Comparison operators
• && Logical AND, || Logical OR
• ~ Case-insensitive regex match (supported by some implementations)
• in Membership in a set

JSONPath Examples

Suppose we have the following JSON data:

{
  "store": {
    "books": [
      {"title": "Clean Code", "price": 59, "category": "Programming"},
      {"title": "Design Patterns", "price": 79, "category": "Programming"},
      {"title": "The Mythical Man-Month", "price": 49, "category": "Management"}
    ],
    "location": "Beijing"
  }
}

Extract All Book Titles

$.store.books[*].title
// Result: ["Clean Code", "Design Patterns", "The Mythical Man-Month"]

Get Books Priced Above 50

$.store.books[?(@.price > 50)]
// Result: [{title:"Clean Code",...}, {title:"Design Patterns",...}]

Recursively Find All price Fields

$.store..price
// Result: [59, 79, 49]

jq Command-Line Tool In Depth

jq is a lightweight command-line JSON processor, often called "the Swiss Army knife of the command line." Built on JSONPath principles, it offers richer functionality including pipes, variables, and programming features like functions.

Installing jq

# macOS
brew install jq

# Ubuntu/Debian
sudo apt install jq

# CentOS/RHEL
sudo yum install jq

# Windows (using scoop)
scoop install jq

Basic Queries

# Pretty-print output
cat data.json | jq '.'

# Extract a field
echo '{"name":"Alice","age":30}' | jq '.name'
# Output: "Alice"

# Strip quotes (-r flag)
echo '{"name":"Alice"}' | jq -r '.name'
# Output: Alice

# Array index
echo '[1,2,3,4,5]' | jq '.[2]'
# Output: 3

Pipes & Composition

One of jq's most powerful features is pipe operations, similar to Unix pipes:

# Extract all book titles and prices
cat books.json | jq '.store.books[] | {title, price}'

# Count books
cat books.json | jq '.store.books | length'

# Extract all prices and sum them
cat books.json | jq '[.store.books[].price] | add'

Advanced Filtering

# select() filter
cat data.json | jq '.store.books[] | select(.price > 50)'

# map() transformation
echo '[1,2,3,4,5]' | jq 'map(. * 2)'
# Output: [2,4,6,8,10]

# group_by() grouping
cat data.json | jq '.store.books | group_by(.category)'

# unique_by() deduplication
cat data.json | jq '.store.books | unique_by(.category)'

# sort_by() sorting
cat data.json | jq '.store.books | sort_by(.price)'

Object Operations

# Construct a new object
echo '{"name":"Alice","age":30}' | jq '{user: .name, years: .age}'

# Add a field
echo '{"name":"Alice"}' | jq '. + {"email": "alice@example.com"}'

# Delete a field
echo '{"name":"Alice","age":30,"city":"BJ"}' | jq 'del(.city)'

# Iterate key-value pairs
echo '{"a":1,"b":2}' | jq 'to_entries[]'

JSONPath vs jq: How to Choose?

Pro Tips

1. Processing Large JSON Files

When JSON files are very large, use streaming to avoid memory overflow:

cat huge.json | jq -c '.[]' | while read item; do
  echo "$item" | jq -r '.id'
done

2. JSON ↔ CSV Conversion

# JSON to CSV (for flat data)
cat data.json | jq -r '.[] | [.name, .age] | @csv'

# CSV to JSON
cat data.csv | jq -R -s 'split("\n") | .[1:] | map(split(",")) |
  map({"name": .[0], "age": (.[1] | tonumber)})'

3. Merging Multiple JSON Files

jq -s 'add' file1.json file2.json
jq -s 'flatten' array1.json array2.json

FAQ