Telemetry Devices

Apache Solr Benchmark includes telemetry devices for collecting server-side metrics. Six devices are always enabled. Eight additional devices can be enabled on demand with --telemetry.

Both Solr 9.x (JSON format) and Solr 10.x (Prometheus text format) are supported. Format detection is automatic via the HTTP Content-Type header.

solr-jvm-stats

Collects JVM statistics from each Solr node via the Solr metrics API.

Enable:

solr-benchmark run --telemetry solr-jvm-stats ...

Metrics collected:

Metric	Unit	Description
`jvm_heap_used_bytes`	bytes	JVM heap memory currently used
`jvm_heap_max_bytes`	bytes	Maximum JVM heap size
`jvm_gc_count`	count	Total GC collections across all collectors
`jvm_gc_time_ms`	ms	Total GC wall time across all collectors
`jvm_gc_young_count`	count	Young-generation GC collection count
`jvm_gc_young_time_ms`	ms	Young-generation GC wall time
`jvm_gc_old_count`	count	Old-generation GC collection count
`jvm_gc_old_time_ms`	ms	Old-generation GC wall time
`jvm_thread_count`	count	Current JVM thread count
`jvm_thread_peak_count`	count	Peak JVM thread count since startup
`jvm_buffer_pool_direct_bytes`	bytes	Direct byte buffer pool memory used
`jvm_buffer_pool_mapped_bytes`	bytes	Memory-mapped buffer pool memory used

solr-node-stats

Collects Solr node-level and OS statistics.

Enable:

solr-benchmark run --telemetry solr-node-stats ...

Metrics collected:

Metric	Unit	Description
`cpu_usage_percent`	%	Process CPU load (0–100)
`os_memory_free_bytes`	bytes	Free physical OS memory
`node_file_descriptors_open`	count	Currently open file descriptors
`node_file_descriptors_max`	count	Maximum allowed file descriptors
`node_http_requests_total`	count	Total HTTP requests processed by Jetty
`query_handler_requests_total`	count	Total `/select` query handler requests
`query_handler_errors_total`	count	Total `/select` query handler errors
`query_handler_avg_latency_ms`	ms	Rolling average `/select` request latency

solr-collection-stats

Collects per-collection document count, segment count, and deleted doc count.

Enable:

solr-benchmark run --telemetry solr-collection-stats ...

Metrics collected (all tagged with collection metadata):

Metric	Unit	Description
`num_docs`	docs	Current document count
`num_deleted_docs`	docs	Number of deleted (soft-deleted) documents
`segment_count`	count	Number of Lucene segments
`index_size_bytes`	bytes	Total index size on disk

Notes: Collection stats are polled every 30 seconds (configurable). Uses both the Collections API and the Luke request handler (/admin/luke) for full statistics.

solr-query-stats

Collects query latency percentiles and filter cache hit ratio.

Enable:

solr-benchmark run --telemetry solr-query-stats ...

Metrics collected:

Metric	Unit	Description
`query_latency_p50_ms`	ms	50th percentile `/select` request latency
`query_latency_p99_ms`	ms	99th percentile `/select` request latency
`query_latency_p999_ms`	ms	99.9th percentile `/select` request latency
`query_requests_total`	count	Total `/select` handler request count
`query_errors_total`	count	Total `/select` handler error count
`query_cache_hit_ratio`	ratio	Filter cache hit ratio (0.0–1.0)

solr-indexing-stats

Collects update handler and merge metrics.

Enable:

solr-benchmark run --telemetry solr-indexing-stats ...

Metrics collected:

Metric	Unit	Description
`indexing_requests_total`	count	Total `/update` handler requests
`indexing_errors_total`	count	Total `/update` handler errors
`indexing_avg_time_ms`	ms	Rolling average `/update` request time
`index_merge_major_running`	count	Currently running major merges
`index_merge_minor_running`	count	Currently running minor merges

solr-cache-stats

Collects hit/miss/eviction and memory statistics for the three primary Solr caches.

Enable:

solr-benchmark run --telemetry solr-cache-stats ...

Metrics collected (all tagged with cache metadata: queryResultCache, filterCache, documentCache):

Metric	Unit	Description
`cache_hits_total`	count	Cache hits since Solr start
`cache_inserts_total`	count	Cache inserts since Solr start
`cache_evictions_total`	count	Cache evictions since Solr start
`cache_memory_bytes`	bytes	RAM used by this cache
`cache_hit_ratio`	ratio	Hit ratio (0.0–1.0)

Optional devices

Optional devices must be explicitly requested with --telemetry <name>. REST-based optional devices work with all pipelines. JVM/process devices inject flags into SOLR_OPTS before Solr starts and require the docker or from-distribution pipeline — they are silently skipped on benchmark-only.

segment-stats

Captures per-collection segment statistics via the Solr Luke request handler.

Pipeline: All pipelines Enable: --telemetry segment-stats

Collected on benchmark stop. Results are written to segment_stats.log in the benchmark log directory.

Metric (in log file)	Description
`numDocs`	Current document count
`maxDoc`	Maximum doc ID (includes deleted docs)
`deletedDocs`	Number of soft-deleted documents
`segmentCount`	Number of Lucene segments
`sizeInBytes`	Total index size on disk

shard-stats

Polls CLUSTERSTATUS and Core STATUS for each shard leader and records per-shard metrics. Skipped silently on standalone (non-SolrCloud) Solr.

Pipeline: All pipelines Enable: --telemetry shard-stats

Metric	Unit	Description
`shard_{name}_num_docs`	count	Document count for the named shard
`shard_{name}_size_bytes`	bytes	Index size for the named shard

The default poll interval is 60 seconds. Override with --telemetry-params shard-stats-sample-interval:30.

cluster-environment-info

Records Solr version, JVM version, and hardware info as run metadata. Called once at benchmark start.

Pipeline: All pipelines Enable: --telemetry cluster-environment-info

Metadata key	Description
`distribution_version`	Solr version string (e.g. `9.7.0`)
`jvm_version`	JVM version (e.g. `21.0.1`)
`jvm_vendor`	JVM vendor name (e.g. `OpenJDK 21`)
`cpu_logical_cores`	Number of logical CPU cores on the node
`cluster_node_count`	Number of live SolrCloud nodes

jfr

Enables Java Flight Recorder. Injects -XX:StartFlightRecording=... into SOLR_OPTS.

Pipeline: docker or from-distribution only Enable: --telemetry jfr

The flight recording is written to profile.jfr in the benchmark log directory. Requires OpenJDK 11 or later. To use a custom recording template, pass --telemetry-params recording-template:/path/to/template.jfc.

gc

Enables GC logging (Java 9+ -Xlog: format). Injects the GC log flags into SOLR_OPTS.

Pipeline: docker or from-distribution only Enable: --telemetry gc

GC output is written to gc.log in the benchmark log directory. The default log configuration is gc*=info,safepoint=info,age*=trace. Override with --telemetry-params gc-log-config:gc*=debug.

jit

Enables JIT compiler logging. Injects -XX:+LogCompilation and related flags into SOLR_OPTS.

Pipeline: docker or from-distribution only Enable: --telemetry jit

JIT output is written to jit.log in the benchmark log directory.

heapdump

Captures a heap dump (jmap -dump) from the Solr JVM when the benchmark finishes.

Pipeline: docker or from-distribution only Enable: --telemetry heapdump

The heap dump is written to heap_at_exit_{pid}.hprof in the benchmark log directory. On Docker pipelines, docker exec is used automatically.

disk-io

Measures disk I/O consumed by the Solr process during the benchmark run. Always active on provisioned pipelines (not opt-in).

Pipeline: docker or from-distribution (always active)

Metric	Unit	Description
`disk_io_read_bytes`	bytes	Bytes read by the Solr process
`disk_io_write_bytes`	bytes	Bytes written by the Solr process

Using multiple devices

# Always-on devices (no --telemetry needed):
solr-benchmark run \
  --pipeline benchmark-only \
  --target-hosts localhost:8983 \
  --workload nyc_taxis

# Add optional REST devices on any pipeline:
solr-benchmark run \
  --pipeline benchmark-only \
  --target-hosts localhost:8983 \
  --workload nyc_taxis \
  --telemetry segment-stats,shard-stats,cluster-environment-info

# Add JVM profiling devices on provisioned pipelines:
solr-benchmark run \
  --pipeline docker \
  --distribution-version 9.10.1 \
  --workload nyc_taxis \
  --telemetry gc,jfr

Telemetry output location

Telemetry metrics are written to the same results.json file as workload metrics, under a "telemetry" key alongside the standard per-task metrics.