Performance Benchmarks

Comprehensive performance data for Product-FARM’s rule evaluation engine, demonstrating enterprise-grade throughput and latency characteristics.

Executive Summary

Methodology

Hardware Specifications

Benchmarks were conducted on:

Test Environment

• DGraph: v23.x running locally
• Connection: Unix socket (minimal network overhead)
• Cache: Warm cache (preloaded data)
• Compilation: Release mode with LTO

Measurement Approach

• Tool: Criterion.rs microbenchmark framework
• Iterations: 10,000+ per benchmark
• Warmup: 1,000 iterations before measurement
• Statistics: Mean, median, standard deviation reported

Rule Evaluation Benchmarks

Simple Rules

Single-operation rules (arithmetic, comparison):

Example Rule:

{
  "expression": { "*": [{"var": "base_rate"}, {"var": "factor"}] }
}

Conditional Rules

Rules with branching logic:

Example Rule:

{
  "expression": {
    "if": [
      {"<": [{"var": "age"}, 25]}, {"*": [{"var": "base"}, 1.5]},
      {"<": [{"var": "age"}, 35]}, {"*": [{"var": "base"}, 1.2]},
      {"*": [{"var": "base"}, 1.0]}
    ]
  }
}

Complex DAG Rules

Multi-level dependency chains:

Throughput Benchmarks

Single-Threaded Performance

Continuous evaluation on a single thread:

Multi-Threaded Performance

Parallel evaluation across CPU cores:

Throughput vs Latency Trade-offs

Memory Profile

Cache Memory Usage

Memory consumption by cache component:

Compilation Overhead

Memory and time for bytecode compilation:

Garbage Collection

Product-FARM uses Rust’s ownership model - no GC pauses:

• Zero GC pauses - Deterministic memory management
• Predictable latency - No stop-the-world events
• Memory safety - Compile-time guarantees

Comparison with Alternatives

vs. Hardcoded Logic

vs. Other Rule Engines

Test Coverage

Backend Tests

E2E Test Suites

Running Your Own Benchmarks

Quick Benchmark

Run the standard benchmark suite:

cd backend
cargo bench

Specific Benchmarks

Target specific benchmark groups:

# Rule evaluation benchmarks
cargo bench --bench evaluation

# DAG execution benchmarks
cargo bench --bench dag

# Throughput benchmarks
cargo bench --bench throughput

Custom Benchmarks

Create benchmarks for your specific rules:

use criterion::{criterion_group, criterion_main, Criterion};
use product_farm::evaluate;

fn my_benchmark(c: &mut Criterion) {
    let product = load_product("my-product");
    let inputs = json!({"age": 30, "income": 75000});

    c.bench_function("my_rule_evaluation", |b| {
        b.iter(|| evaluate(&product, &inputs))
    });
}

criterion_group!(benches, my_benchmark);
criterion_main!(benches);

Interpreting Results

Criterion output example:

rule_evaluation/bytecode
                        time:   [325.42 ns 330.15 ns 335.21 ns]
                        thrpt:  [2.9834 Melem/s 3.0290 Melem/s 3.0728 Melem/s]
                 change: [-1.2% +0.3% +1.8%] (p = 0.72 > 0.05)
                        No change in performance detected.

• time: [lower, mean, upper] confidence interval
• thrpt: Throughput (millions of evaluations per second)
• change: Comparison to baseline (if available)

Optimization Recommendations

For Maximum Throughput

1. Enable bytecode compilation for all production rules
2. Use batch evaluation for multiple inputs
3. Scale threads to physical core count
4. Increase cache sizes for large rule sets
5. Use gRPC instead of REST for evaluation

For Minimum Latency

1. Warm the cache before traffic arrives
2. Pre-compile rules during product activation
3. Use single-thread mode for predictable latency
4. Monitor P99 latency not just mean
5. Avoid GC (Rust handles this automatically)

For Memory Efficiency

1. Tune cache sizes based on working set
2. Use lazy compilation for infrequently used rules
3. Monitor memory usage with metrics
4. Set appropriate limits per product

Next Steps

• Features - Deep dive into capabilities
• Architecture - System design
• API Reference - Complete API docs
• Quick Start - Get running in 5 minutes